Python Explain the principle of execution

I am a Python thread , My job is to explain what the executive programmer wrote Python code .

The reason is to explain and implement , Because Python It's a high-level language ,CPU I don't know that guy Python code , Dynamic translation when running CPU instructions .

I put Python Source code process “ compile ” in the future , Into a bytecode file :.pyc, This is a binary file , Human beings can't understand , Only I can understand .

Then my job is simple , Constantly take out the data in the bytecode file “ instructions ” Interpretation and execution , Until all instructions are executed , I can rest .


later , Multithreaded programming technology has become popular , I'm not the only thread in the process , Some new partners have come .

I thought we could live in peace , I didn't expect to be in trouble , We'll go our separate ways , Often make problems with memory space , I can't find anyone to carry the pot .

Finally one day , I can't stand it anymore , Call on everyone to discuss a solution .

“ Guys , We can't go our separate ways anymore , We are one Team, We should cooperate with each other , A thread can go very fast , But a group of threads can go further together !”

“ boss , Just say what you think ”, Another thread said .

“ or , Let's put a lock on it ! Simple and fast , Each thread executes code , Have to apply for this lock , The application cannot be executed until it is received , Or you'll have to wait ”, I said .

“ When will it be released ?”

“ Why don't you get a counter , Count to per thread 100 Just release , This ensures that others have the opportunity to perform ?”

“ How do you count ? Do you count every byte ?”

“ sure , But don't be so rigid , Some instructions are relatively simple , It will be finished soon , You don't have to count ”

“ Good is good , But if you haven't counted enough 100, But it's executing I/O Operation blocked , If you still occupy the lock , Isn't that a waste of resources ?”

The little friends were talking about it .

I thought about it and wanted to say :“ That's it , Combine the two situations , Normally count to 100 Just release the lock once , But if you encounter a blockage , Also release the lock in advance , how ?”

Everyone nodded , An agreement was reached , Then we gave the lock a name : Global interpreter lock GIL.

Since I used it GIL, Our big guys work more regularly , There is no longer any case of damaging public resources .

Upgraded version GIL

Later , Multi core technology began to rise , One CPU It can execute multiple threads at the same time . The friends spread the news happily .

“ boss , Now? CPU How many cores are there , We can do it together , Can you take that GIL Get rid of it , In this way, we can take advantage of multi-core .”

“ Yeah , next door Java Threads always laugh at us for looking crowded , There is actually only one thread executing ”

Easier said than done , That's how we've worked for so many years , It should be removed suddenly , If something goes wrong, no one dares to be responsible .

“ But boss , Now this GIL Lock unfair ”, The new thread complained .

“ What's unfair ?”

“ I'm going to execute the code , Found it locked in your hand , I had to wait in place , I fell asleep after waiting for a long time , It's hard to wait until you're released , The operating system woke me up , Ready to apply for a lock , It turns out you grabbed it again , What a waste of time ”, The new thread is full of grievances .

“ Yeah , boss , I found out, too , This is not an accident , I've been watching for a long time , It happens all the time ! Frequently awakened , But I found that I was busy in vain , Waste in vain CPU resources , Everyone is very angry ”, Another thread partner also said .

I'm a little embarrassed ,“ Um , This is really a problem ”

“ It's not over yet ”, The new thread continues :“ Now count according to the number of bytecode instructions , But some scripts are simple , Some are complicated , Cause the same number to 100, Some threads can run for a long time , And some of them run quickly , It's not fair ”

The questions raised by the partners are very important , Looks like it's time, right GIL An upgrade has been made .

After a heated discussion , We improved the original GIL, With a new strategy : No longer count , Switch to time slice : The execution time slice of each thread is 5000 Microsecond .

To ensure release GIL after , Don't be robbed by yourself right away , A lock is added to force thread switching

After improvement , Now it's fair , You guys have nothing to say , You can work at ease again .


Python It is a language of interpretation and execution , It has powerful third-party libraries and cross platform capabilities , In recent years Python A second spring , Sweep reptile ,web development , Machine learning and many other fields .

But for a long time ,Python What is most criticized is that it has a lock :GIL, This lock makes Python Can't really implement multi-threaded execution , Unable to utilize multi-core CPU High performance .

actually , This lock heel Python It has nothing to do with half a dime , But responsible for interpretation and implementation Python Interpreter for :CPython Pot .

CPython Yes C Written in Python interpreter , It is also the most widely used Python interpreter , Generally, when there is no special instruction , say Python That's it CPython interpreter .

Python At the beginning of its birth , Multithreading technology is far less popular than it is today , Even multicore CPU Also Python Many years after its birth . In the early interpreters, in order to support multiple threads , Used rough GIL To control , Convenient and simple at the same time , Also became CPython A huge historical burden .

stay Python3.2 before ,Python Use a simple counting method to count and control the execution time of each thread . After that , A fairer time slice method is introduced to upgrade and replace .

The past two decades , There were many big cattle trying to get rid of it completely GIL, But there was no perfect success .

although Python Not completely removed GIL, Fortunately , Several other ways are provided “ save the nation by a devious path ” Achieve concurrency :Ctypes
By writing C Language extension and Python interactive , stay C Language level bypass GIL Realize multi-core utilization .

MultiProcess Python Provided MultiProcess, Bypass through multiple processes GIL

Synergetic process Co process is also called user thread ,Python3.4 Support for collaborative process is added after version , It also provides a choice for performance improvement .

This article tells the story in first person vernacular GIL stay CPython Working principle of interpreter , Does everyone understand ?