I would like to test how my application responds to functions that hold the GIL. Is there a convenient function that holds the GIL for a predictable (or even a significant) amount of time?
My ideal function would be something that operated like time.sleep except that, unlike sleep, it would hold the GIL
A simple, but hacky, way to hold the GIL is to use the re module with a known-to-be-slow match:
import re
re.match(r'(a?){30}a{30}', 'a'*30)
On my machine, this holds the GIL for 48 seconds with Python 2.7.14 (and takes almost as long on 3.6.3). However, this relies on implementation details, and may stop working if the re module gets improvements.
A more direct approach would be to write a c module that just sleeps. Python C extensions don't automatically release the GIL (unlike, say, ctypes). Follow the hellomodule example here, and replace the printf() with a call to sleep() (or the Windows equivalent). Once you build the module, you'll have a GIL holding function you can use anywhere.
You can use a C library's sleep function in "PyDLL" mode.
# Use libc in ctypes "PyDLL" mode, which prevents CPython from
# releasing the GIL during procedure calls.
_libc_name = ctypes.util.find_library("c")
if _libc_name is None:
raise RuntimeError("Cannot find libc")
libc_py = ctypes.PyDLL(_libc_name)
...
libc_py.usleep(...)
(See https://gist.github.com/jonashaag/d455671003205120a864d3aa69536661 for details on how to pickle the reference, for example if using in a distributed computing environment.)
Related
I found this sentence about GIL on the Python wiki:
Luckily, many potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL.
Is there a list of functions outside the GIL (at least in Python standard libraries)?
Or how can I know whether a specific function is outside the GIL?
Starting from the original naming and tracing to the current implementation you can find these private functions:
_gil_initialize()
gil_created()
create_gil()
destroy_gil()
recreate_gil()
drop_gil()
take_gil()
with these you can trace up to:
PyEval_AcquireLock()
PyEval_ReleaseLock()
etc
in the ceval.c file. If you grep for those, you'll get to the parts of code that acquire or release the lock. If there is a release, you might assume GIL present in that module at least somewhere. The other side is looking up modules that do not have the lock acquiring, thus do not manipulate the GIL / run out of it.
This should give you some steps to start tracing it, if you really want to go that way. However, I doubt there's a finite list of functions even for the standard library as the codebase is quite large/volatile to even keep a documentation for that. I'd like to be proven wrong though.
Also there are these two macros, as pointed in the comments:
Py_BEGIN_ALLOW_THREADS
Py_END_ALLOW_THREADS
which should find more matches in the code. (GitHub link might require login)
Alternatively, in case it's completely locked out by mandatory login screen:
git clone --depth 1 https://github.com/python/cpython
grep -nr -C 5 Py_BEGIN_ALLOW_THREADS cpython
For the quote you have:
Luckily, many potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL.
I'd rather go with the explanation that performance-dependent tasks are implemented in lower-level language (such as C; compared to Python) such as I/O, calculations, etc. And the modules implemented in C that do the hard work try not to acquire the lock (or release it beforehand) when working hard, then acquiring it when manipulating with the Python (interpreter's) context/variables so the result can be stored. Thus keeping the hard work on the performance level of its C implementation, not being slowed down by communicating with the interpreter's internals.
I am working on some changes to a library which embeds Python which require me to utilize sub-interpreters in order to support resetting the python state, while avoiding calling Py_Finalize (since calling Py_Initialize afterwards is a no-no).
I am only somewhat familiar with the library, but I am increasingly discovering places where PyGILState_Ensure and other PyGILState_* functions are being used to acquire the GIL in response to some external callback. Some of these callbacks originate from outside Python, so our thread certainly doesn't hold the GIL, but sometimes the callback originates from within Python, so we definitely hold the GIL.
After switching to sub-interpreters, I almost immediately saw a deadlock on a line calling PyGILState_Ensure, since it called PyEval_RestoreThread even though it was clearly already being executed from within Python (and so the GIL was held):
For what it's worth, I have verified that a line that calls PyEval_RestoreThread does get executed before this call to PyGILState_Ensure (it's well before the first call into Python in the above picture).
I am using Python 3.8.2. Clearly, the documentation wasn't lying when it says:
Note that the PyGILState_* functions assume there is only one global interpreter (created automatically by Py_Initialize()). Python supports the creation of additional interpreters (using Py_NewInterpreter()), but mixing multiple interpreters and the PyGILState_* API is unsupported.
It is quite a lot of work to refactor the library so that it tracks internally if the GIL is held or not, and seems rather silly. There should be a way to determine if the GIL is held! However, the only function I can find is PyGILState_Check, but that's a member of the forbidden PyGILState API. I'm not sure it'll work. Is there a canonical way to do this with sub-interpreters?
I've been pondering this line in the documentation:
Also note that combining this functionality with PyGILState_* APIs is delicate, because these APIs assume a bijection between Python thread states and OS-level threads, an assumption broken by the presence of sub-interpreters.
I suspect that the issue was that there's something involving thread local storage on the PyGILState_* API.
I've come to think that it's actually not really possible to tell if the GIL is held by the application. There's no central static place where Python stores that the GIL is held, because it's either held by "you" (in your external code) or by the Python code. It's always held by someone. So the question of "is the GIL held" isn't the question the PyGILState API is asking. It's asking "does this thread hold the GIL", which makes it easier to have multiple non-Python threads interacting with the interpreter.
I overcame this issue by restoring the bijection as best I could by creating a separate thread per sub-interpreter, with the order of operations being very strictly as follows:
Grab the GIL in the main thread, either explicitly or with Py_Initialize (if this is the first time). Be very careful, the thread state from Py_Initialize must only ever be used in the main thread. Don't restore it to another thread: Some module might use the PyGILState_* API and the deadlock will happen again.
Create the thread. I just used std::thread.
Spawn the subinterpreter with Py_NewInterpreter. Be very careful, this will give you a new thread state. As with the main thread state, this thread state must only be used from this thread.
Release the GIL in the new thread when you're ready for Python to do its thing.
Now, there's some gotchas I discovered:
asyncio in Python 3.8-3.9 has a use-after-free bug where the first interpreter loading it manages some resources. So if that interpreter is ended (releasing those resources) and a new interpreter grabs asyncio, there will be a segfault. I overcame this by manually loading asyncio through the C API in the main interpreter, since that one lives forever.
Many libraries, including numpy, lxml, and several networking libraries will have trouble with multiple subinterpreters. I believe that Python itself is enforcing this: An ImportError results when importing any of these libraries with: Interpreter change detected - This module can only be loaded into one interpreter per process. This so far seems to be an insurmountable issue for me since I do require numpy in my application.
I am using Python languages and I use CPU threads from the threading thread.Threading wrapper. In some way, the Python interpreter converts my code into PYC byte code with its JIT. (Please provide a reference to Python bytecode standard, but as far as I know standard does not exist. As well it does not exist a standard for a language).
Then these virtual commands are executed. The real commands for Intel'd CPUs are x86/x64 instructions, and for ARM's CPU are AArch64/AArch32 instructions.
My problem - I want to make an action within the Python programming language, that enforces an ordering constraint between the memory operations.
What I want to know:
Q1: How I can emit a command
mfence if Python program is running in x86/x64 CPU
Or instruction like atomic_thread_fence() for LLVM-IR
Q2: How I can specify that some memory is volatile and should not be put into the CPU register for optimization purposes.
CPython does not have a JIT - though it may do one day.
So your code is only converted into bytecode, which will be interpreted, and not into actual Intel/etc. machine code.
Additionally, Python has what's known as the GIL - Global Interpreter Lock - meaning that even if you have multiple Python threads in a process, only one of them can be interpreting code at once - though this may also change one day. Threads were frequently useful for doing I/O, because I/O operations are able to happen at the same time, but these days asyncio is a good competitor for doing that.
So, in response to Q1, it doesn't make any sense to "put an mfence in Python code" (or the like).
Instead, what you probably want to do, if you want to enforce ordering constraints between one bit of code being executed and another, is use more high-level strategies, such as Python threading.Lock, queue.Queue, or similar equivalents in the asyncio world.
As for Q2, are you thinking of the C/C++ keyword volatile? This is frequently mistakenly thought of as a way to make memory access atomic for use in multithreading - it isn't. All that C/C++ volatile does is ensure that memory reads and writes happen exactly as specified rather than being possibly optimised out. What is your use case? There are all sorts of strategies one can use in Python to optimise code, but it's an impossible question to answer without knowing what you're actually trying to do.
Answers to comments
The CPU executes instructions. Somebody should emit this instruction. I'm calling a JIT a part inside a Python interpreter that emits instructions at the end of the day.
CPython is an interpreter - it does not emit instructions. JITs do emit instructions, but as stated above, CPython does not have a JIT. When you run a Python script, CPython will compile your text-based .py file into bytecode, and then it will spend the rest of its time working through the bytecode and doing what the bytecode says. The only machine instructions being executed are those that are emitted by whoever compiled CPython.
If you compile a Python script to a .pyc and then execute that, CPython will do exactly the same, it will just skip the "compile your text-based .py file into bytecode" part as it's already done it - the result of that step is stored in the .pyc file.
I was a bit vague in naming. Do you mean in Python, the instruction is reexecuted each time the interpreter meats the instruction?
A real CPU executing machine code will "re-execute" each instruction as it reads it, sure. CPython will do the same thing with Python bytecode. (CPython will execute a whole bunch of real machine code instructions each time it reads one Python instruction.)
Thanks, I have found this notice https://docs.python.org/3/extending/extending.html "CPython implementation detail: In CPython, due to the Global Interpreter Lock, only one thread can execute Python code at once ". Ok, so when Python thread will go with C++/C bindings into native code then what will happen? Q1-A - can in that time Python another thread be executed? Q1-B - If inside C++ code I will create another thread what will happen?
Native code can release the GIL but must lock it again before returning to Python code.
Typically, native code that does some CPU-intensive work or does some I/O that requires waiting would release the GIL while it does that work or waits on that I/O. This allows Python code on another Python thread to run at the same time. But at no point does Python code run on two threads at once. That's why it makes no sense to put native ordering instructions and the like in Python code.
Native code that needs to use native ordering instructions for its own purposes will obviously do that, but that is C/C++ code, not Python code. If you want to know how to put native ordering instructions in a C/C++ Python extension, then look up how to do it in any C/C++ code - the fact that it's a Python extension is irrelevant.
Basically, either write Python code and use high-level strategies like what I mentioned above, or write a native extension in C/C++ and do whatever you need to do in that language.
I need to learn more about GIL and seems that there is a good study of GIL from David Beazley https://dabeaz.com/python/UnderstandingGIL.pdf But #Keiji - you maybe can be wrong with Q1 - CPython Threads seems to be a real thread, and if implementor of C++/C extensions (Almost all libraries for Python) will decide to release GIL lock - it's possible to do... So Q1 still has sense...
I've covered this above. Python code can't interact with native code in a way that would require putting native ordering instructions in Python code.
Back to the question - I mean volatile in sense of C++ getting rid of compiler optimization to not allow optimized variables to be put into the register. In C++ it does not guarantee atomicity and it does not guarantee memory fence. So question regarding volatile how I can specify for integer variable or user-defined type?
If you want to make something in C/C++ be volatile, use the volatile keyword. If you're writing Python code, it doesn't make any sense to make something volatile.
About Python Threads:
The Python thread first of all is tricky. Interpreters use real POSIX/WinAPI threads. Threads thread.Threading under the hood is the real threads.
The thread execution model is pretty specific and can be called "cooperative multitasking" due to one enthusiast (David Beazley, https://www.dabeaz.com/about.html)
As David Beazley stated https://www.dabeaz.com/python/UnderstandingGIL.pdf
When a thread is in the I/O waiting for the Thread release global lock (called GIL lock). David Beazley stated that there is a way to release the lock after the system call.
Next, there is a "tick" instruction in Python VM instructions. If some thread is CPU-bound then the thread will execute that "tick" instruction. (Roughly speaking it occurs every 100ms)
In tick, each thread tries to release GIL and acquire "tick" one more time.
There is no thread scheduler in Python
Multithreading in Python is in fact hurts performance.
The reason between Java and Python threads is
Java is designed to lock on the resources
Python is designed to lock the thread itself(GIL)
So Python's implementation performs better on a machine with singe core processor. This is fine 10-20 years before. With the increasing computing capacity, if we use multiprocessor machine with same piece of code, it performs very badly.
Is there any hack to disable GIL and use resource locking in Python(Like Java implementation)?
P.S. My application is currently running on Python 2.7.12. It is compute intensive with less I/O and network blocking. Assume that I can't use multiprocessing for my use case.
I think the most straight way for you, that will give you also a nice performance increase is to use Cython.
Cython is a Python superset that compiles Python-like code to C code (which makes use of the cPython API), and from there to executable. It allows one to optionally type variables, that then can use native C types instead of Python objects - and also allows one direct control of the GIL.
It does support a with nogil: statement in which the with block runs with the GIL turned off - if there are other threads running (you use the normal Python threading library), they won't be blocked while code is running on the marked with block.
Just keep in mind that the GIL is there for a reason: it is thanks to it that global complex objects like lists and dictionaries work without the danger of getting into an inconsistent state between treads. But if your "nogil" blocks restrict themselves to local data structures, you should have no problems.
Check the Cython project - and here is an specific example of turning off the GIL:
https://lbolla.info/blog/2013/12/23/python-threads-cython-gil
As title say, does Python cStringIO protect their internal structures for multithreading use?
Thank you.
Take a look at an excellent work on explaining GIL, then note that cStringIO is written purely in C, and its calls don't release GIL.
It means that the running thread won't voluntarily switch during read()/write() (with current virtual machine implementation). (The OS will preempt the thread, however other Python threads won't be able to acquire GIL.)
Taking a look at the source: Python-2.7.1/Modules/cStringIO.c there is no mention about internals protection. When in doubt, look at source :)
I assume you are talking about the CPython implementation of Python.
In CPython there is a global interpreter lock which means that only a single thread of Python code can execute at a time. Code written in C will therefore also be effectively single threaded unless it explicitly releases the global lock.
What that means is that if you have multiple Python threads all using cStringIO simultaneously there won't be any problem as only a single call to a cStringIO method can be active at a time and cStringIO never releases the lock. However if you call it directly from C code which is running outside the locked environment you will have problems. Also if you do anything more complex than just reading or writing you will have issues, e.g. if you start using seek as your calls may overlap in unexpected ways.
Also note that some methods such as writelines can invoke Python code from inside the method so in that case you might get other output interleaved inside a single call to writelines.
That is true for most of the standard Python objects: you can safely use objects from multiple threads as the individual operations won't break, but the order in which things happen won't be defined.
It is as "thread-safe", as file operations can be (which means — not much). The Python implementation you're using has Global Interpreter Lock (GIL), which will guarantee that each individual file operation on cStringIO will not be interrupted by another thread. That does not however guarantee, that concurrent file operations from multiple threads won't be interleaved.
No it is not currently thread safe.