[Python-Dev] GIL, Python 3, and MP vs. UP

Wed Sep 21 00:10:59 CEST 2005

On Sep 20, 2005, at 5:43 PM, Guido van Rossum wrote:
> On 9/20/05, John J Lee <jjl at pobox.com> wrote:
>>> threading is not the only, nor the best, concurrency model.
>> But maybe these chips designed with threading in mind blow that 
>> argument
>> out of the water. I don't know enough to know whether that's true or
>> not...
>>>> I don't know that any chips are designed with threading in mind. Fast
> threading benefits from fast context switches which benefits from
> small register sets. I believe the trend is towards ever large
> register sets. Also, multiple processors with shared memory don't
> scall all that well; multiple processors with explicit IPC channels
> scale much better. All arguments for multi-processing and against
> multi-threading.

Well, in many operating systems, there isn't a whole lot of 
difference between threads and processes (both are kernel threads). 
When using threads, you can typically still use IPC, so you could 
scale similarly.
I think the biggest argument for threading is simply that lots of 
existing C/C++ code wants to use threads. What we have now works OK, 
but you can't reliably use Python in a real-time thread (e.g. a 
CoreAudio callback) or (reliably) in a signal handler because it 
might block too long because of something else going on. Of course, 
a lot of other design decisions in Python would prevent you from 
using it in that context too, so GIL-free threading wouldn't be a 
panacea.
Personally my biggest issue with the way the CPython VM works is that 
you can't reliably do multiple interpreters in a single process. If 
that worked well, you could start an independent interpreter per 
thread and with a per-interpreter GIL you'd have pretty much 
everything you needed... but this would horribly break the C API and 
may be a performance hit.
My use case for this isn't so much about threads, but plug-ins. 
Writing multiple Python-based plug-ins for an application is always a 
mess, because they share too much (sys.modules, sys.path, etc.). 
PyObjC would benefit greatly from this feature, because you can write 
Python-based plug-ins for any Cocoa app that supports plug-ins, even 
if they're otherwise unaware of Python's existence. There are 
workarounds, of course, with import hooks and similar hacks. I think 
that mod_python would also benefit from this, and probably other such 
systems.
-bob