homepage

The package level imports from the new multiprocessing package exhibit
some very strange behaviour because they are actually functions
pretending to be classes:
Python 2.6b1+ (trunk:64945, Jul 14 2008, 20:00:46)
[GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing as mp
>>> isinstance(mp.Lock(), mp.Lock)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: isinstance() arg 2 must be a class, type, or tuple of classes
and types
>>> mp.Lock.__name__
'Lock'
>>> mp.Lock.__module__
'multiprocessing'
>>> mp.Lock().__class__.__name__
'Lock'
>>> mp.Lock().__class__.__module__
'multiprocessing.synchronize'
The delayed import functions in multiprocessing.__init__ look like a
serious misfeature to me. I'd be inclined to replace them with "from
.synchronize import *" and "from .process import *" (leaving anything
which isn't covered by those two imports to be retrieved directly from
the relevant mp submodule)

Setting to deferred blocker, since this really needs to be dealt with
for RC1 (probably too close to b3 to get it discussed and dealt with for
that).

> The delayed import functions in multiprocessing.__init__ look like a
> serious misfeature to me. I'd be inclined to replace them with "from
> .synchronize import *" and "from .process import *"
+1
(or, even better, qualified than wildcard imports)

Patch attached that removes the misleading "convenience" functions,
replacing them with explicit imports of the appropriate names.
The patch also adds docstrings to some of the original class definitions
that were missing them.
No changes were needed to the multiprocessing tests - they all still
passed with this change, and the docs are still accurate as well (I
would actually say this change makes the docs MORE accurate).
Python 2.6b3+ (trunk:66083, Aug 31 2008, 19:00:32)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import multiprocessing as mp
>>> isinstance(mp.Lock(), mp.Lock)
True
>>> mp.Lock.__name__
'Lock'
>>> mp.Lock.__module__
'multiprocessing.synchronize'

Interesting - in some of the other work I was doing regarding the PEP 8
compliant alternative threading API, I noticed that the threading module
contains similar gems such as:
def Event(*args, **kwds):
 return _Event(*args, **kwds)
Using a factory function to discourage subclassing is one thing, but why
name the factory function as if it was a still a class?

This is why multiprocessing had them nick - the threading module does
On Sep 1, 2008, at 9:07 AM, Nick Coghlan <report@bugs.python.org> wrote:
>
> Nick Coghlan <ncoghlan@gmail.com> added the comment:
>
> Interesting - in some of the other work I was doing regarding the 
> PEP 8
> compliant alternative threading API, I noticed that the threading 
> module
> contains similar gems such as:
>
> def Event(*args, **kwds):
> return _Event(*args, **kwds)
>
> Using a factory function to discourage subclassing is one thing, but 
> why
> name the factory function as if it was a still a class?
>
> _______________________________________
> Python tracker <report@bugs.python.org>
> <http://bugs.python.org/issue3589>
> _______________________________________

Patch reviewed/tested and I also confirmed that this doesn't affect the 
examples. I submitted the patch in r66114 

Reopening, there's a bug that the tests/examples/etc didn't catch (and 
nor did I), after the patch application:
woot:python-trunk jesse$ ./python.exe 
Python 2.6b3+ (trunk:66112:66114M, Sep 1 2008, 13:00:19) 
[GCC 4.0.1 (Apple Inc. build 5484)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import _multiprocessing
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "/Users/jesse/open_source/subversion/python-
trunk/Lib/multiprocessing/__init__.py", line 148, in <module>
 from multiprocessing.synchronize import (Lock, RLock, Condition, 
Event,
 File "/Users/jesse/open_source/subversion/python-
trunk/Lib/multiprocessing/synchronize.py", line 29, in <module>
 SEM_VALUE_MAX = _multiprocessing.SemLock.SEM_VALUE_MAX
AttributeError: 'module' object has no attribute 'SemLock'
>>>

Ben is backing out the patch now

Given how long I've been using the threading module without realising it
does the same thing, I'm actually prepared to live with the wrapper
functions rather than messing with this so close to release.
As Fredrik noted in the python-dev thread, the threading versions of
these are already explicitly documented as being factory functions
rather than classes (and as a reference to _thread.allocate_lock,
threading.Lock has good reason to be a factory function rather than a
class), so it may be appropriate to do the same thing for multiprocessing.

Sounds good to me. :)

Le lundi 01 septembre 2008 à 21:22 +0000, Nick Coghlan a écrit :
> As Fredrik noted in the python-dev thread, the threading versions of
> these are already explicitly documented as being factory functions
> rather than classes (and as a reference to _thread.allocate_lock,
> threading.Lock has good reason to be a factory function rather than a
> class), so it may be appropriate to do the same thing for multiprocessing.
"Foolish consistency, etc.". :-)
I think we should fix this before the releases rather than live with a
stupid indirection that doesn't seem to serve any useful purposes.
Unless the fix is too complicated of course.

Not so much "too complicated" as "potentially touches a lot of code and
not something I want to fiddle with this close to release".
As I noted on python-dev, it's actually a change that can easily be
handled through the normal API deprecation process, so it can be
reconsidered at a later date.