homepage

If several threads try to start a multiprocessing.Pool at the same time when no pool has been started before this often fails with an exception like this (the exact import varies):
Exception in thread Thread-2:
Traceback (most recent call last):
 File "/tmp/py3.9.0rc1/lib/python3.9/threading.py", line 950, in _bootstrap_inner
 self.run()
 File "/tmp/py3.9.0rc1/lib/python3.9/threading.py", line 888, in run
 self._target(*self._args, **self._kwargs)
 File "/tmp/py3.9.0rc1/lib/python3.9/multiprocessing/context.py", line 118, in Pool
 from .pool import Pool
ImportError: cannot import name 'Pool' from partially initialized module 'multiprocessing.pool' (most likely due to a circular import) (/tmp/py3.9.0rc1/lib/python3.9/multiprocessing/pool.py)
This happens even if Pool was imported before starting the threads and is new in 3.9. It's easy to work around by starting a pool in the main thread before starting the other threads.
I have attached a minimal example that triggers it. Tested on Debian stable and FreeBSD 11.3.

I see the same in Python 3.10 on windows 10.
If I change the relative imports to absolute imports in a couple of functions in multiprocessing.context as below, the attached (pool_error_on_3.9.py) script not longer raises the exception.
 def SimpleQueue(self):
 '''Returns a queue object'''
 from multiprocessing.queues import SimpleQueue
 return SimpleQueue(ctx=self.get_context())
 def Pool(self, processes=None, initializer=None, initargs=(),
 maxtasksperchild=None):
 '''Returns a process pool object'''
 from multiprocessing.pool import Pool
 return Pool(processes, initializer, initargs, maxtasksperchild,
 context=self.get_context())

I think this is no a bug, on the basis that multiprocessing.Pool is not thread-safe.

multiprocessing module used to work multithread environment until https://bugs.python.org/issue35943 merged.
I guess we can make multiprocessing thread-safe again if we move local imports to global.
Does it make sense?

Some my observation at https://discuss.python.org/t/differences-between-3-8-and-3-9-in-importing-module/5520 .

Example code (fails):
import os, concurrent.futures
def parallel_callback( arg ):
 return os.getpid()
def parallel( *args ):
 def thread_callback( param ):
 with concurrent.futures.ProcessPoolExecutor(max_workers=1) as executor:
 future = executor.submit( parallel_callback, param )
 pid = future.result()
 print( 'pid:', pid )
 return pid
 with concurrent.futures.ThreadPoolExecutor(max_workers=len(args)) as executor:
 future = executor.map( thread_callback, args )
 results = list(future)
 print( 'DONE' )
parallel( 1, 2, 3 )