homepage

Using Python 2.7 x32 on Windows XP
Attempting to create a multiprocessing.pool.ThreadPool
in a child thread created using threading.Thread, an
AttributeError is thrown. A ThreadPool created in the 
main thread can be passed to the child thread and used.
Exact text of exception
-------------------
 File "D:\Dev\Python27\lib\multiprocessing\dummy\__init__.py", line 47, in star
t
 self._parent._children[self] = None
AttributeError: 'Thread' object has no attribute '_children'
Demonstration Code
-------------------
import unittest
from threading import Thread
from multiprocessing.pool import ThreadPool
def f(x):
 return x*x
def create_and_run(cb, pool = None):
 if not pool:
 pool = ThreadPool(2)
 r = pool.map_async(f, range(10))
 cb(r.get())
class TestThreadPool(unittest.TestCase):
 def setUp(self):
 self.expected = [f(x) for x in range(10)]
 def callback(self, data):
 self.data = data
 def test_creating_pool_in_mainthread(self):
 """Test multiprocessing.pool.ThreadPool from main thread"""
 self.data = None
 create_and_run(self.callback)
 self.assertEqual(self.data, self.expected)
 def test_creating_pool_in_subthread(self):
 """Test multiprocessing.pool.ThreadPool from a child thread."""
 self.data = None
 t = Thread(target=create_and_run, args=[self.callback])
 t.start()
 t.join()
 self.assertEqual(self.data, self.expected)
 def test_creating_pool_in_subthread_workaround(self):
 """Test running a ThreadPool created in main thread, used in child."""
 self.data = None
 pool = ThreadPool(2)
 t = Thread(target=create_and_run, args=[self.callback, pool])
 t.start()
 t.join()
 self.assertEqual(self.data, self.expected)
if __name__ =='__main__':
 suite = unittest.TestLoader().loadTestsFromTestCase(TestThreadPool)
 unittest.TextTestRunner(verbosity=2).run(suite)

I can not, for the life of me, remember why ThreadPool is there, except as a fallback. It's also not part of the documented interface as well. Additionally, in Python 3 we now have futures.

A workaround would be to call the following in the thread you want to use ThreadPool:
if not hasattr(threading.current_process(), "_children"):
 threading.current_process()._children = weakref.WeakKeyDictionary()
(putting this in Process could be a very simple patch)

I think that workaround should be:
if not hasattr(threading.current_thread(), "_children"):
 threading.current_thread()._children = weakref.WeakKeyDictionary()

#14881 seems to be a duplicate of this. It was closed in May 2012 after 2.7, 3.2, and 3.3 were patched. The three tests in potential_issue_demo.py now pass with 2.7.5 and 3.3.2. So closing.