0

I want to use multiprocessing for a class method. I found out from this answer that Pool in multiprocessing cannot pickle class methods directly but there is a workaround for that by defining a function outside the class, and adds an additional argument(s) to that function (Similar suggestion is also on this blog). Hence, I tried to achieve that by the following simple program which has MyClass where I want to parallel fun. However, I am not getting any results (there is no bug). It seems I am missing something but I feel I am almost there! Any fix is really appreciated.

import multiprocessing
class MyClass:
 def __init__(self):
 pass
 def fun(self, myList):
 print myList
def unwrap_fun(obj, myList):
 return obj.fun(myList)
obj = MyClass()
mlp = multiprocessing.Pool(processes=multiprocessing.cpu_count())
mlp.imap_unordered(unwrap_fun, (obj, range(1, 10)))
asked Sep 15, 2017 at 15:31

1 Answer 1

3

You should call close() and join() from your main process. Try this:

import multiprocessing
class MyClass:
 def fun(self, myList):
 print myList
def unwrap_fun(myList):
 obj = MyClass()
 return obj.fun(myList)
if __name__ == '__main__':
 mlp = multiprocessing.Pool(processes=multiprocessing.cpu_count())
 mlp.imap_unordered(unwrap_fun, range(1, 10))
 mlp.close()
 mlp.join()
answered Sep 15, 2017 at 15:50
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

Draft saved
Draft discarded

Sign up or log in

Sign up using Google
Sign up using Email and Password

Post as a guest

Required, but never shown

Post as a guest

Required, but never shown

By clicking "Post Your Answer", you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.