Using Pool map with a method of a class and a list

Wed Aug 7 05:47:06 EDT 2013

On 7 August 2013 09:33, Luca Cerone <luca.cerone at gmail.com> wrote:
> To correct my example:
>> from multiprocessing import Pool
>> class A(object):
> def __init__(self,x):
> self.value = x
> def fun(self,x):
> return self.value**x
>> l = range(100)
> p = Pool(4)
> op = p.map(A(3).fun, l)
>> doesn't work neither in Python 2.7, nor 3.2 (by the way I can't use Python 3 for my application).

Are you using Windows? Over here on 3.3 on Linux it does. Not on 2.7 though.
>> You will find that
>> http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod-> > when-using-pythons-multiprocessing-pool-ma
>> explains the problem in more detail than I understand. I suggest
>> reading it and relaying further questions back to us. Or use Python 3
>> :) Thanks, but of course I googled and found this link before posting. I don't understand much of the details as well, that's why I posted here.
>> Anyway, thanks for the attempt :)

Reading there, the simplest method seems to be, in effect:
from multiprocessing import Pool
from functools import partial
class A(object):
 def __init__(self,x):
 self.value = x
 def fun(self,x):
 return self.value**x
def _getattr_proxy_partialable(instance, name, arg):
 return getattr(instance, name)(arg)
def getattr_proxy(instance, name):
 """
 A version of getattr that returns a proxy function that can
 be pickled. Only function calls will work on the proxy.
 """
 return partial(_getattr_proxy_partialable, instance, name)
l = range(100)
p = Pool(4)
op = p.map(getattr_proxy(A(3), "fun"), l)
print(op)