Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Problems with multiprocessing #121

Open
Assignees
Labels
questionFurther information is requested
@risboo6909

Description

Hi everyone.

I'd like to use pygad's parallel_processing feature to enable parallelization with multiprocessing like that:

parallel_processing=["process", 2]

Then I have my fitness function which is a closure which has access to its outer function variables and meets pygad's requirement to have only 2 arguments at the same time like this:

def do_recognize(img):
 def inner(inp, _):
 best_fitness, _ = scan(img, inp, debug=False)
 return best_fitness
 return inner

When I try to use approach from above example I'm getting an error that it is impossible to pickle inner function:

AttributeError: Can't pickle local object 'do_recognize.<locals>.inner

Ok, my next attempt is to use functools.partial like this:

fitness = partial(do_recognize, img)

But this doesn't work too, because of pygad's check that fitness function has exactly 2 arguments which it does by using __code__co_argcount property which seems to be absent for partial return type.

I came up with the following ugly hack:

 Dummy = namedtuple('Dummy', 'co_argcount')
 find_board = partial(do_recognize, img)
 find_board.__code__ = Dummy(co_argcount=2)

this perfectly satisfies both multiprocessing and pygad but it doesn't look cool at all.

I'd like to propose to avoid using __code__co_argcount function for arity check and switch to something less unusual.
For example inspect package could be helpful:

import inspect
assert len(inspect.getargspec(f).args) == 2

which at least works perfectly fine with partial.

Metadata

Metadata

Labels

questionFurther information is requested

Projects

No projects

Milestone

No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      AltStyle によって変換されたページ (->オリジナル) /