Python – Passes instance methods to the python multiprocessing process

Passes instance methods to the python multiprocessing process… here is a solution to the problem.

Passes instance methods to the python multiprocessing process

If I pass a reference to an instance method instead of a reference to a module-level method to multiprocessing. Process, when I call its start method, is it an exact copy of the parent process instance passed in, or is the constructor called again? What happens to Deep instance member objects? Exact copy or default?

Solution

Instances are not passed between processes because instances are per Python VM process.

The value passed between processes is pickled. Unpickling typically restores an instance by calling a method other than __init__; As far as I can tell, it directly sets property values to resurrect one instance, and resolves references to and from other instances.

Assuming you run the same code in either process (and use multiprocessing, which you do), it restores references to the correct instance methods in the recovered instance chain.

This means that if an __init__ in the object chain does something with side effects, it will not be done on the receiver, that is, in a child process. Initialize this class explicitly.

All in all, the easiest way is to (efficiently) share immutable objects between parallel processes and reconcile the results after join() all objects.

Related Problems and Solutions