Passes instance methods to the python multiprocessing process
If I pass a reference to an instance method instead of a reference to a module-level method to multiprocessing. Process
, when I call its start method, is it an exact copy of the parent process instance passed in, or is the constructor called again? What happens to Deep instance member objects? Exact copy or default?
Solution
Instances are not passed between processes because instances are per Python VM process.
The value passed between processes is pickle
d. Unpickling typically restores an instance by calling a method other than __init__
; As far as I can tell, it directly sets property values to resurrect one instance, and resolves references to and from other instances.
Assuming you run the same code in either process (and use multiprocessing
, which you do), it restores references to the correct instance methods in the recovered instance chain.
This means that if an __init__
in the object chain does something with side effects, it will not be done on the receiver, that is, in a child process. Initialize this class explicitly.
All in all, the easiest way is to (efficiently) share immutable objects between parallel processes and reconcile the results after join()
all objects.