Interprocess communication between independent Python3 scripts

Question

Can we use standard multiprocessing.Queue class for communicating between independent Python3 scripts like

bash% ./aaa.py &
bash% ./bbb.py &

If no, Is there another library like standard multiprocessing module for communicating with independent scripts.

• shiina: I chose multiprocessing.managers module, its sample code url is docs.python.org/3.5/library/…

Answer 1

Take a look at the section on "Listeners and Clients" in the multiprocessing docs; unlike the higher level simpler APIs, these APIs allow you to establish connections by address and authenticate the paired process, which allows two Python scripts to cooperate without having a parent/child relationship.

• shiina: I know multiprocessing.connection module. Sorry for not explaining enough. I'm seeking the higher level simpler APIs.
• ShadowRanger: @shiina: The simpler APIs don't handle named addresses. I'd look at subprocess and/or multiprocessing's source code (the high level stuff is all in Python anyway) to get an idea of what they do and the proper way to solve your specific problem.
• shiina: But, I didn't know multiprocessing.managers module. Your answer led me to the module. Thanks.

Answer 2

It's a self answer. My minimum example is below.

aaa.py

from multiprocessing.managers import BaseManager
import queue
queue_a = queue.Queue()
queue_b = queue.Queue()
BaseManager.register('queue_a', callable=lambda: queue_a)
BaseManager.register('queue_b', callable=lambda: queue_b)
m = BaseManager(address=('', 50000), authkey=b'qwerty')
m.start()
shared_queue_a = m.queue_a()
shared_queue_b = m.queue_b()
shared_queue_a.put("How are you?")
msg = shared_queue_b.get()
print("%s: %s" % (__file__, msg))
m.shutdown()

bbb.py

from multiprocessing.managers import BaseManager
BaseManager.register('queue_a')
BaseManager.register('queue_b')
m = BaseManager(address=('localhost', 50000), authkey=b'qwerty')
m.connect()
queue_a = m.queue_a()
queue_b = m.queue_b()
msg = queue_a.get()
print("%s: %s" % (__file__, msg))
queue_b.put("I'm fine.")

Run these scripts.

./aaa.py &; sleep 1; ./bbb.py

Prints

./bbb.py: How are you?
./aaa.py: I'm fine.

• Dr_Zaszuś: It's a pity you've published the self answer without publishing an explanation. It is not clear what is a BaseManager, how it works, and why are there 2 queues and not one?
• Brian C.: queue_a and queue_b are two different queues. Each queue represents messages coming from either aaa.py or bbb.py. For BaseManager, see the python docs at docs.python.org/3/library/…