Python multiprocessing messaging code

Question

I am just learning how to do some multiprocessing with Python, and I would like to create a non-blocking chat application that can add/retrieve messages to/from a database. This is my somewhat primitive attempt (note: It is simplified for the purposes of this post); however, I would like to know how close/far off off the mark I am. Any comments, revisions, suggestions, etc. are greatly appreciated.

Thank you!

import multiprocessing
import Queue
import time
# Get all past messages
def batch_messages():
 # The messages list here will be attained via a db query
 messages = [">> This is a message.", ">> Hello, how are you doing today?", ">> Really good!"]
 for m in messages:
 print m
# Add messages to the DB 
def add_messages(q): 
 # Retrieve from the queue
 message_to_add = q.get()
 # For testing purposes only; perfrom another DB query to add the message to the DB
 print "(Add to DB)"
# Recieve new, inputted messages.
def receive_new_message(q, new_message):
 # Add the new message to the queue:
 q.put(new_message)
 # Print the message to the screen
 print ">>", new_message
if __name__ == "__main__":
 # Set up the queue
 q = multiprocessing.Queue()
 # Print the past messages
 batch_messages()
 while True: 
 # Enter a new message
 input_message = raw_input("Type a message: ")
 # Set up the processes
 p_add = multiprocessing.Process(target=add_messages, args=(q,))
 p_rec = multiprocessing.Process(target=receive_new_message, args=(q, input_message,))
 # Start the processes 
 p_rec.start()
 p_add.start()
 # Let the processes catch up before printing "Type a message: " again. (Shell purposes only)
 time.sleep(1)

• Friendly King: That's quite odd that it works fine when I run it...I will look further into this. But, thank you for the feedback.
• Friendly King: I can explicitly pass it to the functions that require it, and then I assume it will work for as well, correct?
• Janne Karila: Yes, you should pass q as a parameter.

Answer 1

• Don't create a new process for every task. Put a loop in add_messages; q.get will wait for the next task in the queue.
• receive_new_message is not doing anything useful in your example. Place q.put(new_message) right after raw_input instead.

:

def add_messages(q): 
 while True:
 # Retrieve from the queue
 message_to_add = q.get()
 print "(Add to DB)"
if __name__ == "__main__":
 q = multiprocessing.Queue()
 p_add = multiprocessing.Process(target=add_messages, args=(q,))
 p_add.start()
 while True: 
 input_message = raw_input("Type a message: ")
 # Add the new message to the queue:
 q.put(input_message)
 # Let the processes catch up before printing "Type a message: " again. (Shell purposes only)
 time.sleep(1)

• Friendly King: Thank you for the revision! I appreciate your help. On a side note, I included the receive_new_message process with the thought that I would need to output messages from the queue to users' screens, and that this process should be separate from the add_messages (to the db) process. Is this "concern" a necessary one? Or can the outputting to the screen be coupled with the database querying?
• Janne Karila: @JohnZ If you need the two subprocesses, you would use two queues. For example, receive_new_message might read from q1, do some processing and forward the results to add_messages via q2.
• Friendly King: Ahhh okay. That makes sense. Thank you for the feedback!