Python Multithreading Queue

Question

import threading
from queue import Queue
print_lock = threading.Lock()
def job(worker):
 with print_lock:
 with open('messages.txt') as f:
 for line in f:
 print(line)
def reader():
 while True:
 worker = q.get()
 job(worker)
 q.task_done()
q = Queue()
for x in range(10):
 t = threading.Thread(target=reader)
 t.daemon = True
 t.start()
for worker in range(1):
 q.put(worker)
q.join()

So what i want is each thread reads different messages,

Answer 1

Queue is thread safe

so, threadling lock does not needed

Answer 2

your trying too many things to learn in same code snippet like 1) Multi-Threading 2) Queue Data Structure 3) Thread Synchronization Mechanisms 4) Locking etc.

Let me answer regarding multi-threading only.

In your case, every thread is reading all messages because target function "job" is opening file and reading all the data and every thread is calling that target function.

Let me simplify the stuff bit.

1. You want to read each line of file in new thread. So, instead of opening file in every thread and read it, we will open file one time and put data in list.
2. Now, every thread will read one line from list and print it. Also, it will remove that printed line from list.
3. Once, all the data is printed and still thread trying to read, we will add the exception.

Code :

import threading
import sys
#Global variable list for reading file data
global file_data
file_data = []
#Create lock
lock = threading.Lock()
def reader():
 while len(file_data) != 0:
 print threading.currentThread().getName() + " --- "
 try:
 lock.acquire()
 #Get one line from list and print it
 a = file_data.pop()
 print a
 except:
 #Once data is not present, let's print exception message
 print "------------------------"
 print "No data present in file"
 sys.exit() 
 lock.release()
#Read data from file and put it into list
with open("messages.txt") as fh:
 file_data = fh.readlines()
for x in range(2):
 name = "Thread_"+str(x)
 t = threading.Thread(name=name,target=reader)
 t.start()

Output:

 C:\Users\dinesh\Desktop>python demo.py
Thread_0 --- Thread_1 ---
Each thread read each message
Thread_1 --- I am great
Thread_0 --- How Are you ?
Thread_1 --- Grey
Thread_0 --- Hey
Thread_1 --- Dinesh
Thread_0 --- Hello
------------------------
No data present in file
C:\Users\dinesh\Desktop>
 C:\Users\dinesh\Desktop>

NOTE : I know sue of global is not recommended. But for learning purpose it is good.

• Grey Byte: Okay i almost got it But the problem with your code is , if i need to read 10 lines i need to create 10 threads what am looking for the soloution even if i have 10 message and 2 threads , they need to read that 10 messages
• Grey Byte: Okay thanks a lot for helping me but still i didn't got a perfect answer With your updated code , Only one thread reading all the message This is the output i got while adding a thread name hey Thread-1 hey Thread-1 hey Thread-1 yo Thread-1 hey Thread-1 hello Thread-1 So i think thread 1 is reading all the message Thanks again
• Dinesh Pundkar: Please check my updated code now. I have also added Thread name. But I am getting both threads accessing parallely.