Capturing stdout in a QThread and update GUI

Question

I would like to know opinions about the design I came up with to handle stdout capture to show progress in a PyQt5 application.

So, I have this module which is meant to be run as a command line application. When called with some arguments, it takes time to run, giving progress to stdout and in the end returning some data.

In my PyQt5 application, I'm able to load that module, run it, and get its data. However, since it takes time to run, I'd like to give progress feedback.

I'm handling this using contextlib.redirect_stdout. However, the data fetching happens in a sub-thread and must update widgets in the main thread, so I must use signals. Also, to be able to capture multiple progress messages in one single function call of the data fetching function in the command line module, I need to have a listener for stdout, which I put in a second sub-thread.

Below is a minimalist code representing the real case. Note the global variable out - this is what I came up with to be able to handle communication between 2 sub threads. I don't particularly like having this global variable, but it did solve the problem and looks simple. Any suggestions here would be welcome.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import io
import sys
import requests
from contextlib import redirect_stdout
from PyQt5 import QtCore, QtGui, QtWidgets
# target for stdout - global variable used by sub-threads
out = io.StringIO()
class OutputThread(QtCore.QThread):
 # This signal is sent when stdout captures a message
 # and triggers the update of the text box in the main thread.
 output_changed = QtCore.pyqtSignal(object)
 def run(self):
 '''listener of changes to global `out`'''
 while True:
 out.flush()
 text = out.getvalue()
 if text:
 self.output_changed.emit(text)
 # clear the buffer
 out.truncate(0)
 out.seek(0)
class FetchDataThread(QtCore.QThread):
 # Connection between this and the main thread.
 data_fetched = QtCore.pyqtSignal(object, object)
 def __init__(self, url_list):
 super(FetchDataThread, self).__init__()
 self.url_list = url_list
 def update(self, url_list):
 self.url_list = url_list
 def run(self):
 for url in self.url_list:
 # Can write directly to `out`
 out.write('Fetching %s\n' % url)
 # This context manager will redirect the output from
 # `sys.stdout` to `out`
 with redirect_stdout(out):
 data = fetch_url(url)
 out.write('='*80 + '\n')
 # Send data back to main thread
 self.data_fetched.emit(url, data)
def fetch_url(url):
 '''This is a dummy function to emulate the behavior of the module
 used in the real problem.
 The original module does lots of things and prints multiple 
 progress messages to stdout.
 '''
 print('Fetching', url)
 page = requests.get(url)
 print('Decoding', url)
 data = page.content.decode()
 print('done')
 return data
class MyApp(QtWidgets.QMainWindow):
 def __init__(self, parent=None):
 super(MyApp, self).__init__(parent)
 # ###############################################################
 # ### GUI setup
 self.centralwidget = QtWidgets.QWidget(self)
 self.setCentralWidget(self.centralwidget)
 self.button = QtWidgets.QPushButton('Go', self.centralwidget)
 self.button.clicked.connect(self.go)
 self.output_text = QtWidgets.QTextEdit()
 self.output_text.setReadOnly(True)
 layout = QtWidgets.QVBoxLayout(self.centralwidget)
 layout.addWidget(self.button)
 layout.addWidget(self.output_text)
 # ### end of GUI setup
 # ###############################################################
 self.url_list = ['http://www.duckduckgo.com',
 'http://www.stackoverflow.com']
 self.url = list()
 self.data = list()
 # Thread to update text of output tab
 self.output_thread = OutputThread(self)
 self.output_thread.output_changed.connect(self.on_output_changed)
 # Start the listener
 self.output_thread.start()
 # Thread to fetch data
 self.fetch_data_thread = FetchDataThread(self.url_list)
 self.fetch_data_thread.data_fetched.connect(self.on_data_fetched)
 self.fetch_data_thread.finished.connect(lambda: self.button.setEnabled(True))
 def go(self):
 if not self.fetch_data_thread.isRunning():
 self.button.setEnabled(False)
 self.fetch_data_thread.update(self.url_list)
 self.fetch_data_thread.start()
 def on_data_fetched(self, url, data):
 self.url.append(url)
 self.data.append(data)
 def on_output_changed(self, text):
 self.output_text.append(text.strip())
if __name__ == "__main__":
 app = QtWidgets.QApplication(sys.argv)
 window = MyApp()
 window.setGeometry(100, 50, 1200, 600)
 window.show()
 sys.exit(app.exec_())

Answer 1

OutputThread.run contains a busy loop: it is continuously checking for new input. That's a huge waste of CPU time. The least you could do to avoid that would be insert a time.sleep(0.1) inside the loop. A better approach would be to redirect stdout to a custom object that emits signals when written to. AFAIK it would suffice to implement write and flush methods in the custom object.

• Raf: Good spotted regarding the busy loop - I had corrected that like you mention. I'll give it a try in your suggested approach and see how it goes.