Reassure user that python code is running during long imports

Question

I have created a large module for downloading data from SAP and editing and displaying it on a plotly dashboard. While the code is running as desired there is one problem that bugs me:

The submodules are themselves dependent on large third-party modules, some of which take a very long time to load. This is also due to the fact that end users start the app via portable python from a network drive. While the application is loading, the user sits in front of an uneventful console window for several minutes. To prevent the user from thinking that the application froze or something of the sort, I would like to display the loading progress in some way. I have now solved this with print statements:

print('[#-------] Lade Daten...', end='\r')
from .data import Config, Data
print('[##------] Bereite GUI vor...', end='\r')
from .gui import MainWindow
print('[###-----] Bereite Dashboard vor...', end='\r')
from .dashboard import Dashboard
from . import __version__ as version
class FertigungDashboardApp:
 """
 Combination of:
 CustomTkinter Desktop App to load data from SAP and editing configuration.
 Plotly Dash dashboard on Flask development server to display local dashboard.
 """
 def __init__(self):
 print('[####----] Lade Einstellungen... ', end='\r')
 config = Config()
 print('[#####---] Importiere Daten... ', end='\r')
 data = Data()
 print('[######--] Erstelle Dashboard... ', end='\r')
 dashboard = Dashboard(version, config, data)
 print('[#######-] Starte GUI... ', end='\r')
 gui = MainWindow(config, data, version, dashboard)
 print('[########] Die Anwendung ist geladen!')
 gui.mainloop()
if __name__ == '__main__':
 FertigungDashboardApp()

While this achieves the desired effect, I'm still unhappy about placing the print statements above and between the import statements.

PEP 8: E402 module level import not at top of file

Is there a more elegant way to solve this problem?

• Peilonrayz: Your question really rings like a Generic Best Practice sort of question. One response is "the UNIX way is to accept no feedback means "all's good"", another is "why does E402 matter when you are solving an actual problem?", finally another is to meet in the middle "why can't your main have one print("please wait we're doing stuff") then you just have one E402 violating import?" I won't close your question as I don't think your question is sufficiently clearly off-topic. I still do have my concerns.
• Toby Speight: How do you know this is the best way? Or is your title intended to be clickbait?
• stefan: Most probably you do too many stuff on import. Do explicit instantiation/Data loading etc. after the fast imports. This should also make your code more flexible to work with test data.
• Jan_B: @TobySpeight my title is supposed to be a question asking for the best way to solve this kind of problem. Should I add a questionmark?

Answer 1

A strange game. The only winning move is not to play.

-- War Games, Joshua / WOPR

I don't know what your set of imports is.

... the user sits in front of an uneventful console window for several minutes.

But it's just Wrong. Don't play that game.

We can compute things at "compile time", during an import, or at "run time", when a caller invoked us. Your imports appear to be ignoring that distinction, computing Far Too Much at import time. Resist the temptation to pre-populate tables and such. Lazily defer evaluation until such results are actually needed. The use of @lru_cache decorators can help.

I'm still unhappy about placing the print statements above and between the import statements.

Yes, good, that should produce discomfort.

Defer execution until the appropriate time.

---

As a separate item, it sounds like you have some routines that have "heavy" initialization costs. Consider all the usual alternatives.

• Are we initializing for a routine that might never be called? Be lazy!
• Is python the right language for this, or should we call out to a Rust routine?
• Could this datastructure be written to disk, and then be cheaply read back in?
• Can some init activity be put in the background, surrounded by a mutex?
• Can we pre-spawn instances, and hand them out to new users as needed?
• What is the dollar cost of "slow startup", and what will "fast startup" cost us?

---

If import a, b, c takes a "long" time, then sprinkling print("be patient") admonitions in the middle is not the right approach. Deal with the root cause. Each of those modules should be parsing class definitions and def functions. If profiling reveals they are doing something else, then edit the offending code so its execution is deferred until after import time.

Put another way, your Requirements Document says that you are serving "interactive" results to human beings. Psychology of humans suggests they will become impatient soon after clicking on something, so it's important to be responsive on their time scale. If your time scale is quite different, then it's time to rethink the match between your technology and your customers.

For example, if there are unavoidable startup costs, then maybe you want to offer a thin client to customers, which sends API queries to some flask micro-service that spun up an instance more than a minute ago.

• Jan_B: Thank you for your detailed answer! I will definitely try to decouple the loading and editing of data from the instantiation of the GUI in the hope that this will shorten the startup process. However, the startup of the application runs much faster locally on my computer than from the portable python version on the network drive. I don't work in my company's IT department, so I have to rely on some workarounds to make this tool available. As a small plant in an international group with centralized IT, a "non-IT" tool will not get very far through official channels.
• J_H: When I hear "network drive" I think "per-file latency". And an import can request the open()ing of many files. Two things you could look into: (1.) publish on pypi.org , so folks can pip install your_pkg, and (2.) running from a .zip file full of lots of little *.py files may reduce the number of round trips needed during network file access.