Threading a c++ program in python

Question

I have a Python application written in Kivy that uses a C++ program for a high speed calculation, then it returns a value and my Python application uses that.

The C++ program is wrapped in PyBind11 and imported into the application and then called from Python.

My issue is when the C++ program is executed, my application stops for a short while and I'd still like things to be going on in the background.

I naively thought this could be solved by threading the C++ call, but on second thoughts I think the issue lies in the GIL. Must I unlock the GIL and how could I achieve this?

• Nullman: if you have a multithreaded application, then you wont get much speed up from the multithreading if your c++ code isnt unlocking the GIL
• ivan_pozdeev: Please add a minimal reproducible example that shows how your C++ and Python code interact and how you do threads. Otherwise we're having a pointless discussion here since without knowing that, we can't say anything specific.

Answer 1

Without seeing any code, I can only deduce that your Python code is waiting for the C++ code to complete before doing anything else. Which can mean either or both of the following:

• you are not unlocking the GIL in the C++ code

  • According to Global Interpreter Lock (GIL) — Miscellaneous — pybind11 2.2.3 documentation, with pybind, this is supposed to be done like this:

    py::gil_scoped_release release;
    long_running_method();
    py::gil_scoped_acquire acquire;
    

    Note that you need the GIL to access any Python machinery (including returning the result). So before releasing it, make sure to convert all the data you need from Python types to C++ types.

• you don't have any other active Python threads, so there's no other Python activity programmed in to do anything while the C++ call is in progress

• ljelliot: Amazing, both answers are correct!! I've implemented the first answer and I'm deciding on the activity to be undertaken while the calculation has taken place. Thank you so much for your response, I'd upvote it but I don't have the required reputation yet... :/ !!
• ivan_pozdeev: @ljelliot you can accept it instead.