Calling Python script from C++ and using its output

Question

I want to call a python script from C++ and wish to use the output .csv file generated by this script back into C++. I tried this in main():

std::string filename = "/home/abc/xyz/script.py";
std::string command = "python ";
command += filename;
system(command.c_str());

This does call and execute the python script.

The print commands in the Python are being executed. Things are being printed on the screen when the script is called. So far so good. However, it is not creating the .csv file (part of the same script).

Example: I had a training.csv file with 100 entries. I called the Python script, with little changes to the script so that the training.csv file now should contain only 50 entries instead of 100. It’s overwritten. However, no such thing happening. Rest of the commands in the script (print, etc) are working perfectly.

The training.csv file is to be read with C++ normally using fstream and getline.

Any idea how to do it (using Linux)?

• Christian Kiewiet: You may want to look at ways of embedding (and extending) python instead of dealing with invocations of the python interpreter via shell.
• user9876: Depends on the platform. Windows and Unix have different approaches. Which are you on? And there's nothing Python specific about this.
• StoryTeller - Unslander Monica: This should get you started: stackoverflow.com/questions/9405985/…
• Gauthier Boaglio: Also, for shorter sample stackoverflow.com/a/478960/1715716
• StoryTeller - Unslander Monica: If the python script creates a csv file. Then you don't need to redirect the scripts output. Just run it, and open the file it created using the standard C++ api.

Answer 1

Here is a solution to embed the execution of your python module from within your C++ application. It's not better or worst than forking/executing your python script through a system call, it just is a different way to do it. Whether it is best or not depend on your context and usage.

Some time ago I have coded a way to load python modules as plugins to a C++ application, here's the interesting part.

Basically, you need to #include <Python.h>, then Py_Initialize() to start your python interpreter.

Then you do import sys, using : PyRun_SimpleString("import sys");, and you can load your plugin by doing PyRun_SimpleString('sys.path.append("path/to/my/module/")').

To exchange values between C++ and Python, things get harder, you have to to transform all your C++ objects into python objects (starting line 69 in my script).

Then you can call your function using PyObject_Call_Object(...), using all the python objects you created as arguments.

You get the return value, and transforms all those values in C++ objects. And don't forget the memory management in all that!

To end your python interpreter, a simple call to Py_Finalize().

It really looks harder than it is really, but you have to be really careful doing this, because it could lead to leaks, security issues etc..

Answer 2

Try using POSIX's popen() instead of system(). It pipes stdin/stdout of child process to returned file handle.

FILE* in = popen(command.c_str(), "r");
fscanf(in, ... // or some other method of reading
pclose(in);

• learner: @Vovanium not helping :( could you please show a sample...thanks!