Controlling C++ Output from Python Script

Question

I have a bit of an issue here. I have a Python script which calls binaries compiled from C++. The Python script has its own set of outputs (to standard out and error), which are easily disable-able. The C++ binaries have their own set of outputs (to standard out and error, among others) as well; the source can be altered, but I am not the original author. This is an issue because I do not want the C++ output in my final program, and I also don't want future users to need to edit the C++ source.

What I'd like to be able to do is have some Python method which will catch the C++ code's output that is sent to standard out or error. Is this possible? If so, could someone point me in the right direction?

Thank you!!

• bcc32: This depends on the way you're calling the binaries. For example if you're using os.system you can pipe the output, but otherwise there are probably some methods to look into. How is the calling written in your script?
• learner: Sorry, I should have posted this in the message body: they are imported through a Python interface, so that Python thinks it's calling other Python modules.

Answer 1

One way to do this is:

• Duplicate in python the file descriptors for stdout and stderr using os.dup.
• Redirect the original stdout and stderr using reopen (from C's stdio) to write to a file of your choice.

Note: reopen isn't available directly from python, but you should be able to call it as in the example below or using any other wrapper available.

After this is done:

• Every write to cout and cerr in C++ will write to the output files.
• Every print statement in python will write to the output files.

However, since the original descriptors are duplicated, you can still (see example below):

• Print to the original stdout/stderr using sdout.write and stdout.err
• Use logging methods after configuring properly the stream parameter

The following code uses instant library to test real C++ code that is wrapped into python using SWIG and that should be similar to the library that you have:

import sys, os
import logging
from instant import inline
print 'This is printed from python to stdout'
stdout = os.fdopen(os.dup(sys.stdout.fileno()), 'w')
stderr = os.fdopen(os.dup(sys.stderr.fileno()), 'w')
logging.basicConfig(stream=stderr, level=logging.DEBUG)
redirect = inline(""" 
void redirect(void) { 
 freopen("my_stdout.txt", "w", stdout); 
 freopen("my_stderr.txt", "w", stderr); 
} 
""")
redirect()
cout = inline(""" 
void cout(void) { 
 std::cout << "This is written from C++ to my_stdout.txt" << std::endl; 
 std::cerr << "This is written from C++ to my_stderr.txt" << std::endl; 
} 
""")
cout()
print 'This is written from python to my_stdout.txt'
stdout.write('This is printed from python to stdout\n')
stderr.write('This is printed from python to stderr\n')
logging.info('This is printed to stderr from python using logging')

The output for this example is:

$ python test.py
This is printed from python to stdout
This is printed from python to stdout
This is printed from python to stderr
INFO:root:This is printed to stderr from python using logging
$ cat my_stdout.txt 
This is written from C++ to my_stdout.txt
This is written from python to my_stdout.txt
$ cat my_stderr.txt 
This is written from C++ to my_stderr.txt

Note: First time the code is executed, you might get gcc compilation messages (I've removed them to make the example clearer).

• learner: If I understand what you've done, this will redirect all output to logging, both Python and C++, right? That would be OK, if so, as the useful output of this program is actually written to file.
• jcollado: Not exactly, the StringIO objects contain all the output that would have been printed to sys.stdout and sys.stderr (because they have been patched). The real sys.stderr is known by logging so all logging statements print to the real sys.stderr. I assume the wrapper to your C++ library doesn't use logging. If that's a problem, you remove the logging part and print with _stdout.write and _stderr.write directly.
• Winston Ewert: Will that actually affect the c++ side? I thought that would just change where python code printed...
• learner: Perhaps I am misunderstanding your suggestion, but after resetting standard error/out (a la _stdout = sys.stdout) I see no change in output.
• jcollado: @WinstonEwert You're right, C++ code doesn't see the patching of sys.stdout and sys.stderr. I've rewritten my answer using reopen and testing with real C++ code and now it works fine.

Answer 2

Are you using subprocess to compile the C++? If so, you can set where stderr and stdout go:

nowhere = StringIO()
subprocess.call("exit 1", shell=True, stdout=nowhere, stderr=nowhere)

• sligocki: If you are not using subprocess you're going to need to tell us what you are doing.
• learner: I am not using subprocess. I am calling C++ binaries through a Python interface for them.