Hello World using Python embedded in C++, with RAII library initialization

Question

I have written a small Hello World application that uses Python's C-API. The C library requires some functions to be called for global library (de-)initialization and the objects created by the library eventually need to be released by passing the pointer to a specific function. I think this is a fairly common pattern in C libraries.

I attempted to use RAII, to get automatic cleanup, even when exceptions interfere with the planned control flow.

#include <tr1/memory>
#include <Python.h>
// helper to get function pointer for the macro
void Python_DecreaseRefCount(PyObject* p) {
 Py_DECREF(p);
}
int main() {
 // declare struct and create instance that handles global initialization and
 // deinitialization of Python API
 static struct PythonApi {
 PythonApi() { Py_Initialize(); }
 ~PythonApi() { Py_Finalize(); }
 } python_api;
 // create object
 // use smart pointer for automatic refcount decrement at scope end
 std::tr1::shared_ptr<PyObject> string(
 PyString_FromString("Hello World!\n"),
 &Python_DecreaseRefCount);
 // do something with object
 PyObject_Print(string.get(), stdout, Py_PRINT_RAW);
}

Compiles and runs with:

$ g++ test.cc `pkg-config --cflags --libs python2` && ./a.out
Hello World!

I'm interested in all kinds of feedback, as to whether my attempt makes sense and if there is room for improvement. Are there best practices, or certain idioms that facilitate this, that I might have missed in my approach?

Answer 1

1. Are you sure also scoping de-initialization of Python to main (by removing static) is not an option?
   As-is, you cannot use Python in a globals initializer or destructor anyway.
   Having taken a look at Py_Finalize, I actually would not deinitialize it at all, at least not in a release-build, as none of the reasons that option is provided apply to you:

   This function is provided for a number of reasons. An embedding application might want to restart Python without having to restart the application itself. An application that has loaded the Python interpreter from a dynamically loadable library (or DLL) might want to free all memory allocated by Python before unloading the DLL. During a hunt for memory leaks in an application a developer might want to free all memory allocated by Python before exiting from the application

   Bugs and caveats: [... lots ...]

2. Next, you shouldn't be using a function-pointer, use an empty function-object instead.
   Using a function-object facilitates inlining and as it's empty, reduces memory-overhead.

3. Actually, are you sure you actually need / want a std::shared_ptr?
   Either because you are already irretrievably locked into using that type, or because you actually might need weak pointers somewhere?

   If the answer is no, look into boost::intrusive_ptr, as that will save you an allocation every time you hand a PyObject to a smart-pointer's oversight, and reduces smartpointer-size by half, to raw-pointer-size.

   // Hooks in same scope as PyObject (global scope), so ADL in intrusive_ptr finds them:
   void intrusive_ptr_add_ref(PyObject* p) noexcept { Py_INCREF(p); }
   void intrusive_ptr_release(PyObject* p) noexcept { Py_DECREF(p); }
   // typedef for ease of use:
   typedef boost::intrusive_ptr<PyObject> PyPtr;
   // Use as:
   PyPtr string(PyString_FromString("Hello World!\n"), false);
   

• moooeeeep: Thanks for your input! I'm not sure whether I really need std::shared_ptr here. Normally, with full C++11 support, I'd have used std::unique_ptr. But I didn't find it included in TR1 and I wanted to avoid compiling with -std=c++11 this time. I will try to find an illustrative example of boost::intrusive_ptr to see if it might be an option.
• moooeeeep: What exactly is scoping (de-)initialization of Python to main ?
• Deduplicator: Well, removing the static so leaving main deinitializes Python.

Answer 2

C++11

If your compiler has proper C++11 support or better, you don't need, and shouldn't, use the stuff from std::tr1. shared_ptr is a member of namespace std and lives under the include file <memory>.

Global init/shutdown

Your approach is valid, it ensures proper termination if an exception is thrown and never caught. Still you have to be careful to avoid ever declaring a global object that might need to call the Py_ API in its constructor, because global constructors will be called before main is entered. There is no silver bullet there. The best course of action is to just avoid global objects with constructors & destructors that rely on other globals or APIs that require explicit init/shutdown.

Type alias?

Since all of your objects are stored as pointers to PyObject, you can make life easier and code shorter by defining an alias:

using CppPyObjectPtr = std::shared_ptr<PyObject>;

And to avoid resupplying the deleter every time, you can define a helper akin to std::make_unique. I also suggest using a functor structure or a lambda rather than a function pointer, the compiler should have better chances of optimizing it:

CppPyObjectPtr makePyObjectPtr(PyObject* obj)
{
 return CppPyObjectPtr(obj, [](PyObject* toFree) { Py_DECREF(toFree); });
}
// Usage:
auto pyString = makePyObjectPtr(PyString_FromString("Hello World!\n"));
PyObject_Print(pyString.get(), stdout, Py_PRINT_RAW);

• moooeeeep: I kind of deliberately restricted myself to C++03 in this example. Otherwise I would have used std::unique_ptr where I can specify the custom deleter as a template argument. After thinking about it, I think the library initialization should maybe better be done by a static global variable. For the reason you have stated, and to remove noise in main().