|
| 1 | +.. toctree:: |
| 2 | + :maxdepth: 2 |
| 3 | + |
| 4 | +.. _cpp_and_numpy: |
| 5 | + |
| 6 | +==================================== |
| 7 | +C++ and the Numpy C API |
| 8 | +==================================== |
| 9 | + |
| 10 | +`Numpy <http://www.numpy.org>`_ is a powerful arrary based data structure with fast vector and array operations. It has a fully featured `C API <https://docs.scipy.org/doc/numpy/reference/c-api.html>`_. This section describes some aspects of using Numpy with C++. |
| 11 | + |
| 12 | +------------------------------------ |
| 13 | +Initialising Numpy |
| 14 | +------------------------------------ |
| 15 | + |
| 16 | +The Numpy C API must be setup so that a number of static data structures are initialised correctly. The way to do this is to call ``import_array()`` which makes a number of Python import statements so the Python interpreter must be initialised first. This is described in detail in the `Numpy documentation <https://docs.scipy.org/doc/numpy/reference/c-api.array.html#miscellaneous>`_ so this document just presents a cookbook approach. |
| 17 | + |
| 18 | + |
| 19 | +------------------------------------ |
| 20 | +Verifying Numpy is Initialised |
| 21 | +------------------------------------ |
| 22 | + |
| 23 | +``import_array()`` always returns ``NUMPY_IMPORT_ARRAY_RETVAL`` regardless of success instead we have to check the Python error status: |
| 24 | + |
| 25 | +.. code-block:: cpp |
| 26 | + |
| 27 | + #include <Python.h> |
| 28 | + #include "numpy/arrayobject.h" // Include any other Numpy headers, UFuncs for example. |
| 29 | + |
| 30 | + // Initialise Numpy |
| 31 | + import_array(); |
| 32 | + if (PyErr_Occurred()) { |
| 33 | + std::cerr << "Failed to import numpy Python module(s)." << std::endl; |
| 34 | + return NULL; // Or some suitable return value to indicate failure. |
| 35 | + } |
| 36 | + |
| 37 | +In other running code where Numpy is expected to be initialised then ``PyArray_API`` should be non-NULL and this can be asserted: |
| 38 | + |
| 39 | +.. code-block:: cpp |
| 40 | + |
| 41 | + assert(PyArray_API); |
| 42 | + |
| 43 | +------------------------------------ |
| 44 | +Numpy Initialisation Techniques |
| 45 | +------------------------------------ |
| 46 | + |
| 47 | + |
| 48 | +Initialising Numpy in a CPython Module |
| 49 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 50 | + |
| 51 | +Taking the simple example of a module from the `Python documentation <https://docs.python.org/3/extending/extending.html#a-simple-example>`_ we can add Numpy access just by including the correct Numpy header file and calling ``import_numpy()`` in the module initialisation code: |
| 52 | + |
| 53 | +.. code-block:: cpp |
| 54 | + |
| 55 | + #include <Python.h> |
| 56 | + |
| 57 | + #include "numpy/arrayobject.h" // Include any other Numpy headers, UFuncs for example. |
| 58 | + |
| 59 | + static PyMethodDef SpamMethods[] = { |
| 60 | + ... |
| 61 | + {NULL, NULL, 0, NULL} /* Sentinel */ |
| 62 | + }; |
| 63 | + |
| 64 | + static struct PyModuleDef spammodule = { |
| 65 | + PyModuleDef_HEAD_INIT, |
| 66 | + "spam", /* name of module */ |
| 67 | + spam_doc, /* module documentation, may be NULL */ |
| 68 | + -1, /* size of per-interpreter state of the module, |
| 69 | + or -1 if the module keeps state in global variables. */ |
| 70 | + SpamMethods |
| 71 | + }; |
| 72 | + |
| 73 | + PyMODINIT_FUNC |
| 74 | + PyInit_spam(void) { |
| 75 | + ... |
| 76 | + assert(! PyErr_Occurred()); |
| 77 | + import_numpy(); // Initialise Numpy |
| 78 | + if (PyErr_Occurred()) { |
| 79 | + return NULL; |
| 80 | + } |
| 81 | + ... |
| 82 | + return PyModule_Create(&spammodule); |
| 83 | + } |
| 84 | + |
| 85 | +That is fine for a singular translation unit but you have multiple translation units then each has to initialise the Numpy API which is a bit extravagant. The following sections describe how to manage this with multiple translation units. |
| 86 | + |
| 87 | +Initialising Numpy in Pure C++ Code |
| 88 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 89 | + |
| 90 | +This is mainly for development and testing of C++ code that uses Numpy. Your code layout might look something like this where ``main.cpp`` has a ``main()`` entry point and ``class.h`` has your class declarations and ``class.cpp`` has their implementations, like this:: |
| 91 | + |
| 92 | + . |
| 93 | + └── src |
| 94 | + └── cpp |
| 95 | + ├── class.cpp |
| 96 | + ├── class.h |
| 97 | + └── main.cpp |
| 98 | + |
| 99 | +The way of managing Numpy initialisation and access is as follows. In ``class.h`` choose a unique name such as ``awesome_project`` then include: |
| 100 | + |
| 101 | +.. code-block:: cpp |
| 102 | + |
| 103 | + #define PY_ARRAY_UNIQUE_SYMBOL awesome_project_ARRAY_API |
| 104 | + #include "numpy/arrayobject.h" |
| 105 | + |
| 106 | +In the implementation file ``class.cpp`` we do not want to import Numpy as that is going to be handled by ``main()`` in ``main.cpp`` so we put this at the top: |
| 107 | + |
| 108 | +.. code-block:: cpp |
| 109 | + |
| 110 | + #define NO_IMPORT_ARRAY |
| 111 | + #include "class.h" |
| 112 | + |
| 113 | +Finally in ``main.cpp`` we initialise Numpy: |
| 114 | + |
| 115 | +.. code-block:: cpp |
| 116 | + |
| 117 | + #include "Python.h" |
| 118 | + #include "class.h" |
| 119 | + |
| 120 | + int main(int argc, const char * argv[]) { |
| 121 | + // ... |
| 122 | + // Initialise the Python interpreter |
| 123 | + wchar_t *program = Py_DecodeLocale(argv[0], NULL); |
| 124 | + if (program == NULL) { |
| 125 | + fprintf(stderr, "Fatal error: cannot decode argv[0]\n"); |
| 126 | + exit(1); |
| 127 | + } |
| 128 | + Py_SetProgramName(program); /* optional but recommended */ |
| 129 | + Py_Initialize(); |
| 130 | + // Initialise Numpy |
| 131 | + import_array(); |
| 132 | + if (PyErr_Occurred()) { |
| 133 | + std::cerr << "Failed to import numpy Python module(s)." << std::endl; |
| 134 | + return -1; |
| 135 | + } |
| 136 | + assert(PyArray_API); |
| 137 | + // ... |
| 138 | + } |
| 139 | + |
| 140 | +If you have multiple .h, .cpp files then it might be worth having a single .h file, say ``numpy_init.h`` with just this in: |
| 141 | + |
| 142 | +.. code-block:: cpp |
| 143 | + |
| 144 | + #define PY_ARRAY_UNIQUE_SYMBOL awesome_project_ARRAY_API |
| 145 | + #include "numpy/arrayobject.h" |
| 146 | + |
| 147 | +Then each implementation .cpp file has: |
| 148 | + |
| 149 | +.. code-block:: cpp |
| 150 | + |
| 151 | + #define NO_IMPORT_ARRAY |
| 152 | + #include "numpy_init.h" |
| 153 | + #include "class.h" // Class declarations |
| 154 | + |
| 155 | +And ``main.cpp`` has: |
| 156 | + |
| 157 | +.. code-block:: cpp |
| 158 | + |
| 159 | + #include "numpy_init.h" |
| 160 | + #include "class_1.h" |
| 161 | + #include "class_2.h" |
| 162 | + #include "class_3.h" |
| 163 | + |
| 164 | + int main(int argc, const char * argv[]) { |
| 165 | + // ... |
| 166 | + import_array(); |
| 167 | + if (PyErr_Occurred()) { |
| 168 | + std::cerr << "Failed to import numpy Python module(s)." << std::endl; |
| 169 | + return -1; |
| 170 | + } |
| 171 | + assert(PyArray_API); |
| 172 | + // ... |
| 173 | + } |
| 174 | + |
| 175 | +Initialising Numpy in a CPython Module using C++ Code |
| 176 | +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
| 177 | + |
| 178 | +Supposing you have laid out your source code in the following fashion:: |
| 179 | + |
| 180 | + . |
| 181 | + └── src |
| 182 | + ├── cpp |
| 183 | + │ ├── class.cpp |
| 184 | + │ └── class.h |
| 185 | + └── cpython |
| 186 | + └── module.c |
| 187 | + |
| 188 | +This is a hybrid of the above and typical for CPython C++ extensions where ``module.c`` contains the CPython code that allows Python to access the pure C++ code. |
| 189 | + |
| 190 | +The code in ``class.h`` and ``class.cpp`` is unchanged and the code in ``module.c`` is essentially the same as that of a CPython module as described above where ``import_array()`` is called from within the ``PyInit_<module>`` function. |
0 commit comments