homepage

Following the documentation at https://docs.python.org/3/c-api/buffer.html, if we write a custom object in C with a getbufferproc that simply returns PyBuffer_FillInfo(...), then it works fine up to Python 3.2 but segfaults in Python 3.3 depending on what we do with it. The segfault is caused by memoryobject.c:last_dim_is_contiguous(), which reads from the "strides" array without checking that it is not NULL.
Attached the simplest example of C extension module I could make. When executing this:
>>> import xy
>>> m=memoryview(bytearray(b"abcdef"))
>>> m[:5] = xy.gm
...it segfaults in Python 3.3. (I'm told it works fine in 3.2 and segfaults in 3.4 as well, but didn't confirm it.)

The flags from mb_getbuf() have to be passed to PyBuffer_FillInfo():
 return PyBuffer_FillInfo(view, (PyObject *)self,
 internal, 5,
 /*readonly=*/0, flags);
The flags contain the actual request that the consumer
sends to the buffer provider, so they cannot be hardcoded.
In this example, memoryview sends the PyBUF_FULL request to
mb_getbuf(). If the request is successful, it can assume
non-NULL strides.
Should the documentation be improved?

Ah! Thanks. I systematically missed the "flags" arguments passed in the getbufferproc function.
(I thought I had to tell PyBuffer_FillInfo() what kind of format we actually provide, but it seems that the interface is the other way around. I don't see how I would implement a getbufferproc that forces a specific format, but I don't need that, so it's fine.)

Yes, PyBuffer_FillInfo() is a bit confusing: The canonical usage *looks* as
if "flags" were somehow used in the Py_buffer struct to mark a buffer as having
certain capabilities:
Modules/_io/bufferedio.c:
-------------------------
 if (PyBuffer_FillInfo(&buf, NULL, start, len, 0, PyBUF_CONTIG) == -1)
 return -1;
 memobj = PyMemoryView_FromBuffer(&buf);
What really goes on since Python 3.3 is that memobj is a memoryview
with (shape,strides,format), since missing Py_buffer values are
reconstructed.
So all memoryviews -- when acting as an exporter as in this example --
have PyBUF_FULL capabilities. To make that clear, we should actually
change the PyBUF_CONTIG in the above example to PyBUF_FULL. Then
the mental model is:
 PyBuffer_FillInfo(..., PyBUF_FULL)
 ^
 | "Request" full buffer to export.
 memobj
 ^
 | Request anything from PyBUF_SIMPLE to PyBUF_FULL. Note
 | that e.g. with PyBUF_SIMPLE the consumer's shape and
 | strides will be NULL!
 Consumer
And that model is consistent with the usage of PyBuffer_FillInfo() inside
a getbufferproc, where "flags" of course can be anything.

I think it's worth adding a note to the docs about passing the flags unmodified inside a getbufferproc.

New changeset 49cdb04bfcc6 by Stefan Krah in branch '3.4':
Issue #21778: Clarify use of flags if PyBuffer_FillInfo() is used inside a
http://hg.python.org/cpython/rev/49cdb04bfcc6