[Python-Dev] Understanding the buffer API

Sat Aug 4 17:25:49 CEST 2012

Jeff Allen <ja...py at farowl.co.uk> wrote:
> I still don't see why, if the consumer says "I'm assuming 1-D
> unsigned bytes", and that's what the data is, memoryview_getbuf
> could not provide a shape and strides that agree with the data.

In most cases it won't matter. However, a consumer is entitled to rely
on shape==NULL in response to a PyBUF_SIMPLE request. Perhaps there
is code that tests for shape==NULL to determine C-contiguity.
This is an example that might occur in C. You hinted at the fact that not
all of this may be relevant for Java, but on that I can't comment.
> When you say such-and-such is disallowed, but the PEP or the data
> structures seem to provide for it, you mean memoryview_getbuf()
> disallows it, since you've concluded it is not sensible?

The particular request of PyBUF_SIMPLE|PyBUF_FORMAT, when applied to
any array that is not one-dimensional with format 'B' would lead to a
contradiction: PyBUF_SIMPLE implies 'B', but format would be set to
something else.
It is also a useless combination, since a plain PyBUF_SIMPLE suffices.
> >I think the protocol would benefit from changing the getbuffer rules to:
> >
> > a) The buffer gets a 'flags' field that can store properties like
> > PyBUF_SIMPLE, PyBUF_C_CONTIGUOUS etc.
> >
> > b) The exporter must *always* provide full information.
> >
> > c) If a buffer can be exported as unsigned bytes but has a different
> > layout, the exporter must perform a full cast so that the above
> > mentioned invariants are kept.
> >
> Just like PyManagedBuffer mbuf and its sister view in memoryview?
> I've thought the same things, but the tricky part is to do it
> compatibly.
>> a) I think I can achieve this. As I have interfaces and polymorphism
> on my side, and a commitment only to logical equivalence to CPython,
> I can have the preserved flags stashed away inside to affect
> behaviour. But it's not as simple as saving the consumer's request,
> and I'm still trying to work it out what to do, e.g. when the
> consumer didn't ask for C-contiguity, but in this case it happens to
> be true.
>> In the same way, functions you have in abstract.c etc. can be
> methods that, rather than work out by inspection of a struct how to
> navigate the data on this call, already know what kind of buffer
> they are in. So SimpleBuffer.isContiguous(char order) can simply
> return true.

Avoiding repeated calls to PyBuffer_IsContiguous() was in fact the main
reason for storing flags in the new MemoryViewObject.
It would be handy to have these flags in the Py_buffer structure, but
that can only be considered for a future version of Python, perhaps no
earlier than 4.0. The same applies of course to all three points that
I made above.
Stefan Krah