homepage

Hello,
in my private repo I've changed memoryview's getbufferproc to be PEP-3118
compliant. test_multiprocessing does the equivalent of the following sequence,
which is not allowed by PEP-3118:
>>> import array, io
>>> a = array.array('i', [1,2,3,4,5])
>>> m = memoryview(a)
>>> m.format
'i'
>>> buf = io.BytesIO(bytearray(5))
>>> buf.readinto(m)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
TypeError: expected an object with a writable buffer interface
>>>
The error occurs in Objects/abstract.c:315:
 ((*pb->bf_getbuffer)(obj, &view, PyBUF_WRITABLE) != 0))
Here, PyObject_AsWriteBuffer() requests a simple writable buffer of unsigned
bytes *without format information* from the memoryview. The memoryview's
getbufferproc is required to return an error:
"If format is not explicitly requested then the format must be returned
 as NULL (which means "B", or unsigned bytes)."
But the underlying buffer has format 'i' and not 'B', hence the error.
Antoine, is it correct that io.BytesIO should only be used with bytearray
buffers?
If so, this is a bug in the tests (patch attached).

Of course, there is another interpretation:
[PyBUF_FORMAT]
"The returned buffer must have true format information if this flag is provided. This would be used when the consumer is going to be checking for what 'kind' of data is actually stored. An exporter should always be able to provide this information if requested. If format is not explicitly requested then the format must be returned as NULL (which means "B", or unsigned bytes)"
So, the returned buffer may have false format information ('B' vs. 'i'
in this case) if this flag is not provided.
Do you agree with this? I'll then make it explicit in the documentation.

> Antoine, is it correct that io.BytesIO should only be used with bytearray
> buffers?
BytesIO does a copy of the original object, it does not touch the
original buffer.

I've come to think that PyBUF_SIMPLE requests were really intended
as a way of casting contiguous buffers with arbitrary formats to
one-dimensional buffers of unsigned bytes. At least that is what Numpy
does. 
Then test_multiprocessing would be correct. I've asked the general
question on the Numpy mailing list:
http://mail.scipy.org/pipermail/numpy-discussion/2011-August/058189.html 

Please also see issue5231.

The current interpretation in the PEP-3118 repo is that a request
without PyBUF_FORMAT means "implicit cast to unsigned bytes".
This makes the behavior of PyObject_AsWriteBuffer() correct, so I'm
closing this.