homepage

Reporting this as requested by Antoine Pitrou: Under certain circumstances in Python 3.2 (r32:88445) it's possible for buffered I/O to lose data before it is written and/or return the wrong results when reading. I tripped over this issue whilst writing an assembler which first writes out a bunch of binary data and then goes back over it in a somewhat arbitrary order patching addresses.
The following code demonstrates the issue:
[code]
START = 0
MID = 1
LENGTH = 4
def test(buffering):
 f = open("test.bin", "w+b", buffering = buffering)
 for i in range(LENGTH):
 f.write(b'\x00')
 f.seek(MID)
 f.read(1)
 f.write(b'\x00')
 f.seek(MID)
 f.write(b'\x01')
 f.seek(START)
 f.seek(MID)
 print(f.read(1))
 f.close()
print("Buffered result: ")
test(-1)
print("Unbuffered result:")
test(0)
[end code]
Output on both Gentoo and Vista is:
 Buffered result:
 b'\x00'
 Unbuffered result:
 b'\x01'
Expected output is b'\x01' in both cases.
The issue is reproducible with larger files provided that the constants are increased ~proportionally (START 0, MID 500, LENGTH 1000 for example). Transposing the buffered/unbuffered tests and/or using different buffer sizes for the buffered test seem have no effect. 
The lengthy code at this URL also demonstrates the issue: http://pastebin.com/xqrzKr5D The above was produced from this code, which was autogenerated by intercepting my assembler's output.

If you add "from _pyio import open" to the example, unbuffered and buffered tests give the same result :-)

Here is a patch with assorted tests.

open("test.bin", "w+b", buffering=-1) creates a 
BufferedRandom object. In the _pyio module, BufferedRandom overrides the write() method to "undo readahead", whereas the _io module reuses bufferedwriter_write() for bufferedrandom_methods and bufferedwriter_methods.
I suppose that the problem is "just" that _io.BufferedRandom.write() doesn't undo readahead.

New changeset 89f77afac947 by Antoine Pitrou in branch '3.1':
Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
http://hg.python.org/cpython/rev/89f77afac947
New changeset 47ca1244a929 by Antoine Pitrou in branch '3.2':
Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
http://hg.python.org/cpython/rev/47ca1244a929
New changeset 05e0227e3879 by Antoine Pitrou in branch 'default':
Issue #12062: Fix a flushing bug when doing a certain type of I/O sequence
http://hg.python.org/cpython/rev/05e0227e3879 

New changeset 0d24d4c537a6 by Antoine Pitrou in branch '2.7':
Issue #12062: In the `io` module, fix a flushing bug when doing a certain
http://hg.python.org/cpython/rev/0d24d4c537a6 

Should be fixed now. Again, thank you reporting this bug!