homepage

The following code fails on an assertion error (Python exception for _pyio, C assertion for io):
------------------
with io.BytesIO(b'abcd') as raw:
 with _pyio.TextIOWrapper(raw, encoding='ascii') as f:
 f.read(1)
 f.write('2')
 f.tell()
------------------
I found this assertion while testing interlaced read-write on TextIOWrapper:
------------------
with io.BytesIO(b'abcd') as raw:
 with _pyio.TextIOWrapper(raw, encoding='ascii') as f:
 f.write("1")
 # read() must call writer.flush()
 assertEqual(f.read(1), 'b')
 # write() must rewind the raw stream
 f.write('2')
 assertEqual(f.read(), 'd')
 f.flush()
 assertEqual(raw.getvalue(), b'1b2d')
with io.BytesIO(b'abc') as raw:
 with _pyio.TextIOWrapper(raw, encoding='ascii') as f:
 self.assertEqual(f.read(1), b'a')
 # write() must undo reader readahead
 f.write(b"2")
 assertEqual(f.read(1), b'c')
 f.flush()
 assertEqual(raw.getvalue(), b'a2c')
------------------
These tests fails on "read, write, read" path: write() breaks TextIOWrapper internal state if a read() occured just before. Note that _pyio.TextIOWrapper.write() contains the following comment...
# XXX What if we were just reading?
See also the issue #12213 for BufferedRandom and BufferedRWPair.

textiowrapper_interlaced_read_write.patch: TextIOWrapper.write() calls self.seek(self.tell()) if it has a decoder or if snapshot is not None.
I suppose that we can do better, but at least it does fix this issue.
"read(); write(); write()" only calls self.seek(self.tell()) once, at the first write. So I don't think that it changes anything with performances.
In which case snapshot can be not different than None, whereas decoder is None?

For c stdio files, intermixed reads and writes require a file positioning operation. This is a nuisance and source of program bugs. I do not see any such limitation documented for our io module. So for both reasons, it will be nice to not have the limitation in the code.
If I understand, the essence of the patch is to do the file positioning automatically internally when needed.

> If I understand, the essence of the patch is to do
> the file positioning automatically internally when needed.
My patch is just a proposition to fix the issue. I wrote "I suppose that we can do better": self.seek(self.tell()) is more a workaround than a real fix. I don't understand why it does fix this bug :-)

Perhaps the stdio requirement was based on an underlying OS (*nix?) requirement, which io has to fulfill even if it does not use stdio.
Stdio was, I presume, optimized for speed. In the relatively rare case of mixed read/write, it *should* put the burden on the programmer. Python is a bit different.

Does anybody want to follow up on this? #12213 was closed as fixed, #12513 is still open.

Consider codecs that maintain an internal buffer (UTF-7) or other state (ISO-2022). When you call TextIOWrapper.read() and then tell(), the I think the returned number is supposed to hold the _decoder_ state, so you can seek back and read again. But I don’t think the number holds any _encoder_ state, so seek() cannot set up the encoder properly for these more awkward codecs.
I don’t think it is practical to fix this problem using the incremental codec API. You would need to construct the encoder’s state from the decoder. There are a couple of bugs marked as duplicates of this. What are the real-world use cases?

Given that nobody complains the last 9 years (since Python 3.0 was released), I'm not sure that it's worth it to fix this corner case.
If you consider that I'm wrong, please reopen the issue ;-)

For the record, the more recent bug I mentioned was a complaint from 2015 (one and a half years before Victor’s comment). Even if it is not worth supporting writing after reading, the problem could be documented.