homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Python io implementation doesn't flush with write_through=True unlike C implementation
Type: behavior Stage: resolved
Components: IO Versions: Python 3.4, Python 3.5
process
Status: closed Resolution: fixed
Dependencies: Superseder:
Assigned To: Nosy List: akira, martin.panter, pitrou, python-dev
Priority: normal Keywords: patch

Created on 2014年04月30日 10:36 by akira, last changed 2022年04月11日 14:58 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
io-write_through-c-vs-python-issue21396.patch akira, 2014年05月01日 10:27 make C io implementation behave as documented like _pyio already does, add corresponding tests review
Messages (7)
msg217600 - (view) Author: Akira Li (akira) * Date: 2014年04月30日 10:36
related: msg217596 (bufsize=1 is broken if subprocess module uses Python io)
TextIOWrapper.write behavior:
_pyio.py [1]:
 if self._line_buffering and (haslf or "\r" in s):
 self.flush()
textio.c [2]:
 if (self->write_through)
 needflush = 1;
 else if (self->line_buffering &&
 (haslf ||
 PyUnicode_FindChar(text, '\r', 0, PyUnicode_GET_LENGTH(text), 1) != -1))
 needflush = 1;
C implementation calls flush() if write_through=True, Python implementation doesn't.
[1]: http://hg.python.org/cpython/file/0b2e199ad088/Lib/_pyio.py#l1636
[2]: http://hg.python.org/cpython/file/0b2e199ad088/Modules/_io/textio.c#l1333 
msg217654 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014年04月30日 21:04
Thanks for the report, Akira. Feel free to submit a patch! (with a test)
msg217684 - (view) Author: Akira Li (akira) * Date: 2014年05月01日 10:27
I've uploaded the patch that makes C implementation behave according
to the docs like Python implementation with the corresponding tests.
Issue #21332 is a dependency for this issue: subprocess' test_universal_newlines needs to be updated to work with Python version.
For Reference
-------------
issue #12591 introduces `write_through` to support subprocess' stdin
pipe in text mode with bufsize=0 i.e., TextIOWrapper.buffer is
unbuffered (raw) and Python and C implementation behave the same in
this particular case.
C implementation (pseudo-code)::
 if self.write_through:
 flush_text_buffer()
 buffer.flush() # <-- undocumented
Python implementation::
 if self.write_through:
 pass # _pyio.TextIOWrapper.write() calls buffer.write() directly
behaves according to the current documentation [1]:
 If *write_through* is ``True``, calls to :meth:`write` are guaranteed
 not to be buffered: any data written on the :class:`TextIOWrapper`
 object is immediately handled to its underlying binary *buffer*
 
[1]: hg.python.org/cpython/file/2a56d3896d50/Doc/library/io.rst
For reference, here's how subprocess.py uses write_through [2]::
 self.stdin = io.open(pipe, 'wb', bufsize)
 if universal_newlines=True:
 self.stdin = io.TextIOWrapper(self.stdin,
 write_through=True,
 line_buffering=(bufsize == 1)) # <-- issue #21332
http://hg.python.org/cpython/rev/9ce8fa0a0899/ - introduce io.TextIOWrapper in subprocess
http://hg.python.org/cpython/rev/5cc536fbd7c1 - introduce write_through
[2]: http://hg.python.org/cpython/file/2a56d3896d50/Lib/subprocess.py#l849 
msg218129 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014年05月08日 22:17
The patch looks basically fine. I will make a few tweaks to the comments in the test case.
msg218130 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014年05月08日 22:28
Actually, with the patch, the universal_newlines tests in test_subprocess hang (quite logically, since they lack a flush()). I will fix them as well.
msg218132 - (view) Author: Roundup Robot (python-dev) (Python triager) Date: 2014年05月08日 22:33
New changeset 39f2a78f4357 by Antoine Pitrou in branch '3.4':
Issue #21396: Fix TextIOWrapper(..., write_through=True) to not force a flush() on the underlying binary stream.
http://hg.python.org/cpython/rev/39f2a78f4357
New changeset 37d0c41ed8ad by Antoine Pitrou in branch 'default':
Issue #21396: Fix TextIOWrapper(..., write_through=True) to not force a flush() on the underlying binary stream.
http://hg.python.org/cpython/rev/37d0c41ed8ad 
msg218133 - (view) Author: Antoine Pitrou (pitrou) * (Python committer) Date: 2014年05月08日 22:34
Thank you for the patch!
History
Date User Action Args
2022年04月11日 14:58:03adminsetgithub: 65595
2014年05月08日 22:34:18pitrousetstatus: open -> closed
resolution: fixed
messages: + msg218133

stage: needs patch -> resolved
2014年05月08日 22:33:29python-devsetnosy: + python-dev
messages: + msg218132
2014年05月08日 22:28:47pitrousetmessages: + msg218130
2014年05月08日 22:17:54pitrousetmessages: + msg218129
2014年05月02日 01:53:10martin.pantersetnosy: + martin.panter
2014年05月01日 10:27:33akirasetfiles: + io-write_through-c-vs-python-issue21396.patch
keywords: + patch
messages: + msg217684
2014年04月30日 21:04:35pitrousetstage: needs patch
messages: + msg217654
versions: - Python 3.3
2014年04月30日 16:57:51ned.deilysetnosy: + pitrou
2014年04月30日 10:36:57akiracreate

AltStyle によって変換されたページ (->オリジナル) /