homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: Support unbuffered TextIOWrapper
Type: Stage:
Components: IO Versions: Python 3.7
process
Status: open Resolution:
Dependencies: Superseder:
Assigned To: Nosy List: serhiy.storchaka, vstinner
Priority: normal Keywords:

Created on 2017年10月12日 15:09 by vstinner, last changed 2022年04月11日 14:58 by admin.

Messages (7)
msg304250 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017年10月12日 15:09
It seems like that some methods of the io.TextIOWrapper class requires that its buffer object has the read1() method, whereas the constructor checks if the buffer has a read1() method and the TextIOWrapper _read_chunk() method is able to call buffer.read() if buffer doesn't have read1().
This issue may help to get fully unbuffered sys.stdin, at least when replaced manually:
stdin = sys.stdin
sys.stdin = open(0, "r", buffering=0, encoding=stdin.encoding, errors=stdin.errors, newline=stdin.newline)
msg304252 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017年10月12日 15:12
I created this issue because of this comment in create_stdio() of Python/pylifecycle.c:
---
 /* stdin is always opened in buffered mode, first because it shouldn't
 make a difference in common use cases, second because TextIOWrapper
 depends on the presence of a read1() method which only exists on
 buffered streams.
 */
 if (Py_UnbufferedStdioFlag && write_mode)
 buffering = 0;
 else
 buffering = -1;
---
msg304253 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017年10月12日 15:25
Oh, currently it's not possible to create an unbuffered read-only TextIOWrapper object:
haypo@selma$ python3
Python 3.6.2 (default, Oct 2 2017, 16:51:32) 
>>> open("/etc/issue", "r", 0)
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
ValueError: can't have unbuffered text I/O
msg304258 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017年10月12日 15:30
The falling back to read() was implemented in issue12591. What methods still require read1()?
msg304260 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017年10月12日 15:46
> The falling back to read() was implemented in issue12591. What methods still require read1()?
I read the Python implementation of TextIOWrapper in _pyio:
* seek() calls buffer.read() "if chars_to_skip:"... not sure that it can happen if buffer is a FileIO?
* read(None) and read(-1) to read "everything" (until EOF): buffer.read()
It seems like read(None) also calls buffer.read() in the C implementation, _io_TextIOWrapper_read_impl():
 /* Read everything */
 PyObject *bytes = _PyObject_CallMethodId(self->buffer, &PyId_read, NULL);
 PyObject *decoded;
 if (bytes == NULL)
 goto fail;
and in _io_TextIOWrapper_seek_impl():
 /* Just like _read_chunk, feed the decoder and save a snapshot. */
 PyObject *input_chunk = _PyObject_CallMethodId(
 self->buffer, &PyId_read, "i", cookie.bytes_to_feed);
msg304262 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2017年10月12日 15:48
Context: I created this issue as a follow-up of the discussion on the -u command line option, bpo-28647, which still leaves sys.stdin *buffered* in the current master branch.
msg304266 - (view) Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) Date: 2017年10月12日 16:19
What methods still require read1()?
History
Date User Action Args
2022年04月11日 14:58:53adminsetstatus: pending -> open
github: 75956
2017年10月28日 12:09:57serhiy.storchakasetstatus: open -> pending
2017年10月12日 16:19:18serhiy.storchakasetmessages: + msg304266
2017年10月12日 15:48:40vstinnersetmessages: + msg304262
2017年10月12日 15:46:21vstinnersetmessages: + msg304260
2017年10月12日 15:30:50serhiy.storchakasetnosy: + serhiy.storchaka
messages: + msg304258
2017年10月12日 15:25:04vstinnersetmessages: + msg304253
2017年10月12日 15:12:26vstinnersetmessages: + msg304252
2017年10月12日 15:09:32vstinnercreate

AltStyle によって変換されたページ (->オリジナル) /