Commit b3aaf7b

committed

apveyor, #519: FIX incomplete Popen pump

+ The code in `_read_lines_from_fno()` was reading the stream only once per invocation, so when input was larger than `mmap.PAGESIZE`, bytes were forgotten in the stream. + Replaced buffer-building code with iterate-on-file-descriptors. + Also set deamon-threads.

1 parent d12fdca commit b3aaf7bCopy full SHA for b3aaf7b

File tree

2 files changed

+18

-6

lines changed

git
- cmd.py
- test
  - test_git.py

2 files changed

+18

-6

lines changed

`‎git/cmd.py`

Lines changed: 13 additions & 3 deletions

Original file line number	Diff line number	Diff line change
`@@ -193,14 +193,24 @@ def _deplete_buffer(fno, handler, buf_list, wg=None):`
`193`	`193`	`else:`
`194`	`194`	`# Oh ... probably we are on windows. select.select() can only handle sockets, we have files`
`195`	`195`	`# The only reliable way to do this now is to use threads and wait for both to finish`
	`196`	`+ def _handle_lines(fd, handler, wg):`
	`197`	`+ for line in fd:`
	`198`	`+ line = line.decode(defenc)`
	`199`	`+ if line and handler:`
	`200`	`+ handler(line)`
	`201`	`+ if wg:`
	`202`	`+ wg.done()`
	`203`	`+`
`196`	`204`	`# Since the finalizer is expected to wait, we don't have to introduce our own wait primitive`
`197`	`205`	`# NO: It's not enough unfortunately, and we will have to sync the threads`
`198`	`206`	`wg = WaitGroup()`
`199`		`- for fno, (handler, buf_list) in fdmap.items():`
	`207`	`+ for fd, handler in zip((process.stdout, process.stderr),`
	`208`	`+ (stdout_handler, stderr_handler)):`
`200`	`209`	`wg.add(1)`
`201`		`- t = threading.Thread(target=lambda: _deplete_buffer(fno, handler, buf_list, wg))`
	`210`	`+ t = threading.Thread(target=_handle_lines, args=(fd, handler, wg))`
	`211`	`+ t.setDaemon(True)`
`202`	`212`	`t.start()`
`203`		`-# end`
	`213`	`+`
`204`	`214`	`# NOTE: Just joining threads can possibly fail as there is a gap between .start() and when it's`
`205`	`215`	`# actually started, which could make the wait() call to just return because the thread is not yet`
`206`	`216`	`# active`

`‎git/test/test_git.py`

Lines changed: 5 additions & 3 deletions

Original file line number	Diff line number	Diff line change
`@@ -238,9 +238,11 @@ def counter_stderr(line):`
`238`	`238`	`stdin=None,`
`239`	`239`	`stdout=subprocess.PIPE,`
`240`	`240`	`stderr=subprocess.PIPE,`
`241`		`- shell=False)`
	`241`	`+ shell=False,`
	`242`	`+ creationflags=Git.CREATE_NO_WINDOW if sys.platform == 'win32' else 0,`
	`243`	`+ )`
`242`	`244`
`243`	`245`	`handle_process_output(proc, counter_stdout, counter_stderr, lambda proc: proc.wait())`
`244`	`246`
`245`		`- assertcount[1]==line_count`
`246`		`- assertcount[2]==line_count`
	`247`	`+ self.assertEqual(count[1], line_count)`
	`248`	`+ self.assertEqual(count[2], line_count)`

0 commit comments

Comments

(0)

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Commit b3aaf7b

File tree

2 files changed

2 files changed

`‎git/cmd.py`

`‎git/test/test_git.py`

0 commit comments