Message 234931 - Python tracker

➜

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

In-reply-to
Author	ned.deily
Recipients	ned.deily, neologix, pitrou, vstinner
Date	2015年01月29日.00:58:32
SpamBayes Score	-1.0
Marked as misclassified	Yes
Message-id	<1422493112.13.0.637480910452.issue23285@psf.upfronthosting.co.za>

Content
It turns out the times are not important; the hangup is the default size of the socket buffers on OS X and possibly BSD in general. In my case, the send and receive buffers are 8192, which explains why the chunks written are so small. I somewhat arbitrarily changed the sizes of the buffers in _test_send with: rd.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, support.SOCK_MAX_SIZE // 3) wr.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, support.SOCK_MAX_SIZE // 3) The results were: test_send (__main__.SocketEINTRTest) ... rd SO_RCVBUF default was 8192, wr SO_SNDBUF default was 8192 len(data) = 50331651 sent = 5592405, written = 5592405 chunk = 5592405, total read = 5592405 chunk = 5592405, total read = 11184810 chunk = 5592405, total read = 16777215 sent = 16777215, written = 22369620 chunk = 5592405, total read = 22369620 chunk = 5592405, total read = 27962025 chunk = 5592405, total read = 33554430 chunk = 5592405, total read = 39146835 sent = 22369620, written = 44739240 chunk = 5592405, total read = 44739240 sent = 5592411, written = 50331651 chunk = 5592405, total read = 50331645 chunk = 6, total read = 50331651 ok test_sendall (__main__.SocketEINTRTest) ... rd SO_RCVBUF default was 8192, wr SO_SNDBUF default was 8192 len(data) = 50331651 chunk = 5592405, total read = 5592405 chunk = 5592405, total read = 11184810 chunk = 5592405, total read = 16777215 chunk = 5592405, total read = 22369620 chunk = 5592405, total read = 27962025 chunk = 5592405, total read = 33554430 chunk = 5592405, total read = 39146835 chunk = 5592405, total read = 44739240 sent = None, written = 50331651 chunk = 5592405, total read = 50331645 chunk = 6, total read = 50331651 ok test_sendmsg (__main__.SocketEINTRTest) ... rd SO_RCVBUF default was 8192, wr SO_SNDBUF default was 8192 len(data) = 50331651 sent = 5592405, written = 5592405 chunk = 5592405, total read = 5592405 chunk = 5592405, total read = 11184810 chunk = 5592405, total read = 16777215 chunk = 5592405, total read = 22369620 chunk = 5592405, total read = 27962025 sent = 27962025, written = 33554430 chunk = 5592405, total read = 33554430 chunk = 5592405, total read = 39146835 chunk = 5592405, total read = 44739240 sent = 16777221, written = 50331651 chunk = 5592405, total read = 50331645 chunk = 6, total read = 50331651 ok I dunno if a value that large will work in all environments, so the code to call setsockopt might need to be smarter.

Content

It turns out the times are not important; the hangup is the default size of the socket buffers on OS X and possibly BSD in general. In my case, the send and receive buffers are 8192, which explains why the chunks written are so small. I somewhat arbitrarily changed the sizes of the buffers in _test_send with:
rd.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, support.SOCK_MAX_SIZE // 3)
wr.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, support.SOCK_MAX_SIZE // 3)
The results were:
test_send (__main__.SocketEINTRTest) ... rd SO_RCVBUF default was 8192, wr SO_SNDBUF default was 8192
len(data) = 50331651
 sent = 5592405, written = 5592405
 chunk = 5592405, total read = 5592405
 chunk = 5592405, total read = 11184810
 chunk = 5592405, total read = 16777215
 sent = 16777215, written = 22369620
 chunk = 5592405, total read = 22369620
 chunk = 5592405, total read = 27962025
 chunk = 5592405, total read = 33554430
 chunk = 5592405, total read = 39146835
 sent = 22369620, written = 44739240
 chunk = 5592405, total read = 44739240
 sent = 5592411, written = 50331651
 chunk = 5592405, total read = 50331645
 chunk = 6, total read = 50331651
ok
test_sendall (__main__.SocketEINTRTest) ... rd SO_RCVBUF default was 8192, wr SO_SNDBUF default was 8192
len(data) = 50331651
 chunk = 5592405, total read = 5592405
 chunk = 5592405, total read = 11184810
 chunk = 5592405, total read = 16777215
 chunk = 5592405, total read = 22369620
 chunk = 5592405, total read = 27962025
 chunk = 5592405, total read = 33554430
 chunk = 5592405, total read = 39146835
 chunk = 5592405, total read = 44739240
 sent = None, written = 50331651
 chunk = 5592405, total read = 50331645
 chunk = 6, total read = 50331651
ok
test_sendmsg (__main__.SocketEINTRTest) ... rd SO_RCVBUF default was 8192, wr SO_SNDBUF default was 8192
len(data) = 50331651
 sent = 5592405, written = 5592405
 chunk = 5592405, total read = 5592405
 chunk = 5592405, total read = 11184810
 chunk = 5592405, total read = 16777215
 chunk = 5592405, total read = 22369620
 chunk = 5592405, total read = 27962025
 sent = 27962025, written = 33554430
 chunk = 5592405, total read = 33554430
 chunk = 5592405, total read = 39146835
 chunk = 5592405, total read = 44739240
 sent = 16777221, written = 50331651
 chunk = 5592405, total read = 50331645
 chunk = 6, total read = 50331651
ok
I dunno if a value that large will work in all environments, so the code to call setsockopt might need to be smarter.

History
Date	User	Action	Args
2015年01月29日 00:58:32	ned.deily	set	recipients: + ned.deily, pitrou, vstinner, neologix
2015年01月29日 00:58:32	ned.deily	set	messageid: <1422493112.13.0.637480910452.issue23285@psf.upfronthosting.co.za>
2015年01月29日 00:58:32	ned.deily	link	issue23285 messages
2015年01月29日 00:58:32	ned.deily	create

homepage