Message166043
| Author |
serhiy.storchaka |
| Recipients |
Arfrever, eli.bendersky, jcon, meador.inge, ncoghlan, pitrou, serhiy.storchaka, tshepang |
| Date |
2012年07月21日.15:54:30 |
| SpamBayes Score |
-1.0 |
| Marked as misclassified |
Yes |
| Message-id |
<201207211854.14377.storchaka@gmail.com> |
| In-reply-to |
<1342865621.3492.1.camel@localhost.localdomain> |
| Content |
> It's under 64-bit Linux, Intel Core i5 CPU. Are you sure you're testing
> in non-debug mode?
I use 32-bit Linux.
> That said, the numbers under Windows suggest me that Eli's original idea
> (append and then join at the end) would be more robust.
I agree, it would be more robust, but much more complex solution. I will try to implement this approach too. Victor Stinner in their experiments with formatting preferred
overallocation approach (_PyUnicodeWriter), therefore, the solution probably will be a hybrid.
However, even in itself the patch deserves attention. It not only significant speeds up writing under Linux, but also speeds up the initialization, which leads to faster reading.
./python -m timeit -s "import io; d=(b'a'*99+b'\n')*10000" "s=io.BytesIO(d); r=s.readline" "while r(): pass"
Unpatched: 100 loops, best of 3: 5.92 msec per loop
Patched: 100 loops, best of 3: 3.95 msec per loop
I will try to preserve this advantage.
> By the way, here is Matt Mackall's take on realloc():
Note, that list also uses realloc() inside and therefore you get the same behavior with other constant factor. Actually, appending to a list less than sizeof(void*) bytes at a
time you will run into this sooner. Perhaps this is the cause that append/join approach under Windows slower than under Linux.
Thank you for measurements, this shows the scale of the issue. |
|