Message224562
| Author |
serhiy.storchaka |
| Recipients |
barry, jader.fabiano, r.david.murray, serhiy.storchaka, tshepang |
| Date |
2014年08月02日.13:26:55 |
| SpamBayes Score |
-1.0 |
| Marked as misclassified |
Yes |
| Message-id |
<1406986015.56.0.523010824829.issue21448@psf.upfronthosting.co.za> |
| In-reply-to |
| Content |
Parser reads from input file small chunks (8192 churacters) and feed FeedParser which pushes data into BufferedSubFile. In BufferedSubFile.push() chunks of incomplete data are accumulated in a buffer and repeatedly scanned for newlines. Every push() has linear complexity from the size of accumulated buffer, and total complexity is quadratic.
Here is a patch which fixes problem with parsing long lines. Feel free to add comments if they are needed (there is an abundance of comments in the module). |
|