Timeline for python - memory error
Current License: CC BY-SA 3.0
8 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 20, 2020 at 9:12 | history | edited | Community Bot |
Commonmark migration
|
|
| Aug 6, 2014 at 2:43 | vote | accept | user3893623 | ||
| Aug 6, 2014 at 2:43 | comment | added | user3893623 | This worked, but it printed the count of 51 million lines (not just one) | |
| Aug 6, 2014 at 2:40 | comment | added | Matt3o12 |
Ok, I now have a 2GB file with only zeros, but python only took 2GB of my memory but crashed eventually: SystemError: Negative size passed to PyString_FromStringAndSize. 2,489,971,200 is too big for an integer. Anyway, with 16GB Ram, there should be a memory issue. Reading in chunks works fine, though (and it is quicker than I thought).
|
|
| Aug 6, 2014 at 2:29 | comment | added | Matt3o12 |
dd: count: illegal numeric value. Anyway, I now have a third testcase that is almost 1GB and python still doesn't need more than 20MB.
|
|
| Aug 6, 2014 at 2:27 | history | edited | Robφ | CC BY-SA 3.0 |
added 1 character in body
|
| Aug 6, 2014 at 2:25 | comment | added | Robᵩ |
@Matt3o12 - In your comment to the original post, you say that you created lines 110,000 bytes long. That's relatively short. Trying creating a 1GB file with no newlines in it. For example: dd if=/dev/zero of=/tmp/list.txt bs=1M count=1K.
|
|
| Aug 6, 2014 at 2:19 | history | answered | Robφ | CC BY-SA 3.0 |