This issue tracker has been migrated to GitHub ,
and is currently read-only.
For more information,
see the GitHub FAQs in the Python's Developer Guide.
Created on 2015年03月24日 22:32 by rkuska, last changed 2022年04月11日 14:58 by admin. This issue is now closed.
| Files | ||||
|---|---|---|---|---|
| File name | Uploaded | Description | Edit | |
| leak.py | rkuska, 2015年03月24日 22:32 | not so minimal reproducer | ||
| report | rkuska, 2015年03月24日 22:32 | report from valgrind | ||
| leak2.py | vstinner, 2015年03月24日 23:29 | |||
| leak3.py | rkuska, 2015年03月25日 09:15 | |||
| leak4.py | vstinner, 2022年01月28日 02:41 | |||
| Messages (6) | |||
|---|---|---|---|
| msg239193 - (view) | Author: Robert Kuska (rkuska) * | Date: 2015年03月24日 22:32 | |
Leaks happen only when both testDoctestFile and testDoctestSuite are run. Run with Python 3.4.2 and 3.4.1 with same result. I have extracted those two tests into `leak.py` (attached). > $ valgrind --suppressions=/../cpython/Misc/valgrind-python.supp python3 leak.py ==17896== Memcheck, a memory error detector ==17896== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==17896== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==17896== Command: python3 leak.py ==17896=わ=わ =わ=わ17896=わ=わ =わ=わ17896== HEAP SUMMARY: ==17896== in use at exit: 1,599,328 bytes in 11,595 blocks ==17896== total heap usage: 283,757 allocs, 272,162 frees, 37,891,147 bytes allocated ==17896== ==17896== LEAK SUMMARY: ==17896== definitely lost: 30 bytes in 1 blocks ==17896== indirectly lost: 0 bytes in 0 blocks ==17896== possibly lost: 597,418 bytes in 2,319 blocks ==17896== still reachable: 1,001,880 bytes in 9,275 blocks ==17896== suppressed: 0 bytes in 0 blocks ==17896== Rerun with --leak-check=full to see details of leaked memory ==17896== ==17896== For counts of detected and suppressed errors, rerun with: -v ==17896== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Note that when I remove support.modules_cleanup(*modules_before) from leak.py valgrind reports no leaks (in original test_zipimport those are run in setUp and tearDown). Output of valgrind --suppressions=/home/rkuska/upstream/cpython/Misc/valgrind-python.supp --leak-check=yes -v python3 leak.py attached as `report`. |
|||
| msg239194 - (view) | Author: Robert Kuska (rkuska) * | Date: 2015年03月24日 22:36 | |
Summary for valgrind python3 test_zipimport.py ==18608== ==18608== HEAP SUMMARY: ==18608== in use at exit: 1,596,390 bytes in 11,536 blocks ==18608== total heap usage: 343,849 allocs, 332,313 frees, 59,355,776 bytes allocated ==18608== ==18608== LEAK SUMMARY: ==18608== definitely lost: 90 bytes in 3 blocks ==18608== indirectly lost: 0 bytes in 0 blocks ==18608== possibly lost: 594,488 bytes in 2,258 blocks ==18608== still reachable: 1,001,812 bytes in 9,275 blocks ==18608== suppressed: 0 bytes in 0 blocks ==18608== Rerun with --leak-check=full to see details of leaked memory ==18608== ==18608== For counts of detected and suppressed errors, rerun with: -v ==18608== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) |
|||
| msg239196 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2015年03月24日 23:29 | |
When calling gc.collect() after each test, I don't see any leak anymore. doTest() has a small leak: it prepends a path to sys.path, but it never removes it. Try attached leak2.py: it displays something like +254 kB. Uncomment the two following lines and the output will be close to +0 kB. #sys.path = old_path #gc.collect() |
|||
| msg239238 - (view) | Author: Robert Kuska (rkuska) * | Date: 2015年03月25日 09:15 | |
I tried leak2.py with valgrind, I've uncommented the lines you mentioned. > $ valgrind python3 leak2.py ==28421== Memcheck, a memory error detector ==28421== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==28421== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==28421== Command: python3 leak2.py ==28421=わ=わ =わ=わ28421=わ=わ =わ=わ28421== HEAP SUMMARY: ==28421== in use at exit: 1,599,354 bytes in 11,594 blocks ==28421== total heap usage: 284,971 allocs, 273,377 frees, 37,976,898 bytes allocated ==28421== ==28421== LEAK SUMMARY: ==28421== definitely lost: 0 bytes in 0 blocks ==28421== indirectly lost: 0 bytes in 0 blocks ==28421== possibly lost: 597,482 bytes in 2,319 blocks ==28421== still reachable: 1,001,872 bytes in 9,275 blocks ==28421== suppressed: 0 bytes in 0 blocks ==28421== Rerun with --leak-check=full to see details of leaked memory ==28421== ==28421== For counts of detected and suppressed errors, rerun with: -v ==28421== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) I've removed `import readline` from leak2.py, everything else left as in previous run (also ofc removed tracemalloc). > $ valgrind python3 leak3.py ==28515== Memcheck, a memory error detector ==28515== Copyright (C) 2002-2013, and GNU GPL'd, by Julian Seward et al. ==28515== Using Valgrind-3.10.1 and LibVEX; rerun with -h for copyright info ==28515== Command: python3 leak2.py ==28515=わ=わ =わ=わ28515=わ=わ =わ=わ28515== HEAP SUMMARY: ==28515== in use at exit: 1,599,384 bytes in 11,595 blocks ==28515== total heap usage: 285,648 allocs, 274,053 frees, 38,126,379 bytes allocated ==28515== ==28515== LEAK SUMMARY: ==28515== definitely lost: 30 bytes in 1 blocks ==28515== indirectly lost: 0 bytes in 0 blocks ==28515== possibly lost: 597,375 bytes in 2,317 blocks ==28515== still reachable: 1,001,979 bytes in 9,277 blocks ==28515== suppressed: 0 bytes in 0 blocks ==28515== Rerun with --leak-check=full to see details of leaked memory ==28515== ==28515== For counts of detected and suppressed errors, rerun with: -v ==28515== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) Duh? Why does readline have this effect? |
|||
| msg325731 - (view) | Author: Serhiy Storchaka (serhiy.storchaka) * (Python committer) | Date: 2018年09月19日 07:55 | |
zipimport has been rewritten in pure Python (issue25711). |
|||
| msg411963 - (view) | Author: STINNER Victor (vstinner) * (Python committer) | Date: 2022年01月28日 02:41 | |
> valgrind reports leaks for test_zipimport It's not strictly a memory leak: running the same code multiple time doesn't leak memory. It's just that Python allocates memory once and never releases it (doesn't releast it at exit). --- leak4.py: Updated and simplified script. If the code is run 10 times, it doesn't leak. With 1, 10 or 100 loops (LOOPS=100), Python still says the same at exit: $ ./python -I -X showrefcount leak4.py [1574 refs, 664 blocks] Using cannot_deallocate2.patch of bpo-46417, I can see that multiple static types are not cleared at Python exit: --- Cannot deallocate type 'Exception': it still has subclasses * error Cannot deallocate type 'BaseException': it still has subclasses * Exception Cannot deallocate type 'dict': it still has subclasses * collections.defaultdict Cannot deallocate type 'object': it still has subclasses * dict * BaseException * itertools.accumulate * itertools.combinations * itertools.combinations_with_replacement * itertools.cycle * itertools.dropwhile * itertools.takewhile * itertools.islice * itertools.starmap * itertools.chain * itertools.compress * itertools.filterfalse * itertools.count * itertools.zip_longest * itertools.pairwise * itertools.permutations * itertools.product * itertools.repeat * itertools.groupby * itertools._grouper * itertools._tee * itertools._tee_dataobject * collections.deque * _collections._deque_iterator * _collections._deque_reverse_iterator * _collections._tuplegetter * _struct.Struct * _struct.unpack_iterator --- The _collections, itertools and _struct extensions implement types as static types which are not cleared at Python exit. It explains why Python says that there are still 664 memory blocks allocated at Python exit (664 blocks). I consider that this issue is a duplicate of bpo-40077. |
|||
| History | |||
|---|---|---|---|
| Date | User | Action | Args |
| 2022年04月11日 14:58:14 | admin | set | github: 67957 |
| 2022年01月28日 02:41:02 | vstinner | set | status: open -> closed files: + leak4.py resolution: duplicate messages: + msg411963 superseder: Convert static types to heap types: use PyType_FromSpec() stage: resolved |
| 2018年09月19日 07:55:23 | serhiy.storchaka | set | nosy:
+ serhiy.storchaka messages: + msg325731 |
| 2015年03月25日 09:15:11 | rkuska | set | files:
+ leak3.py messages: + msg239238 |
| 2015年03月24日 23:29:32 | vstinner | set | files:
+ leak2.py nosy: + vstinner messages: + msg239196 |
| 2015年03月24日 22:36:12 | rkuska | set | messages: + msg239194 |
| 2015年03月24日 22:32:45 | rkuska | set | files: + report |
| 2015年03月24日 22:32:14 | rkuska | create | |