Unable to find required GLIBC private symbols

Question

I am trying to analyze a memory dump obtained with dotnet-dump from a Ubuntu container in WinDbg to find an unmanaged memory leak.

I have been able to load the dump, but whenever I try to run !heap -s, which seems to be the first step, I get the following error:

************************************************************************************************************************
 GLIBC ptmalloc2 Heap Stats
************************************************************************************************************************
ERROR: Unable to find required GLIBC private symbols; unable to walk native heaps

I have hunted around but cannot find where these symbols are to download and reference them. Is there a download location or some other method to analyze this?

• Thomas Weller: When I started with WinDbg 13 years ago, I never imagined, anyone would ever debug Linux with it. Times are so changing...
• William Messmer: Yes -- it's definitely a bit weird doing a bunch of Linux (and/or Mac, Android, etc...) debugging in WinDbg. We've done a whole lot of work to enable that for a lot of scenarios ranging from user mode to kernel mode (Linux only) to cross-plat .NET and more. It's pretty wild debugging the Linux kernel running in QEMU at a source level with WinDbg.

Answer 1

You need to have the libc.so.6 executable (or whatever libc* is being used) and debug package in your symbol search path in order for this to work. A few notes for this:

• Ubuntu frequently publishes the debug packages to their DebugInfoD server but does NOT frequently publish the executables in the same way. You can add the federated DebugInfoD server via ".sympath+ DebugInfoD*https://debuginfod.elfutils.org" and get the debug package (assuming we have access to the executable). You can also download the libc debug package from the Ubuntu package manager and manually copy it into a local directory that's in the WinDbg search path.

• Assuming that the DebugInfoD server won't serve up the executable, you'll need to make sure that's in your sympath via other means (e.g.: copy the container's version of libc.so.6 out of the container and into a local directory that's in the WinDbg search path). If you are debugging the process live (e.g.: via a GDBServer connection), we'll pull the binary across the RSP via vFile requests and this isn't as important.

Once you have the executable and the libc debug package in the search path, you should see something like the below (from a silly test app in my Ubuntu WSL environment):

0:000> !heap -s
************************************************************************************************************************
 GLIBC ptmalloc2 Heap Stats
************************************************************************************************************************
Main Arena Heap Sys Mem Size Alloc Range
------------------------------------------------------------------------------------------------------------------------
 ** 00007ffff7d27ac0 132 Kb 0000555555559000 - 000055555557a000
Allocator Information:
----------------------------------------
 Current mmap (large block) threshold: 128 Kb
 Number of mmap blocks : 0
 Size of mmap blocks : 0
 Original SBRK base : 0x555555559000
0:000> !heap -s -ar 0x7ffff7d27ac0
Arena 0x7ffff7d27ac0 (main arena):
 System Memory: 132 Kb
 Chunks: 259
 Allocated: 80 Kb in 258 chunks
 Free: 51 Kb in 1 chunks
 In Use: 80 Kb in 258 chunks
 Not In Use: 51 Kb in 1 chunks
 Fast Bin: 0 in 0 chunks
 TCACHE: 0 in 0 chunks
 Range (bytes) Chunks # Allocated Alloc Bytes Avg Size
 --------------------------------------------------------------------------------
 0 - 1023 257 257 80 Kb 34
 1024 - 2047 0 0 80 Kb 0
 2048 - 3071 0 0 80 Kb 0
 3072 - 4095 0 0 80 Kb 0
 4096 - 5119 0 0 80 Kb 0
 5120 - 6143 0 0 80 Kb 0
 6144 - 7167 0 0 80 Kb 0
 7168 - 8191 0 0 80 Kb 0
 8192 - 9215 0 0 80 Kb 0
 9216 - 10239 0 0 80 Kb 0
 10240 - 11263 0 0 80 Kb 0
 11264 - 12287 0 0 80 Kb 0
 12288 - 13311 0 0 80 Kb 0
 13312 - 14335 0 0 80 Kb 0
 14336 - 15359 0 0 80 Kb 0
 15360 - 16383 0 0 80 Kb 0
 16384 - 17407 0 0 80 Kb 0
 17408 - 18431 0 0 80 Kb 0
 18432 - 19455 0 0 80 Kb 0
 19456 - 20479 0 0 80 Kb 0
 20480 - 21503 0 0 80 Kb 0
 21504 - 22527 0 0 80 Kb 0
 22528 - 23551 0 0 80 Kb 0
 23552 - 24575 0 0 80 Kb 0
 24576 - 25599 0 0 80 Kb 0
 25600 - 26623 0 0 80 Kb 0
 26624 - 27647 0 0 80 Kb 0
 27648 - 28671 0 0 80 Kb 0
 28672 - 29695 0 0 80 Kb 0
 29696 - 30719 0 0 80 Kb 0
 30720 - 31743 0 0 80 Kb 0
 31744 - 32767 0 0 80 Kb 0
 32768 - <inf> 2 1 72 Kb 73744
0:000> !heap -ar 0x7ffff7d27ac0 -sg 0x0 -sl 0x3ff
 Chunk User Addr Size State
 --------------------------------------------------------------------------------
 0000555555559000 0000555555559010 656 in use, allocated
 000055555556b2a0 000055555556b2b0 32 in use, allocated
 000055555556b2c0 000055555556b2d0 32 in use, allocated
 000055555556b2e0 000055555556b2f0 32 in use, allocated
 000055555556b300 000055555556b310 32 in use, allocated
 000055555556b320 000055555556b330 32 in use, allocated
 ... <snipped for brevity> ...

Hopefully, that helps.