heap fragmentation?

Tue Jun 4 16:22:00 GMT 2002

While testing some applications with 3.1, I've noticed that
steady-state heap usage is significantly greater than with Sun's VM, by
roughly a factor of three. (I would've expected a precise GC to be more
space efficient than a conservative non-moving collector, but not quite
this much.)
Can someone tell me if I understand this output correctly:
Initiating full world-stop collection 291 after 25312 allocd bytes
0 bytes in heap blacklisted for interior pointers
--> Marking for collection 291 after 25312 allocd bytes + 0 wasted bytes
Collection 290 reclaimed -6736 bytes ---> heapsize = 32661504 bytes
World-stopped marking took 150 msecs
Bytes recovered before sweep - f.l. count = -41312
Immediately reclaimed -41312 bytes in heap of size 32661504 bytes
364600 (atomic) + 5346520 (composite) collectable bytes in use
Total heap size is ~32MB, with 5-6MB in use. No blacklisting, which is
good.
***Static roots:
...
Total size: 6367157
***Heap sections:
...
Total heap size: 32661504
This concurs with the heap size above.
***Free blocks:
Free list 1 (Total size 2007040):
 0x101876000 size 8192 not black listed
 0x101694000 size 8192 not black listed
 0x101582000 size 8192 not black listed
 ...
Free list 2 (Total size 1228800):
Free list 3 (Total size 737280):
...
Free list 32 (Total size 278528):
Free list 33 (Total size 360448):
Free list 60 (Total size 5652480):
Total of 13369344 bytes on free list
I see only 5MB or so allocated between collections. Why does the GC
maintain more free blocks than needed from one collection to the next?
Shouldn't MERGE_SIZES help?
***Blocks in use:
(kind(0=ptrfree,1=normal,2=unc.,3=stubborn):size_in_bytes, #_marks_set)
(4:64,21)(4:64,3)(4:256,25)(1:112,3)(4:96,16)(4:512,16)(4:32,23)
...
blocks = 2268, bytes = 19292160
These blocks are mostly empty, it seems. (HBLKSIZE is 8192 on sparc).
My observations:
a) A good deal of heap is lying unused. 32MB isn't a large amount,
however this could increase by at least an order of magnitude if our usage
patterns change. Also, significant fragmentation worsens locality, right?
We're counting on gcj to help reduce our memory overhead. On the code
size it certainly does. But heap size more than offsets that savings, and
heap can't be shared with other processes.
b) Building with SMALL_CONFIG should help, as does incremental
collection. But those two cannot be used together.
c) Presently I'm building 64-bit executables, which no doubt contributes
to heap size. The VM I'm comparing to is a 32-bit VM, which isn't exactly
a fair comparison. However, byte/char arrays are responsible for a good
part of the heap, which you don't see above because I don't have hash
synchronization enabled.
d) Hash synchronization will help by trimming 8 bytes from each object.
Any other strategies to avoid fragmentation? I couldn't find much in the
documentation available.
Jeff