Heap profiling, of sorts

Thu Mar 11 23:57:00 GMT 2004

This is an initial pass at a patch to make heap backtraces, as they were
already supported by the collector, work with gcj. This works for me
on X86, though it could certainly be improved. (It also "works" on Itanium,
but it needs to be taught to take advantage of libunwind to be more
useful.)
The patch also cleans up the libgcj collector interface, so that libgcj no
longer includes GC-private headers.
Some of these changes required GC changes. They are included in this patch,
and will be in the next standalone GC release.
The patch further includes a GC patch to reduce the address range for blacklisting.
This was recently discussed on the gc@linux.hpl.hp.com list
(http://www.hpl.hp.com/hosted/linux/mail-archives/gc/2004-March/000350.html),
and may help with some of the space usage complaints for large heaps.
In my opinion, this should go into 3.5, after more testing, and with adjustment for
any suggestions. Except possibly for the GC blacklisting part, it should not go into 3.4.
Usage model:
To get heap backtraces you need to build the GC with --enable-full-debug. This
will also define LIBGCJ_GC_DEBUG for the libjava build, which will cause
libjava to use the debug allocation routines.
The easiest way to actually get heap backtraces is to then set the GC_BACKTRACES
environment variable to N before you run the Java application. This will cause
the collector to pick N random addresses of live objects in the heap after each GC,
and try to generate trace of the reference chain that keeps it alive. Each object
in the reference chain will be described by the call stack at its allocation
(by default 8 frames on X86). The trace will appear on stderr. It may be
incorrect if the objects are concurrently changed, but should be statistically useful.
The backtraces are currently rather verbose, so expect a fair amount of output.
(Clearly there is room for a better UI here.)
Generally this gives you a good idea of why things in the heap are live. I find
it more useful than a breakdown of the heap composition by type, though that would
clearly be useful, too. (And getting Java types into the backtraces would perhaps
be even nicer.) It allows you to determine why objects are being retained,
even in cases in which that's due to pointer misidentification, finalization issues,
etc.
Note that a libgcj built in this manner is NOT suitable for production use. For very
allocation intensive code, you may see a factor of 5-10 slowdown. And that's without
actually turning on backtraces (which are even slower). Instances of
java.lang.Object become 64 bytes. Even if you don't turn on backtraces, you're
getting some other checking in addition to the large objects. Unfortunately, this
does mean that you need a separate build for heap debugging, which is very
unfortunate. (Fixing this seems to be hard without adding to the allocation path.)
Comments/bug reports are greatly appreciated, though it may take me a day or two to get
to them.
Hans
Boehm-gc ChangeLog:
2004年03月11日 Hans Boehm <Hans.Boehm@hp.com>
	* alloc.c
	(GC_should_collect): Add GC_collect_at_heapsize.
	(GC_expand_hp_inner): Set expansion_slop more conservatively.
	Warn if heap too large.
	* configure, configure.in: Set LIBGCJ_GC_DEBUG with --enable-full-debug.
	* dbg_mlc.c
	(GC_print_backtrace): Add newlines in prints.
	(GC_generate_random_backtrace): Split off
	GC_generate_random_backtrace_no_gc.
	(GC_debug_header_size): New.
	* finalize.c
	(GC_notify_or_invoke_finalizers): Check GC_backtraces.
	* misc.c
	(GC_backtraces): New.
	(GC_init_inner): Set GC_backtraces from GC_BACKTRACES in environment.
	(GC_new_free_list_inner, GC_new_free_list): New.
	(GC_new_kind_inner, GC_new_kind): New.
	(GC_new_proc_inner, GC_new_proc): New.
	* gcj_mlc.c
	(GC_init_gcj_malloc): Use GC_new_kind_inner() and friends.
	(GC_debug_gcj_malloc): Correct comment.
	* doc/README.environment: Add GC_BACKTRACES.
	* include/gc_mark.h:
	(GC_debug_header_size, GC_USR_PTR_FROM_BASE): New.
	(GC_new_free_list_inner, GC_new_free_list): New.
	(GC_new_kind_inner, GC_new_kind): New.
	(GC_new_proc_inner, GC_new_proc): New.
	(GC_generic_malloc): Add declaration.
	* include/javaxfc.h: Delete everything except GC_finalize_all.
	* include/private/dbg_mlc.h
	(USR_PTR_FROM_BASE): Delete.
	* include/private/gc_priv.h
	(GC_generic_malloc): comment out declaration.
	(GC_backtraces): Add declaration.
	* Tests/test.c: Don't include private/dbg_mlc.h.
	(USR_PTR_FROM_BASE): Rename to GC_USR_PTR_FROM_BASE.
	
Libjava ChangeLog:
2004年03月11日 Hans Boehm <Hans.Boehm@hp.com>
	* boehm.cc: Include gc_mark.h, javaxfc.h, but no GC private files.
	(GC_DEBUG): Set if LIBGCJ_GC_DEBUG is set.
	(GC_finalize_all, GC_debug_generic_malloc): Don't declare.
	(disable_gc_mutex): Delete along with all references.
	(_Jv_MarkObj, _Jv_MarkArray): Use public types,
	adjust for debug header size.
	(_Jv_AllocObj, _Jv_allocPtrFreeObj): Define out of line for
	debug case.
	(_Jv_AllocArray): Declare min_heap_addr only if needed.
	(_Jv_InitGC): Use GC_new_free_list, GC_new_proc, and GC_new_kind.
	* include/boehm-gc.h:
	(_Jv_AllocObj, _Jv_allocPtrFreeObj):
	Don't define, but declare, for debug case.
	* java/lang/natObject.cc:
	(GC_DEBUG): Define if LIBGCJ_GC_DEBUG is set.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gc_diffs
Type: application/octet-stream
Size: 36391 bytes
Desc: not available
URL: <http://gcc.gnu.org/pipermail/java/attachments/20040311/2e13c496/attachment.obj>