maximum memory for gcj-compiled executable?

Thu Apr 3 03:49:00 GMT 2003

> -----Original Message-----
> From: Lars Andersen [mailto:lars@rimfaxe.com]
> ...
> First of all, I don't claim that live data is sharply below 
> 10 MB at all
> time. I debug used/free heap at short intervals, and there is always a
> lot free. This is a peculiar program : it recieves a http request once
> every 30secs. (Not average, literally every 30secs). Plus, it 
> sends out
> a http request every 10 minutes. This might or might not start a short
> period of activity. The program sleeps thru periods of inactivity.
>> The development on the heap goes like this :
>> eat 20 kb (30sec request)
> eat 20 kb
> eat 20 kb
> ...
> this goes on for a while
> ...
> eat 20 kb
> free 3000kb ( the garbage collector kicks in)
>> this is just great, and how it is supposed to work. But sometimes a
> little extra is allocated to the heap. This might be due to increased
> activity. 
>> Clearly, when a heap is almost full, and the GC can free almost all of
> it, why not a smaller heap, conserving memory?
The heap is sized so that the amount of GC work per byte allocated is roughly constant. I think it's important to preserve that property, at least in the default configuration; otherwise allocations can become arbitrarily expensive.
You can adjust the constant, and trade off time for space, by adjusting GC_free_space_divisor (see gc.h).
The behavior you describe looks normal for a fairly small heap, with default GC_free_space_divisor. Gcj collections may not take place much more often than every 3MB of allocation because the root set is often bigger than that (unfortunately). This should not substantially affect applications that need a big heap; it does matter for applications that could otherwise live with a very small one. It can be adjusted with GC_free_space_divisor. (The latest version contains a setter function for that; I think gcj does not.)
>>> Hey, would incremental garbage collecting do the trick, you think?
No. It'll probably worsen this problem.
>> If the heap was full of non-freeable stuff, it would be a different
> story. But this is not the case.
>> I realize that the current behaviour is desired for most apps. But Sun
> included the -mx -ms options for a reason, don't you think?
I suspect that part of the original reason for -mx (with a small default) was an implementation weakness in some old JVMs. It's easier to write the collector if you can preallocate a contiguous heap of the right size. But I think we agree: An option along these lines is sometimes very useful, but by default the bound on heap size should be as large as possible.
> Well I'm not partial about it as long as it works. Currently only
> initial heap size works. Other people have suggested the 
> option path, I
> just wanted to move things along.
You should be able to use the collector C hook (GC_set_max_heap_size) for setting the maximum heap size. That should work after the program has started, though it won't reduce the heap size.
If an environment variable is OK, I think the easiest path is to wait for the current GC to be integrated. It already has that facility.
>> Why set a default max heap size. There isn't one now!
> If the max heap size option is present, use it. Otherwise don't.
>> > There are also some known reasons that heap size is 
> currently suboptimal, but these 
> > shouldn't normally add more than 10MB or so to the heap:
> > 
> > 1) The root size is way too big. This has been discussed 
> here before, and I think we agree 
> > on at least an approach for fixing it. This causes the collector to
> > grow the heap to much to compensate for the root scanning time. 
> > (You can tune the collector to compensate, but then it runs slowly.)
> > 
> > 2) Under win32, it tends to scan system malloced memory as roots. 
> >
> > This is a misfeature, since it can't be relied on. My 
> current version 
> > attempts to fix it, but the fix is heuristic, since I 
> don't know of a
> > way to identify malloced memory sections. (The fix works well on a
> > small toy example :-) .) This affects only Windows.
>> Well, the worst cases all manifested themselves on win32...
>> This is an important matter, and I'm spending a lot of time getting to
> the bottom of it. 
>> I suspect my program uses less than 2-3MB effective heap. 
> Setting a max
> size to 3MB might make the GC run wild, but if I'm wrong I would get a
> outofmemory exception. In the current situation I don't know whether
> there is stray objects on the heap, errors in the gcj or what.
You should be able to do that with a small amount of C++ glue code to call GC_set_max_heap_size (or possibly the _Jv version). The GC log should also tell you something.
>> ...
> BTW, any idea when that max heap environment setting makes it 
> into gcj?
>I think it's still on Tom's (long) list. It requires either someone with CVS expertise or for someone (like me) to have enough time to acquire the expertise. (At this point, it might also make sense to wait for a real GC 6.2, which should happen shortly.)
Hans