[OT] Re: Controlling the garbage collector (GC) at RT?

Fri Mar 18 20:13:00 GMT 2005

Hi Hans,
> It's probably best to move this discussion to some linux kernel
> group, since it no longer seems to be related to gcj.
I have done so - I've taken it to the linux.kernel.mm-group - hoping it 
is the correct place... :-)
> I would try it on the newest available kernel, with as few other
> applications
> running as possible. It is conceivable that the problem is
>> - Some other application which somehow convinces the kernel it needs
> less
> memory than it actually does.
>> - A temporary bug in 2.6.5.
Oh, but my 2.6.5 kernel works as it is supposed to - that's what I meant 
by the strange sentence: "I verified that that it should work on an 2.6.5".
But as stated in the other thread posting I did earlier today, this is 
on my desktop-Linux with swap...
> I think the basic mechanism should work if the /proc/sys file is there.
> I assume overcommit_memory reads back as the value you echoed into it?
Sure - it reads back the correct value if I cat it...
// Martin
>>-----Original Message-----
>>From: java-owner@gcc.gnu.org [mailto:java-owner@gcc.gnu.org] 
>>On Behalf Of Martin Egholm Nielsen
>>Sent: Thursday, March 17, 2005 11:45 PM
>>To: java@gcc.gnu.org
>>Subject: Re: [OT] Re: Controlling the garbage collector (GC) at RT?
>>>>>>Hi,
>>>>>>>I tried the attached program on Linux 2.6.7rc3 running on a 
>>>>4GB IA64 
>>>>>machine with 1GB swap space. It failed with a signal with 
>>>>the default 
>>>>>overcommit policy of 0.
>>>>>>After an
>>>>>>echo 2 > /proc/sys/vm/overcommit_memory
>>>>>>as root it failed with sbrk returning zero. My 
>>>>interpretation is that 
>>>>>the overcommit policy zero heuristic may need a bit of work, but 
>>>things basically work as described.
>>>>Not at my place :-)
>>>>>>>I repeated the experiment with an old X86 machine (256M mem + 512M 
>>>swap) running 2.4.18, with basically identical results. 
>>>>This seems to 
>>>>>be a system configuration issue.
>>>>So you think it requires some kernel configuration?
>>>>But then it's weird the kernel bothers to create the 
>>/proc/sys/vm/overcommit_memory file, if I have it disabled somehow...
>>>>But I verified that it should work on an 2.6.5/i386...
>>>>// Martin
>>>>>>>>-----Original Message-----
>>>>From: Martin Egholm Nielsen [mailto:martin@egholm-nielsen.dk]
>>>>Sent: Thursday, March 17, 2005 1:15 AM
>>>>To: Boehm, Hans
>>>>Subject: [OT] Re: Controlling the garbage collector (GC) at RT?
>>>>>>>>>>>>Hi Hans,
>>>>>>>>>>>>>>>>>>>You also need to touch the resulting memory. Try something like
>>>>>>>char *v = sbrk( 1000000 ); for (char *p = v; p < p + 
>>>>>>>>1000000; ++p)
>>>>>>>>>>>>>>>*p = 42; in the loop.
>>>>>>>>>>>>I'll try something similar tomorrow, thanks alot - 
>>>>although I will 
>>>>>>>>need modify the ever true statement "p < p + 1M" to "p < 
>>>>v + 1M"...
>>>>>>>So, now I tried this:
>>>>>>>>>>#include <unistd.h>
>>>>>>>>>>int main( int i )
>>>>>{
>>>>> while ( 1 ) {
>>>>> char *v = sbrk( 1000000 );
>>>>> char *p;
>>>>> for (*p = v; p < v + 1000000; ++p) {
>>>>> *p = 42;
>>>>> } // for
>>>>>>>>>> printf( "%x\n\n", v );
>>>>> } // while
>>>>>} // main
>>>>>>>>>>And still nothing happens with the memory consumption, 
>>>>although the 
>>>>>>>pointer ends up at 0xffffffff...
>>>>>>>>Nooow, after modifying the program to
>>>>>>>>for ( p = v; ...
>>>>>>>>it "works" - the application is terminated by the kernel. This goes 
>>>>regardless what [0,1,2] is echo'ed into 
>>>>"/proc/sys/vm/overcommit_memory"...
>>>>The kernel reports:
>>>>>>>>Out of Memory: Killed process 36 (sbrktest)
>>>>>>>>But what should the outcome really had been?
>>>>>>>>I tried the same on an elder kernel on a different target -
>>>>2.4.17 and 
>>>>i386 - and the same outcome...
>>>>>>>>BR,
>>>> Martin Egholm
>>>>>>>>>>>>>>>>>>>>-----Original Message-----
>>>>>>>>From: java-owner@gcc.gnu.org [mailto:java-owner@gcc.gnu.org] On
>>>>>>>>Behalf Of Martin Egholm Nielsen
>>>>>>>>Sent: Wednesday, February 23, 2005 12:04 AM
>>>>>>>>To: java@gcc.gnu.org
>>>>>>>>Subject: Re: Controlling the garbage collector (GC) at RT?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>It seems to me that the only problem here is that this
>>>>>>>>>>>>>>>>still seems to
>>>>>>>>>>>>>>>>>>>>>>>>>happen with overcommit-accounting set to 2.
>>>>>>>>>I would expect that you can reproduce this problem with a
>>>>>>>>>>>>>>>>program that
>>>>>>>>>>>>>>>>>>>>>>>>>alternately allocates a few MB with sbrk, and then touches the
>>>>>>>>>allocated memory. If you can't, there's something really 
>>>>>>>>>>>>>>>>weird going
>>>>>>>>>>>>>>>>>>>>>>>>>on here. If you can, it'll give you a test case for the kernel
>>>>>>>>>people.
>>>>>>>>>>>>>>>>I tried the following:
>>>>>>>>>>>>>>>>#include <unistd.h>
>>>>>>>>>>>>>>>>int main( int i )
>>>>>>>>{
>>>>>>>> while ( 1 ) {
>>>>>>>> void *v = sbrk( 100000 );
>>>>>>>> // sleep( 1 );
>>>>>>>> } // while
>>>>>>>>} // main
>>>>>>>>>>>>>>>>But that doesn't result in anything - the memory usage for the
>>>>>>>>application does not grow...
>>>>>>>>Are there anything else I should do to allocate the memory?
>>>>>>>>>>>>>>>>BR,
>>>>>>>>Martin
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Hans
>>>>>>>>>>>>>>>>>>On 2005年2月16日, Martin Egholm Nielsen wrote:
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Hi Hans,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>It would indeed be interesting to know why the Linux kernel
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>kills the application rather than returning failure.
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Sure, but how to do that? Any guidelines?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>What do you see on the console? Anything in the system log?
>>>>>>>>>>>Does strace tell you anything?
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>Below is the last part of "strace -f -F -i -v". It doesn't
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>really look
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>like there's anything of value?
>>>>>>>>>>>>>>>>>>>>// Martin
>>>>>>>>>>>>>>>>>>>>[pid 75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>8388608\n", 29***
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>MEM CHUNK TAKEN: 8388608
>>>>>>>>>>) = 29
>>>>>>>>>>[pid 75] [0f839834] brk(0x12d15000) = 0x12d15000
>>>>>>>>>>[pid 75] [0f839834] brk(0x12d25000) = 0x12d25000
>>>>>>>>>>[pid 75] [0f81126c] getpid() = 75
>>>>>>>>>>[pid 75] [0f799444] kill(77, SIGPWR <unfinished ...>
>>>>>>>>>>[pid 76] [0f840e2c] <... poll resumed> [{fd=3, 
>>>>>>>>events=POLLIN,
>>>>>>>>>>>>>>>>>>revents=POLLIN}
>>>>>>>>>>], 1, 2000) = 1
>>>>>>>>>>[pid 75] [0f799444] <... kill resumed> ) = 0
>>>>>>>>>>[pid 76] [0f81127c] getppid() = 75
>>>>>>>>>>[pid 76] [0f833548] read(3,
>>>>>>>>>>"20円7円/234円0円0円0円4円17円374円0ドル20円7円/240円$0円0円B17円3円
>>>>>>>>>>72j(177円"..., 148) = 148
>>>>>>>>>>[pid 76] [0f840e2c] poll( <unfinished ...>
>>>>>>>>>>[pid 75] [0f799444] kill(77, SIGXCPU) = 0
>>>>>>>>>>[pid 75] [0f839834] brk(0x13525000) = 0x13525000
>>>>>>>>>>[pid 75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>8388608\n", 29***
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>MEM CHUNK TAKEN: 8388608
>>>>>>>>>>) = 29
>>>>>>>>>>[pid 75] [0f839834] brk(0x13535000) = 0x13535000
>>>>>>>>>>[pid 75] [0f839834] brk(0x13545000) = 0x13545000
>>>>>>>>>>[pid 75] [0f839834] brk(0x13d45000) = 0x13d45000
>>>>>>>>>>[pid 75] [0f833558] write(1, "*** MEM CHUNK TAKEN: 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>8388608\n", 29***
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>MEM CHUNK TAKEN: 8388608
>>>>>>>>>>) = 29
>>>>>>>>>>[pid 76] [0f840e2c] <... poll resumed> [{fd=3, 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>events=POLLIN}], 1,
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>2000) = 0
>>>>>>>>>>[pid 76] [0f840e2c] --- SIGTERM (Terminated) ---
>>>>>>>>>>#
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>