Space required to keep largest integer array in memory

Question

I am reviewing an algorithm and it keeps an array of integers, the size of input is dynamic. So according to my calculations it can take as much as

 integer MAX_VALUE * int size = ? 
 2^31 integers * 4 bytes = ?
2147483648 integers * 4 bytes = 8 Gigabytes

Is this calculation correct ? would the JVM use this much contiguous space to store the int array or are there other things that one needs to consider ?

• Gene: Yes the basic calculation is correct, but the jvm must also be configured for it, and the underlying machine needs enough ram to support that configuration. See e.g. stackoverflow.com/questions/31382531/….
• mcvkr: oh, great to know @Gene thanks
• mcvkr: yes the concern is the RAM of the machines that runs this software
• Eugene: Is it an int[] or Integer[]?
• mcvkr: it is an int[] @Eugene

Answer 1

The theoretical size of the array would be :

• numberOfElementsInTheArray * 4 bytes

• 12 bytes of headers (int[] is an Object). Actually the size of headers depends on the flags you used and on the JVM version you are running this

• 4 bytes to keep the length of the array

• padding.

For example: (I am going to use JOL for this):

 int [] x = new int[10];
 for(int i=0;i<10;++i){
 x[i] = 9999;
 }
 System.out.println(GraphLayout.parseInstance((Object)x).toPrintable()); 

will output:

 [I@7a81197dd object externals:
 ADDRESS SIZE TYPE PATH VALUE
 70fe45268 56 [I [9999, 9999, 9999, 9999, 9999, 9999, 9999, 9999, 9999, 9999]

So it has 56 bytes:

• 40 for the values themselves (10 ints * 4 bytes)
• 12 for headers
• 4 for length
• 0 for padding

---

If you change this array to Integer, things change dramatically. Integer is an Object so you will store a reference inside the array (which could be 4 or 8 bytes, depending on UseCompressedOops flag), plus each of the Integer instances will require 2 headers (each Integer is an Object).

 Integer[] y = new Integer[10];
 for(int i=0;i<10;++i){
 y[i] = 9999;
 }
 System.out.println(GraphLayout.parseInstance((Object)y).toFootprint());

which will show:

 [Ljava.lang.Integer;@369f73a2d footprint:
 COUNT AVG SUM DESCRIPTION
 1 56 56 [Ljava.lang.Integer;
 10 16 160 java.lang.Integer
 11 216 (total)

A total of 216 bytes:

• 4 bytes for each reference (I have UseCompressedOop turned on), 40 bytes total
• 12 bytes headers of the array
• 4 bytes length of the array
• 0 bytes padding

Each reference from that array points to an Integer, each of those Objects will have 16 bytes:

• 4 bytes for the inner int they hold
• 12 bytes headers
• 0 bytes padding

• mcvkr: this makes it so clear why we need to do tasks with primitives as much as possible. great details!
• Eugene: @mcvkr just notice that there is a JEP in java where Objects might drop headers in special cases, but it seems there is a long way there
• mcvkr: can public see the JEP or does it need a openJDK account?
• Eugene: @mcvkr project ValHalla. you can accept the answer, btw.
• mcvkr: Thanks again, looks like, the specific JEP for this is : openjdk.java.net/jeps/169 in scope of the ValHalla project

Answer 2

Array size maximum < Integer.MAX_VALUE

No, your maximum is incorrect.

The limit on number of elements in an array in Java is a little bit less than Integer.MAX_VALUE (2,147,483,647), depending on the version of Java, the host OS, and how Java was compiled. See this Answer by Ivan Mamontov on the Question, Why I can't create an array with large size?.

Yes, biggest array of int ≈ 8 gigs

So the size of a maximum array of int will be roughly ( Integer.MAX_VALUE - 8L ) * 32L bits which is 68,719,476,448 bits which is 8,589,934,556 octets.

So yes, around 8 gigs of memory. And remember: this is contiguous memory for an array. So:

• There may be significant work on the part of the JVM and host OS to produce such an array depending how fragmented is memory at that moment during runtime.
• If the host hardware does not have sufficient real memory, you'll be lapsing into virtual memory where the resulting paging may lead to terrible performance.

Always make real-world tests if you really are pushing these limits in your work. And you may want to consider alternative implementations of Java designed for very large memory, such as Zing by Azul Systems.

• mcvkr: and what if it is an Integer array @Basil or a List of Integers? Because one can easily switch to those from an int array
• Basil Bourque: @mcvkr Objects take up more room than primitives, and execute a bit slower than primitives. So when minimizing memory or execution time is paramount, use primitive int and arrays. Otherwise, for more convenience, use class Integer and collections such as List and Set.
• Basil Bourque: FYI: Work is underway, led by Oracle, to largely eliminate the difference between primitives and objects — but won’t arrive soon.
• Eugene: a lot more...