Array Partitioning

Question

I have a method in Java that I'm using to partition byte[] arrays into a number of fixed-size sub-arrays. It's not necessary for the last array to be that size, but all preceding ones must be, and the maximum size of a sub array is the partition size.

Here's the code I've been using:

 private static byte[][] partition(byte[] in, int partitionSize) {
 int partitionCount = (int)Math.ceil((double)in.length / (double) partitionSize);
 byte[][] temp = new byte[partitionCount][partitionSize];
 for (int i = 0; i < partitionCount; i++) {
 if (in.length < (partitionSize * (i + 1))) {
 temp[i] = new byte[(in.length - (partitionSize * i))];
 }
 for(int j = 0; (j < partitionSize && (partitionSize * i + j) < in.length); j++) {
 temp[i][j] = in[(partitionSize * i + j)];
 }
 }
 return temp;
 }

Are there any obvious ways I could optimize this for speed? The partitionSize I've been using so far is 971, and even though I was expecting it to take a while for large byte arrays (around a gigabyte in size) I'm hoping to be able to squeeze out a bit more performance.

• FreezePhoenix: Why are we not using the native Array.forEach?
• Lev Knoblock: @FreezePhoenix I believe that's not an option? stackoverflow.com/q/35518471/9319868?
• FreezePhoenix: Forgive my ignorance on that. I did not know.
• somebody: Well really the ideal (i.e. probably fastest) way is to not have a partitioned array at all, just use arithmetic to determine the appropriate location to index into: row * row_length + column
• Lev Knoblock: @somebody Sadly, that's not an option that I have. I'm using this to separate large messages into separate packets, so splitting them up at some point is a requirement.

Answer 1

You have to use System.arraycopy() to get best performance on array copies.

Another Small Point : Your point of 'final array can be less in size' is ignored during the 2d array creation. i.e., the second dimension cannot be partitionSize. It has to be dynamic based on the which partition it is.

The below code is high performance and also it is taking care of this last partition size also in the final returned 2d array.

private static byte[][] partition2(byte[] in, int partitionSize)
{
 int partitionCount = (int)Math.ceil((double)in.length / (double) partitionSize);
 byte[][] temp = new byte[partitionCount][];
 for (int p = 0; p < partitionCount; p++)
 {
 int start = p * partitionSize;
 int len = (p != partitionCount - 1) ? partitionSize : in.length - start;
 byte[] partition = new byte[len];
 System.arraycopy(in, start, partition, 0, len);
 temp[p] = partition;
 } 
 return temp;
}

With this snippet, you will be able to feel the difference if the input array size is significant.

• Lev Knoblock: Definitely seeing a difference! Also, just to point out, my code was handling the final array being of a different size in the if statement.
• FreezePhoenix: I transpiled this to javascript, and noticed that it is slower than one of my implementations of a similar thing in JS. Is it possible that this could be improved even more? Also note that in.length should be static, so we don't need to constantly retrieve it every time.
• somebody: @FreezePhoenix We aren't constantly retrieving it
• FreezePhoenix: @somebody In every iteration they retrieve that value again.
• somebody: @FreezePhoenix ... No? it resolves to partitionSize and never retrieves in.length except in the last iteration...