Merge three ByteArrays together and then split resulting ByteArrays

Question

I have a ByteArray value as avroBinaryValue , Schema Id value as short schemaId and Last Modified Date value as lastModifiedDate in long.

short schemaId = 32767;
long lastModifiedDate = "1379811105109L";
byte[] avroBinaryValue = os.toByteArray();

Now, I will write schemaId , lastModifiedDate and avroBinaryValue together into a single ByteArray and then deserialize that final ByteArray value to extract schemaId , lastModifiedDate and avroBinaryValue from it.

Below is the code, I have got so far...

public static void main(String[] args) throws Exception {
 String os = "whatever os is";
 byte[] avroBinaryValue = os.getBytes();
 long lastModifiedDate = 1379811105109L;
 short schemaId = 32767;
 ByteArrayOutputStream byteOsTest = new ByteArrayOutputStream();
 DataOutputStream outTest = new DataOutputStream(byteOsTest);
 outTest.writeShort(schemaId); // first write schemaId
 outTest.writeLong(lastModifiedDate); // second lastModifiedDate
 outTest.writeInt(avroBinaryValue.length); // then attributeLength
 outTest.write(avroBinaryValue); // then its value
 byte[] allWrittenBytesTest = byteOsTest.toByteArray();
 DataInputStream inTest = new DataInputStream(new ByteArrayInputStream(allWrittenBytesTest));
 short schemaIdTest = inTest.readShort();
 long lastModifiedDateTest = inTest.readLong();
 int sizeAvroTest = inTest.readInt();
 byte[] avroBinaryValue1 = new byte[sizeAvroTest];
 inTest.read(avroBinaryValue1, 0, sizeAvroTest);
 System.out.println(schemaIdTest);
 System.out.println(lastModifiedDateTest);
 System.out.println(new String(avroBinaryValue1));
 writeFile(allWrittenBytesTest);
}

I am trying to see whether there is any efficient way of doing this in Java or this is the only correct way of doing it in Java?

The way I am serializing all the three ByteArrays into one ByteArray and the way I am deserializing the resulting ByteArrays to extract the schemaId, lastModifiedDate, avroBinaryValue looks correct or not?

Answer 1

The way you are doing it is fine, however it seems like a better idea to just put all of the data you want to serialize into a Serializable object and then just serialize that automatically. For example you could do something like :

public static void main(String[] args) throws IOException,
 ClassNotFoundException {
 String os = "whatever os is";
 long lastModifiedDate = 1379811105109L;
 short schemaId = 32767;
 Thing myData = new Thing(os, lastModifiedDate, schemaId);
 ByteArrayOutputStream byteOsTest = new ByteArrayOutputStream();
 ObjectOutputStream outTest = new ObjectOutputStream(byteOsTest);
 outTest.writeObject(myData);
 byte[] allWrittenBytesTest = byteOsTest.toByteArray();
 ObjectInputStream inTest = new ObjectInputStream(new ByteArrayInputStream(
 allWrittenBytesTest));
 Thing myDataTest = (Thing) inTest.readObject();
 short schemaIdTest = myDataTest.getSchemaId();
 long lastModifiedDateTest = myDataTest.getLastModifiedDate();
 String avro = myDataTest.getAvro();
 System.out.println(schemaIdTest);
 System.out.println(lastModifiedDateTest);
 System.out.println(avro);
 writeFile(allWrittenBytesTest);
}
// in its own java file
public final class Thing implements Serializable {
 private final String avro;
 private final long lastModifiedDate;
 private final short schemaId;
 public Thing(String avro, long lastModifiedDate, short schemaId) {
 this.avro = avro;
 this.lastModifiedDate = lastModifiedDate;
 this.schemaId = schemaId;
 }
 public String getAvro() {
 return avro;
 }
 public long getLastModifiedDate() {
 return lastModifiedDate;
 }
 public short getSchemaId() {
 return schemaId;
 }
}

• WeaponsGrade: The only problem with using Java serialization is that it wouldn't contain a "pure" representation - it would also contain the class, field, and serial version information. But I agree it's a good method if you just want to transport that data from Java-Java.You could implement Externalizable, though and achieve a pure data representation.

Answer 2

Instead of writing them to a byte array first, why not just write them directly to the file?

public void writeInfo(final OutputStream fileStream) throws IOException {
 final DataOutputStream data = new DataOutputStream(fileStream);
 data.writeShort(schemaId);
 data.writeLong(lastModifiedDate);
 data.writeInt(avroBinary.length);
 data.write(avroBinaryValue);
 data.flush();
 // if it's okay to close the underlying stream:
 data.close();
}

Of course, the method names the stream a fileStream, but really it's just an untyped OutputStream (to write directly to a file pass in a FileOutputStream). This is also testable since you can still pass in a ByteArrayOutputStream.