python pickle.dumps and struct.pack in Java

Question

I'm trying to feed a carbon(Graphite) server with data from a Java application. I want to use the pickle protocol instead of the oneline protocol because it seems to be much faster.

I've done this in a small python script that a invoke from my Java application. But I want to write this in native Java.

The python script looks like this:

listOfMetricTuples = [('test', (1, 1352903620)), ('test', (2, 1352903620))]
payload = pickle.dumps(listOfMetricTuples)
header = struct.pack("!L", len(payload))
message = header + payload

It would be great to not need to include any libraries.

Anyone got a solution for this?

• Jeremy Roman: The pickle format is not really used outside Python, and is somewhat complicated (it involves writing a little bytecode VM, as I understand it). I would suggest using a different format (e.g. JSON) to encode your data if possible. Otherwise, it's probably better to find a library that already implements the pickle format than trying to do it yourself.
• brange: I cant change the server. So I must use pickle.dumps. But I found this: irmen.home.xs4all.nl/pyrolite That I might be able to use.

Answer 1

Its now solved.

I solved it by using Jython and the following code.

try{
 Socket s = null;
 try{
 s = new Socket("debian-srv", 2004);
 }catch(UnknownHostException e){
 e.printStackTrace();
 }catch(IOException e){
 e.printStackTrace();
 }
 if (s == null) {
 return -1;
 }
 PyTuple t = new PyTuple(new PyString("Test.brange-debian.mojo"), new PyTuple(new PyInteger(1352975858), new PyInteger(56)));
 PyTuple t2 = new PyTuple(new PyString("Test.brange-debian.mojo"), new PyTuple(new PyInteger(1352975858-60), new PyInteger(43)));
 PyTuple t3 = new PyTuple(new PyString("Test.brange-debian.mojo"), new PyTuple(new PyInteger(1352975858-2*+60), new PyInteger(65)));
 PyList list = new PyList();
 list.append(t);
 list.append(t2);
 list.append(t3);
 PyString payload = cPickle.dumps(list);
 byte[] bytes = ByteBuffer.allocate(4).putInt(payload.__len__()).array();
 s.getOutputStream().write(bytes);
 s.getOutputStream().write(payload.toBytes());
 s.getOutputStream().flush();
 s.close();
}
catch (Exception e) {
 e.printStackTrace();
}

• Adam Gent: You should do some benchmarks to see if that little exercise was worth your time :)

Answer 2

Another alternative, according to How do I serialize a Java object such that it can be deserialized by pickle (Python)?, might be to use pyrolite. The footprint might be smaller than having to use Jython.