Packing data (was Re: [PROPOSAL 5.4] alternate way of format/pack/unpack (in the string library))
[
Date Prev][
Date Next][
Thread Prev][
Thread Next]
[
Date Index]
[
Thread Index]
- Subject: Packing data (was Re: [PROPOSAL 5.4] alternate way of format/pack/unpack (in the string library))
- From: Sean Conner <sean@...>
- Date: 2017年5月26日 19:22:24 -0400
It was thus said that the Great tobias@justdreams.de once stated:
>
> Quoting Sean Conner <sean@conman.org>:
>
> > I don't find the pack/unpack minilanguage all that bad, per se.
> > Lowercase
> >letters are signed quantities, uppercase unsigned and there's some
> >mneumonic
> >meaning to the letters used. But it can get silly (sample from an SMPP
> >parser):
> >
> > result.service_type,
> > result.source.addr_ton,
> > result.source.addr_npi,
> > result.source.addr,
> > result.dest.addr_ton,
> > result.dest.addr_npi,
> > result.dest.addr,
> > result.esm_class,
> > result.protocol_id,
> > result.prority,
> > result.schedule_time,
> > result.validity_period,
> > result.registered_delivery,
> > result.replace_if_present,
> > result.data_coding,
> > result.sm_default_msg_id,
> > result.message =
> > string.unpack(">z I1 I1 z I1 I1 z I1 I1 I1 z z I1 I1 I1 I1
> > s1",blob,13)
> >
> > It was hard to debug, and the obvious solution:
> >
> > result.service_type,pos = string.unpack(">z",blob,pos)
> > result.source.addr_ton,pos = string.unpack(">I1",blob,pos)
> > result.source.addr_npi,pos = string.unpack(">I1",blob,pos)
> > --- and so on
>
> This is exactly what I'm trying to solve with lua-t library Pack module. It
> doesn't support zero terminated strings yet but once it does your code could
> look like:
>
> snppParser = Pack(
> { service_type = ">z" },
> { source = Pack(
> { addr_ton = 'I1' },
> { addr_npi = 'I1' },
> { addr = 'z' }
> ) },
> { dest = Pack(
> { addr_ton = 'I1' },
> { addr_npi = 'I1' },
> { addr = 'z' }
> ) },
> { esm_class = 'I1' },
> { protocol_id = 'I1' },
> { prority = 'I1' },
> { schedule_time = 'z' },
> { validity_period = 'z' },
> { registered_delivery = 'I1' },
> { replace_if_present = 'I1' },
> { data_coding = 'I1' },
> { sm_default_msg_id = 'I1' },
> { message = 's1' }
> )
>
> result = snppParser( blob )
Actually, that's just one of several different packet types for SMPP, and
not included there are the optional parameters. I'm reading your
documentation on Pack [1] and I'm not seeing anything that can actually
handle the optional part of the SUBMIT_SM packet (what I'm actually encoding
there). You list:
atomic A single instance
sequence Multiple values in a given order
array Multiple values of the same type
struct Ordered collection of named values
The optional parts of the SUBMIT_SM are a series of values that can appear
in any order. Now, they all follow a similar format:
tag 2 byte integer network byte order
length 2 byte integer network byte order
value variable, based upon length
For an even weirder example, we have a custom binary format.
tag 1 or 3 bytes
length 1 or 3 bytes
value variable length
For the tag or length, if the byte is less than 255, then that's the
value; if it is 255, then the actual value is the next two bytes, in network
byte order. The order of tags does not matter, *except* that tag value of 1
should always be first.
CBOR [2] is similar---integer values can be 1 byte, 2 bytes, 3 bytes, 5
bytes or 9 bytes in size, and the type is encoded with the value in some
cases.
Then there are protocols like IP, where some fields are only a few bits in
size, but it looks like you handle that somewhat.
-spc
[1] https://github.com/tobbik/lua-t/blob/master/docs/Pack.rst
[2] Concise Binary Object Representation RFC-7049
http://cbor.io/