cbor: Updated serialization. #20

pukkamustard commented 2023年11月19日 17:38:07 +01:00

This is an incompatible update to the CBOR serialization (it could be made backwards compatible).

It's inspired by #11 - an appendable serialization. The changes make the CBOR serialization usable as CBOR Sequence (RFC 8742) and thus appendable.

Furthermore it allows including only references to blocks. This might be useful for transporting a manifest of blocks without the actual block content.

We can make these changes backwards compatible by adding the previous map in the ERISContent definition:

ERISContent = [ Block / BlockWithReference / Reference / ReadCapability / BlockMap]
BlockMap = { * bstr => bstr }

I'm unsure if we should, given there are only very few (known) implementations.

This is an incompatible update to the CBOR serialization (it could be made backwards compatible). It's inspired by https://codeberg.org/eris/eer/pulls/11 - an appendable serialization. The changes make the CBOR serialization usable as CBOR Sequence ([RFC 8742](https://www.rfc-editor.org/rfc/rfc8742.html)) and thus appendable. Furthermore it allows including only references to blocks. This might be useful for transporting a manifest of blocks without the actual block content. We can make these changes backwards compatible by adding the previous map in the `ERISContent` definition: ``` ERISContent = [ Block / BlockWithReference / Reference / ReadCapability / BlockMap] BlockMap = { * bstr => bstr } ``` I'm unsure if we should, given there are only very few (known) implementations.

pukkamustard commented 2023年11月19日 17:46:26 +01:00

Probably should just make it backwards-compatible if we can...

Probably should just make it backwards-compatible if we can...

emery commented 2023年11月19日 18:40:49 +01:00

A { * Reference => Block } with one pair has the same encoded size as [ Reference, Block ], so why not use maps rather than pair arrays?

A `{ * Reference => Block }` with one pair has the same encoded size as `[ Reference, Block ]`, so why not use maps rather than pair arrays?

emery commented 2023年11月19日 19:16:34 +01:00

Backwards compatibility would be lost in any case because the top-level item is not a tagged array but a tagged byte-string containing our tagged array?

Backwards compatibility would be lost in any case because the top-level item is not a tagged array but a tagged byte-string containing our tagged array?

pukkamustard commented 2023年11月20日 08:47:14 +01:00

A { * Reference => Block } with one pair has the same encoded size as [ Reference, Block ], so why not use maps rather than pair arrays?

Very good point. Let's just use maps then.

> A `{ * Reference => Block }` with one pair has the same encoded size as `[ Reference, Block ]`, so why not use maps rather than pair arrays? Very good point. Let's just use maps then.

pukkamustard commented 2023年11月20日 09:10:39 +01:00

Backwards compatibility would be lost in any case because the top-level item is not a tagged array but a tagged byte-string containing our tagged array?

No, it's still a array (or tagged array).

CBOR Sequence basically replaces the array and should/can not be tagged. The CBOR Sequence is just a concatenation of blocks, references, read capabilities or CBOR Maps from references to blocks. It's a bit confusing because there is no CDDL to define a top-level CBOR Sequence.

It's also not clear how CBOR Sequences should be tagged. I've added a note with a suggestion:

tagged-eris-cbor-sequence = [ #6.1701996915(null), * eris-cbor-element ]

It's not pretty as now the tag can also be used on null. But it works and makes on-disk CBOR sequences detectable.

> Backwards compatibility would be lost in any case because the top-level item is not a tagged array but a tagged byte-string containing our tagged array? No, it's still a array (or tagged array). CBOR Sequence basically replaces the array and should/can not be tagged. The CBOR Sequence is just a concatenation of blocks, references, read capabilities or CBOR Maps from references to blocks. It's a bit confusing because there is no CDDL to define a top-level CBOR Sequence. It's also not clear how CBOR Sequences should be tagged. I've added a note with a suggestion: ```cddl tagged-eris-cbor-sequence = [ #6.1701996915(null), * eris-cbor-element ] ``` It's not pretty as now the tag can also be used on null. But it works and makes on-disk CBOR sequences detectable.