ZBuf

Stupidly easy (de)serialization for both JSON text and binary.

Documentation

https://7games.codeberg.page/ZBuf/@docs

API

• DeltaType() - Get the type used for delta compressing updates to a struct.
• applyDelta() - Apply delta information to a struct to update it to a newer version.
• deserializeBin() - Deserialize binary data into a struct using a reader.
• deserializeBinNoAlloc() - Deserialize binary data into a struct using a reader, but the resulting struct will have zero allocations to free.
• deserializeBinBuffer() - Deserialize binary data into a struct from a byte buffer.
• deserializeBinBufferNoAlloc() - Deserialize binary data into a struct from a byte buffer, but the resulting struct will have zero allocations to free.
• deserializeJson() - Deserialize text JSON data into a struct using a reader.
• deserializeJsonNoAlloc() - Deserialize text JSON data into a struct using a reader, but the resulting struct will have zero allocations to free.
• deserializeJsonBuffer() - Deserialize text JSON data into a struct from a byte buffer.
• deserializeJsonBufferNoAlloc() - Deserialize text JSON data into a struct from a byte buffer, but the resulting struct will have zero allocations to free.
• freeDeserialized() - Free dynamically allocated data from a deserialization.
• getDelta() - Get the difference between 2 structs into a result that can be serialized.
• serializeBin() - Serialize a struct into binary data using a writer.
• serializeBinBuffer() - Serialize a struct into binary data returned into a byte buffer using an allocator.
• serializeJson() - Serialize a struct into text JSON data using a writer.
• serializeJsonBuffer() - Serialize a struct into text JSON data returned into a byte buffer using an allocator.

Features

• Easy (de)serialization of almost any struct out of the box.
• Deserialization options to try and parse data even with errors (invalid type ignoring, safe defaults, use last or first options for duplicates, etc).
• Great for recovering as much data as possible from frequently updating formats (just use the lenient mode with safe defaults).
• Serialize struct deltas to only send changed data over the network.
• Comptime magic.

Install

• Run zig fetch --save=zbuf git+https://codeberg.org/7Games/ZBuf#v2.0.1
• Place in your build.zig:

my_module.addImport("zbuf",b.dependency("zbuf",.{.target=target,.optimize=optimize},).module("zbuf"));

Build Options

• Run zig build test to run the unit tests.
• Run zig build to build all binaries.
• Run zig build run_{example_name} to run an example. Example names include:
  • benchmark

FAQ

• Why use this over zig's JSON or bufzilla or some other serialization method?
  • ZBuf wants to provide a simple to use interface that can handle changes in the format. For example, if you add new fields to a format or change the type of a field the old file should still work with the lenient deserialization option in ZBuf as long as you set safe defaults for the new or changed fields. This makes ZBuf suitable for environments with both rapid prototyping and for when more strict format adherence must exist with the strict deserialization option.
• Why does JSON deserialization take 2 allocators?
  • Since JSON is text, one allocator is used to parse the data into values where the other is used to store allocations used in the resulting structure. Since the lifetime of values is temporary and likely outlives the lifetime of the data of the returned struct, the option to use different allocators is given.
• What is the leaky JSON allocator option?
  • If the allocator used for the temporary JSON parsing data is a structure that you can freely leak like an arena, memory optimizations in parsing can be made by not having to create an arena during parsing.
• What are the pros/cons for the binary and JSON forms?
  • Both should have the same level of soundness against structure updates, but JSON is great in situations where you may want to have the user edit the data outside the program. JSON on average is faster to serialize, but slower to deserialize. Binary is slower to serialize, but faster to deserialize. Binary is the best choice for runtime speed, smaller file sizes, and supports small delta updates.
• Why does every field get serialized for JSON delta updates?
  • Due to the limitations of JSON serialization, delta updates with JSON do not save much space. Use binary instead.

Credits

• Gota7 - ZBuf design and bufzilla modifications.
• theseyan - Bufzilla used for binary serialization and deserialization.