The 1414 C I/O Streams Library

A library providing generic I/O interface over etherogenous data streams.

Purpose

dfstream reads and writes data in many popular binary formats (zlib, bzip, LZMA, Zstandard, and possibly others).

The main abstraction unit in dfstream is the stream. Each stream deals with a specific format, data source, or data sink. At the very least, a stream provides the basic operations defined by the struct df_stream interface:

• int (*read)(void *streamp, void *dest, size_t n, size_t *nactual)
• int (*write)(void *streamp, const void *data, size_t n, size_t *nactual)

No more, no less.

struct df_stream is explicitly minimal, since it only offers the bare minimum required to perform I/O on a stream. Code that only needs this much, may read or write data on any stream. When generic code is not necessary, a specific stream type may expose more specific operations, for example a compressor may include functions to customize the compression strategy, the memory usage, and more.

Why?

Because every library wants to push its own flavor of how you should read and write data.

Unix solved this problem decades ago. One should be able to read() and write() over anything that can be seen as a file. dfstream does exactly this. It gives the opportunity to define a stream type, while also providing uniform generic primitives for basic read and write operations. This means that any code that only needs those basic operations can work with any kind of stream.

Design principles

• Performance - the library should impose little to no overhead compared to raw I/O over the specific data source, e.g. it should be just as fast as using raw zlib or raw bzip.

• Static linking - only static linking is supported, this allows to discard any unnecessary functionality, e.g. if zlib compression is never used, any zlib compression functionality is stripped from the final executable. You don't pay for what you don't use.

• Simplicity - it shouldn't do anything more than dealing with popular binary data formats.

• Modularity - split every stream into its own library.

• GNU C only - the library uses some GNU C specific extensions to improve safety.

Build

dfstream requires a GNU C compatible compiler, such as GCC, clang, Intel C Compiler. You can build dfstream using make, running the following command:

$ make

Or, equivalently:

$ make all

You can customize the compilation flags using the CFLAGS variable, for example:

$ make CFLAGS="-Os"

Other familiar variables, such as CC, AR, ARFLAGS are also supported.

You can optionally select to only build a single module by using the appropriate make target:

$ make libdfbzip.a

The following is a summary of the supported stream types with the corresponding target:

To remove any file generated by a previous build, run:

$ make clean

You can use dfstream in your project by linking to the generated libraries, plus any specific library necessary for the specific data format.

License

MIT, see LICENSE file for details.