4
1
Fork
You've already forked eer
1

eris-fs: Simplify encoding to an index #21

Merged
pukkamustard merged 6 commits from eris-fs-2 into main 2023年12月15日 08:27:57 +01:00

This updates the ERIS-FS format by reducing it to an index that carries references to file contents which are themselves encoded using ERIS.

Imho this simplifies the format considerably and allows much simpler parallelization. The requirement for a random-access decoder has been lifted. Encoding/decoding of individual files can be parallelized. Implementation in general is simpler as there is no need to count block offsets or do any padding.

Support for compression has been dropped. It can be added later with additional entry types. Removing compression has the advantage that this format can be implemented much easier on constrained platforms (e.g. browsers). It also makes creation of ERIS-FS file system much easier in GUI applications.

This change was motivated by considerable performance issues when working with real-world use-cases (Guix substitutes).

This is a breaking change, the version number in the index has been incremented.

This updates the ERIS-FS format by reducing it to an index that carries references to file contents which are themselves encoded using ERIS. Imho this simplifies the format considerably and allows much simpler parallelization. The requirement for a random-access decoder has been lifted. Encoding/decoding of individual files can be parallelized. Implementation in general is simpler as there is no need to count block offsets or do any padding. Support for compression has been dropped. It can be added later with additional entry types. Removing compression has the advantage that this format can be implemented much easier on constrained platforms (e.g. browsers). It also makes creation of ERIS-FS file system much easier in GUI applications. This change was motivated by considerable performance issues when working with real-world use-cases (Guix substitutes). This is a breaking change, the version number in the index has been incremented.
Author
Owner
Copy link

direct link to the md: d5624d9167/eer/eris-fs/index.md

direct link to the md: https://codeberg.org/eris/eer/src/commit/d5624d9167f86ab126dc5d07419eb35cec0a8fa2/eer/eris-fs/index.md
Owner
Copy link

Looks good, I'll implement it when I get the chance.

Looks good, I'll implement it when I get the chance.
Owner
Copy link

Let's specific a MIME type while we are here.

The ERIS-FS MIME type is application/x-eris-fs+cbor

Let's specific a MIME type while we are here. > The ERIS-FS MIME type is application/x-eris-fs+cbor
Author
Owner
Copy link

Let's specific a MIME type while we are here.

The ERIS-FS MIME type is application/x-eris-fs+cbor

Added.

> Let's specific a MIME type while we are here. > > > The ERIS-FS MIME type is application/x-eris-fs+cbor Added.
Owner
Copy link

I have an implementation (without symlink support) for eris-go on an eris-fs-1 branch.

There is a mkfs and dumpfs command, and a FUSE interface at /fs/urn:eris:....

I have an implementation (without symlink support) for eris-go on an [eris-fs-1](https://codeberg.org/eris/eris-go/src/branch/eris-fs-1) branch. There is a `mkfs` and `dumpfs` command, and a FUSE interface at `/fs/urn:eris:...`.
Author
Owner
Copy link

I have an implementation (without symlink support) for eris-go on an eris-fs-1 branch.

Nice. I've added to the list of implementations and will merge.

> I have an implementation (without symlink support) for eris-go on an [eris-fs-1](https://codeberg.org/eris/eris-go/src/branch/eris-fs-1) branch. Nice. I've added to the list of implementations and will merge.
Sign in to join this conversation.
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
eris/eer!21
Reference in a new issue
eris/eer
No description provided.
Delete branch "eris-fs-2"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?