Event Sourcing

Event sourcing system that provides:

• uniform meta data
• pull-based event streaming (lite_events/event_log/consume_* functions)
• push-based event streaming (lite_events/subscribe_* functions)
• storing in SqLite database per event/object type (first meaning of 'lite')
• you determine meta and event decoding (second meaning of 'lite')
• No projections, not even a default one (third meaning of 'lite')

See bottom of this page for rationale.

We have made two example apps in dev/direct_counter.gleam and dev/supervised_counter.gleam of this repository. You can run them with gleam run -m direct_counter resp. gleam run -m supervised_counter. The latter explicitly crashes the counter, where a normal crash would be inadvertant. The key is that our lite events processes can be supervised according to BEAM principles.

Features

• Run supervised, either as a worker, or with its own private supervisor
• chunked retrieval on consume* (and therefore on subscribe*), preventing DB overload, enabling streaming for applications

TODO

• Update docs with subscribe* changes
• s/thing that maintains (itself) where a subscriber is at. This can also make an error log. A hard crash is a different thing.
• Decide where timestamp of command is created
• refactor
• experiment with monitors (first see what the spans on death do)
• for nice deployments - and for compilation speed - switch to postgres?
• make it easy to get hold of the "previous" aggregate. state changes are a powerful source of information. No - that's a job of the "aggregate", that is, the apply function! Or a wrapper of some sorts.

Lite - no en/decoding for you

First, metadata is entirely up to you. You only need a user ID? fine. You need organisations, roles and more? Also fine. All you need to provide is a codec.

For a coherent set of event logs, you can pass the same meta codec to each event log.

Second, encoding and decoding the events is up to you.

Gleam has no meta-programming, so you have to do encoding and decoding of both meta data and event types. A JSON codec is as good as any other.

Realize that when your events change, your decoder needs to be able to handle this, so for important domain data, you may want to incorporate a version field in your event encoder from day one.

Lite - no projections

Event Sourcing provides a natural Command Query Separation: the commands result in an event log, which can be queried.

However, most practical systems require more advanced querying. Lite events does not even provide a canonical aggregate, but event_log does contain a helper function get which takes an apply function that will be folded over all events.

It gets more interesting (and harder) when you want to query a collection, or a combination of two objects of a different type, one or more or all from an event log.It is up to you, to process event logs and construct (and possibly persist) those projections. Thus, we do not restrict possibilities. To help you, we provide both push and pull mechanisms so you can do the projections at your own leisure.

Lite - no logging

We provide no logging. We encourage use of telemetry, e.g. opengleametry. If necessary, you can use this in the checker function that you provide.

When you do log, you probably want to log incoming requests, as such. These should result in commands, which you may log (or the reason why no command resulted). You may want to log the results (events or reason why no event resulted) of your checkers.

We will probably provide telemetry for framework action such as (re)start of actors.

lite - no Replay of commands or errors

It may be useful to keep track of errors in a way that you can replay them after fixing your application. That means persisting the failed command and the accompanying meta data. This is not logging. It is also not something that lite events provides - we persist events, not commands. You have to build this yourself. This makes sense, since it depends very much on your domain whether such a service provides value.

Store the commands in a DB, which you can query in the ways that make sense for "recovery".

Lite - no assistance for external effects

Say you want to send e-mails to external parties. We do not provide a framework for doing this. You are in the lead to collect the e-mails (when consuming events, those could map directly to e-mails that need to be sent, you know which you handled - but you have to persist the sequence ID yourself!).

Since sending e-mails can fail, and some e-mails might be more important than others, you may need more than just remembering which events you handled. Then you can persist e-mails that need to be sent and try to send those with highest priority, first. Retry on failure. Report errors when a recipient does not exist.

Again, we are not helping, and we are not in your way. You can therefore tailor the outgoing effects.

Lite - maybe concurrent event logs

It is possible we will introduce per-id processes in the future. This would result in a more complex library, but it would leverage BEAM concurrency for when your independent evebt logs would otherwise get in each others way.

Do not expect this until gleam OTP gains a dynamic supervisor, i.e. a supervisor that can start and stop children as desired.