-
Notifications
You must be signed in to change notification settings - Fork 0
The problem is real
Date and BigInt are the friction people will actually hit. Today the log is
JSON, so a Date comes back as a string and a BigInt throws
ERR_ENTRY_NOT_SERIALIZABLE. That is documented, but "serialise deliberately"
only works when the payload shape is yours — it is no help when you are
forwarding a webhook body you did not author.
Why not a pluggable codec
The obvious answer is a codec seam so people can drop in superjson or CBOR. I do
not think that is a feature; I think it is a v2, because it breaks things the
package currently guarantees:
- Framing. Recovery depends on records being newline-terminated. CBOR has no
newline framing, sohealTail— which scans backwards for the last\n—
stops working. - The repair procedure.
docs/durability.mdtells you that every line is
independent JSON and gives you anode -eone-liner to find the bad ones. A
binary codec makes that page wrong. - Validation.
scanAccountingandwriteSurvivorsparse JSON lines to
check that sequence numbers increase and to decide what survives compaction.
That is the on-disk format, the recovery path and the operational story, not an
extension point.
The narrower version that fits
replacer and reviver options, passed through to JSON.stringify and
JSON.parse:
const wal = createWal({ replacer(key, value) { return this[key] instanceof Date ? { $date: value } : value; }, reviver(key, value) { return value?.$date ? new Date(value.$date) : value; }, });
Framing is untouched — still one JSON object per line — so healing, compaction,
sequence validation and the repair procedure all keep working unchanged. No
runtime dependency either: the caller brings the transform.
Three sharp edges, verified rather than assumed
1. The replacer never sees a Date. toJSON() runs first, so the replacer
receives the ISO string. Detecting a Date requires this[key], which means the
replacer cannot be an arrow function:
valor recibido: string "1970-01-01T00:00:00.000Z" | this[k]: Date real
2. A replacer does fix BigInt. Confirmed — without one JSON.stringify
throws TypeError, with one it produces {"n":{"$bigint":"1"}}.
3. The reviver sees the envelope, not just the value. Parsing a record calls
it with the keys ["seq","d","value",""]. A careless reviver can corrupt seq.
Options: apply it to the whole line and let the existing validation reject a
mangled seq, or double-parse to isolate the value, which costs a round trip on
every read.
Open questions
- Does a mismatched replacer/reviver pair across a restart need guarding? Today
the on-disk format is fixed; this would make it caller-dependent, with no
version marker in the file to catch a changed transform against old records. - Is edge 1 acceptable, or does the ergonomic answer need a different shape than
rawJSON.stringifysemantics? - Is this worth it at all, given
wal.append({ at: date.toISOString() })already
works for payloads you control?
Not scheduled. Opening it here so the reasoning exists before anyone asks, and
so the codec version gets rejected for a stated reason rather than silently.