Summary

MemPalace's backend abstraction (RFC 001 BaseBackend / BaseCollection) makes
it possible to swap the vector store. This proposes a second, opt-in backend
built on turbovec (a CPU, 4-bit
TurboQuant ANN) paired with a durable SQLite sidecar, packaged as a standalone
library — turbovecdb — with MemPalace as its first customer. ChromaDB stays
the default; this is selected with MEMPALACE_BACKEND=turbovec.

Motivation

The interesting properties for a local-first, "verbatim, 100% recall" system:

• No HNSW corruption class. A meaningful chunk of the Chroma backend exists to
  detect and quarantine corrupted/stale HNSW segments. A design where the durable
  state is plain SQLite rows and the ANN index is a rebuildable cache sidesteps
  that failure mode entirely.
• "100% recall" is a store property, not an index property. With SQLite as the
  source of truth (exact float32 vectors + verbatim text), a lossy quantized index
  can only reorder neighbours, never change what text comes back — and it can be
  rebuilt from the store after any crash.
• Cheap to build, small, CPU-resident. Useful on modest/local hardware.

What it is

turbovecdb is a small embedded vector DB:

• SQLite is the source of truth — documents, metadata, the string-id↔uint64
  map, and the exact float32 vectors. The turbovec .tvim index is a derived
  cache; if missing or stale it rebuilds from SQLite on open (crash-safe).
• Exact-cosine re-rank — turbovec returns an approximate candidate pool;
  turbovecdb recomputes true cosine from the stored vectors and returns
  distance ∈ [0, 2]. So callers (and the hybrid ranker in searcher.py) get a
  correct cosine distance, and end-to-end quality matches an exact index.
• Full metadata filters — $eq/$ne/$in/$nin/$gt/$gte/$lt/$lte, $and/$or,
  and where_document $contains; unsupported operators raise rather than silently
  match everything.
• Multi-process safe — a cross-process write lock plus lock-free reader cache
  coherence (a store_gen counter), so the hooks miner / MCP server / CLI can
  share a palace.

In MemPalace, backends/turbovec.py is a thin adapter over the library; no engine
code lives in MemPalace itself.

Evidence (measured, local, honest)

Local parity vs ChromaDB through the MemPalace adapter, 15,805 real text
drawers, ONNX MiniLM (384-d), identical vectors fed to both backends:

≈ ×ばつ faster build, ×ばつ faster query (p50/p95), ×ばつ smaller on disk, with
retrieval quality a wash (hybrid hit@k tied within noise). A standalone,
reproducible synthetic benchmark (docs/performance/benchmark.py in turbovecdb)
shows the same build win and recall@10 = 1.0 (via re-rank) vs ChromaDB's
default-ef 0.67.

Caveats, stated plainly:

• The size result is workload-dependent. On a pure-vector workload (no
  documents) turbovecdb is actually slightly larger than Chroma, because it
  stores exact float32 vectors; the ×ばつ win above comes from the document-heavy
  regime where Chroma's HNSW graph overhead dominates. Measure on your data.
• Recall numbers are default-vs-default; Chroma's recall is tunable via
  hnsw:search_ef at a latency cost.
• Benchmarks are one machine; ratios shift with N, dim, and hardware.

Compatibility / risk

• Default unchanged. palace.py resolves the backend from MEMPALACE_BACKEND
  (default chroma); a normal run is byte-for-byte identical.
• Tests: full suite 2044 passed, 14 skipped, 1 pre-existing unrelated
  failure (present on base). turbovecdb has its own 39-test suite (contract,
  persistence, filters, embedder, real multi-process concurrency).
• Conforms to the BaseBackend/BaseCollection contract; results are the typed
  QueryResult/GetResult.

Open questions for the list

1. Interest level — is a second, opt-in backend something the project wants to
   carry, or should this live purely as a third-party plugin discovered via the
   mempalace.backends entry point?
2. Dependency packaging — the [turbovec] extra currently points at
   turbovecdb (not yet on PyPI). Preference between (a) publishing turbovecdb to
   PyPI, or (b) a git-URL dependency for now?
3. Contract gaps worth closing before anyone relies on it: batch query, an
   embedder-identity guard (refuse a model/dim change on reopen), and validation
   at 100K+ drawers (MemPalace's stress regime).
4. Conformance — would a parametrized cross-backend conformance suite (run
   against both chroma and turbovec) be welcome? It'd make adding future
   backends safer.

References

• PR: Add turbovec backend (powered by turbovecdb), opt-in via MEMPALACE_BACKEND kostadis/mempalace#18
• Library: https://github.com/kostadis/turbovecdb
  (docs: docs/core/architecture.md, docs/performance/)
• Underlying index: https://github.com/RyanCodrai/turbovec