provgraf

A "bank of verified facts" for AI agents — a W3C PROV knowledge graph on plain Postgres, with automatic staleness propagation.

⚠️ This is a proof-of-concept / engineering demo, not a supported product. It was designed, evaluated and deployed for one real client engagement (anonymised here), and is published to document the idea and its implementation. Expect rough edges; issues are welcome, a roadmap is not promised.

If you let an AI agent write client-facing documents, the model is not the risk — the facts are. A language model will happily "correct" a true, contract-grade number (a rent rate, a buyout rule, a share-capital figure) toward whatever the internet believes. In our real deployment, an overnight agent once tried to fix six verified facts to match plausible-but-wrong public sources.

Not every certain fact arrives as a PDF. Plenty of what you are told is true and consequential — a director confirms on a call how a scheme works, and you act on it. People also change their minds. So a source here is either a document with a file, or a testimony: a dated, attributed record that on this day this person vouched for this. That is what lets you stop relitigating a settled question, and it is why the integrity check differs by shape rather than demanding a file for everything.

There is a second, more mundane version of the same problem. An agent left to its own devices dumps everything into Markdown files as it goes — notes, half-verified numbers, copies of copies — with no validation at the door. If you're disciplined about your files, maybe you can live with that. If you're messy (I am), the workspace silently rots: three files disagree about the same number and none of them says where it came from. That was the actual trigger for building this.

provgraf is the counter-measure: a small database of atomic facts, each with provenance (which official document it came from, who put it there, when), plus one mechanism you don't get from a notes file:

Change a source → the system tells you everything downstream that just went stale.

It cuts both ways: the validating write path keeps the human's workspace from rotting, and the read-only agent surface keeps the agent honest — it can quote the bank, but it can't quietly "improve" it.

• Facts are nodes (entity), sources are nodes, people/software are agents, decisions are activitys — W3C PROV core plus documented extensions (provenance_class, entity_status; PROV sanctions extension via prov:type subtyping).
• Derived facts (sums, report figures) link to their inputs via wasDerivedFrom and store an inputs_hash of the input values.
• When a source fact is revised, a recursive SQL CTE walks the derivation graph and flags every transitively dependent fact as stale (measured: a 2-level cascade in ~2.5 ms).
• Versioning is a single trick: valid_to + a partial unique index (WHERE valid_to IS NULL). That gives as-of queries over the bank's own history for free.
• The bank is bitemporal: valid_from/valid_to is transaction time (what the bank believed when), world_valid_from/world_valid_to is world time (when the fact holds per its source). Only both axes together answer "per what we knew on 15 June, what was in force in May" — the question a backdated correction creates. The four-timestamp model is borrowed from Graphiti.
• Contradictory sources coexist as disputed alternates; a human resolves them with a recorded decision (the rejected alternative stays in the graph as a trail).
• No triplestore, no graph database. Postgres gives constraints, transactions, pg_dump, and a recursive CTE is all the graph traversal this problem needs.

graph LR
 D1["📄 datasheet<br/>(source doc)"] --> F1["riverside.units = 152"]
 D1 --> F2["hillside.units = 84"]
 D1 --> F3["lakeside.units = 58"]
 F1 --> T["units_total = 294<br/>inputs_hash ✓"]
 F2 --> T
 F3 --> T
 T --> R["report.units = 294"]
 D2["📄 NEW datasheet<br/>units 152 → 154"] -.revise.-> F1
 F1 -.cascade.-> T
 T -.cascade.-> R
 style D2 fill:#fdd,stroke:#c33
 style T stroke:#c33,stroke-dasharray: 5 5
 style R stroke:#c33,stroke-dasharray: 5 5

The build was gated, not vibes-driven:

1. PRD first, reviewed by a panel of independent AI reviewer agents; their objections (e.g. "at ~30 facts Postgres barely beats a JSON file") were recorded as open tensions, not deleted.
2. A 1-day spike on flat files to prove the staleness-cascade design before writing any DDL.
3. A go/no-go milestone on real data: cascade correctness, version windows, invariants that actually reject bad writes, hash parity Python↔SQL.
4. Standards check: the PROV-JSON export deserializes cleanly with the reference W3C library, enforced by a test. Scope stated honestly: that proves the serialization is well-formed PROV-JSON, not PROV-CONSTRAINTS conformance (typing, causality loops, provenance travelling backwards in time), which needs a validator we have not run.
5. In production the bank grew to ~150 facts from 22 source documents and was used to fill investor-facing and grant documents, with a validator that blocks any hard number lacking a bank qname tag.

Requirements: Docker, uv.

docker compose -f infra/postgres/docker-compose.yml up -d --wait
uv run provgraf init
uv run pytest # 27 tests: hash parity, invariants, cascade, as-of, conflicts
bash examples/demo_cascade.sh # seed a fictional company → check → revise a source → watch the cascade

(Run the tests before the demo — they read global check state, so a seeded database makes two of them fail. examples/reset.sh wipes and re-seeds; it connects through DATABASE_URL and refuses to touch a database holding anything other than the demo owner.)

The demo seeds a fictional social-housing company ("Acme Community Housing"), builds a 2-level derivation chain, plants a source conflict and an overdue fact, then revises one source number and shows check flagging the transitively dependent facts. It closes on the bitemporal query: a rent recorded today but in force since 1 June answers for June and correctly finds nothing in force in May.

Optional extras:

uv sync --group rag # local embeddings + reranker (configure models in .env)
uv run provgraf embed acme-housing && uv run provgraf search "how much is the rent"
uv run --group dashboard streamlit run dashboard/app.py
uv run --group mcp provgraf-mcp # read-only MCP server for AI agents

The default embedding/reranker models in .env.example are Polish (sdadas/mmlw-retrieval-roberta-large, sdadas/polish-reranker-large-ranknet) because the original deployment was Polish-language; swap them for any sentence-transformers-compatible pair.

• Boring storage was the right call — and historically the norm. PROV is a data model, not a technology choice, and most deployed provenance recorders (Karma, Komadu, the IVOA provenance store) were relational too. Postgres gives constraints, transactions and pg_dump; a recursive CTE is all the graph traversal this problem needs. Presenting "PROV without a graph database" as a discovery would be a tell that you hadn't read the field.
• Hash parity enforced by tests. The staleness hash exists in Python and in a SQL function; a unit test feeds both the same fixtures. Divergence = the whole staleness feature silently lies.
• Read/write asymmetry for agents. Agents get a read-only MCP surface; writes go through a validating CLI with a human in the loop. Prompts can't enforce this — architecture can.
• Auto-glosses for retrieval. One dictionary entry per field type generates the embedded description for every fact of that type. New field → one JSON entry, not N hand-written descriptions.
• Decisions are nodes. Resolving a conflict creates an activity with an agent and a basis; precedents searches past decisions before you resolve a new dilemma.

Before publishing this I ran a prior-art survey across ~20 systems. Every ingredient here exists somewhere; the assembly is what does not. Being specific about that is more useful than a novelty claim:

• Graphiti / Zep — the closest neighbour and the one that beat us on an axis: genuine bitemporality with four timestamps per edge (which is why provgraf now has it). It has per-fact provenance back to the ingested episode. What it does not have is a staleness cascade — and its conflict handling is the mirror image of this project's: a small LLM decides at write time which contradicting fact loses and silently expires it, and its MCP surface hands the agent add_memory, delete_entity_edge and clear_graph.
• Dagster — hashes code and input data versions to mark downstream assets stale. That is inputs_hash + cascade, shipping and battle-tested, at the granularity of assets and tables. provgraf's difference is the unit (one number, with a citation) and the human in the loop, not the mechanism.
• Truth maintenance systems (Doyle 1979, de Kleer 1986) and content-addressed build systems (Nix, Bazel) are the real ancestors of justification-plus-invalidation. Nothing here is new under the sun; it is applied to facts instead of beliefs or build artifacts.
• TrustGraph — real prov: vocabulary in shipped code and a provenance CLI. Instructive twice over: it wrote this project's staleness feature down as a motivating use case and never built it, and it deliberately retreated from per-triple provenance to per-chunk containment because reification got expensive.
• WhyHow.AI — shipped per-triple → chunk → page-offset provenance back in 2024. The repositories have been untouched since, and the domain no longer resolves. Building this is demonstrably possible and demonstrably not sufficient on its own.
• Knowledge-base tools (Notion, Guru, Slab, Slite, Document360, GitBook) — verification exists, but the trigger is a calendar interval or an LLM judging that a document drifted from a connected source, and the unit is a page or a card. None of them holds a typed number, and none computes transitive dependents of a changed input. The industry is converging on probabilistic drift detection with accept/dismiss; the cascade here is deterministic and replayable. (Guru's "an edit by a non-owner unverifies the card" is where this project's agent gating comes from.)
• Agent memory (Mem0, Letta, LlamaIndex, LangMem, Google Memory Bank) — contradiction is resolved by overwriting or deleting, decided by a model. Mem0's prompt says it outright: if the retrieved facts contradict the memory, delete it. Coexisting disputed alternates plus a recorded human decision with a basis appears nowhere in that category.
• Not database provenance in the Green–Karvounarakis–Tannen sense (ProvSQL, GProM). That field annotates query results with semiring lineage; this is retrospective/workflow provenance. Different camp, easy to confuse.

Uncontested, as far as the survey found: fact-level staleness propagation, and conflicts that survive as alternates until a human records a decision. Everything else in the list above is prior art we are standing on.

Honest list (an adversarial code review ran before publishing; the notable leftovers):

• Clean PROV-JSON deserialization is tested; PROV-CONSTRAINTS conformance is not checked.
• No incremental recomputation: the cascade tells you what went stale, it does not recompute derived values for you.
• Single-user by design: no auth, no concurrency story beyond Postgres transactions.
• No rm-doc: documents are never deleted, only reported as dangling (with the orphaned facts they would take down).

• Query-time provenance — recording which facts fed a particular answer or document (TrustGraph does this). Likely belongs in the application consuming the bank rather than in the engine.
• LLM-resolved contradictions and auto-repairing memory. Both are well-trodden elsewhere and both defeat the point of a bank whose contents a human vouched for.

If you are an AI agent operating on this bank, read AGENTS.md first: what you may read, what you may not write, and the two ways the machinery will stop you on purpose.

MIT