a dimmed sha-256 string with one segment locked in signal green — threshold locked before run.
I maintain PRML — an open standard that locks the success criteria of an AI evaluation to a SHA-256 hash before the run, so a claim about a model can be checked by a third party afterwards. Reference implementation and public registry at falsify.dev .
It proves the bar was locked. Never the result.
The claim below is a real PRML v0.1 manifest — manifest.yaml in this repository. It commits this profile to something checkable: at least 4 reference implementations, byte-identical across the spec's 13 conformance vectors.
version: prml/0.1 claim_id: 0198f2c4-7a1e-7000-8000-4d5e6f7a8b9c created_at: "2026年08月20日T21:30:00Z" metric: byte_equivalent_reference_implementations comparator: ">=" threshold: 4 dataset: id: spec.falsify.dev/v0.1/test-vectors hash: cd7d22ee5bda1b5834c74f955fe3452b49e5357d5a7c9933805d0d0d99aa2a8e seed: 0 producer: id: github.com/sk8ordie84
Don't take my word for any of it:
# 1 — the file is what this page says it is (no install needed) curl -s https://raw.githubusercontent.com/sk8ordie84/sk8ordie84/main/manifest.yaml | shasum -a 256 # → 960b0483f7e503e87f4d14cf1421d7385e67954712cf445d931acd7a622fa37e # 2 — the PRML canonical hash, from the reference implementation pip install falsify curl -sO https://raw.githubusercontent.com/sk8ordie84/sk8ordie84/main/manifest.yaml falsify hash manifest.yaml # → 4f1ebcc8ddc249ff656d7d040367fcef64ced224e2a90941a68b18d8fd9261e7 # 3 — the claim itself: 4 byte-equivalent implementations clear the locked bar falsify verify manifest.yaml \ --expected-hash 4f1ebcc8ddc249ff656d7d040367fcef64ced224e2a90941a68b18d8fd9261e7 \ --observed 4 # → PASS (edit any field first and it prints TAMPERED instead — try it)
The dataset.hash binds the 13 v0.1 conformance vectors (of 21 in the spec) the claim is measured against, and CI re-runs the byte-equivalence check on every push.
| Claim | Check it |
|---|---|
| Openly specified, CC BY 4.0, archived | DOI 10.5281/zenodo.20177839 |
| 4 reference implementations — Python, JS, Go, Rust — byte-equivalent | conformance CI · falsify-js |
| JSON Schema in the SchemaStore catalogue | catalog.json → "PRML manifest" |
| Listed in the UK AI Security Institute's Inspect extensions catalogue | inspect.aisi.org.uk/extensions — "Inspect falsify", Analysis, 1 of 30 |
| OpenSSF Best Practices badge: passing | bestpractices.dev/projects/13539 |
| Public registry: Ed25519-signed receipts, RFC 3161 countersigned, mirrored to Rekor | registry.falsify.dev · /pubkey |
| Works inside real eval harnesses — 12 tools | examples: lm-eval-harness, promptfoo, Langfuse, Opik, Braintrust, DeepEval, lighteval, Mastra, Laminar, hud, Moonshot, RewardHackingAgents · Inspect ships as falsify-inspect |
Falsify OÜ · Tallinn · hello@falsify.dev — the discipline is optional; the hash is not.