Deterministic runtime for agent evaluation
-
Updated
Mar 25, 2026 - Python
Deterministic runtime for agent evaluation
KayaDB distributed key-value storage engine
Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench
Deterministic net.Conn/net.Listener fault injection for Go — the testing/synctest philosophy applied to the network layer.
Local-first C# project for deterministic prompt versioning, A/B evaluation, and evidence-based promotion using structured scoring.
Reliability testing for MCP servers: deterministic faults, bounded recovery, and executable regressions.
A TypeScript harness for deterministic frontend testing and development with repeatable scenarios, local fixtures, and browser verification for coding agents.
Deterministic UI testing framework for TS/JS — canvas-first. Virtual time, scene contracts, model-driven discovery.
Record normalized DeepSeek Harness tool results and replay them offline with strict matching and zero tool-body execution.
Safe Rust metaprogramming and adversarial testing: typed code generation with receipts, proc macros, generated trials, mutation, replay, and reproducible evidence.
WordleBench — Deterministic AI Wordle benchmark. Compare 34+ LLMs (GPT-5, Claude 4.5, Gemini, Grok, Llama) head-to-head on accuracy, speed, and cost across 50 standardized words.
Reproduce OpenAI Agents SDK runtime bugs without API keys — real Runner, scripted models, verifiable certificates.
LogOS A closed-loop cognitive Operating surface: written in Rust, deployed via self-verifying 'narrow-waist' Nix OS + Mirage OS Uniquernel to Google Cloud Run/Kubernetes,
Local-first persistent Chromium automation harness for AI agents, built in Rust on raw CDP with semantic snapshots, trusted input, deterministic waits, checkpoints, and background jobs.
Document-driven E2E test harness with AI-assisted compilation and deterministic Hurl execution
Deterministic Rust testing utility for simulation and stochastic workflows
Bounded deterministic schedule exploration for cooperative asyncio code
A Raft consensus library built from scratch in Rust, with a deterministic network simulator for testing leader election, log replication, and partition recovery.
A black-box evaluator that checks agent behavior and evidence independently, then produces reproducible reports.
To associate your repository with the deterministic-testing topic, visit your repo's landing page and select "manage topics."