Meta-harness optimization loop wired onto Islo sandboxes. POC: 0/5→5/5 in four proposer steps. Built on islo.dev.
-
Updated
May 5, 2026 - HTML
#
agent-eval
Here are 19 public repositories matching this topic...
Language: All
Filter by language
Sort: Most stars
Sort options
AI-operated company. Building agent-friend: universal tool adapter for AI agents. @tool → OpenAI, Claude, Gemini, MCP. Live 24/7 on Twitch.
python twitch structured-logging interactive-cli exponential-backoff human-in-the-loop zero-dependencies open-startup ai-agent autonomous-ai building-in-public llm-tools agent-security mcp-security personal-ai-agent agent-eval agent-friend
-
Updated
Mar 26, 2026 - Python
Project page for Meta-harness on Islo (POC). https://zozo123.github.io/meta-harness-on-islo-page/
-
Updated
May 5, 2026 - JavaScript
Live, open-source benchmark for comparing AI coding agents on real GitHub issues
benchmark machine-learning developer-tools awesome-list dev-tools live-data ai-research ai-benchmarks ai-engineering ai-tools auto-updated ai-evaluation llm-testing agent-evaluation swe-bench llm-benchmarks agent-eval ai-coding-agent-benchmark codex-vs-opencode coding-agent-benchmark
-
Updated
Jun 15, 2026 - Python
A reasoning benchmark runner for comparing LLMs as OpenClaw agents use them. 52 prompts, 3 eval sets, 11 traps, LLM-as-judge, tier-based leaderboard.
-
Updated
Apr 11, 2026 - Python
Scenario Testing for AI Agents
-
Updated
Jun 4, 2026 - Python
Transcript-first evaluation tool for comparing coding-agent sessions across Codex, Claude Code, and Pi.
-
Updated
May 29, 2026 - TypeScript
Framework-agnostic evaluation harness for Go — test your MCP servers and AI agents with scored, CI-ready checks.
-
Updated
Jun 15, 2026 - Go
Vendor-neutral research umbrella for measuring AI plugin, agent, and MCP server quality across CLI runtimes (Claude Code, Gemini CLI, Copilot CLI, Codex CLI).
mcp skill-discovery opentelemetry ai-evaluation gemini-cli claude-code plugin-testing cross-cli agent-eval invocation-rate
-
Updated
Jun 15, 2026 - Python
🚀 基于Java的开源AI自动化评测框架 / An open source AI automation evaluation framework based on Java
-
Updated
Jun 11, 2026 - Java
A curated list of benchmarks, harnesses, leaderboards, and tools for evaluating AI coding agents.
benchmark leaderboard evaluation awesome-list codex ai-agent llm aider claude-code coding-agent swe-bench agent-eval ai-coding-agent-benchmark coding-agent-benchmark
-
Updated
Jun 8, 2026
Local-first synthetic finance anomaly trainer for agent evals.
-
Updated
Jun 13, 2026 - JavaScript
Trace-first evaluation harness for deciding whether AI agents deserve more tokens, permissions, and trust
-
Updated
May 16, 2026 - Python
agent-convergence-scorer is a CLI and Python library that scores how lexically similar N agent or LLM outputs are: exact-match rate, Jaccard token overlap, divergence point, and a composite convergence score over any list of runs. An eval primitive for measuring reproducibility and fan-out collapse. Lexical, not semantic. Zero deps.
cli benchmark consistency evaluation similarity multi-agent convergence reproducibility agents jaccard divergence llm llm-evaluation ai-reliability eval-harness agent-eval
-
Updated
Jun 7, 2026 - Python
开源通用 AI Agent 真实任务评测 · 同 Prompt、客观开奖、评分细则全公开 | Open-source evaluation of general-purpose AI Agents on real-world tasks with verifiable outcomes — by PingWest / 硅星人
-
Updated
Jun 13, 2026
Regression testing for AI agents — snapshot tool-calls, diff in CI, fail on regressions. A GitHub Action. By Viprasol Tech.
testing typescript ci snapshot-testing regression-testing ai-agents github-action llm llmops agent-eval
-
Updated
Jun 7, 2026 - TypeScript
Agent and LLM evaluation harness — golden datasets, multi-scorer execution, regression detection across model versions, cost-quality leaderboards, and CI gates for model promotion.
express typescript platform-engineering regression-detection ml-ops ai-platform ai-governance llm-eval agent-eval ci-gate
-
Updated
Jun 11, 2026 - TypeScript
Lightweight CI-native regression and behavior-aware evaluation toolkit for black-box agent workflows.
-
Updated
May 9, 2026 - Python
Binary-criteria evaluation harness for Claude skills with planned extension to plugins, agents, and MCP servers. Score every change yes/no across 7 layers — package integrity, trigger quality, functional quality, regression protection, baseline value, model variance, rollout safety. Never gradients.
mcp regression-testing skill-evaluation ai-evaluation llm-eval claude-code plugin-testing eval-harness agent-eval binary-criteria
-
Updated
Jun 15, 2026 - TypeScript
Improve this page
Add a description, image, and links to the agent-eval topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the agent-eval topic, visit your repo's landing page and select "manage topics."