1
0
Fork
You've already forked autocollie
0
autonomous tuning for collimator, inspired by autoresearch
  • Go 99.8%
  • Makefile 0.2%
Thomas Stromberg 9dd2a052b8 update deps
2026年07月04日 09:08:29 -04:00
cmd/autocollie update deps 2026年07月04日 09:08:29 -04:00
internal tiny-model false-positive tuning 2026年06月06日 19:24:03 -04:00
.gitignore genesis 2026年05月07日 14:37:03 -04:00
.golangci.yml genesis 2026年05月07日 14:37:03 -04:00
go.mod update deps 2026年07月04日 09:08:29 -04:00
go.sum update deps 2026年07月04日 09:08:29 -04:00
knobs.json --try-knobs: support directed experimentation 2026年07月01日 06:39:12 -04:00
LICENSE Initial commit 2026年05月07日 19:55:50 +02:00
Makefile genesis 2026年05月07日 14:37:03 -04:00
README.md swap away from llm-agent to openai direct for less overhead 2026年05月14日 11:08:02 -04:00
skill.md simplify prompts 2026年05月15日 08:36:28 -04:00

autocollie

Autonomous experiment loop for collimator. Proposes JSON specs via a local OpenAI-compatible /v1/chat/completions endpoint, runs them as make experiment in ../collimator, and writes per-route summaries.

Make targets in ../collimator

target purpose
autocollie-dryrun ROUTES=... Call the LLM, validate specs, print, run nothing
autocollie-screen ROUTES=... EXPERIMENTS=N Run accepted specs via make experiment
autocollie-confirm KEY=... Re-run a key with SEED=43 to check signal reproducibility
autocollie-promote KEY=... Confirm + full-train at inflated profile + 3-gate comparison
autocollie ROUTES=... EXPERIMENTS=N PASSES=P The full ladder: screen + auto-promote per route, looped
autocollie-loop ROUTES=... Same as autocollie with PASSES=0 (loop until Ctrl-C)

The agent's instruction manual is skill.md; the LLM knob allowlist is knobs.json. Some allowed feature families may require later litmus runtime support before deployment.

By default autocollie calls http://127.0.0.1:8000/v1/chat/completions with model Qwen/Qwen3.6-27B. Override with AUTOCOLLIE_OPENAI_BASE_URL and AUTOCOLLIE_OPENAI_MODEL when the local endpoint uses different settings.

Routes

Pass ROUTES= (comma-separated) to either Make target. Trailing slash expands a prefix against routes seen in out/experiments/azoth/runs/*.json:

  • ROUTES=filetypes/javascript — one explicit route
  • ROUTES=filetypes/rust,filetypes/go,general
  • ROUTES=filetypes/ — every filetypes/* ever run (~50)
  • ROUTES=filegroups/,general — all filegroup specialists plus general