-
Notifications
You must be signed in to change notification settings - Fork 2
End-to-end latency benchmark: key press → transcription → paste, fully local #34
Open
Description
Existing benchmarks cover parts of the pipeline: make benchmark (encode + provider + network) and make bench-local (local inference only). Neither measures what the user actually feels: key release → text pasted, including hotkey handling, capture teardown, encode, inference, clipboard, and paste.
Add a repeatable end-to-end benchmark that:
- runs fully locally (local model, no network), so results are stable and machine-comparable
- measures press-to-paste wall time, ideally as a Go benchmark (
testing.B) sobenchstatcan compare runs; a manual/semi-manual harness is acceptable if real key events make a pure Go benchmark impractical (e.g. feed a canned WAV through the post-release path) - can be re-run from time to time and appended as a labelled baseline (same spirit as
make bench-save) to see what we lose or gain across changes
The per-segment timings already logged to diagnostics (press_to_record_ms, paste_copy_ms, ...) are a good starting point — the benchmark mostly needs to drive the pipeline and aggregate them.
🤖 Generated with Claude Code
Activity
Metadata
Metadata
Assignees
Labels
No labels