Skip to content

Navigation Menu

Sign in
Sign up

End-to-end latency benchmark: key press → transcription → paste, fully local #34

Open

Description

Existing benchmarks cover parts of the pipeline: make benchmark (encode + provider + network) and make bench-local (local inference only). Neither measures what the user actually feels: key release → text pasted, including hotkey handling, capture teardown, encode, inference, clipboard, and paste.

Add a repeatable end-to-end benchmark that:

  • runs fully locally (local model, no network), so results are stable and machine-comparable
  • measures press-to-paste wall time, ideally as a Go benchmark (testing.B) so benchstat can compare runs; a manual/semi-manual harness is acceptable if real key events make a pure Go benchmark impractical (e.g. feed a canned WAV through the post-release path)
  • can be re-run from time to time and appended as a labelled baseline (same spirit as make bench-save) to see what we lose or gain across changes

The per-segment timings already logged to diagnostics (press_to_record_ms, paste_copy_ms, ...) are a good starting point — the benchmark mostly needs to drive the pipeline and aggregate them.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      AltStyle によって変換されたページ (->オリジナル) /