- Shell 69.9%
- Python 17.9%
- Rust 12%
- Go 0.1%
fast-ctx
fzf-style Rust client for Microsoft's FastContext
repository explorer. You start the model; this drives its Read/Glob/Grep loop and
returns path:line-range citations. The directory to explore is the first argument.
Use
Download the weights once, serve the model, then point the client at code.
mise run install # cargo build --release
mise run pull # ~8 GB into models/
mise run serve # vLLM on :30000 (CUDA; mise run stop to shut it down)
mise run repl -- /path/to/code # interactive TUI
mise run ask -- /path/to/code -q "where is auth handled?" # headless
Without mise: cargo run --release -- <dir> (TUI) or add -q "..." for headless.
rg must be on PATH. Flags: --headless --json --citation --verbose --max-turns --base-url --model --api-key --traj (--help).
Serving
serve defaults suit a 24 GB Turing card (fp16, 32k context, Triton attention). Override via
FASTCONTEXT_{DTYPE,MAXLEN,GPU_UTIL}; a newer card may use bfloat16 and a longer context.
Apple silicon (MLX)
mise run pull # fetch HF weights
mise run convert-mlx # → models/FastContext-1.0-4B-SFT-mlx (FASTCONTEXT_MLX_QBITS=4 for 4-bit)
mise run serve-mlx # OpenAI endpoint on :30000
mise run repl -- /path/to/code --model "$FASTCONTEXT_MLX_MODEL"
MLX has no served-model-name, so pass the converted path as --model.
Variants
Microsoft ships only the 4B, in two flavours; the int4 build is a community quantisation.
pull/serve take a variant argument (both default to SFT); for vLLM the served API id stays
FASTCONTEXT_MODEL, so the client is unchanged whichever you load.
| Variant | Repo | Size | Notes |
|---|---|---|---|
| SFT | microsoft/FastContext-1.0-4B-SFT |
~8 GB | default |
| RL | microsoft/FastContext-1.0-4B-RL |
~8 GB | RL-tuned |
| int4 | INC4AI/FastContext-1.0-4B-SFT-AutoRound |
~2.7 GB | GPTQ int4 (CUDA); frees VRAM for longer context |
mise run pull -- microsoft/FastContext-1.0-4B-RL
mise run serve -- FastContext-1.0-4B-RL
Container
The client runs in a container; the model server stays on the host.
mise run container-build # podman (Linux) or docker/colima (Mac)
mise run container-run -- /path/to/code
FASTCTX_ARGS="-q 'where is auth?' --citation" mise run container-run -- /path/to/code
On Mac, reach a host server with FASTCTX_ARGS="--base-url http://host.docker.internal:30000/v1".
As a plugin
See SKILLS.md to wire fast-ctx into an agent (Claude/Codex/Hermes/Pi) as a
code-search tool returning citations.
Develop
mise run check (clippy pedantic + fmt + typos, then tests). Requires rg on PATH.