Cognitum Learn — build, chat, and voice-query your knowledge from anywhere in your home

You pick a subject. Cognitum watches the videos and reads the documents for you, then answers your questions — by text on your Mac or out loud through Siri, Alexa, or Google. Everything stays on devices you own.

(You don't need the Cognitum Seed hardware to try it — your Mac can do everything by itself. The Seed just lets the answers follow you around your home.)

How Cognitum Learn fits together at a glance

 ┌─────────────────────┐
 │ YOU — your topic │
 │ "sous vide cooking" │
 └──────────┬──────────┘
 │
 ┌─────────────────────┼─────────────────────┐
 │ │ │
 ▼ ▼ ▼
 BUILD CHAT VOICE
(your Mac) (Mac CLI + UI) (anywhere at home)
learn study learn ask "Hey Siri,
learn ingest learn chat Cognitum..."
learn import learn ui "Alexa, ask
 Cognitum..."
 │ │ │
 └─────────────────────┼─────────────────────┘
 ▼
 ┌──────────────────────┐
 │ COGNITUM ONE SEED │
 │ ~/Docs/KB/*.rvf │ ← also on Mac
 │ /var/lib/cognitum/ │ ← on Seed
 │ │
 │ One file per topic. │
 │ You own both copies. │
 └──────────────────────┘

This is not just a knowledge-base tool. It is three things that work together:

• Build — your Mac downloads videos, transcribes audio, and pushes a searchable expert to your Seed.
• Chat — talk to your knowledge base from your computer, via the command line or a local web dashboard.
• Voice — ask Siri, Alexa, or Google a question and hear the answer spoken back. Anywhere in your home.

No cloud account. No subscription. Your knowledge lives on your Seed, on your network.

Before you start — you need a free Anthropic API key. learn ask and learn chat turn your knowledge base into a written answer using Claude, so they need a key. Get one in two minutes at https://console.anthropic.com/ (sign up → API Keys → Create Key), then add it to your shell profile so it's always set:

echo 'export ANTHROPIC_API_KEY=sk-ant-...' >> ~/.zshrc # or ~/.bashrc
source ~/.zshrc

Building and quizzing a KB (learn study, learn ingest, learn quiz) works without a key — you only need it the moment you ask for a written answer.

# 1. Install on your Mac (prebuilt binary, no developer tools required)
T=$(mktemp -d) && curl -L https://github.com/stuinfla/cognitum-learn/releases/latest/download/learn-aarch64-apple-darwin.tar.gz | tar xz -C "$T" && "$T/learn-aarch64-apple-darwin/install.sh"
brew install yt-dlp ffmpeg
# 2. Bind your Seed (one time — SKIP if you don't have a Seed yet)
# No Seed? Cognitum Learn still builds, chats, and quizzes locally on your Mac.
# You only need a Seed for voice from around the house and cross-device sharing.
# `cognitum-XXXX.local` is the mDNS name printed on the bottom of your Seed; replace XXXX with your unit's four-char ID.
learn config set seed.address cognitum-XXXX.local
learn config set seed.auto_push true
learn doctor
# 3. Build a knowledge base
learn study "sous vide cooking for beginners"
# 4. Chat with it from your Mac
learn chat sous-vide
# OR open the web dashboard
learn ui
# 5. Set up voice — follow docs/voice-setup-manual.md (Apple/Alexa/Google CLI, ~30 min one-time)
# Apple Siri works today. Alexa is in flight in v0.5.8. A one-command browser wizard ships in v0.6.0.

The five-step quickstart visualised

 1. INSTALL 2. BIND SEED 3. BUILD KB 4. CHAT 5. VOICE
 ─────────── ──────────── ────────── ────── ──────
 curl ... | tar xz learn config set learn study learn chat learn voice
 install.sh seed.address ... "sous vide" sous-vide setup
 brew install ... learn doctor ─or─ ↓
 learn ui Hey Siri,
 Cognitum...
 one-time Mac one-time per per-topic everyday use once per
 install Seed pairing ingest ecosystem

After step 5, "Hey Siri, Cognitum what temperature for medium-rare steak" speaks the answer back to you from any iPhone, HomePod, or CarPlay device. (For Echo, the wording is "Alexa, ask Cognitum"; for Google, ask via a Cognitum Routine.)

Your knowledge base is one file: ~/Docs/KB/sous-vide.rvf. The Seed gets an identical copy at /var/lib/cognitum/rvf-store/. You own both.

Don't have Apple Silicon? Linux x86_64, Linux aarch64, and Windows x86_64 binaries are on the latest release page.

The first time you run learn study "sous vide cooking for beginners", the terminal walks you through every stage so you can see the KB taking shape:

$ learn study "sous vide cooking for beginners"
[1/8] discovering candidate videos ......................... 12 found
[2/8] you picked 4; downloading captions + audio ........... 4/4
[3/8] smart frame decision (pHash variance) ................ 2 visual, 2 talking-head
[4/8] transcribing (Whisper.cpp, on-device) ................ 18m23s of audio
[5/8] chunking (~300 tokens, 50-token overlap) ............. 412 chunks
[6/8] embedding with BGE-small-en-v1.5 (384-dim, on-device) 412 vectors
[7/8] indexing into HNSW + Ed25519 witness chain ........... ok
[8/8] auto-summary (3-5 takeaways via Claude) .............. ok
wrote ~/Docs/KB/sous-vide.rvf (4.7 MB)
auto-push → cognitum-9842.local ... ok (verified by checksum)
✓ Done. Try one of these next:
 learn ask sous-vide "what temperature for medium-rare steak?"
 learn chat sous-vide
 learn ui

If a step shows red, learn doctor will tell you exactly which dependency is missing (yt-dlp, ffmpeg, model cache, Seed reachability) and how to fix it.

Mac builds. Seed hosts. Voice anywhere.

Three modes — Mac builds the KB, Seed hosts it, voice surfaces it across your home

 MAC (the workshop) COGNITUM SEED (the house) YOUR HOME (the voice)
 ───────────────────── ─────────────────────────── ────────────────────────
 learn ingest <url> ──► /var/lib/cognitum/rvf-store/ ◄── "Hey Siri, Cognitum..."
 learn study "topic" single file per topic "Alexa, ask Cognitum..."
 learn chat <topic> answers questions locally "Hey Google, run Cognitum check"
 learn ui (dashboard) with cited timestamps

The Mac does the heavy work — once per topic. It finds videos, downloads them, runs them through Whisper or VTT captions, breaks the transcript into bite-sized passages, turns each into a 384-number search fingerprint using a tiny open-source model (BGE-small) that runs on your Mac, and writes everything into a single .rvf file. If seed.auto_push: true, the file is pushed to your Seed automatically when the ingest completes.

learn study "Japanese knife sharpening" # autonomous discovery + ingest
learn ingest "https://youtu.be/QZMljuD10sU" # one video
learn import ~/Downloads/lectures/ # local files (PDF, MP3, MP4, TXT, MD)

Once a KB is built, you can talk to it from your computer two ways: CLI or browser.

Mac dashboard — chat with your KB

learn ask knife-sharpening "what angle for a 210mm gyuto?"
learn apply knife-sharpening "give me a 20-minute sharpening routine for 3 knives"
learn chat knife-sharpening # multi-turn dialog, session-persistent
learn ui # opens http://127.0.0.1:7878 in your browser

Leaving a chat session: type /quit (or /exit), or press Ctrl-D. /help lists the in-chat slash commands (/cite, /save, /quit).

learn ui is the friendliest entry — a self-contained React dashboard served by the built-in Axum bridge. Pick a topic, watch ingest progress live, chat with the KB, no terminal required.

This is the new piece in v0.5.7+. After running the voice setup, you can ask your KB questions out loud and hear the cited answer spoken back. Three ecosystems, same KB:

The voice surface is the same KB you built on your Mac. No second copy, no re-indexing.

Cognitum Learn isn't trying to be smarter than the cloud assistants on general knowledge. It's trying to be the right assistant for your knowledge — the things you watched, the documents your grandmother typed up, the lectures from your specific course — and to make that assistant reachable from the kitchen, the car, and the bedside table without uploading any of it.

Whatever you trained the KB on. A few real examples from Stuart's home:

At the CLI:

learn ask sous-vide "what temperature for a medium-rare steak?"
# → 54°C for 1–4 hours [Sous Vide Everything @ 3:12]
learn ask family-recipes "how long does Nonna's tomato sauce simmer?"
# → 3 hours covered, then 1 hour uncovered [grandma-cookbook.pdf p.14]
learn ask type-2-diabetes "is intermittent fasting safe with metformin?"
# → Generally yes if monitored; consult MD. [Dr. Berg @ 12:04, Mayo Clinic @ 4:33]

In the dashboard chat (no terminal):

"Walk me through a croissant lamination schedule for tomorrow morning. My kitchen is 68°F."

The dashboard hands the question to the KB, picks the best 10 chunks, and gives you a grounded answer with every step linked to the exact video moment that taught it.

Out loud, anywhere in your home:

"Hey Siri, Cognitum what knife angle for a gyuto."

"Alexa, ask Cognitum about beurrage temperature."

"Hey Google, is the room safe?" (Google Routine — uses presence sensors + KB context.)

Manual setup (v0.5.7 / v0.5.8 — current): voice access today is a CLI procedure. The full step-by-step is in docs/voice-setup-manual.md — Apple Shortcut install, Alexa Custom Skill publish, Google Routines, plus a voice-proxy LaunchAgent on your Mac. About 30 minutes if you've done it once, longer first time.

Wizard (v0.6.0 — designed, implementation in Phase 2.0): a single command will replace the 30+ manual steps with an interactive browser wizard.

learn voice setup # opens http://127.0.0.1:7878/voice-setup
learn voice setup --ecosystem apple # one ecosystem at a time

The wizard does the work and shows you what's happening — embedded OAuth callbacks (no paste-this-code-back-into-the-terminal), live progress over SSE, dynamic QR codes for iPhone hand-off, and pre-recorded GIFs for the Alexa app and Google Home app screens. Architecture is in ADR-CL-004 (design doc lives in the private cognitum-home-integration repo); the seven UI assets already exist in assets/voice-setup/. Implementation lands across cognitum-learn v0.6.0a → v0.6.0d.

How each ecosystem talks to your Seed — Apple, Alexa, Google compared side-by-side

 APPLE ALEXA GOOGLE (Nest)
 ───── ───── ─────────────
Free-form Q&A ✓ Siri Shortcut ✓ Custom Skill ✗ ecosystem-limited
Setup time ≤4 min ≤5 min ≤6 min (Routines only)
Cloud cost 0ドル AWS free tier 0ドル
Privacy footprint tunnel only tunnel + Lambda tunnel + HA notify
Hardware reach iPhone/HomePod/ Echo Dot/Show/ Nest Mini/Hub +
 CarPlay/Watch Auto HA-fanned speakers
GA status (today) v0.5.7 ✓ v0.5.8 in flight pre-defined Routines

Apple voice flow

Apple (≤4 min). A Shortcut on your iPhone calls your Mac's voice-proxy over HTTPS via a cloudflared tunnel; the proxy calls learn ask and returns the cited answer for Siri to speak. Works on iPhone, HomePod, CarPlay, Apple Watch. Free, LAN-friendly, no developer account needed. GA in v0.5.7.

Alexa voice flow

Alexa (≤5 min). A private Custom Skill on your Amazon developer account proxies "Alexa, ask Cognitum about X" to a Lambda function, which calls your Mac's voice-proxy via the same tunnel. Free under AWS free tier (1M requests/month). In flight tonight — the Haiku-fast-path Lambda handler is being isolated in a separate codepath so the slow-path full-Sonnet retrieval doesn't time out Alexa's 8-second response window.

Google voice flow

Google (≤6 min, optional). Three pre-defined Routines in the Google Home app: "Hey Google, run Cognitum check", "is the room safe", "good morning Cognitum". Each fires a script in Home Assistant that broadcasts a TTS answer back through notify.google_assistant_sdk. Sensor fanout (presence, room state) works first-class; arbitrary-slot Q&A on Nest hardware is not possible — Google sunset Conversational Actions on June 13, 2023 with no replacement. Use Siri or Alexa for free-form questions.

Voice setup progress states

• Your knowledge lives on your hardware. The .rvf file is on your Mac at ~/Docs/KB/<topic>.rvf and on your Seed at /var/lib/cognitum/rvf-store/. Copy it, back it up, delete it — you control it.
• Your audio never leaves your machine. Whisper and BGE run on-device. The only outbound network call is learn ask's text completion to Anthropic, and you can replace that with on-device RuVLLM by setting LEARN_SYNTH_LOCAL=1.
• The voice path stays on your LAN. Siri, Alexa, and Google all hit your Mac's voice-proxy through a cloudflared tunnel — your KB is never uploaded to Apple, Amazon, or Google. The tunnel only carries one HTTPS round-trip per spoken question.
• No telemetry. Cognitum Learn does not phone home. Ever.

Cognitum Learn is a three-tier system. Each tier has one job.

Cognitum Seed workflow — Mac is the workshop, Seed is the house, voice is the everyday surface

 YOUR MAC (the workshop) YOUR SEED (the house) YOUR HOME (the voice)
 ───────────────────────── ───────────────────────── ───────────────────────────
 Find videos, download them, Hold the KB. Answer questions Siri / Alexa / Google sends
 transcribe audio, chunk text, with the same retrieval + the spoken question to your
 embed with BGE-small (384-dim), synthesis pipeline. Native Mac's voice-proxy via a
 build .rvf, push to Seed. RVF; no conversion needed. cloudflared tunnel. The proxy
 calls `learn ask`, returns text,
 the assistant speaks it.
 Hardware: M-series Mac Hardware: Pi Zero 2W Hardware: iPhone / Echo / Nest
 Software: learn CLI + Whisper Software: cognitum-agent + Software: voice-proxy on Mac +
 Storage: ~/Docs/KB/*.rvf rvf-store + 100+ tool MCP proxy ecosystem skill / shortcut

Why this split: the heavy one-time work (ingest) needs CPU and the internet. The everyday work (answer questions) needs to be on the Seed where the knowledge actually lives. The voice surface needs to be wherever the user is — kitchen, car, bedroom — and so rides through the cloud-borne assistants that already have microphones in those places. Each tier does what its hardware is good at.

The three-ecosystem architecture is documented in ADR-CL-003 (private cognitum-home-integration repo). Sensor fanout (the 21 RuView entities — presence, temperature, etc.) rides the same path through Home Assistant; voice Q&A is the simpler lane that bypasses HA entirely.

URL or local file in, searchable expert out. Eight stages, all on your Mac, all on-device except the final auto-summary.

Ingest pipeline — eight stages from URL to a tamper-evident .rvf knowledge file

Source URL or path
 ↓
 ACQUIRE — yt-dlp pulls captions first; falls back to audio
 ↓
 SMART FRAME DECISION — pHash variance decides whether to extract
 visual frames (demos) or skip (talking heads)
 ↓
 TRANSCRIBE — VTT captions where available, else Whisper.cpp on-device
 ↓
 CHUNK — sentence-aware, ~300 tokens, 50-token overlap
 ↓
 EMBED — BGE-small-en-v1.5 (384 dimensions, ONNX, on-device)
 ↓
 INDEX — append-only HNSW segments inside the .rvf file,
 with an Ed25519 witness chain per chunk for tamper evidence
 ↓
 AUTO-SUMMARY — Claude generates 3-5 key takeaways
 ↓
 ~/Docs/KB/<topic>.rvf ←auto-push→ /var/lib/cognitum/rvf-store/<topic>.rvf

The same retrieval-and-synthesis pipeline answers a CLI question, a dashboard chat turn, and a Siri request — only the input shape and output shape change.

Query path — six stages from a question to a cited answer or spoken response

Question (typed or spoken)
 ↓
 EXPAND — HyDE generates a hypothetical answer as a second query vector
 ↓
 HYBRID RETRIEVE — dense (BGE) + BM25, fused with reciprocal rank fusion → top 50
 ↓
 RERANK — cross-encoder picks the top 10
 ↓
 MMR + SOURCE-CAP — diversity λ=0.7, no more than 3 chunks from any single video
 ↓
 SYNTHESIZE — cited prompt; abstain if signal is weak; AIMDS scan in and out
 ↓
 Cited answer with [Title @ MM:SS](url&t=Xs) links (or spoken summary for voice)

One file per topic. You can copy, back up, or delete a topic by touching one file. Per-topic isolation is total.

Storage layout — Mac mirrors Seed; one .rvf per topic plus per-topic side-cars

~/Docs/KB/ ← on your Mac
├── sous-vide.rvf ← chunks, embeddings, HNSW, witness chain
├── french-cooking.rvf
├── sous-vide.summary.md ← auto-generated takeaways
├── _graph/sous-vide.graphdb ← claims, entities, relations
├── _meta/sous-vide.json ← per-video state
└── _chat/sous-vide/ ← session JSONL files
/var/lib/cognitum/rvf-store/ ← on your Seed (after push)
└── <topic>.rvf
~/.cognitum-learn/ ← Mac-side state
├── voice-setup-state.json ← (v0.6.0) wizard resume state
└── voice-proxy.token ← (v0.5.7+) bearer for voice surface

Architecture — 17 crates split across CLI, ingestion, retrieval, synthesis, chat, serve, voice

 ┌─────────────────────────────────────────────────────────────────┐
 │ learn (CLI binary) │
 │ learn-cli │
 │ 25 subcommands, voice setup entry (v0.6.0) │
 └────────┬──────────┬──────────┬──────────┬──────────┬─────────┘
 │ │ │ │ │
 ▼ ▼ ▼ ▼ ▼
 INGESTION RETRIEVE SYNTH CHAT SERVE/VOICE
 acquire retrieve synth chat serve
 asr (BM25+ (cited (REPL, (Axum + MCP
 frames dense, answers, JSONL + v0.6.0
 chunk rerank, AIMDS sessions) wizard)
 embed MMR) in/out) voice-proxy
 index (scaffolding)
 graph
 └──────────┴──────────┴──────────┴──────────┘
 │
 ▼
 learn-core
 (shared types, errors,
 topic slug logic)

Every query teaches the Seed which chunks were useful and which weren't. Over time, the SONA per-topic adapter quietly retrains and your KB gets better at answering your kind of question without you ever leaving the loop.

Learning flywheel — query → judge → distill → adapt → better answer next time

 ┌──────────────────────┐
 │ 1. You ask a │
 │ question │
 └──────────┬───────────┘
 ▼
 ┌──────────────────────┐
 ┌────────┤ 2. Seed retrieves + │
 │ │ cites an answer │
 │ └──────────┬───────────┘
 │ ▼
 │ ┌──────────────────────┐
 │ │ 3. You read it, │
 │ │ click a citation, │
 │ │ or ask follow-up │
 │ └──────────┬───────────┘
 │ ▼
 │ ┌──────────────────────┐
 │ │ 4. SONA adapter │
 │ │ learns which │
 │ │ chunks won │
 │ └──────────┬───────────┘
 │ ▼
 │ ┌──────────────────────┐
 └───────►│ 5. Next answer is │
 │ a little sharper │
 └──────────────────────┘

"My Seed isn't reachable" / learn doctor red on seed.reachable. Confirm the address with ping cognitum-XXXX.local or use the raw IP printed on the Seed. Some routers drop .local names; fall back to learn config set seed.address 192.168.x.x. Mac and Seed must be on the same LAN. The auth token, if your Seed requires one, lives at ~/.config/cognitum/seed.token and is read automatically.

"Dimension mismatch" when pushing to the Seed. Cognitum Learn embeds at 384 dims (BGE-small). A fresh Seed often shows dimension: 8 because its sensor pipeline wrote first. The Seed is not hardware-limited to any particular dimension — fix it by editing the agent's --dimension flag and wiping the old store. Full procedure in docs/voice-setup-manual.md under "Dimension fix." Never chase this as a firmware bug.

"I built a KB with an older version and now learn ask returns nothing." Versions before v0.2.17 used a 1024-dim embedder; new 384-dim query vectors won't match. Re-ingest: learn ingest <url> --topic <topic>. learn compact does not re-embed.

iPhone Shortcut returns nothing (most common). The voice-proxy is bound to 127.0.0.1 instead of 0.0.0.0, so the cloudflared tunnel can't reach it. Set COG_VOICE_BIND=0.0.0.0 in ~/Library/LaunchAgents/com.cognitum.voice-proxy.plist, then launchctl unload && launchctl load it. Verify with lsof -nP -i :7879 — you want *:7879 (LISTEN), not 127.0.0.1:7879.

Apple Shortcut won't trigger at all. Name the Shortcut Cognitum (NOT Ask Cognitum); avoid Siri-reserved verbs like Ask, Tell, Call, Text — Siri routes those to Contacts/Messages instead of your Shortcut. The v0.6.0 wizard detects both automatically and offers a re-install QR.

Apple Shortcut won't trigger. The Shortcut needs your voice-proxy URL embedded at install time. If you renamed the proxy or changed the cloudflared tunnel, re-install the Shortcut. The v0.6.0 wizard's troubleshooting screen (above) detects this automatically and offers a re-install QR.

Alexa skill returns "Cognitum is having trouble." Almost always a Lambda cold-start or timeout. The v0.5.8 release adds the Haiku-fast-path that returns within Alexa's 8-second window for short queries; long queries still fall back to a "let me think about that for a moment" response. Watch the Lambda logs (aws logs tail /aws/lambda/ask-cognitum --follow) for the actual error.

Google Routine fires but speaker stays silent. The TTS broadcast hop (notify.google_assistant_sdk) needs your homegraph service account credentials at ~/.homeassistant/google_service_account.json. If the file is missing, only the scene fires — there's no spoken response. Re-run the Google branch of the wizard, or drop the JSON manually at ~/.homeassistant/google_service_account.json (generate from Google Cloud Console → Home Graph API service account).

cloudflared tunnel URL changes every reboot. Quick tunnels (free) get a new *.trycloudflare.com on each invocation. Amazon won't accept that — its OAuth redirect URI is fixed per skill. Fix: either keep the tunnel up (the v0.6.0 wizard supervises it via launchd), or migrate to a Cloudflare account with a named tunnel (free, requires domain ownership).

Whisper or BGE didn't download. Models auto-fetch into ~/.cache/learn-rs/models/. If that failed, learn doctor shows which are missing; delete the cache and re-run to force a refresh.

This README reflects what is shipped tonight and what is queued. Cognitum Learn is moving fast and we'd rather tell you what's real than make promises.

learn status <topic> # chunk count, file size, coherence KPI
learn list <topic> # videos in the topic
learn who-said <topic> "Julia Child"
learn timeline <topic> "beurrage"
learn compare <topic-a> <topic-b>
learn cloud <topic> # SVG word cloud
learn map # 2D galaxy of all your topics
learn summarize <topic> # key takeaways

learn push <topic> # push KB to your Cognitum One Seed
learn serve <topic> # MCP server for Claude Code
learn ui # local web dashboard
# learn voice setup is the v0.6.0 wizard — not in this release; see docs/voice-setup-manual.md
learn watch <topic> # monitor a YouTube channel
learn eval <topic> # regression Q&A
learn forget <topic> <video-id>
learn compact <topic> # dedupe + optimize the index
learn doctor # health check

learn setup # guided first-run wizard
learn config set seed.address 192.168.x.x
learn config set seed.auto_push true
learn config get seed.address # or seed.auto_push, seed.token — bare `seed` is not a valid key

cargo fmt --check
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo build --release --workspace

Licensed under the PolyForm Noncommercial License 1.0.0. Free for personal, research, and other noncommercial use. For a commercial license, contact the author.

This license is intentional. Cognitum Learn is built for the people who own a Cognitum Seed and want it to be smart about something that matters to them. It is not built for someone to wrap and resell as a hosted service. If your use is personal, educational, or research, you're in the right place.

Contributions welcome. Open an issue before sending a PR larger than ~50 lines so we can align on approach. CI gate must be green.

Built with RuVector · PolyForm Noncommercial 1.0.0 · Releases