Senti iris logo

What Am I Looking At?
A local-first visual assistant for macOS — it sees the scene, names the objects, and answers your questions. On your Mac. Nothing uploaded.

MIT License Python 3.11+ macOS Apple Silicon Local-first privacy pytest

Quick start · How it works · Stack · Docs · License

Senti — local-first visual assistant on macOS

Most visual assistants ship camera frames to a cloud API. Senti does not.

It is built for Apple Silicon so the fast path — object detection and tracking — stays at interactive rates, while the slow path — a local vision-language model via Ollama — only runs when the scene actually changes or you ask a question.

Two loops keep the UI live. The camera never waits on the language model.

Camera → detect → understand → speak

flowchart LR
 Cam["📷 Camera"] --> Fast
 subgraph Fast["⚡ Fast loop — 15–30 FPS"]
 YOLO["🎯 YOLO26 + tracking"]
 Scene["🧭 Scene change + stability"]
 YOLO --> Scene
 end
 Fast --> Slow
 subgraph Slow["🌙 Slow loop — on change or question"]
 Frame["🖼️ Best-frame selection"]
 OCR["🔤 Optional OCR"]
 VLM["🧠 Local VLM"]
 Frame --> OCR --> VLM
 end
 Slow --> UI["🖥️ Desktop UI + TTS"]
 Mic["🎙️ Push-to-talk"] --> UI
 UI --> Ask["💬 Ask / Focus / Analyze"]
 Ask --> Slow

1. ⚡ Fast loop — frames go to YOLO26, then tracking and scene-change detection. Target: 15–30 FPS.
2. 🌙 Slow loop — when the scene reaches READY, Senti picks the sharpest, most stable frame from a rolling buffer and sends it to the local VLM (and OCR, if enabled). Follow-ups reuse scene memory when they can.

State machine: WATCHING → SCENE_CHANGED → WAITING_FOR_STABILITY → READY

Full package map and thread model: Architecture .

Every runtime dependency, with a badge that opens its official site. Click through — these are the projects Senti stands on.

Python macOS PySide6 Qt NumPy OpenCV Ultralytics YOLO26 PyTorch MLX yolo-mlx Ollama EasyOCR faster-whisper sounddevice python-dotenv pytest

Pinned versions live in requirements.txt.

• 🍎 macOS on Apple Silicon (developed on M2 Pro)
• 🐍 Python 3.11 or newer
• 📷 Built-in or Continuity Camera
• 🔐 Camera permission (and microphone if voice input is on)
• 🦙 Ollama with a vision model for scene descriptions

git clone https://github.com/TeslaNeuro/Senti.git
cd Senti
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env

Pull a local vision model, then launch:

ollama pull gemma4
./scripts/run.sh

Or:

source .venv/bin/activate
python -m app

On first launch, macOS will ask for camera access. Grant it to Terminal (or your IDE) if you start Senti from the command line. Ultralytics downloads yolo26n.pt into models/ automatically (~6 MB) unless that file is already there.

Optional native Metal path: install yolo-mlx with pip install "yolo-mlx[tracking,convert]", then set YOLO_RUNTIME=mlx.

💡 If the preview is black, another app (FaceTime, Zoom, Chrome, ...) likely holds the camera. Quit it and relaunch.

Status bar shows camera, YOLO device, VLM activity, FPS, and inference latency. Hover a pill for the full detail.

Step-by-step walkthrough: Usage .

Copy .env.example to .env. Important defaults:

Optional features (OCR, TTS, voice) are off until you turn them on. Full reference: Configuration .

OCR_ENABLED=true
TTS_ENABLED=true
TTS_RUNTIME=auto
VOICE_ENABLED=true
VOICE_MODEL=base

source .venv/bin/activate
pytest tests/ -q

Senti is local-first:

• 🚫 Camera frames are never written to disk
• 🧠 The frame buffer is bounded and in-memory only
• 💻 YOLO, OCR, Whisper, and the VLM run on-device (Ollama on localhost)
• 📡 No telemetry, no cloud uploads, no account

See SECURITY.md for the threat model and reporting.

Senti/
├── app/ Application package
│ ├── camera/ Qt / AVFoundation capture + in-memory buffer
│ ├── detection/ YOLO26 worker
│ ├── tracking/ ByteTrack / BoT-SORT monitor
│ ├── perception/ Scene change, stability, best-frame selection
│ ├── vision/ Local VLM, scheduler, object crops
│ ├── scene/ Conversational scene memory
│ ├── ocr/ EasyOCR worker
│ ├── speech/ Qt TTS + macOS say
│ ├── voice/ Push-to-talk + faster-whisper
│ └── ui/ Native desktop window
├── docs/ Architecture, usage, configuration
│ └── assets/ README graphics (logo, banner, pipeline)
├── models/ YOLO26 weights (gitignored *.pt / *.npz)
├── scripts/run.sh Create venv, install, launch
├── tests/ Unit tests
└── resources/Info.plist Camera and microphone usage strings

Senti is released under the MIT License.

Copyright (c) 2026 Arshia Keshvari.

Senti
_{Built on-device. Nothing leaves your Mac.}