FaceID — self-hosted face recognition for Frigate + Home Assistant

FaceID is a small, self-hosted service that adds reliable, trainable face recognition on top of Frigate. It uses the same model family as Immich and CompreFace (InsightFace buffalo_l: SCRFD detection + ArcFace embeddings) and was built because Frigate's built-in face recognition UX didn't cut it:

• No train-tab treadmill. Matching is nearest-neighbor over face embeddings — every image you assign is a visible reference point, with no training cycles and no queue that refills with already-known faces. To be clear: this is not immune to bad data — an imbalanced or mislabeled gallery still degrades matching (a person with many reference images wins borderline matches more often). The difference is that the failure mode is an image you can see and delete, not an opaque model update.
• Strangers are first-class. Unknown faces are collected, auto-clustered (DBSCAN, the same trick photo apps use) and reviewed in a web UI: one click assigns a whole cluster to a person — or ignores it. People you enroll (the mailman you want notifications for) get recognized; people you ignore go permanently silent.
• An ignore list that actually sticks. Ignored faces stay as negative anchors: never notified, never matched to your family, never resurfacing in review — and FaceID learns their new looks over time (only on unambiguous matches with a clear margin over every enrolled person, visibly marked "auto", deletable anytime). Anchors are grouped per person; groups can be merged, curated, or released into a real person with one click if you change your mind.
• Train from anywhere, without bloat. Assign faces from your cameras, upload photos from your photo library, or enroll whole folders via CLI. A per-person photo cap keeps galleries lean: when it's exceeded, the reference most similar to the rest is set aside (so unusual angles are kept, not lost) — visibly, on the person card, where you can restore it. The cap is adjustable in the Settings tab.
• Home Assistant native. MQTT discovery sensors per camera (presence-window state like Alice, Bob → nobody), plus a per-recognition event topic for automations — and a notification blueprint that puts the recognised name into the Frigate notification you already get, by replacing it once the name is known — independent of which Frigate version puts the name where, and carrying the score and zones with it. (Why a Frigate-side blueprint often shows no name.)
• Tags flow back to Frigate. Recognized names are written as sub_label, so you can filter clips by person in Frigate's Explore view — including retroactively: the history scan tags past events, and assigning a face in the review UI tags its original event too.
• Yours to keep. A Settings tab holds the matching thresholds (live-editable) plus one-click backup & restore of your gallery, and an optional built-in daily auto-backup — your hand-curated face data is the one irreplaceable thing, so it's easy to safeguard.

What it is — Screenshots · How it works · Local-only, and what gets downloaded

Setting it up — Requirements · Install as a Home Assistant app · Install standalone · Connecting to Frigate · Getting started

Living with it — Ignoring people · Sharper reference photos · How training stays healthy · Backup & restore

When it does not recognise someone — When the snapshot has no face · Events MQTT never announces · Calibrating the threshold · Seeing what it is doing · Measuring instead of guessing

Home Assistant — Home Assistant · Getting the name into your Frigate notification

Reference — Updates · Security & privacy notes · Configuration reference

All faces below are AI-generated (StyleGAN) — no real persons.

Unknown review — new faces arrive auto-clustered; assign a whole cluster with one click, or scan your camera history to bootstrap the gallery:

Unknown review with auto-clustered faces

Persons — your gallery; upload photos, rename a person by clicking their name, or send faces back to review:

Person gallery

Frigate --MQTT frigate/events--> FaceID
 ^ | snapshot.jpg?crop=1 (person crop)
 | v
 +--API sub_label---------- InsightFace buffalo_l -> cosine match vs. gallery
 |
 match >= 0.50 -> publish person + tag Frigate event
 below -> review queue (clustered in the web UI)
MQTT -> Home Assistant: sensor.faceid_<camera> + faceid/event (JSON)

FaceID performs all recognition locally on your hardware — no cloud APIs, no accounts, no telemetry. The only thing ever fetched from the internet is the open-source recognition model itself, once, on first start:

• What: InsightFace buffalo_l model pack (SCRFD face detection + ArcFace recognition, the same open models Immich and CompreFace use)
• From where: the official InsightFace GitHub release
• Size: ~300 MB, cached on disk afterwards (survives restarts and app updates)

After that download, FaceID works completely offline. Your camera images and face data never leave your machine.

• Frigate 0.16+ (snapshot + sub_label APIs), reachable over HTTP
• An MQTT broker (the one Frigate already uses is fine)
• A CPU with AVX (any Intel/AMD from the last decade; no GPU needed). Running HAOS/your host in a VM? The default virtual CPU model (e.g. Proxmox kvm64) hides AVX — set the VM CPU type to host and cold-restart the VM, or the recognition runtime will refuse to start.

Measured on an idle-to-normal household setup (4 cameras, ~60 events a week):

Most of that RAM is the recognition model, and it is not tunable. Loading buffalo_l accounts for 572 MB of it — the libraries themselves (Python, numpy, opencv, onnxruntime) come to only 64 MB. Within that, the ArcFace model w600k_r50 alone costs 333 MB: it is a ResNet-50 whose ~25M parameters are unpacked into memory as float32. FaceID already skips the three model files it does not need (3D landmarks, 106-point landmarks, gender/age), which would add another ~140 MB.

Things that sound like they should help but were measured and do not: thread count (OMP_NUM_THREADS 1/2/4), det_size, and disabling onnxruntime's CPU memory arena. All three left the figure unchanged. Anyone reporting ~1 GB is seeing normal behaviour, and tools built on the same models (Immich, CompreFace) sit in the same range.

det_size is the CPU lever, not a RAM lever. Same memory, very different speed:

Lower values find small and distant faces less reliably, so trade it against the Why do events yield no face? tool (Tools tab, or scripts/why-no-face.py) rather than by feel.

If ~650 MB is genuinely too much, the only real reduction is a smaller model family (buffalo_s, MobileFaceNet instead of ResNet-50, roughly a third of the memory) at the cost of recognition quality — and switching would invalidate every stored embedding, so the whole gallery would need re-enrolling.

(Apps were formerly known as apps.)

1. Add this repository to your app store — one click:

   Add repository to my Home Assistant

   (or manually: Settings → Apps → App Store → ⋮ → Repositories → add https://github.com/SkyTechNerds/faceid)

2. Install the FaceID app, set your Frigate URL in the options (MQTT is picked up automatically from the Mosquitto app) and start it.

3. Open the FaceID panel in the sidebar. First start downloads the model (~300 MB).

The app is built locally on your machine (amd64/aarch64). See faceid-addon/DOCS.md for all options.

Tested on Debian 12/13 and Ubuntu 22.04+; any Linux with Python 3.10+ works.

1. System packages

apt install python3-venv python3-dev build-essential libglib2.0-0 libgl1 libxcb1 libgomp1

2. Get the code and install the Python environment

git clone https://github.com/SkyTechNerds/faceid /opt/faceid
cd /opt/faceid
python3 -m venv venv
venv/bin/pip install -r requirements.txt

3. Configure

cp docs/example-config.yaml config.yaml
nano config.yaml # set: Frigate URL, MQTT host + credentials, your camera names

4. Run as a service

cp faceid.service /etc/systemd/system/
systemctl daemon-reload
systemctl enable --now faceid

5. Verify

The first start downloads the model pack (~300 MB, one time — see above). Follow the log with journalctl -u faceid -f until you see MQTT verbunden (Success), then open http://<host>:8600 and check that the header shows your person/queue counters.

By default FaceID uses Frigate's API on port 5000, which is unauthenticated — fine on a trusted LAN, and it needs no configuration. Frigate also serves an authenticated API on port 8971:

frigate:
 url: https://192.168.1.10:8971
 user: faceid
 password: secret
 verify_tls: false # Frigate's default certificate is self-signed

One thing to know before you switch, measured rather than assumed: a viewer account can read everything FaceID needs, but cannot write sub_label back into Frigate (Role viewer not authorized. Required: admin). So authenticating costs you either the names in Frigate's Explore view, or requires admin credentials in a config file.

Trade-offs, TLS, what FaceID actually requests, and which setup fits which network: docs/frigate-connection.md .

1. Scan your camera history (optional but recommended): in the Unknown tab, click "Scan camera history" — faces from past Frigate events land in the review queue, pre-clustered per person, and already-known people are tagged in Frigate retroactively. (CLI alternative: venv/bin/python -m app.backfill --days 14)
2. Assign clusters in the UI (Unknown tab): pick a name per cluster — select individual tiles first if a cluster contains a stray face. The ⛶ button shows the full snapshot for context.
3. Once a few people exist, use "apply suggestions" to bulk-assign everything the gallery already recognizes with ≥ 50 % similarity. Repeat as the gallery grows.
4. Optionally upload 5–10 clear photos per person (Persons tab) as clean anchors. Photos with several people are handled: FaceID picks the face matching that person's existing references rather than the largest one, and skips the photo if none of them does. Or enroll a folder: venv/bin/python -m app.enroll "Alice" /path/to/photos.

Not everyone deserves a notification. FaceID distinguishes three actions on an unknown face, and the difference matters:

Ignored faces become negative anchors, grouped by person in the Ignored tab:

• Reappearances are silently dropped (a log line is all you get), and genuinely new looks are auto-learned into the right group — guarded so a household member can never silently become an anchor (requires high similarity and a clear margin over every enrolled person; auto anchors are marked and deletable). Opt out with ignore_learning: false.
• Curate groups: merge two groups when they're the same person, move selected anchors between groups, or send a face back to the review queue.
• Change your mind: release a whole group into an existing or brand-new person — the anchors become that person's reference gallery and tracking starts immediately.
• You can also ignore an entire enrolled person via "ignore person" on their card.

Training tips: camera snapshots beat photo-library images (same lens, angle and light as at recognition time). Diversity beats volume. Create dedicated persons for regular strangers (mailman, neighbors) instead of discarding them — that keeps them from being force-matched to your family.

Frigate runs detection on a downscaled stream (often 1280x960 or even 640x360) but records in full camera resolution (e.g. 2560x1920). Snapshots come from the detect stream, so faces arrive smaller and softer than they need to be.

With Settings → Sharper reference photos enabled (default), a face heading for the review queue is re-fetched from the recording instead: measured across a dozen real events, faces came out roughly twice as large (e.g. 84px → 178px). Better references mean better recognition — and, as a bonus, similarity scores spread out, making genuine duplicates easy to tell from "same person, other angle".

Details: FaceID downloads the event clip once and scans frames across it, because Frigate picks its snapshot from a moment that can't be queried afterwards. Every candidate face is compared against the snapshot face and the best identity match wins — so with several people in frame, the wrong face can't be enrolled. When no frame yields a usable face (roughly one event in three), the original snapshot is kept.

Live recognition tries the fast snapshot path first; only when that yields nothing does it fall back to a full-resolution frame — see docs/recognition-pipeline.md. Needs recordings enabled for the camera.

Frigate picks its snapshot by highest person score, which is not the same as "a face is visible" — often it is the moment someone turns away. Measured over seven days of real events, only 21% of snapshots held a usable face.

FaceID therefore falls back, in order, and only when the step before found nothing:

On the setup measured here this lifts usable events from a fifth to about four in five — but the gain depends almost entirely on the camera angle, both fallbacks can be limited per camera, and the live one needs go2rtc reachable on port 1984.

→ docs/recognition-pipeline.md covers the whole thing: the measurements behind each stage, why cropping to the person box makes things worse, how several people in one frame are handled, what it costs, and how to read the log.

Sometimes the detect snapshot holds no usable face at all and the event is skipped entirely. The history scan can go back over those via the clip:

python -m app.backfill --days 28 --rescue

About one in five such events yields a face this way. Because there is no snapshot face to check identity against, the only guard is detection quality — and it has to be strict: in a measured run, finds below ~0.8 were overwhelmingly back-of-head shots, motion blur and false positives (a church spire scored 0.57), while finds above it were real faces. The default is --rescue-min-det 0.85; lowering it multiplies the queue faster than it adds usable references.

Expect a clip download (8–32 MB) and several seconds per affected event, so this is a manual run, not something the live pipeline does.

Keep --days within your recording retention. Rescue needs the clip, so events older than record.retain yield nothing and only cost time — a 28-day run against a 10-day retention spends most of its hour on events it cannot help.

FaceID listens on frigate/events. Events created through Frigate's API are not tracked objects, so they never appear there — FaceID simply never learns about them.

This matters if you use a camera's own person detection as a reliability bridge: an automation sees the camera report a person and creates a Frigate event for it. Those events carry a snapshot and usually a clip, but stay invisible to FaceID.

Set poll_interval (seconds, e.g. 30) to also poll Frigate's event API:

faceid:
 poll_interval: 30

On one installation that added ~22 events a day at the front door, of which 12 in 20 held a usable face — roughly doubling what FaceID could learn from. Polled events run through the same pipeline; they carry no bounding box, so their snapshot is the full frame rather than a person crop. Off by default, since it costs one API request per interval.

Setting up such a bridge, checking beforehand whether it pays off, and the pitfalls of box-less snapshots: docs/camera-bridge.md .

The defaults are deliberately cautious, and on a real gallery that caution turned out to be expensive. Measure yours rather than guessing — the Tools tab produces the numbers without a terminal, as do scripts/measure-recognition.py and scripts/coverage.py.

On a 128-photo household gallery the separation was far wider than the defaults assume:

With a default threshold of 0.50 sitting exactly on the median of correct matches, half of all genuine recognitions were discarded to keep a distance nothing ever came close to. Two changes followed, both measured:

• match_top_k 3 → 1. Averaging the best k photos punishes people whose references cover many angles — their own less similar photos drag the score down, so the best-covered people scored worst.
• match_threshold 0.50 → 0.45, still 0.14 above the highest score any stranger ever reached.

Together these lifted recognition on a held-out set of real events from 90% to 100%, and moved the weakest favourite's worst match from 0.01 above the cut-off to 0.09 above it — without a single misassignment.

In the UI: Settings → Does it actually work? runs the same analysis as a background job — no shell needed, which matters if you run FaceID as a Home Assistant app. The scripts remain for scripted or comparative runs (--baseline, --top-k).

The obvious question is "how close does a stranger get?", and it is the wrong one to stop at. Two numbers matter, and the higher of them sets your floor:

Going by strangers alone, 0.25 would look safe. It is not: at 0.25 the two people in this household who resemble each other most become interchangeable — a father and daughter whose galleries overlap at 0.411. And a wrong known name is worse than no name at all, because automations act on it, while "unknown" can simply be ignored.

This is why the analysis reports both, and why lowering the threshold because someone is not being recognised is usually the wrong move. (One case where no threshold helps: small children — a toddler's face is about half the size of an adult's and falls below min_face_px at the same distance. See the pipeline doc.) If a person sits just below the line, the fix is more reference photos of the situations they are actually seen in — that raises their score without moving anyone else closer.

Do not copy any of these numbers. Measure your own gallery: Settings → Does it actually work? reports the ceiling for both cases, and refuses to suggest lowering anything if it finds a misassignment.

The LOG tab shows the last 500 log lines straight in the UI — including the quiet cases that decide whether a setup works: whether Frigate answers at startup, which cameras were announced to Home Assistant, and for every event whether a face was found at all. If nothing is ever recognised, that tab usually says why within a few lines. There is a warnings-only filter and a copy button for pasting into an issue.

Switch it on under Settings → Does it work? → show the Tools tab. It runs the same analyses in the service itself and shows them as tables:

This is not a wrapper around the scripts below, because it could not be: scripts/ is not part of the app image at all, and the delay measurement there reads journalctl, which does not exist in a container without systemd. If you run FaceID as a Home Assistant app, the tab is the only way to get these numbers.

The delay figures therefore come from the history rather than the log, which also means they need no camera access. Recognitions produced by a history scan are excluded instead of being averaged in — they lag their event by weeks — and the tab says how many it dropped.

Three scripts answer the questions that otherwise invite guesswork:

python scripts/why-no-face.py --days 7 --clip 12 # why do events yield no face?
python scripts/coverage.py # what is each person missing?
python scripts/measure-recognition.py --baseline /tmp/old --days 3

Start with why-no-face.py if recognition feels rare. It counts why events are discarded — no face at all, too small, no snapshot — and separates the hopeless cases (person too far away) from the recoverable ones (the snapshot moment was bad). With --clip it re-checks discarded events against the recording, which tells you what clip_fallback is worth on your cameras rather than on mine. Most "it barely recognises anyone" reports turn out not to be gallery problems at all.

coverage.py reports, per person: photo count, diversity, viewing angles from the landmarks, which cameras she was enrolled from, greyscale/IR shots, and a leave-one-out self test — then names the concrete gap.

measure-recognition.py compares the current gallery against an older one (unpack a backup from data/backups) and adds a practical probe against recent Frigate events, including how much headroom each recognition has above the threshold. Events whose face is already in the gallery are excluded — they score ~1.0 and measure nothing.

Running FaceID as a Home Assistant app? You have no shell, so the part that decides your threshold was moved into the UI: Settings → Does it actually work? runs the leave-one-out test and the practical probe as a background job and reports the one number that matters, how high a stranger got. That covers threshold and match_top_k.

Two things still need a shell, and neither is required to run FaceID well:

• the coverage report (coverage.py) — which angles, cameras and IR shots each person is missing
• comparing two galleries (--baseline) — useful after a round of enrolling, but the UI analysis already tells you where you stand today

Recognition is only as good as the reference photos, so FaceID keeps galleries diverse rather than large:

• A successful recognition never adds a photo. Recognising you at the door tags the Frigate event and updates the sensor — but the gallery stays untouched. This is deliberate: a gallery that grows from its own matches reinforces whatever it already believes, and a single wrong match would quietly breed more of the same. References only come from what you assign in the review queue, or upload yourself.

• Ignore anchors can be capped too (max_ignore_anchors). With ignore_learning switched on, every unambiguous ignore match adds an anchor, which grows without bound on a busy street. The cap drops the most redundant auto-learned anchor per group — never a manual one, and never by age, since an old anchor is exactly as valid as a new one and removing it would let that person resurface.

• New photos are only kept if they add something — a near-duplicate of one you already have is skipped.

• A per-person cap (default 40, adjustable in Settings) bounds how many references a person keeps. When exceeded, FaceID sets aside the photo that is most similar to all the others — i.e. the most redundant one — so a rare side/angle shot is preserved while a 30th near-identical front shot is the first to go.

• Remove duplicates on demand: even under the cap, near-identical photos add nothing. Settings → Remove duplicates finds both truly identical images (perceptual hash) and near-identical faces (embedding similarity) and sets them aside (live preview), keeping the gallery diverse — as a button, so you stay in control. (Camera crops score lower than phone photos, so useful sensitivity is ~0.60–0.70.)

• Nothing vanishes silently: trimmed photos appear on the person card with a short reason and a one-click restore (or delete), and the set-aside pile is itself capped (trimmed_keep) so it never grows without bound.

If someone is recognized poorly from a certain angle, just add a photo from that angle — being unusual, it's automatically kept.

Full details: docs/trimming.md explains the why, the exact selection rule (with numbers), and how to restore or curate set-aside photos.

Your gallery (enrolled persons + ignore anchors) is the one thing you can't regenerate — so FaceID makes it easy to keep. Everything is on the Settings tab:

• Download backup — a .tar.gz of persons/ + ignored/ (not the unknown queue).
• Restore — replace everything, or merge in only what's missing (handy for moving people between instances).
• Automatic daily backup — enable it, pick an hour and how many to keep. It runs inside FaceID (no external cron needed); the folder defaults to data/backups, which survives app updates. Point it at a mounted share to get backups off the box.

Automate it yourself if you prefer: the download is a plain endpoint, so any host cron or Home Assistant automation can pull it:

# nightly host cron — keep 14 days
curl -fsS http://<faceid-host>:8600/api/backup -o /backups/faceid-$(date +\%F).tar.gz
find /backups -name 'faceid-*.tar.gz' -mtime +14 -delete

Settings changed here (thresholds + backup) are stored in data/settings.json and override config.yaml / app options, persisting across restarts and updates.

Sensors appear automatically via MQTT discovery (sensor.faceid_<camera> for every camera in discovery_cameras). The state lists everyone recognized within presence_window seconds (Alice, Bob), then falls back to nobody. Attributes carry the person list and the last recognition (score, event id).

For automations, trigger on the faceid/event topic — one JSON message per (Frigate event, person): see docs/ha-automation-example.yaml for a phone-notification automation with the Frigate snapshot attached.

{"person": "Alice", "score": 0.612, "camera": "driveway",
 "event_id": "1786945313.350301-xiec8d", "ts": 1786945318.4,
 "zones": ["driveway_zone"]}

zones lists the Frigate zones the person entered during that event, so an automation can tell where someone was seen — notify for strangers in the driveway, stay quiet for the street, ignore known faces anywhere. The same payload appears as the last attribute on the presence sensor, where it stays until the next recognition replaces it — the state falls back to nobody when nobody is around, but last keeps answering who was seen there most recently. It is absent until the first recognition after a restart.

⚠️ An empty zones does not mean "outside every zone". It also appears when the camera has no zones at all, and when Frigate never registered the person in one — being visible in a region and being counted as in a zone are different things (the zone test uses the object's anchor point and has its own timing rules). On the gallery here, 12 of 28 correct recognitions on a zoned camera carried no zone. Treat empty as unknown, not as elsewhere, and decide deliberately what your automation does with it. If you want zone gating to be strict, do it in Frigate instead — snapshots: required_zones: stops the event from ever reaching FaceID, because FaceID only acts on events that have a snapshot.

If you use a Frigate notification blueprint (SgtBatten's is the common one), you have probably noticed the name never appears in it. Two measured reasons:

• The name is not ready yet. The blueprint fires when the event starts; FaceID first needs a snapshot to exist, a face in it, and a match. Over 155 real recognitions on one household setup the name was ready after a median of 8 seconds, the fastest in one. What sets the floor is Frigate's time to a first usable snapshot, and that varies a lot — it is not a fixed few seconds. Measure your own in FaceID's Tools tab (switch it on under Settings → Does it work?); the numbers depend on your cameras and your hardware.
• Waiting can help — but often does not, for a different reason than I first published. ⚠️ An earlier version of this section claimed Frigate never announces the name over MQTT. That was wrong: the test behind it ran against an already finished event, where nothing more is sent. On a running event, Frigate 0.17.2 forwards it in the same second, as after.sub_label = ["Eli", 0.514] — an array of name and score.
• The catch is where it appears. At least one widely used Frigate notification blueprint reads after.data.sub_labels, which is null on 0.17.2. The name is in the payload, just not at the field being read — so the notification stays nameless and it looks like FaceID never wrote anything.

So do it the other way round — react to FaceID's own event and replace the notification that already went out:

Import blueprint

The trick is the notification tag: give it the same tag your Frigate blueprint uses (the event id, by default) and the phone replaces the earlier message instead of stacking a second one.

⚠️ If the notification arrives without a picture, the Frigate address is almost always the reason. Since v0.20.0 the image comes from Home Assistant by default (/api/frigate/notifications/<event>/snapshot.jpg), which the phone resolves against its own HA connection — leave that setting alone and it works at home and away. If you switch to a Frigate address instead, it has to be reachable from the phone: http://ccab4aaf-frigate:5000 and similar add-on hostnames exist only inside Home Assistant.

⚠️ Already imported an older version? Pasting the URL again does not replace it — Home Assistant keeps the copy it has. Use Settings → Automations & scenes → Blueprints → ⋮ → Re-import blueprint.

Since Frigate does forward the name (see above), a Frigate-side blueprint can show it — it just has to read after.sub_label, which on 0.17.2 is an array: ["Alice", 0.51], so the name is after.sub_label[0].

If you use SgtBatten's blueprint and would rather extend that than run a second automation, @crunchynuts has published a merged version covering both routes — the Frigate payload and faceid/event:

• blueprint_sgtbatten_faceid · thread

A community contribution, not maintained here: I have read it but not run it, and its Signal notification path is written for the author's own setup. Which one to pick:

• Home Assistant app: you get notified automatically when a new version is available (Settings → Apps → FaceID → Update). The changelog is shown right in the update dialog.
• Standalone: cd /opt/faceid && git pull && systemctl restart faceid. Watch the GitHub releases to get notified.

See CHANGELOG.md for the full history.

• The web UI supports optional HTTP Basic Auth (faceid.auth in config.yaml) — strongly recommended for standalone installs; the HA app is protected by ingress and your Home Assistant login instead. Either way: it manages biometric data — keep it on a trusted LAN and don't expose port 8600 to the internet (Basic Auth without TLS is not internet-grade protection).
• All face data stays in data/ on your host (JPEG crops + embeddings). Delete a person and their data is gone.
• Depending on where you live, informing household members/visitors about face recognition on your cameras may be legally required. Be a good human.

See docs/example-config.yaml — every option is commented. The two knobs that matter most:

MIT