Contactless heart-rate and respiratory-rate estimation from face video, processed in 5-second incremental chunks using the Open-rppg FacePhys model family.

• Overview
• Pipeline
• Architecture
• Setup
• Usage
• Accuracy Framework
• Performance Metrics
• Failure Modes
• Code Organization
• References
• AI Disclosure
• Deliverables

This project implements a near real-time remote photoplethysmography (rPPG) system that processes face video in 5-second windows and outputs per-chunk BPM, a final session BPM, respiratory rate, and runtime performance metrics.

Integrated model stack: Open-rppg · rppg.Model('FacePhys.rlap')

Video Input (60s)
 │
 ▼
┌─────────────────────┐
│ Face Detection │ OpenCV Haar Cascade
│ ROI Extraction │ Forehead crop + coverage tracking
└────────┬────────────┘
 │ 5-second windows
 ▼
┌─────────────────────┐
│ Model Inference │ Open-rppg (FacePhys.rlap)
│ │ → chunk BPM + confidence score
└────────┬────────────┘
 │
 ▼
┌─────────────────────┐
│ Robust Aggregation │ Confidence-weighted averaging
│ │ Outlier down-weighting
└────────┬────────────┘
 │
 ▼
┌─────────────────────┐
│ Respiratory Rate │ Model output → PSD fallback
└────────┬────────────┘
 │
 ▼
 JSON Output + Metrics

alt text

• OpenCV Haar Cascade face detector
• Forehead ROI extraction per frame
• Valid-face coverage tracking across chunks

• Open-rppg backend (FacePhys.rlap default)
• Chunk-level inference returning BPM and confidence

• Non-overlapping 5-second chunking
• Confidence-aware robust aggregation for final BPM

• Throughput, latency, and realtime-factor logging
• Ground-truth and proxy-quality evaluation modes
• JSON artifact serialization

Requirements: Python 3.10+, macOS / Linux / Windows

python3 -m pip install -r requirements.txt

Core dependencies: numpy, scipy, opencv-python, open-rppg

Optional (run_application.py only, not needed for the chunked prototype): pypylon, PyQt5, pyqtgraph, scikit-image

python3 - <<'PY'
import cv2, numpy as np
from scipy import signal
import rppg
print("dependency check: ok")
PY

python3 run_chunked_prototype.py \
 --video input/assignment_60s.mp4 \
 --chunk-sec 5 \
 --model-backend open-rppg \
 --open-rppg-model FacePhys.rlap \
 --json-out notes/chunked_rppg_output.json

python3 evaluate_accuracy.py \
 --pred notes/chunked_rppg_output.json \
 --out notes/accuracy_report.json

cp notes/reference_hr_template.csv notes/reference_hr.csv
python3 evaluate_accuracy.py \
 --pred notes/chunked_rppg_output.json \
 --ref-bpm-csv notes/reference_hr.csv \
 --ref-rr 14.8 \
 --out notes/accuracy_report_with_gt.json

python3 -m pip install open-rppg && \
python3 run_chunked_prototype.py \
 --video input/assignment_60s.mp4 --chunk-sec 5 \
 --model-backend open-rppg --open-rppg-model FacePhys.rlap \
 --json-out notes/chunked_rppg_output.json && \
python3 evaluate_accuracy.py \
 --pred notes/chunked_rppg_output.json \
 --out notes/accuracy_report.json

Evaluation operates at three levels: chunk predictions, session aggregation, and runtime behavior.

When no reference is available, the evaluator reports:

• Median confidence score
• Chunk BPM standard deviation
• BPM interquartile range
• Outlier chunk ratio

Note: notes/reference_hr_template.csv is a small workflow-validation sample. For trustworthy benchmarks, replace it with a larger manually annotated dataset.

RTF ≥ 1.0 confirms at-least-real-time throughput.

• Motion & pose — Fast head movement and large pose changes reduce per-chunk stability and confidence.
• Illumination — Flicker and compression artifacts degrade the rPPG signal.
• Short windows — 5-second chunks improve responsiveness but increase BPM variance.
• Latency vs. accuracy — The deep model backend improves physiological modelling but raises per-chunk compute time.
• Production hardening — Deployments should apply confidence gating and graceful fallback for low-quality frames.

.
├── run_chunked_prototype.py # Entry point — chunked pipeline CLI
├── evaluate_accuracy.py # Accuracy evaluator (GT + proxy modes)
├── requirements.txt # Core dependencies
├── src/
│ ├── open_rppg_backend.py # Open-rppg model adapter
│ └── rppg_toolbox_pos.py # Alternate POS signal backend
├── input/
│ └── assignment_60s.mp4
└── notes/
 ├── chunked_rppg_output.json # Primary inference output
 ├── chunked_rppg_output_open_rppg.json
 ├── accuracy_report.json # Proxy-quality evaluation
 ├── accuracy_report_with_gt.json # GT evaluation
 └── reference_hr_template.csv

• Open-rppg — primary integrated model backend (FacePhys family)
• rPPG-Toolbox — alternate signal processing reference
• heartbeat — lightweight rPPG reference implementation
• Meta-rPPG — meta-learning rPPG approach

AI tools were used extensively throughout this project, including for:

• Chunked pipeline architecture and CLI design (run_chunked_prototype.py)
• Open-rppg model adapter implementation (src/open_rppg_backend.py)
• Confidence-aware aggregation and evaluation warnings (evaluate_accuracy.py)
• Chunk outlier rejection logic tuning
• Accuracy framework definition and metrics selection
• README structure and reproducibility documentation

• Integrated CV + rPPG model pipeline
• 5-second incremental chunk processing
• Chunk BPM and final session BPM outputs
• Runtime and performance metrics
• Respiratory rate estimation with frequency-domain fallback
• Formal accuracy protocol with GT and proxy-quality modes
• Reproducible commands and structured documentation