EyeSense — Blink to Click Chrome Extension

Control your browser with your eyes. EyeSense uses MediaPipe Face Mesh to detect blinks in real-time via your webcam and fires a mouse click at your cursor position — all processed locally in the browser, with zero data leaving your device.

eye-blink-extension/
├── manifest.json # Chrome Extension MV3 config
├── background.js # Service worker — orchestrates start/stop, tab tracking
├── content.js # Injected into active tab — webcam, MediaPipe, blink logic
├── popup.html # Extension popup UI
├── popup.css # Popup styles (dark cyberpunk theme)
├── popup.js # Popup logic — settings, stats polling, calibration
├── styles-injected.css # CSS injected into tracked tab (overlay, badge, flash)
└── icons/
 ├── icon16.png
 ├── icon48.png
 └── icon128.png

sequenceDiagram
 participant U as User
 participant C as Camera
 participant S as System
 participant AI as Eye Model
 participant UI as Display
 loop Real-Time Tracking
 U->>C: Look or move eyes
 C->>S: Capture frame
 S->>AI: Process frame
 AI-->>S: Return eye position
 S->>S: Analyze attention
 S->>UI: Update state
 UI-->>U: Show focus or alert
 end

1. Download / clone this folder to your computer.

2. Open Chrome and navigate to:

   chrome://extensions
   

3. Enable Developer Mode (toggle in the top-right corner).

4. Click "Load unpacked" and select the eye-blink-extension/ folder.

5. The EyeSense icon will appear in your Chrome toolbar.

1. Navigate to any regular webpage (not chrome:// pages).
2. Click the EyeSense icon in the toolbar → popup opens.
3. Click "Start Tracking" — Chrome will ask for webcam permission; allow it.
4. A small webcam overlay appears in the bottom-right of the page showing your face mesh.
5. Blink deliberately (hold ~2 frames) to trigger a click at your cursor position.
6. Click "Stop Tracking" to disable.

EAR is a single number that describes how "open" an eye is, derived from 6 landmark points:

 p2 p3
 p1 · · p4
 p6 p5

EAR = (||p2−p6|| + ||p3−p5||) / (2 ×ばつ ||p1−p4||)

• Eye open: EAR ≈ 0.25–0.32
• Eye blinking: EAR drops below ~0.21
• Eye closed: EAR < 0.15

Frame N: EAR < threshold → blink_counter++
Frame N+1: EAR < threshold → blink_counter++ (= 2 frames, threshold met)
Frame N+2: EAR >= threshold → BLINK CONFIRMED → fire click, reset counter

The "Confirm Frames" setting controls how many consecutive closed-eye frames are needed before a blink is registered. Higher values reduce false positives from lighting changes.

1. Start Tracking first.
2. Open settings → click "Run Calibration".
3. Look at the screen normally for 3 seconds (keep eyes open, don't blink).
4. The extension measures your personal baseline EAR and sets the threshold to 75% of that value — tuned to your eyes and lighting.

Privacy: All face/eye processing happens in your browser using MediaPipe's WASM model. No video frames, landmarks, or personal data are ever sent to any server.

[Webcam] → MediaPipe FaceMesh → landmarks[]
 ↓
 computeEAR(landmarks)
 ↓
 avgEAR < threshold?
 ↓
 consecutiveBlinks >= blinkFrames?
 ↓
 simulateClick(mouseX, mouseY)
 ↓
 MouseEvent dispatched on DOM element

popup.js ──START_TRACKING──► background.js
 │ scripting.executeScript(content.js)
 │ tabs.sendMessage(INIT_TRACKER)
 ▼
 content.js (active tab)
 │
 │ BLINK_DETECTED → background.js (badge flash)
 │ GET_STATS → popup.js (live EAR/counts)

• Gaze Tracking: MediaPipe's refineLandmarks: true exposes iris landmarks (indices 468–477). The iris center can be mapped to screen coordinates via affine transform for cursor control.
• Wink detection: Compute EAR for left and right eyes independently — a left wink vs. right wink could trigger different actions.
• Double-blink for right-click: Track inter-blink timing; two blinks within 400ms → contextmenu event.
• Head-nod scrolling: Track the vertical position of the nose tip across frames to detect nods → scroll events.
• Offscreen document: Move the webcam + MediaPipe into a Chrome Offscreen Document (MV3 API) to avoid injecting a visible overlay into the page.
• WASM caching: Cache the MediaPipe .wasm and .bin model files using a service worker cache for faster startup.

• Chrome Extension Manifest V3
• MediaPipe Face Mesh 0.4 (loaded from CDN, runs as WASM in the browser)
• Vanilla JavaScript (no build step required)
• HTML5 Canvas for landmark rendering
• Web Audio API for click sound feedback

MIT — free to use, modify, and distribute.