←BaldanAll skillsLet's talk
Deep Learning & Vision lab

Computer vision

A driver-fatigue monitor in the browser: eye closure, blinks, PERCLOS and yawns from your webcam, with an alarm.

20

Points used

1.5 s

Default alarm

2 s

Calibration

3.8 MB

Model download

Overview

Drowsiness causes a large share of road accidents, and the first signs are visible in the face: the eyes close for longer, blinks slow down, yawns come more often. Driver-monitoring systems in modern cars watch for exactly these cues.

This project measures them from a webcam. Google's face mesh model tracks 478 points; the detector uses twelve around the eyes to compute the eye aspect ratio, compares it with your own open eyes from a short calibration, and sounds an alarm when your eyes stay closed too long. A dashboard adds blink rate, PERCLOS and yawns.

How it works

From raw file to prediction.

Every number below comes from the training run.

  1. 01

    Camera

    your webcam

    Frames stay on the device

  2. 02

    Face mesh

    478 points

    Google's Face Landmarker

  3. 03

    EAR

    6 points per eye

    Eyelid opening over eye width

  4. 04

    Calibrate

    2 s

    Your open-eye EAR becomes the reference

  5. 05

    Track

    blinks · PERCLOS · yawns

    Over the last minute

  6. 06

    Alarm

    1.5 s closed

    Adjustable, Web Audio tone

Inside the model

From eyelids to an alarm.

Google's network places the face points; the rest is one ratio and a few timers I wrote. The face and the timeline are diagrams; the thresholds are the demo's real settings.

FACE DETECTOR: A FACE IS HERE

The signal

Twelve points around the eyes, thirty times a second.

Drowsy drivers close their eyes for longer, blink more slowly and yawn. All three can be measured from face landmarks: no special hardware, just a webcam and a face mesh model.

MediaPipe Face Landmarker

Google's pretrained face mesh model (Apache-2.0 (Google)), the same one behind Face Mood Mirror. Of its 478 points, this detector uses 12 around the eyes and 8 around the mouth. Everything else is plain geometry and timing, written for this page.

20

points used

2 s

calibration

1.5 s

default alarm

3.8 MB

model download

Model page

Column dictionary

● used by the model

ColumnTypeWhat it means
eye points2 × 6 of 478Corners, upper and lower eyelid of each eye (mesh points 33, 160, 158, 133, 153, 144 and 362, 385, 387, 263, 373, 380)
mouth points8 of 478Mouth corners and three inner-lip pairs, for the yawn ratio
other points458Tracked by the model but not needed for drowsiness

01

Camera frame

Processed in your browser, never uploaded

02

478 face points

Google's Face Landmarker tracks the face every frame

03

EAR per eye

Eyelid opening over eye width, from six points per eye

04

Against your open eyes

Closed below 70% of your own open-eye EAR from a 2-second calibration

05

Timer → alarm

Short closures count as blinks; 1.5 s closed sounds the alarm

Try it

Close your eyes. It will notice.

Start the camera, keep your eyes open for the 2-second calibration, then blink, yawn or close your eyes and wait for the alarm. Turn the sound on for the full effect; there is a simulation if you have no camera.

Live demo

runs in your browser

Turn on your camera

Video stays on this device. Sit as you would at the wheel or your desk, face the camera.

Driver status

No face

Eye aspect ratio, last seconds

EAR 0.000 / closed below 0.000

closed

–

Blinks / min

…

PERCLOS (1 min)

0

Yawns (5 min)

0.0 s

Longest closure

How it measures

One ratio for the eyes, one for the mouth, and a clock.

The eye aspect ratio (Soukupová and Čech, 2016) divides the two vertical eyelid distances by the eye's width. It stays fairly constant while the eye is open and falls towards zero as it closes, whatever the size of your face in the picture.

The eye aspect ratio, by hand

Drag the slider to close the drawn eye and watch the ratio fall. The detector calls an eye closed when its EAR drops below a share of your own open-eye value.

p1p2p3p4p5p6EAR = (|p2−p6| + |p3−p5|) / (2·|p1−p4|) = 0.278
The signals on the dashboard

Researchers combine several cues, because a single long blink or one yawn isn't drowsiness on its own.

SignalHow it's computed here
Eyes closedEAR below 70% of your open-eye EAR
BlinkA closure shorter than 400 ms; shown as blinks per minute
AlarmEyes closed for 1.5 s or more (adjustable)
PERCLOSShare of the last minute with the eyes closed, a standard fatigue measure
YawnMouth aspect ratio above 0.55 for at least 1.2 s
LevelDrowsy at the alarm or PERCLOS above 30%; tired above 15%, or 3 yawns in 5 minutes

Honest notes

A demonstration, not a safety device.

If you feel drowsy while driving, stop and rest. Commercial driver-monitoring systems use infrared cameras that work at night and through sunglasses, and are tested on many drivers; this page shows the core idea in a browser.

What this project is, and isn't

Google's face mesh, my geometry, timers, alarm and dashboard, all on your device.

Here
Face modelGoogle's MediaPipe Face Landmarker (Apache-2.0 (Google)), pretrained and used as published
Built hereEAR and mouth ratio from the landmarks, per-person calibration, blink, PERCLOS and yawn tracking, the alarm, the dashboard and the simulation
AccuracyNot measured on a labelled drowsiness dataset; the default thresholds come from the EAR literature and were checked on simulated eye closures
LimitsNeeds a lit, visible face: darkness, sunglasses, strong side angles or a lowered head defeat it. Looking down at a phone can look like closed eyes
PrivacyVideo is processed in the browser and never uploaded or recorded

Build notes

How it's built, step by step.

The real code behind this demo, taken from the repository when the site is built. Copy any piece.

In your browser · TypeScript

  1. 1

    The eye aspect ratio

    The two vertical eyelid distances over twice the eye's width, measured in pixels so a wide video doesn't distort it.

    src/lib/drowsiness.ts:17 · 4 lines

    export function ear(pts: Pt[], idx: number[], w: number, h: number) {
      const [p1, p2, p3, p4, p5, p6] = idx.map((i) => pts[i]);
      return (dist(p2, p6, w, h) + dist(p3, p5, w, h)) / (2 * dist(p1, p4, w, h));
    }
  2. 2

    The mouth aspect ratio, for yawns

    The mean of three inner-lip openings over the mouth's width.

    src/lib/drowsiness.ts:22 · 6 lines

    /** Mouth aspect ratio: mean inner-lip opening over mouth width. */
    export function mar(pts: Pt[], w: number, h: number) {
      const m = MOUTH;
      const open = (dist(pts[m.top], pts[m.bottom], w, h) + dist(pts[m.top2], pts[m.bottom2], w, h) + dist(pts[m.top3], pts[m.bottom3], w, h)) / 3;
      return open / dist(pts[m.left], pts[m.right], w, h);
    }
  3. 3

    One frame of the detector

    Compares EAR with the calibrated threshold, times closures into blinks or alarms, counts yawns and keeps PERCLOS over the last minute.

    src/lib/drowsiness.ts:50 · 35 lines

    /**
     * One frame of the detector. `baseline` is the visitor's open-eye EAR from calibration.
     * Returns the updated state plus what to show: closed for how long, blink rate, PERCLOS (share of the last minute
     * with eyes closed) and the alarm flag.
     */
    export function step(s: State, t: number, earNow: number, marNow: number, baseline: number, cfg: Settings = DEFAULTS) {
      const threshold = baseline * cfg.closedRatio;
      const closed = earNow < threshold;
      let { closedSince, mouthSince, longest, alarm } = s;
      const blinks = s.blinks.filter((b) => t - b < 60_000);
      const yawns = s.yawns.filter((y) => t - y < 300_000);
      if (closed) closedSince ??= t;
      else if (closedSince !== null) {
        const dur = t - closedSince;
        if (dur <= cfg.blinkMaxMs) blinks.push(t);
        longest = Math.max(longest, dur);
  4. 4

    Awake, tired or drowsy

    Long closures and a high PERCLOS mean drowsy; a raised PERCLOS or frequent yawns mean tired.

    src/lib/drowsiness.ts:86 · 7 lines

    /** A plain fatigue level for the dashboard: long closures dominate, then PERCLOS, frequent yawns and few blinks. */
    export function fatigueLevel(closedMs: number, perclos: number | null, yawns5min: number, alarmMs: number): "Awake" | "Tired" | "Drowsy" {
      const p = perclos ?? 0;
      if (closedMs >= alarmMs || p > 0.3) return "Drowsy";
      if (p > 0.15 || yawns5min >= 3 || closedMs >= alarmMs * 0.5) return "Tired";
      return "Awake";
    }
  5. 5

    The alarm tone

    A short square-wave beep every 400 ms with the Web Audio API, unlocked by the Start click because browsers block sound before a click.

    src/components/demos/drowsiness-demo.tsx:30 · 21 lines

    /** A pulsing warning tone with the Web Audio API (no sound files). Stable across renders. */
    function useAlarm() {
      const ctx = useRef<AudioContext | null>(null);
      const timer = useRef<ReturnType<typeof setInterval> | null>(null);
      const unlock = useCallback(() => {
        ctx.current ??= new AudioContext();
        void ctx.current.resume();
      }, []);
      const set = useCallback((on: boolean) => {
        if (on && !timer.current && ctx.current) {
          const c = ctx.current;
          beep(c);
          timer.current = setInterval(() => beep(c), 400);
        } else if (!on && timer.current) {
          clearInterval(timer.current);
          timer.current = null;

Notes

The decisions behind it.

Approach

  1. 1

    Ran Google's Face Landmarker in the browser (the same self-hosted model as Face Mood Mirror) and took the six EAR points of each eye and eight mouth points.

  2. 2

    Computed the eye aspect ratio in pixels, averaged over both eyes, and set the closed threshold per person: a share of the median EAR from a 2-second open-eye calibration.

  3. 3

    Timed every closure: short ones count as blinks, long ones fill the alarm bar; tracked PERCLOS over the last minute and yawns from the mouth aspect ratio.

  4. 4

    Added an alarm tone with the Web Audio API, a live EAR chart, adjustable thresholds and a slider simulation for visitors without a camera.

Honest note

This is a demonstration, not a safety device: if you feel drowsy while driving, stop and rest. The face model is Google's; the thresholds are mine, taken from the EAR literature and checked on simulated closures, not measured on a labelled drowsiness dataset. It needs a lit, visible face, so darkness, sunglasses, a lowered head or looking down at a phone can fool it.

Reproduce: src/lib/drowsiness.ts · src/components/demos/drowsiness-demo.tsx

dataset source

Next project

Virtual Mouse