A carer cannot be in the room twenty-four hours a day.
Bedridden patients in home and residential care are checked on a schedule. Between those checks, a patient who cannot reach a call button, whether from paralysis, sedation, dementia, or simple exhaustion, has no way to report pain. A patient in pain at 2am may wait until the morning round.
Pulse does not replace a carer. It tells a carer when to come. That is the whole ambition, and it is deliberately small: the gap we are trying to close is measured in minutes, not in staffing budgets. Forty-one seconds, not four hours.
Video in, a phone notification out.
One wide-angle camera, pointed at a bed from a bedside table. No wearables, no contact sensors, no microphone in the demo.
A push notification to one carer's phone: what was detected, how confident, and a still frame from the moment it fired.
A family carer or a night nurse covering several rooms. They set it down, point it at the bed, and otherwise ignore it.
Two branches, because in bed and out of bed aren't the same problem.
If the patient is in bed, the body is under a blanket. The face is the only signal there is, so that branch would read facial expression: a CNN, now complete, and an LSTM, in development in week six, scoring pain from the last 32 frames. If the patient is out of bed, the body is visible instead, so that branch reads posture: a pose model that maps the body to 17 COCO keypoints and tracks them frame to frame. Both detectors already run, each on its own, and the current work is bringing them into one live pipeline that feeds the carer dashboard. Beyond that, the next stage on either side is interpretation: the LSTM that would turn the CNN's features into a pain score, in development now, and whatever decides that a particular set of keypoints is a fall or a limp.
-
A wide-angle frame every 33 milliseconds. Nothing leaves the device: frames are processed and discarded, and no video is stored or streamed.
-
A lightweight detector finds the face and crops it out. This part works and runs on its own today. If there is a face, the patient is in bed, which is the case the CNN and LSTM below are built for: the CNN is complete, and the LSTM is in development this week. If there is no face, that usually means the patient is out of bed, which is the pose model's case. Each detector runs standalone, and the current work is connecting both into one live pipeline that feeds the carer dashboard, including that handoff between them. After that comes the layer that decides what each one is looking at; until then, nothing downstream acts on a detected face or body.
-
Complete as of week five. A convolutional network turns each crop into 512 numbers describing the shape of the expression, not a picture of it: brow tension, eye closure, mouth shape, compressed into a vector. Training surfaced a class imbalance issue, which is now resolved.
-
The CNN now generates these vectors, one per frame. Storing them frame by frame for the live pipeline is part of this week's work, and each one goes into a rolling buffer, because a single frame is a bad witness: people grimace when they sneeze, and a still face can hide sustained pain.
-
In development this week, not trained yet. The idea is a recurrent model that reads the last 32 vectors, about 1.1 seconds of video, and returns one confidence value between 0 and 1: change over time, not a snapshot.
-
This one works and runs on its own, with a demo video on the media page. For the out-of-bed case the model finds a body and maps it to 17 COCO keypoints, tracked frame to frame. Bringing it and face detection into one pipeline that feeds the carer dashboard is the current work. Deciding what those keypoints mean is the next stage. There is no fall detection, no limp detection, nothing that turns a moving skeleton into an alert. That interpretation layer is the next piece of this branch to be written.
-
Planned for week seven, alongside the rest of integration. The idea is a confidence compared against a threshold: above it, an alert goes out, below it, the room stays quiet. At this stage nothing feeds it: the face branch produces a confidence once the LSTM is trained on the CNN's output, and the pose branch produces keypoints rather than a decision about them.
Above threshold, and only above threshold, an alert goes out.
Below it, the room stays quiet. False alarms cost trust. A carer who is woken three times a night by a sneeze stops trusting the fourth alert, and the fourth alert is the one that matters. Everything about this decision is a trade.
Below are twenty illustrative confidence values, not measured ones: real numbers here will come from the LSTM, in development this week. Seven are placed where a genuine-pain moment would sit. Move the threshold and watch what you gain and what you give up.
A concept, not a product. We should say that plainly.
Right now Pulse is software running on a laptop with a webcam. This cube is where we're heading: a Raspberry Pi 5, a wide camera, and a round display, in about 8cm of warm plastic on a bedside table. Nothing clipped to the patient, no wires across the bed.
CAD STAGE · NOT YET BUILTCamera Module 3 Wide, the only sensor the demo needs.
240 × 240 round LCD. It exists to hold the face.
I2S mic for audible distress, as a second opinion.
Thermal, motion, distance, air quality. Roadmap, not demo.
We train on a physiotherapy clinic and aim at a bedroom.
Our models are trained on the UNBC-McMaster Shoulder Pain Expression Archive: video of adult patients undergoing range-of-motion tests on an injured shoulder, frame-coded for facial action units and pain intensity. It is licensed for non-commercial research and teaching only, and we use it under those terms.
The domain gap is real and we are not going to talk around it. That footage is seated adults, lit clinically, facing the camera, in acute pain provoked on purpose. Our target is a person lying down, in a dark room, at an oblique angle, possibly in chronic pain. We align faces before the CNN and augment with rotation and angle jitter. That narrows the gap. It does not close it, and no result we get at the showcase should be read as if it did.
A concept, not a product. We should say that plainly.
- Not a certified medical device. It has no regulatory approval of any kind and is not intended for clinical use.
- Not clinically validated. Our numbers come from a research dataset, not from patients in a bedroom.
- Not a diagnosis. Pulse says a face looks like pain, at a confidence. A carer decides what that means.
- Not a replacement for a carer, and never a reason to check on someone less often.
- Not reading meaning into posture yet. The pose branch tracks 17 body keypoints, but nothing classifies those into a fall or a limp.
- Not built. Week 8 is a software demo. We're not pretending otherwise.
Ten weeks, counted from when we actually started. We're four of them in.
-
Scope: falls, breathing, or pain
-
Face detection and patient detection (17 keypoints) both working
-
Dashboard prototype
-
Dataset access came through; indexed 48,398 frames and checked labels against the paper
-
CNN complete: 512-dimension feature vectors generated, class imbalance issue resolved
-
WE ARE HERE
Wiring the pieces together: frame storage for the CNN pipeline, webcam integration, LSTM development, patient detection refinement
-
Integration: one live inference loop
-
Showcase: live video in, alerts out
-
Threshold tuning, latency, failure cases
-
Write-up, handover, CAD refinement