A carer cannot be in the room twenty-four hours a day.
Bedridden patients in home and residential care are checked on a schedule. Between those checks, a patient who cannot reach a call button, whether from paralysis, sedation, dementia, or simple exhaustion, has no way to report pain. A patient in pain at 2am may wait until the morning round.
Pulse does not replace a carer. It tells a carer when to come. That is the whole ambition, and it is deliberately small: the gap we are trying to close is measured in minutes, not in staffing budgets. Forty-one seconds, not four hours.
Video in, a phone notification out.
One wide-angle camera, pointed at a bed from a bedside table. No wearables, no contact sensors, no microphone in the demo.
A push notification to one carer's phone: what was detected, how confident, and a still frame from the moment it fired.
A family carer or a night nurse covering several rooms. They set it down, point it at the bed, and otherwise ignore it.
Two branches, because in bed and out of bed aren't the same problem.
If the patient is in bed, the body is under a blanket. The face is the only signal there is, so that branch reads facial expression: a CNN with a ResNet18 backbone extracts features from each aligned face crop, and an LSTM reads 30-frame windows of those features to predict a PSPI pain score from 0 to 16. If the patient is out of bed, the body is visible instead, so that branch reads posture: a pose model that maps the body to 17 COCO keypoints and tracks them frame to frame. The pipeline runs live, end to end: camera, face detection and alignment, the CNN and LSTM pain score, pose, posture and movement, the alert engine, and the carer dashboard. Fall detection is the one part that is still experimental: a prototype behind a Settings toggle that is off by default.
-
A wide-angle frame every 33 milliseconds. Nothing leaves the device: frames are processed and discarded, and no video is stored or streamed.
-
A lightweight detector finds the face, aligns it and crops it out. If there is a face, the patient is in bed, which is the case the CNN and LSTM below are built for. If there is no face, that usually means the patient is out of bed, which is the pose model's case. We connected both detectors into one live pipeline that feeds the carer dashboard, including the handoff between them, and the alert engine downstream acts on what each one finds.
-
Trained, and running live in the dashboard. A convolutional network with a ResNet18 backbone turns each aligned face crop into 512 numbers describing the shape of the expression, not a picture of it: brow tension, eye closure, mouth shape, compressed into a vector. Training surfaced a class imbalance issue, which is resolved.
-
The CNN generates these vectors, one per frame. We built the step that stores them frame by frame for the live pipeline, and each one goes into a rolling buffer, because a single frame is a bad witness: people grimace when they sneeze, and a still face can hide sustained pain.
-
Trained, and running live in the dashboard. A recurrent model reads 30-frame windows of feature vectors, about one second of video, and predicts a PSPI pain score from 0 to 16: change over time, not a snapshot. While the window is still filling, the CNN's score is used on its own as a fallback.
-
This one runs live in the dashboard, with a demo video on the media page. For the out-of-bed case the model finds a body and maps it to 17 COCO keypoints, tracked frame to frame, and the dashboard shows posture and movement from them. The pose team has also built a prototype fall detector. It scores evidence such as downward motion, a change in orientation and a transition into lying. It is integrated behind a Settings toggle that is off by default, and it is not clinically validated.
-
Done, and running live. The alert engine turns pain, movement, posture and camera status into OK, CHECK and URGENT patient alarms, plus technical alarms for when the camera is lost or the light is too low. An alert stays on screen until a carer acknowledges it, the audio pauses once they do, and the alert clears once readings stay back in range. The thresholds are demo defaults set through a calibration step, not clinical thresholds.
Above threshold, and only above threshold, an alert goes out.
Below it, the room stays quiet. False alarms cost trust. A carer who is woken three times a night by a sneeze stops trusting the fourth alert, and the fourth alert is the one that matters. Everything about this decision is a trade.
Below are twenty illustrative confidence values, not measured ones. Seven are placed where a genuine-pain moment would sit. Move the threshold and watch what you gain and what you give up. In the working dashboard, the LSTM predicts a PSPI pain score from 0 to 16, and the thresholds are demo defaults set through a calibration step, not clinical thresholds.
A concept, not a product. We should say that plainly.
Right now Pulse is software running on a laptop with a webcam. The camera cube is in progress, and it is where we're heading: a Raspberry Pi 5, a wide camera, and a round display, in about 8cm of warm plastic on a bedside table. Nothing clipped to the patient, no wires across the bed.
CAD STAGE · NOT YET BUILTCamera Module 3 Wide, the only sensor the demo needs.
240 × 240 round LCD. It exists to hold the face.
I2S mic for audible distress, as a second opinion.
Room sensors for thermal, motion, distance and air quality. Planned, not built yet.
We train on a physiotherapy clinic and aim at a bedroom.
Our models are trained on a published clinical pain-expression dataset: video of adult patients in a clinical setting, with pain intensity coded frame by frame. It is licensed for non-commercial research and teaching only, and we use it under those terms.
The domain gap is real and we are not going to talk around it. That footage is seated adults, lit clinically, facing the camera, in acute pain provoked on purpose. Our target is a person lying down, in a dark room, at an oblique angle, possibly in chronic pain. We align faces before the CNN and augment with rotation and angle jitter. That narrows the gap. It does not close it, and no result from our demo should be read as if it did.
A concept, not a product. We should say that plainly.
- Not a certified medical device. It has no regulatory approval of any kind and is not intended for clinical use.
- Not clinically validated. Our numbers come from a research dataset, not from patients in a bedroom.
- Not a diagnosis. Pulse says a face looks like pain, at a confidence. A carer decides what that means.
- Not a replacement for a carer, and never a reason to check on someone less often.
- Not a reliable fall detector. Our fall detection is an unvalidated prototype, behind a Settings toggle that is off by default.
- Not built yet. The camera cube is still in progress, so the working prototype doesn't run on it. We're not pretending otherwise.
Ten weeks, counted from when we actually started. We're eight of them in.
-
Scope: falls, breathing, or pain
-
Face detection and patient detection (17 keypoints) both working
-
Dashboard prototype
-
Dataset access came through; indexed 48,398 frames and checked labels against the paper
-
CNN complete: 512-dimension feature vectors generated, class imbalance issue resolved
-
Wiring the pieces together: frame storage for the CNN pipeline, webcam integration, LSTM development, patient detection refinement
-
Integration: one live inference loop
-
WE ARE HERE
Showcase: live video in, alerts out
-
Threshold tuning, latency, failure cases
-
Write-up, handover, CAD refinement