004building

Tend

A garden you grow by moving. Chair exercises for older adults, with a camera that only checks that you're moving.

pose detection, on-device ML, signal processing, generative art pipeline
health & mobility, games · TypeScript, MediaPipe Pose, Vite, Vitest, FLUX + custom LoRA

Inspiration

I've wanted to build something in the social impact space, and Kellogg has pushed me further that way. The people I kept thinking about were my parents. Mobility matters more every year, and the simplest version of it, moving a little each day, is the version nobody's app is built for.

So the idea got small on purpose. Exercises you can do in a chair, a camera that notices you're moving, and something that grows because you did. Not a fitness app. A garden.

The gardener. Generated with a style trained on 19th-century seed catalogs.

What it is

Tend is for someone in her seventies with an iPad propped across the room. She opens it into her garden, sits, and a voice names the first exercise while a painted gardener demonstrates beside her silhouette. As she moves, a vine grows. When she's done, something blooms. She can skip anything by crossing her arms, and nothing happens if she doesn't come back tomorrow.

No streaks, no countdowns, nothing wilts, and the app never says she did it wrong. The research was clear that time pressure and loss-framed streaks are what older adults abandon apps over, and a camera that grades your form has nothing honest to grade against.

hover to play
My own recording, rendered from the saved landmarks. Overhead arm raise.

How it works

The camera runs a pose model on the device, 33 body landmarks thirty times a second, and no video is ever stored. For each exercise the engine reduces those points to one number per frame, like wrist height against the shoulders, normalized to the person's own torso so distance doesn't matter. Each exercise is then one of three shapes: a threshold you cross and hold, a position you keep, or a slow regular oscillation. Every threshold is a fraction of that person's own calibration. Eighteen exercises have detectors, and the timer only advances while the movement is happening. That timer is the product.

hover to play
Seated cat-cow. The detector looks for a slow, regular rhythm, not a shape.

The social part is a second garden. A son or granddaughter gets a small kingdom next to hers, sees her progress only as growth, never as numbers, and can plant a flower that her sessions tend. When both have been active the same week, fireflies come out.

The partner's garden. They can add and react, never remove.

The hardest problem

Deciding what not to measure. The obvious product checks your form and tells you to reach higher. I wrote down why that's wrong before writing code: there's no shared standard for correct form across bodies and ages, a false "correct" is an injury risk, and "is this movement happening" is the only question a camera can answer honestly. Everything kind about the app follows from that decision.

hover to play
Seated march. Legs are the worst-tracked joints, so this one leans on rhythm.

What broke

The first detectors fired on fidgeting. I recorded myself just shifting in the chair and standing up, and the shoulder-roll detector counted twenty seconds of it as exercise. The fix was rhythm consistency: exercise is regular, fidgeting isn't, so each swing has to look like the last one. That cut the false time to about a second.

The art model fought motion, too. Trained on pressed flowers, it kept turning dynamic poses into dignified portraits and growing flowers out of the watering can.

One plant, five stages. A session of any length is one watering.

What's next

The engine is done enough. The app isn't started. It'll be native and iPad-first. Nobody over seventy has used it yet, and every principle above comes from the literature until they do.

what I learned

The hardest design decision was what not to measure. Once the camera stopped judging form, everything else got kinder.