On-Device AI Is a Feature, Not a Compromise

Two of our products do machine vision. RootMap finds and groups the faces in a photo library. LookoutNVR watches camera feeds and picks out people, vehicles, and animals.

Both run the model on the user's own hardware. Neither ships an image anywhere.

That is not the easy path, and we want to be straight about what it costs before arguing for it.

What it costs

Speed on the first pass. A datacenter GPU will chew through a twenty thousand photo library faster than the machine under your desk. Our first index takes as long as it takes.

Model size. We are bounded by what runs acceptably on ordinary consumer hardware. The frontier models are not an option, so we work with smaller ones and accept the accuracy that comes with them.

No free improvements. A cloud service quietly swaps in a better model and every customer benefits on Tuesday. We ship an update and wait for people to install it.

Support surface. Every machine is a different machine. Cloud inference is uniform by construction. Ours is not.

Those are genuine disadvantages. Anyone who tells you local inference is strictly better is selling something.

Why we do it anyway

Because of what the data is.

RootMap's input is a family photo library. Not stock images. Your kids, your house, the inside of your home, everyone you know, with timestamps and coordinates attached. Face detection over that produces something even more sensitive than the photos: a biometric index of the people in your life.

LookoutNVR's input is continuous video of the inside and outside of a building, running around the clock.

For either of those, "we take security seriously" is not an argument. The strongest possible statement about a dataset is that it never left the building. Every other promise depends on a company's competence, its retention policy, its acquisition, its subpoena response, and its breach history. The bytes staying on your drive does not depend on any of that.

There is a version of this decision that is only marketing. We do not think this one is, because it is enforced by architecture rather than policy. There is no upload path to misconfigure.

The economics nobody mentions

The unglamorous half of this is cost.

Cloud inference is a per-frame, per-image, forever expense. A product that analyzes every frame from an unlimited number of cameras has an operating cost that scales with how much its customers use it, which is why the market is full of camera subscriptions priced per camera per month.

LookoutNVR supports unlimited cameras. We can say that because the compute is the user's, and their electricity bill does not scale as our variable cost. The pricing model people actually want falls out of the architecture.

Same with RootMap. Detection on our servers would give us a bill that grows with library size, and the only ways to cover it are a subscription or advertising against the data. Running it locally is why the app can be free and stay free without either.

Where the line is

We are not zealots. Plenty of work genuinely belongs in the cloud, and we build cloud software for clients all week. Coordination, sync, collaboration, and anything needing a model far larger than a desktop can hold all belong on a server.

The rule we use: if the input is a continuous, intimate record of somebody's private life, the processing goes where the data already is. Photos of your family and video of your home clear that bar easily.

The honest summary is that local inference is slower, smaller, and harder to support, and for these two products it is worth every bit of that.