The technical story here is unremarkable, quantisation improved, memory bandwidth improved, small models got better. The interesting story is that people started asking for it.

Demand arrived before the capability did

Surveys of consumer attitudes to AI features have shown the same pattern for two years: high willingness to use assistance on generic tasks, sharp refusal on anything involving messages, photographs, health records, or finances. Vendors read this as a messaging problem. It was a processing-location problem.

The economic consequence

Cloud inference is a recurring cost that supports a recurring price. On-device inference is a fixed cost recovered once, at the point of hardware sale.

A vendor that sells devices is happy with that trade. A vendor that sells subscriptions is not. Watch which companies emphasise local processing in their marketing over the next year, it maps almost perfectly onto which of them make money from hardware.