AI · Explainer
Small models are not a compromise. They are a different product.
The case for running a two billion parameter model on a laptop or a factory controller has very little to do with saving money, and a great deal to do with latency and control.
Independent coverage
Published 13 September 2026
6 min read
Evidence: Reporting
Small language models are usually introduced as the budget option, which undersells them. The interesting property is not price. It is that they run where the data already is.
On a laptop, a handheld scanner, a clinical cart or a machine controller, the round trip to a datacentre is the expensive part of the transaction. Removing it changes the interaction from a request to a response.
What they are good at
Classification, extraction, routing, structured rewriting, intent detection, and narrow domain question answering after fine tuning. These are unglamorous and they are the majority of production language work.
What they are not good at
Open ended reasoning, long context synthesis, and anything where the correct answer requires knowledge the model was never given. Fine tuning does not fix a knowledge gap. Retrieval does.
The practical test
Take fifty real inputs from the workflow in question. Run them through a small model and a frontier model. Count disagreements, then read only the disagreements. In most narrow workflows the count is low enough that the small model wins on every other axis.
"A model that answers in forty milliseconds on a device changes what you can build. A better model that answers in two seconds does not."
Sources