Case Study
SLM or LLM? The More Interesting Question Came First.
Enterprise AI · AI Architecture · Decision Intelligence
An enterprise decision-intelligence platform was being designed around a simple question:
Should the platform use an SLM or an LLM?
Instead of immediately comparing models, I stepped back and looked at what the platform actually needed the AI to do.
Some tasks were relatively structured — interpreting a business request, identifying relevant information, discovering datasets and generating recommendations. Other tasks required deeper reasoning across business context, evidence, analytics and domain knowledge.
That immediately made "SLM vs LLM" a poor architectural question.
The better question was: which parts of the system actually need a large language model?
Two tasks that looked similar, and weren't
Consider two steps that sit right next to each other in the platform's workflow.
The first: understanding what a senior business user is actually asking for. These requests arrive open-ended, sometimes ambiguous, phrased however the person happens to phrase them — the platform's users weren't going to write careful prompts, and the system had to work with plain language. Getting this step wrong means everything downstream inherits the wrong problem. So this step was optimized for accuracy above cost, and pointed at a larger model that could handle genuine ambiguity.
The next step: matching that request against the right underlying data. This looks similar on the surface — it's still "understanding" something — but it's actually a semantic-matching problem, not a reasoning problem. It happens far more frequently, latency and cost matter more, and the range of things it needs to get right is narrower. A smaller, more focused model handled this step instead.
Same neighborhood in the workflow. Different job. Different model.
Most of the system didn't need a language model at all
Once the platform's components were mapped out task by task, a pattern showed up: the majority of them didn't need AI in any form. User interface, access control, data integration, workflow coordination, quality checks — these are deterministic problems, and adding a language model to them would have added cost and unpredictability without adding capability.
That reframed the whole exercise. The architecture stopped looking like one AI model driving everything, and started looking like this:
Each layer doing what it's actually good at, and most of the system doing what it always did: running deterministically, with AI reserved for the handful of places genuine ambiguity or reasoning showed up.
- Business context
- Knowledge
- Data
- Analytics
- SLMs
- LLMs
The interesting decision wasn't SLM vs LLM
It was deciding where intelligence belongs in the system — and, just as importantly, where it doesn't.
Decide where intelligence belongs
If an AI platform is forming and the first debate is which model to buy, start by mapping which tasks actually need a language model — and which don't.