Ask an LLM, or call a decision API?

The problem, in plain words

Look at the AI calls in a business app and most of them aren't conversations. They're decisions. Is this ticket urgent? Which team owns it? Is this message spam? How bad is the damage in this photo?

The habit is to send those to a chat model with a prompt that says "answer only with one of these labels". It works, mostly. Each answer takes seconds, costs a general model's price, and comes back as text you have to parse. And you can't put a threshold on it: there's no number that says how sure the model is, so you can't tell the confident answers from the guesses.

I hold one belief about this whole field: putting intelligence into software is about reliability. Until a new paradigm arrives, reliability is a function of intelligence, so the price of the frontier models sets the economics. Decision APIs like Jev, and the ones Cloudflare and OpenAI now offer, are the kind of paradigm shift I watch for: they buy reliability from the shape of the question, not from a bigger model.

The picture

One maintenance request at a time, and three things the code can do with the answer.

The options

A rule in code. If a regular expression, a lookup table or a number decides it ("amount over $10,000 needs approval"), write the rule. It's free, instant and explainable. It fails on wording it has never seen.

A decision model. You send the state (the ticket, the email, the screenshot) and typed questions: yes or no, as a probability; pick one of your labels; or a position on your scale. You get back probabilities and a calibrated confidence, in a few hundred milliseconds or less, for cents per million input tokens. It never writes text. Three exist so far: TypeSafe's Jev (early access, $0.042 per million input tokens), Cloudflare's open-weight Clef (Clef Flash at $0.038, output free) and OpenAI's Decisions API (generally available since 22 October 2026, $0.10, output free). All three take the same shape of request.

An LLM with a schema. A general model forced to answer in your JSON Schema. Since 20 October 2026, Claude on Bedrock compiles the schema into a grammar, so the response always parses, with your types and enum values. That fixes the parsing. It doesn't give you a probability to act on.

An LLM, prompted. The habit above: a prompt that begs for JSON, code to strip fences, a retry loop, and still the occasional missing field at 3 am.

A concrete scenario

A strata manager receives 40,000 maintenance requests a month, by email and web form. Three questions on each: urgent or not (a burst pipe at 2 am, or a squeaky door), which trade, and how severe.

On Sonnet 5.5 with a prompt, at about 800 tokens in and 150 out per request, that's roughly $124 a month. On Clef Flash, about $1.22, by my arithmetic from the list prices. At this volume the price doesn't decide anything. What decides it is what the code can do with the answer, as the picture shows: page the plumber above the threshold, send the middle band to a frontier model with the building's history, and leave the rest for a person in the morning. Thresholds per question, set from a labelled sample of last month's requests. A prose answer can't support that design.

Now the same pattern at a marketplace moderating 50 million chat messages a month, about 300 tokens each: 15 billion tokens. Sonnet 5.5 costs about $30,000 a month on input alone; Clef Flash about $570. Now the price is prohibitive too, and it argues for the decision model as well.

The trade-offs

The rule of thumb

If the answer is one of a fixed set, don't ask for prose. If a rule can decide it, write the rule. If it needs judgement, ask a decision model and act above a threshold. If the answer must be text, or the decision needs investigation, that's an LLM or an agent. Whatever you pick, the unsure cases go up a level, never through. In the weekly's words: a confidence score you don't act on is wasted.

When the answer changes with scale

• Low volume: price is noise. Choose by where the data may go. For an Australian client today, I'd start where I start for document extraction: Claude through Bedrock's AU profile, a schema, and the business rules in code.
• High volume: 10 million tokens of support tickets cost about $1 at OpenAI's price. Billions of tokens on a frontier model don't.
• Higher stakes: raise the threshold and send more to a person. The model doesn't change.
• The frontier moves. When frontier prices drop, the escalation path gets cheaper; when a new paradigm lands, the floor moves. Re-run your labelled sample each time.

Before you build on it

The category is weeks old, so I'm watching, not defaulting. Start with Clef Flash: open weights are your exit. It runs on Workers AI, which is a Skip for general model calls; Clef is the exception, because that's where Cloudflare serves it. Use OpenAI's if the app already runs on OpenAI and needs zero data retention or a US or European region. Keep Jev to experiments: it's early access, and a third-party report on 24 October said a leak exposed every Jev customer's outputs (the weekly). Don't send it personal data until TypeSafe explains. Pin a named model version, never a "latest" alias, and put every call behind your own yes-or-no, choice and score interface: the three share a request shape, so switching is a thin adapter.