# Ask an LLM, or call a decision API?

> When my app needs a yes, a no, a label or a score, should it prompt a general model or call a model built to decide? Most AI calls in business apps are decisions: urgent or not, which team, how severe. A decision model (Jev, Cloudflare's Clef, OpenAI's Decisions API) takes your data and typed questions and returns probabilities with a confidence, in milliseconds, for cents per million tokens, and never writes text. Your code acts above a threshold and sends the rest to a bigger model or a person. The category is weeks old, so I'm watching it, with an adapter in front.

- A concept by Nicolas Dao (Cloudless Labs, Sydney): the decision before the tool
- Area: [Putting AI into software](https://cloudlesslabs.com/concepts/areas/ai/index.md)
- Last reviewed: 2 November 2026, by Nicolas Dao
- Web page: https://cloudlesslabs.com/concepts/llm-or-decision-api/

## What you'll be able to decide

For each AI call in your app, whether it is a decision or a piece of writing; whether code, a decision model, an LLM with a schema or an agent should make it; and where the unsure cases go.

## In short

### The options

- **A. A rule in code:** If a regular expression, a lookup or a threshold decides it, nothing smarter is needed.
- **B. A decision model:** Typed questions in (yes or no, pick one, score), probabilities and a confidence out. It never writes text. Tools: [Baseten decision models](https://cloudlesslabs.com/stack/baseten-decision-models/index.md) (Use when it fits), [Jev](https://cloudlesslabs.com/stack/typesafe-jev/index.md) (Watching), [Cloudflare Workers AI](https://cloudlesslabs.com/stack/cloudflare-workers-ai/index.md) (Skip).
- **C. An LLM with a schema:** A general model forced to answer in your JSON Schema: typed fields that always parse, but no probability to act on. Tools: [Amazon Bedrock](https://cloudlesslabs.com/stack/aws-bedrock/index.md) (Watching), [Anthropic API (for client systems)](https://cloudlesslabs.com/stack/anthropic-api/index.md) (Skip), [OpenRouter](https://cloudlesslabs.com/stack/openrouter/index.md) (My default).
- **D. An LLM, prompted:** A general model asked to decide in prose: slow, and you parse what comes back. Tools: [OpenRouter](https://cloudlesslabs.com/stack/openrouter/index.md) (My default).

### The rule of thumb

A confidence score you don't act on is wasted. Set thresholds per question, and route low-confidence answers to a frontier model or a person. From [The Stack Weekly, week 43: Decision models (Jev, and Cloudflare's Clef)](https://cloudlesslabs.com/weekly/2026-w43/index.md).

### How the answer changes with scale

- **A few thousand decisions a month:** Price is noise. Choose by where the data may go and what you already run.
- **Billions of tokens a month:** A frontier model is prohibitive; a decision model or a small model with a schema is not.
- **A wrong answer costs more:** Raise the threshold and send more to a person; the model stays the same.

## The problem, in plain words

Look at the AI calls in a business app and most of them aren't conversations. They're decisions. Is this ticket urgent? Which team owns it? Is this message spam? How bad is the damage in this photo?

The habit is to send those to a chat model with a prompt that says "answer only with one of these labels". It works, mostly. Each answer takes seconds, costs a general model's price, and comes back as text you have to parse. And you can't put a threshold on it: there's no number that says how sure the model is, so you can't tell the confident answers from the guesses.

I hold one belief about this whole field: **putting intelligence into software is about reliability.** Until a new paradigm arrives, reliability is a function of intelligence, so the price of the frontier models sets the economics. Decision APIs like Jev, and the ones Cloudflare and OpenAI now offer, are the kind of paradigm shift I watch for: they buy reliability from the shape of the question, not from a bigger model.

## The picture

One maintenance request at a time, and three things the code can do with the answer.

**Diagram:** A maintenance request goes to a decision model, which answers with probabilities; above the threshold the code acts at once, in the middle band a frontier model reasons for the duty manager, and below it the request waits for a person.

- **Confident:** "Water pouring through the ceiling of unit 4" → Decision model → urgent: 0.97 → Page the after-hours plumber. Milliseconds, no person in between.
- **Unsure:** "The tap in unit 9 drips sometimes" → Decision model → urgent: 0.70 → A frontier model, with the building's history → Its reasoning, for the duty manager. Seconds, at a frontier model's price, for the few per cent in the middle band.
- **Not urgent:** "The garage door squeaks" → Decision model → urgent: 0.12 → The morning queue, for a person.

## The options

**A rule in code.** If a regular expression, a lookup table or a number decides it ("amount over $10,000 needs approval"), write the rule. It's free, instant and explainable. It fails on wording it has never seen.

**A decision model.** You send the state (the ticket, the email, the screenshot) and typed questions: yes or no, as a probability; pick one of your labels; or a position on your scale. You get back probabilities and a calibrated confidence, in a few hundred milliseconds or less, for cents per million input tokens. It never writes text. Three exist so far: TypeSafe's **Jev** (early access, $0.042 per million input tokens), Cloudflare's open-weight **Clef** (Clef Flash at $0.038, output free) and **OpenAI's Decisions API** (generally available since 22 October 2026, $0.10, output free). All three take the same shape of request.

**An LLM with a schema.** A general model forced to answer in your JSON Schema. Since 20 October 2026, Claude on Bedrock compiles the schema into a grammar, so the response always parses, with your types and enum values. That fixes the parsing. It doesn't give you a probability to act on.

**An LLM, prompted.** The habit above: a prompt that begs for JSON, code to strip fences, a retry loop, and still the occasional missing field at 3 am.

## A concrete scenario

A strata manager receives 40,000 maintenance requests a month, by email and web form. Three questions on each: urgent or not (a burst pipe at 2 am, or a squeaky door), which trade, and how severe.

On Sonnet 5.5 with a prompt, at about 800 tokens in and 150 out per request, that's roughly $124 a month. On Clef Flash, about $1.22, by my arithmetic from the list prices. At this volume the price doesn't decide anything. What decides it is what the code can do with the answer, as the picture shows: page the plumber above the threshold, send the middle band to a frontier model with the building's history, and leave the rest for a person in the morning. Thresholds per question, set from a labelled sample of last month's requests. A prose answer can't support that design.

Now the same pattern at a marketplace moderating 50 million chat messages a month, about 300 tokens each: 15 billion tokens. Sonnet 5.5 costs about $30,000 a month on input alone; Clef Flash about $570. Now the price is prohibitive too, and it argues for the decision model as well.

## The trade-offs

| | A rule in code | A decision model | An LLM with a schema | An LLM, prompted |
|---|---|---|---|---|
| What comes back | Your rule's answer | Typed answers with probabilities | Typed fields that always parse | Text you parse |
| A confidence you can threshold | No need: it's exact | Yes | No | No |
| Speed | Instant | Clef 38.8 ms median, Jev 524 ms (vendor tests) | Seconds | Seconds |
| Price per million input tokens | Free | $0.038–0.10, output free | Haiku 5.5 $0.10, Sonnet 5.5 $2 | The same |
| New wording | Breaks | Handles it | Handles it | Handles it |
| Data in Australia | Yes | None confirmed; OpenAI's regional processing covers only the US and Europe | Claude on Bedrock's AU profile, 10% more | The same |
| Maturity | Yours | Weeks old: watching | Generally available | Mature |

## The rule of thumb

**If the answer is one of a fixed set, don't ask for prose.** If a rule can decide it, write the rule. If it needs judgement, ask a decision model and act above a threshold. If the answer must be text, or the decision needs investigation, that's an LLM or an agent. Whatever you pick, the unsure cases go up a level, never through. In the weekly's words: a confidence score you don't act on is wasted.

## When the answer changes with scale

- **Low volume:** price is noise. Choose by where the data may go. For an Australian client today, I'd start where I start for document extraction: Claude through Bedrock's AU profile, a schema, and the business rules in code.
- **High volume:** 10 million tokens of support tickets cost about $1 at OpenAI's price. Billions of tokens on a frontier model don't.
- **Higher stakes:** raise the threshold and send more to a person. The model doesn't change.
- **The frontier moves.** When frontier prices drop, the escalation path gets cheaper; when a new paradigm lands, the floor moves. Re-run your labelled sample each time.

## Before you build on it

The category is weeks old, so I'm **watching**, not defaulting. Start with Clef Flash: open weights are your exit. It runs on [Workers AI](https://cloudlesslabs.com/stack/cloudflare-workers-ai/index.md), which is a Skip for general model calls; Clef is the exception, because that's where Cloudflare serves it. Use OpenAI's if the app already runs on OpenAI and needs zero data retention or a US or European region. Keep Jev to experiments: it's early access, and a third-party report on 24 October said a leak exposed every Jev customer's outputs ([the weekly](https://cloudlesslabs.com/weekly/2026-w44/index.md)). Don't send it personal data until TypeSafe explains. Pin a named model version, never a "latest" alias, and put every call behind your own yes-or-no, choice and score interface: the three share a request shape, so switching is a thin adapter.

## Questions that change the answer

- Is the answer one of a fixed set (yes or no, a label, a score)? Then it's a decision, not a piece of writing.
- Must the data stay in Australia? None of the three decision APIs is confirmed there; Claude on Bedrock's AU profile is.
- What does a wrong answer cost? That sets the threshold, not the model.

Doesn't cover:

- Writing replies, summaries and drafts: that's generation, and [Which model, at what price](https://cloudlesslabs.com/stack/guides/which-model/index.md) covers it.
- Picking between the decision models themselves: that's [Which decision model?](https://cloudlesslabs.com/stack/guides/which-decision-model/index.md).
- Decisions that need investigating across systems: that's an agent, in [System 1 or System 2?](https://cloudlesslabs.com/concepts/system-1-or-system-2/index.md).

## Check your call

- **A form asks "is this a complaint?" of about 2,000 messages a month, and the data must stay in Australia.** Claude through Bedrock's AU profile, with a schema (a yes or no and a category). Price is noise at this volume, and none of the decision APIs is confirmed in Australia.
- **A chat app must hide abusive messages before anyone sees them: 30 million messages a month.** A decision model with a threshold: hide above it, hold the middle band for a person, show the rest. At that volume a frontier model is prohibitive, and the answer has to come in milliseconds.
- **Customers ask for refunds by email, and the app must reply with the decision and the reason.** Two calls. The refund or not is a decision: a decision model or a rule. The reply is writing: an LLM, given the decision.

## The technology on this site

- [Which decision model?](https://cloudlesslabs.com/stack/guides/which-decision-model/index.md): the decision guide
- [Which model, at what price](https://cloudlesslabs.com/stack/guides/which-model/index.md): the decision guide
- [Cloudflare AI Gateway, OpenRouter or LiteLLM?](https://cloudlesslabs.com/stack/guides/ai-gateway-choice/index.md): the decision guide
- [Anthropic API, Bedrock or OpenRouter?](https://cloudlesslabs.com/stack/guides/anthropic-bedrock-or-openrouter/index.md): the decision guide
- [An AI chat or LLM feature](https://cloudlesslabs.com/stack/use-cases/ai-chat/index.md): the use case
- [Week 43: Decision models (Jev, and Cloudflare's Clef)](https://cloudlesslabs.com/weekly/2026-w43/index.md): in The Stack Weekly
- [Week 44: OpenAI Decisions API](https://cloudlesslabs.com/weekly/2026-w44/index.md): in The Stack Weekly
- [Week 44: Structured outputs for Claude on Amazon Bedrock](https://cloudlesslabs.com/weekly/2026-w44/index.md): in The Stack Weekly

## Related

- [System 1 or System 2?](https://cloudlesslabs.com/concepts/system-1-or-system-2/index.md): read next
- [Putting AI into software](https://cloudlesslabs.com/concepts/areas/ai/index.md): the area, and the technology for each answer
- [Every concept](https://cloudlesslabs.com/concepts/index.md)
- [Map of every Markdown file on cloudlesslabs.com, with what each one answers](https://ask.cloudlesslabs.com)