Putting AI into software

The decisions, in order

1. 01 Ask an LLM, or call a decision API? When my app needs a yes, a no, a label or a score, should it prompt a general model or call a model built to decide? Most AI calls in business apps are decisions: urgent or not, which team, how severe. A decision model (Jev, Cloudflare's Clef, OpenAI's Decisions API) takes your data and typed questions and returns probabilities with a confidence, in milliseconds, for cents per million tokens, and never writes text. Your code acts above a threshold and sends the rest to a bigger model or a person. The category is weeks old, so I'm watching it, with an adapter in front. →
2. 02 System 1 or System 2? Which steps of an AI feature should be fast and cheap, which should think, and how does one hand over to the other? System 1 is fast and cheap: code, a decision model or a small model, one call, at volume. System 2 is slow and deliberate: a top model at high reasoning effort, or an agent that loops with tools. Route per step, not per app: start each step on a top model, move it to the fast path once a replay of real cases proves the fast path does as well, and from then on everything takes the fast path first, checks in code catch what it got wrong, and only those cases climb. A person keeps the last word where the stakes or the law require it. →
3. 03 How much freedom should an AI step get? Should each AI step act on its own, ask someone first, or not be AI at all, and what decides it? Models in late 2026 are neither untrustworthy nor made deterministic by a good harness: they're highly reliable, not perfect. Design for exactly that, one step at a time. Three questions set a step's freedom: can code verify the result (then let it loop until the checks pass, within a number of attempts); if not, who can judge it (ideally the user, consenting at each critical fork, with no support person in between); and is a wrong result critical and impossible to undo (then it isn't a job for AI yet). Whatever does the work shouldn't be what checks it: checks in code first, then other agents with a different job. →
4. 04 How should a model get your company's knowledge? Do you put your documents in the prompt, build retrieval, fine-tune a model, or let it find what it needs? Knowledge isn't a system prompt. The system prompt is always loaded, because it sets how the agent thinks; knowledge depends on the need, and no situation needs all of it. So understand the need first, then disclose progressively: an entry point (a README of at most 750 lines, or an llms.txt map) whose links each say what's behind them, followed only where the question leads. The same goes beyond documents: some specific emails, some rows, some passages found by meaning. Retrieval is one source the agent can query, not the strategy. Cache what's always read, check with real questions that it finds the right source, and leave fine-tuning for format. →

If you chose this, use this

A decision with a fixed set of answers

A rule in code, or a decision model behind your own interface; Claude with a schema on Bedrock when the data must stay in Australia

A fast path at volume

Opus 5.5 at medium by default; a small model through OpenRouter once a replay proves it does as well, checked by rules in code: Which model, at what price

A deliberate step or an agent

A top model at high effort, or an agent that waits in a durable function, with a person for what's risky

A step code can verify

Let it loop until the checks pass, within a set number of attempts: Claude Code with tests and Stryker: How much freedom should an AI step get?

Knowledge for a model or an agent

An entry point whose links say what's behind them, Markdown copies, and pgvector for search by meaning: How should a model get your company's knowledge?