How much freedom should an AI step get?

The problem, in plain words

There are two camps on AI in late 2026, and both are wrong. One says the models are flawed and nothing they do can be trusted. The other says that with the right harness, an agent becomes 100% deterministic. Neither is true. The models are highly reliable, and not perfect. The job is to design for exactly that, and for nothing else.

That's why "workflow or agent?" is the wrong question. It treats the choice as black or white: either code fixes every step, or the model chooses them all. Real features have both, and the decision that matters is made one step at a time: how much freedom does this step get, and who checks it?

Three questions answer it. Can code verify the result? If not, who can judge it? And if it's wrong, can it be undone?

The picture

Three steps, three amounts of freedom, and the checks around the first one.

The options

Loop until the checks pass. Some work is verifiable. Code either passes its tests or it doesn't; data matches its schema or it doesn't; totals reconcile or they don't. Whole domains are like that, and so are parts of others. There, the AI can loop until it reaches a result the checks accept, and that loop is what the harness is for: the checks, a limit on the number of attempts, and a spending cap on the key as the backstop. My Claude Code entry says it in one line: an agent with no tests doesn't know when it's done. An essay or a design isn't verifiable in that way, so it needs another answer.

The user consents at each critical fork. When code can't judge the result, the person it's for usually can. The best design I know lets the AI do everything it can, shows the user what it did, and asks for their consent at each fork that matters. It's neither a fixed script nor an agent left alone. It also puts the user in control of their own outcome, instead of hiding the work behind a support person who passes messages along badly. Claude Code works this way: it does the work and stops before the risky actions. So does how I work on HappySkills: the agent does nearly everything, every test has to pass, and I review the outcome. When something slips through, I report it and ask the agent why its checks didn't find it, and we add what was missing.

It has limits, and they decide when to move to the next option:

• The user can't judge it. A patient can't check medical advice, and their consent doesn't make it right.
• Approval fatigue. After twenty approvals, people click yes without reading, so a task needs a few well-chosen forks, not one per step.
• It lands on someone else. The user approves, and another person bears the result: a message to their customer, a colleague's data.
• Someone must be accountable. In regulated work, a named person has to sign off, whatever the user clicked.

A named person signs off. That's the answer when one of those limits applies. The AI still prepares everything; a person who can judge it, and who answers for it, decides.

Plain code, not AI yet. Some results are critical and can't be undone. A refund that's been paid is gone, so I wouldn't put AI in that step at all. Determinism is the requirement there, and only code gives it. The AI can prepare the case; it doesn't release the money.

A concrete scenario

Background agents that fix production bugs are the clearest case of a verifiable loop. They watch the logs. When something breaks, they file a bug report, send the notifications, read the source code to find the cause, and when the fix is contained, merge and deploy it. This is the design I build to:

1. Intentional error traces. Every failure carries the message the code meant it to carry, through catchErrors in my open-source puffy-core package. A trace precise enough to read is a trace precise enough to reproduce.
2. Triage first. Is it a bug in the code, or an outage, a throttle or an expired key? Only a bug gets a fix.
3. Reproduce it with a test before fixing anything. If the agent can't make the error happen in a test, it doesn't get to claim it fixed it.
4. The fix passes every test, unit and end-to-end. Aim for complete coverage. Mutation testing with Stryker runs on the fix when it can finish within about 10 minutes; otherwise it waits for a weekly night run, because on a big codebase a full run takes five to ten hours.
5. A contained blast radius, decided by a script. A change to the database (every database versioned with migration scripts) or to a connector other systems use stops the automation and goes to a person.
6. Test environment, then production, with the end-to-end tests run again in each.

Around that loop sits what I call my super harness: a harness of my own on top of the existing ones. Three parts of it matter here:

• Reviewer agents with a different job (next section).
• A rollback agent. It writes the rollback procedure for each deployment, and nothing deploys without its approval. It rolls back on errors and confirms the errors stopped, and it hands over to a person when a rollback isn't possible or doesn't work.
• A screen in front of the logs. Logs carry whatever users typed, so text written to fool an agent could reach one. A cheap decision model screens them for prompt injection before any agent reads them: d1-ultra, at $0.042 per million tokens, after personal data is removed, because its data terms aren't documented. It's one layer, not the defence: in the one independent test, a planted answer still steered it 28% of the time (Which decision model?).

Whatever does the work shouldn't check it

An agent told to ship is rewarded for shipping. Its incentive is the merge, not the stability of everything around it, so it shouldn't grade its own work. Banks solved this decades ago with maker-checker: one person makes the change, another checks it. With agents, the checkers are other agents, each given a different job and the incentive that comes with it:

Four rules make the checkers worth having:

• Code checks what code can. If an agent edits an old test so its fix goes green, a script that refuses edited tests catches it every time; a reviewer catches it most of the time. Where a test really has to change, another agent questions the one that changed it before anything merges.
• Checkers read the evidence, not the maker's summary: the diff, the trace, the test results. A checker from another vendor's model shares fewer blind spots.
• Permissions beat instructions. In the design I recommend, the maker's credentials can push a branch and nothing more, and only the pipeline merges, once the checks and the checkers agree.
• Checkers get a limit too. Told to find problems, a checker always finds some. Disagreements that outlast the attempts go to a person.

The trade-offs

The rule of thumb

Design for highly reliable, not perfect. Let code verify what it can, let the user judge what it can't, and keep AI away from what can't be undone. Then give the checking to something other than what did the work.

When the answer changes with scale

• Your own product, early: as much freedom as the tests allow. Review the outcome in production, and turn every miss into a new check.
• Paying users who depend on it: checks the maker can't edit, a second agent with a different job, and a rollback written before every deployment.
• Regulated work, or decisions about people outside the loop: a named person signs off what's critical.

Models, limits and the loop

• The model: Opus 5.5 at medium effort, by default. For standard engineering (database changes, test-driven development) it's more than capable. I raise the effort only for novel, creative work, like researching a way of doing something nobody has documented. Saving on the model is penny-wise and pound-foolish when the cost of failing is that much higher: a cheaper model takes over a step only when a replay of real cases proves it does just as well (Which model, at what price).
• The limits: a number of attempts per task, with a spending cap on the key as the backstop for the run that goes wrong in a way the count doesn't see.
• The loop: don't rewrite it. An SDK (the Claude Agent SDK, the OpenAI Agents SDK) gives you the loop and its primitives. Then build your own layer for your domain on top, because no SDK knows your domain.
• Where it runs: the waits and retries of the whole job in a durable function (SQS retries or Durable Functions?), and the agent's loop inside one step, with its own time limit.