# How much freedom should an AI step get?

> Should each AI step act on its own, ask someone first, or not be AI at all, and what decides it? Models in late 2026 are neither untrustworthy nor made deterministic by a good harness: they're highly reliable, not perfect. Design for exactly that, one step at a time. Three questions set a step's freedom: can code verify the result (then let it loop until the checks pass, within a number of attempts); if not, who can judge it (ideally the user, consenting at each critical fork, with no support person in between); and is a wrong result critical and impossible to undo (then it isn't a job for AI yet). Whatever does the work shouldn't be what checks it: checks in code first, then other agents with a different job.

- A concept by Nicolas Dao (Cloudless Labs, Sydney): the decision before the tool
- Area: [Putting AI into software](https://cloudlesslabs.com/concepts/areas/ai/index.md)
- Last reviewed: 9 November 2026, by Nicolas Dao
- Web page: https://cloudlesslabs.com/concepts/ai-autonomy/

## What you'll be able to decide

How much freedom each AI step gets: loop on its own until checks in code pass, act with the user's consent at each critical fork, wait for a named person, or stay plain code; and why whatever does the work shouldn't also be what checks it.

## In short

### The options

- **A. Loop until the checks pass:** Code can verify the result: tests, a schema, totals that must reconcile. The AI tries, the checks answer, and it tries again, within a set number of attempts. Tools: [Claude Code](https://cloudlesslabs.com/stack/claude-code/index.md) (My default), [Stryker mutation testing](https://cloudlesslabs.com/stack/stryker/index.md) (My default).
- **B. The user consents at each critical fork:** Code can't verify it, but the person it's for can judge it. The AI does all it can, shows its work, and asks before each step that matters. Tools: [Claude Code](https://cloudlesslabs.com/stack/claude-code/index.md) (My default), [HappySkills](https://cloudlesslabs.com/stack/happyskills/index.md) (My default).
- **C. A named person signs off:** The user can't judge it, the result lands on someone else, or the rules need someone accountable.
- **D. Plain code, not AI yet:** A wrong result is critical and can't be undone, like money paid out. That needs determinism, and only code gives it.

### The rule of thumb

Design for highly reliable, not perfect: let code verify what it can, let the user judge what it can't, and keep AI away from what can't be undone.

### How the answer changes with scale

- **Your own product, early:** As much freedom as the tests allow. Review the outcome in production, and when something slips through, ask the agent why its checks missed it, then add the check.
- **Paying users who depend on it:** Checks the agent can't edit, a second agent with a different job, and a rollback ready before every deployment.
- **Regulated work, or decisions about people outside the loop:** A named person signs off what's critical, whatever the user clicked.

## The problem, in plain words

There are two camps on AI in late 2026, and both are wrong. One says the models are flawed and nothing they do can be trusted. The other says that with the right harness, an agent becomes 100% deterministic. Neither is true. The models are highly reliable, and not perfect. The job is to design for exactly that, and for nothing else.

That's why "workflow or agent?" is the wrong question. It treats the choice as black or white: either code fixes every step, or the model chooses them all. Real features have both, and the decision that matters is made one step at a time: how much freedom does this step get, and who checks it?

Three questions answer it. Can code verify the result? If not, who can judge it? And if it's wrong, can it be undone?

## The picture

Three steps, three amounts of freedom, and the checks around the first one.

**Diagram:** Four lanes. A bug fix that code can verify: a trace, a test that reproduces it, the fix, tests again, merge when they pass. A draft only the user can judge: the AI drafts, the user adjusts and consents at the fork, then the AI acts. A refund that can't be undone: a rule in code, then the payment, with no AI in the path. And the checks around it: a maker writes the fix and three checkers, a verifier, a steward and an auditor, each check it before the merge.

- **Code can verify it, a bug fix:** An error, with an intentional trace → A test that reproduces it → Fix, then every test again → Merge when the checks pass. Within a set number of attempts. When they run out, a person.
- **Only the user can judge it, a draft:** The AI drafts → The user adjusts → Consent at the fork → The AI acts. As autonomous as possible, with no support person in between.
- **It can't be undone, a refund:** The refund rule, in code → Money paid out, for good. No AI in the path until a wrong answer can be taken back.
- **The maker and its checkers:** Maker: writes the fix → [Verifier: does it do what it claims? / Steward: is the whole still stable? / Auditor: what could it expose?] → Merge. Whatever does the work isn't what checks it.

## The options

**Loop until the checks pass.** Some work is verifiable. Code either passes its tests or it doesn't; data matches its schema or it doesn't; totals reconcile or they don't. Whole domains are like that, and so are parts of others. There, the AI can loop until it reaches a result the checks accept, and that loop is what the harness is for: the checks, a limit on the number of attempts, and a spending cap on the key as the backstop. My [Claude Code](https://cloudlesslabs.com/stack/claude-code/index.md) entry says it in one line: an agent with no tests doesn't know when it's done. An essay or a design isn't verifiable in that way, so it needs another answer.

**The user consents at each critical fork.** When code can't judge the result, the person it's for usually can. The best design I know lets the AI do everything it can, shows the user what it did, and asks for their consent at each fork that matters. It's neither a fixed script nor an agent left alone. It also puts the user in control of their own outcome, instead of hiding the work behind a support person who passes messages along badly. Claude Code works this way: it does the work and stops before the risky actions. So does how I work on [HappySkills](https://cloudlesslabs.com/work/happyskills/index.md): the agent does nearly everything, every test has to pass, and I review the outcome. When something slips through, I report it and ask the agent why its checks didn't find it, and we add what was missing.

It has limits, and they decide when to move to the next option:

- **The user can't judge it.** A patient can't check medical advice, and their consent doesn't make it right.
- **Approval fatigue.** After twenty approvals, people click yes without reading, so a task needs a few well-chosen forks, not one per step.
- **It lands on someone else.** The user approves, and another person bears the result: a message to their customer, a colleague's data.
- **Someone must be accountable.** In regulated work, a named person has to sign off, whatever the user clicked.

**A named person signs off.** That's the answer when one of those limits applies. The AI still prepares everything; a person who can judge it, and who answers for it, decides.

**Plain code, not AI yet.** Some results are critical and can't be undone. A refund that's been paid is gone, so I wouldn't put AI in that step at all. Determinism is the requirement there, and only code gives it. The AI can prepare the case; it doesn't release the money.

## A concrete scenario

Background agents that fix production bugs are the clearest case of a verifiable loop. They watch the logs. When something breaks, they file a bug report, send the notifications, read the source code to find the cause, and when the fix is contained, merge and deploy it. This is the design I build to:

1. **Intentional error traces.** Every failure carries the message the code meant it to carry, through `catchErrors` in my open-source [puffy-core](https://github.com/nicolasdao/puffy-core) package. A trace precise enough to read is a trace precise enough to reproduce.
2. **Triage first.** Is it a bug in the code, or an outage, a throttle or an expired key? Only a bug gets a fix.
3. **Reproduce it with a test before fixing anything.** If the agent can't make the error happen in a test, it doesn't get to claim it fixed it.
4. **The fix passes every test, unit and end-to-end.** Aim for complete coverage. Mutation testing with [Stryker](https://cloudlesslabs.com/stack/stryker/index.md) runs on the fix when it can finish within about 10 minutes; otherwise it waits for a weekly night run, because on a big codebase a full run takes five to ten hours.
5. **A contained blast radius, decided by a script.** A change to the database (every database versioned with migration scripts) or to a connector other systems use stops the automation and goes to a person.
6. **Test environment, then production,** with the end-to-end tests run again in each.

Around that loop sits what I call my super harness: a harness of my own on top of the existing ones. Three parts of it matter here:

- **Reviewer agents with a different job** (next section).
- **A rollback agent.** It writes the rollback procedure for each deployment, and nothing deploys without its approval. It rolls back on errors and confirms the errors stopped, and it hands over to a person when a rollback isn't possible or doesn't work.
- **A screen in front of the logs.** Logs carry whatever users typed, so text written to fool an agent could reach one. A cheap decision model screens them for prompt injection before any agent reads them: d1-ultra, at $0.042 per million tokens, after personal data is removed, because its data terms aren't documented. It's one layer, not the defence: in the one independent test, a planted answer still steered it 28% of the time ([Which decision model?](https://cloudlesslabs.com/stack/guides/which-decision-model/index.md)).

## Whatever does the work shouldn't check it

An agent told to ship is rewarded for shipping. Its incentive is the merge, not the stability of everything around it, so it shouldn't grade its own work. Banks solved this decades ago with maker-checker: one person makes the change, another checks it. With agents, the checkers are other agents, each given a different job and the incentive that comes with it:

| Role | In software | Its question | Outside software |
|---|---|---|---|
| Maker | The developer | How do I get this done? | The writer of a report |
| Verifier | QA | Does it do what it claims? | The fact-checker |
| Steward | DevOps | Is the whole still stable afterwards? | The editor of the whole publication |
| Auditor | Security | What could this expose or break the rules on? | Legal or compliance |

Four rules make the checkers worth having:

- **Code checks what code can.** If an agent edits an old test so its fix goes green, a script that refuses edited tests catches it every time; a reviewer catches it most of the time. Where a test really has to change, another agent questions the one that changed it before anything merges.
- **Checkers read the evidence, not the maker's summary:** the diff, the trace, the test results. A checker from another vendor's model shares fewer blind spots.
- **Permissions beat instructions.** In the design I recommend, the maker's credentials can push a branch and nothing more, and only the pipeline merges, once the checks and the checkers agree.
- **Checkers get a limit too.** Told to find problems, a checker always finds some. Disagreements that outlast the attempts go to a person.

## The trade-offs

| | Loop until the checks pass | The user consents at each fork | A named person signs off | Plain code |
|---|---|---|---|---|
| Fits | Work code can verify | Judgement the user can make | Judgement only an accountable person can make | Critical results that can't be undone |
| Speed | Minutes, unattended | As fast as the user answers | As fast as that person | Instant |
| What goes wrong | Checks that miss, or a maker that games them | Approval fatigue | A queue in front of one person | Rigid when the case is new |
| What keeps it safe | Checks the maker can't edit, an attempt limit, a key cap | Few, well-chosen forks | Their accountability | Tests |

## The rule of thumb

**Design for highly reliable, not perfect.** Let code verify what it can, let the user judge what it can't, and keep AI away from what can't be undone. Then give the checking to something other than what did the work.

## When the answer changes with scale

- **Your own product, early:** as much freedom as the tests allow. Review the outcome in production, and turn every miss into a new check.
- **Paying users who depend on it:** checks the maker can't edit, a second agent with a different job, and a rollback written before every deployment.
- **Regulated work, or decisions about people outside the loop:** a named person signs off what's critical.

## Models, limits and the loop

- **The model:** Opus 5.5 at medium effort, by default. For standard engineering (database changes, test-driven development) it's more than capable. I raise the effort only for novel, creative work, like researching a way of doing something nobody has documented. Saving on the model is penny-wise and pound-foolish when the cost of failing is that much higher: a cheaper model takes over a step only when a replay of real cases proves it does just as well ([Which model, at what price](https://cloudlesslabs.com/stack/guides/which-model/index.md)).
- **The limits:** a number of attempts per task, with a spending cap on the key as the backstop for the run that goes wrong in a way the count doesn't see.
- **The loop:** don't rewrite it. An SDK (the Claude Agent SDK, the OpenAI Agents SDK) gives you the loop and its primitives. Then build your own layer for your domain on top, because no SDK knows your domain.
- **Where it runs:** the waits and retries of the whole job in a durable function ([SQS retries or Durable Functions?](https://cloudlesslabs.com/stack/guides/retries/index.md)), and the agent's loop inside one step, with its own time limit.

## Questions that change the answer

- Can code tell whether the result is right? Then the step can loop on its own.
- If not, can the person it's for judge it, at a few well-chosen moments?
- Can a wrong result be undone? If not, keep AI out of that step for now.
- Who is accountable if it goes wrong, and do they know the AI is acting?

Doesn't cover:

- Which model a step needs and what it costs per task: that's [Which model, at what price](https://cloudlesslabs.com/stack/guides/which-model/index.md).
- Whether a step should answer fast or think: that's [System 1 or System 2?](https://cloudlesslabs.com/concepts/system-1-or-system-2/index.md).

## Check your call

- **Every night, 50,000 bank transactions have to be matched against invoices.** Loop until the checks pass: the totals reconcile or they don't. Let the AI match and try again within a set number of attempts, and send what's still unmatched to a person.
- **An assistant drafts a client's monthly newsletter to 8,000 subscribers.** The user consents at the fork. The AI drafts and adjusts as the client asks, and nothing is sent until the client approves the send, because a sent email can't be taken back.
- **A support agent wants to issue refunds on its own.** Not AI for the payment itself: once the money's gone, it's gone. The AI can prepare the case and the amount; plain code and a person release it.
- **An insurer's assistant tells customers whether a claim is covered.** A named person signs off. A customer can't judge a policy clause, so their consent doesn't make the answer right.

## The technology on this site

- [Which model, at what price](https://cloudlesslabs.com/stack/guides/which-model/index.md): the decision guide
- [SQS retries or Durable Functions?](https://cloudlesslabs.com/stack/guides/retries/index.md): the decision guide
- [Durable functions or Step Functions?](https://cloudlesslabs.com/stack/guides/durable-or-step-functions/index.md): the decision guide
- [AI that runs the code it writes](https://cloudlesslabs.com/stack/use-cases/ai-code/index.md): the use case
- [Building with AI agents](https://cloudlesslabs.com/stack/use-cases/ai-dev/index.md): the use case
- [An AI chat or LLM feature](https://cloudlesslabs.com/stack/use-cases/ai-chat/index.md): the use case
- [HappySkills, the package manager for AI agent skills](https://cloudlesslabs.com/work/happyskills/index.md): in our client work

## Thoughts on it

- [Verifiability is your next AI investment. TDD is just the start.](https://cloudlesslabs.com/thoughts/economics-of-verification/index.md), 12 November 2026: Tests used to slow you down. Now they're the only thing that can speed you up, for two reasons. They got cheap: agents write and run them for you. And they got valuable: your agents can write more code than you'll ever read, so the limit isn't the code any more. It's how much of it you can trust, and only verification raises that ceiling.
- [Stop building software. Build the software factory that builds it.](https://cloudlesslabs.com/thoughts/software-factory-philosophy/index.md), 11 November 2026: A software factory is the environment you build for your coding agents. For me, it's the new software development lifecycle, automated and constrained, from the moment a request arrives until its business outcome is met. Here's how Matt Pocock, Lauren Tan and StrongDM define it, and how I'm building mine.
- [I haven't opened my code editor in three months. I don't miss it.](https://cloudlesslabs.com/thoughts/three-months-without-my-code-editor/index.md), 11 November 2026: I haven't opened Sublime Text in about three months, since my coding agents moved to a server. I review outcomes now, not code. My bet: editors die, VS Code included, and Microsoft is asleep.

## Related

- [System 1 or System 2?](https://cloudlesslabs.com/concepts/system-1-or-system-2/index.md): read next
- [Ask an LLM, or call a decision API?](https://cloudlesslabs.com/concepts/llm-or-decision-api/index.md): read next
- [How should a model get your company's knowledge?](https://cloudlesslabs.com/concepts/progressive-disclosure/index.md): read next
- [Putting AI into software](https://cloudlesslabs.com/concepts/areas/ai/index.md): the area, and the technology for each answer
- [Every concept](https://cloudlesslabs.com/concepts/index.md)
- [Map of every Markdown file on cloudlesslabs.com, with what each one answers](https://ask.cloudlesslabs.com)