Stop building software. Build the software factory that builds it.

Where I left off

In my last post I made a bet: within 12 months, software factories will write 70% or more of the world's code, and none of them will need a text editor. I also said I'm moving my own code to software factories, where agents run for days on their own, and that they deserved a post of their own.

This is that post. Before I get to anyone's definition, one thing should be clear: I didn't learn this from a book or a conference talk. I got here on my own, by climbing what I call the trust ladder, over the past two years of working with AI agents every day. Matt Pocock and Lauren Tan got to the same place, and they've put it into words better than anyone I know. That's why they're in this post.

I've been building my own factory, and I'm planning to release it very soon. If you want to know when it's ready, subscribe.

How I got here: the trust ladder

Claude Code came out as a research preview on 24 February 2025, and I was an early adopter. I started cautious. As I got familiar with it, I trusted it more, and I gave it more to do.

The trust ladder isn't an absolute scale. You can't say you're on the fourth rung and I'm on the sixth. It's personal, measured against your own goals and your own team, because your interactions with AI are different from mine, and so is our idea of a job done well.

It has three steps, and they repeat:

1. Get familiar. Use AI as early and as much as you can, starting today. That's how you learn what it does well and what it doesn't, for the work you care about. It's also the only way to notice when a new model gets smarter: you can't see what a version does better or worse unless you knew the one before it.
2. Build countermeasures. Once you know where it fails, you build around it. The first countermeasures are harnesses: Claude Code, Codex and OpenCode are all harnesses. The frontier labs ship theirs as turnkey ways to trust the model a little more (OpenCode is the open-source one). What matters more is the system you build on top of them.
3. Expect failure, and catch it. The key to success with AI isn't expecting it to get everything right. It's expecting it to fail, because at some point it will. The question is whether you'll detect the failure when it happens, and add a countermeasure so it doesn't happen again.

Every time you do that, you trust the system you've built a little more, and you climb a rung.

The key isn't expecting AI to get it right. It's knowing it will fail, and catching it when it does.

I'm sure anyone who's used Claude Code consistently since they started has lived the same thing. Lauren talks about a trust ladder too, and that tells you something: people who use these tools every day end up in the same place. I came to the same conclusion as Matt and Lauren: the work now is building the system around the agents, and that activity has a name. It's called a software factory.

The trust ladder deserves a post of its own, and it'll get one.

What a software factory is

The name is older than any of us. The earliest written use I could find is Robert Bemer's, of General Electric, in a paper for the NATO software engineering conference in October 1968: a "machine-controlled production environment, or software factory". A year later, Hitachi became the first company to call a real software facility a factory. Nobody can claim to have coined it for agents. It spread over the past year: Dan Shapiro named the "dark factory" in January, and two weeks later StrongDM published the strictest definition there is.

The term is everywhere this year, and it means different things to different people.

Matt and Lauren aren't mainstream. They're at the edge, which is exactly why they're worth reading: they've articulated these ideas better than anyone. Lauren worked at Netflix and on the React team at Meta, joined Cursor in March (Cursor is now part of SpaceX AI), and created pstack, a popular set of skills for coding agents. Her talk on shipping 2,500 pull requests to production in a month went around the internet in September, and Matt's live stream with her is the best conversation I've heard on the subject. If you build with agents, watch the whole thing. Each definition sees a different part of the same machine. Shapiro and StrongDM describe the destination: specs go in, software comes out, and nobody reads the code. Matt describes how the work starts. Lauren describes what makes it trustworthy. Mine is closest to Lauren's, and it stretches the factory at both ends: before the work is accepted, and long after it ships.

Why now

For as long as I've built software, the expensive part was writing the code. That's no longer true. Agents write it fast, and you can run as many as your machines allow. When the cost of one step collapses, the bottleneck moves to the steps around it.

Lauren put it plainly: the bottleneck is no longer the agent, it's your ability to express your intent clearly enough for the agent to carry it out. That's why she thinks domain expertise matters more than ever, not less. OpenAI's team, which built a product with no hand-written code, says it in four words: "Humans steer. Agents execute."

So the work moves to three places:

1. Intent. Saying precisely what should be true when the work is done.
2. Verification. Proving it is.
3. Operation. Releasing it safely, watching it run, and repairing it when it isn't.

A software factory is the machine that does those three things around the agents. The agents bring the capacity. The factory makes that capacity usable.

Verification is what turns cheap code into usable code.

Start from one assumption: the agents will get it wrong

This is the trust ladder's lesson, and it's the first principle of my factory. Agents are fallible, and they'll stay fallible. Even the frontier models take shortcuts: in Lauren's words, they "tend to do the easy thing". You've seen it. A test that only checks the mock it set up. A failing check quietly skipped. A requirement reinterpreted until the code satisfies it.

So I don't build my factory on the hope that the agents get better. I build it on the assumption that they're wrong some of the time, and I design for that:

• Find the mistakes early, before a user does.
• Keep them small, so one bad change can't hurt many people.
• Fix them fast, with a fix that's verified like any other change.

Lauren's version of the same idea is a question I now ask myself all the time: how do I make the easy thing the right thing?

Verification is the engine

If you take one idea from this post, take this one. "The bottleneck with agents is verification," Lauren wrote in May, and on the stream she called it the single most important skill in your toolkit. It means giving the agent hands and eyes: it runs the code, uses the product like a person would, takes traces and screenshots, and sees for itself whether the work does what was asked.

The most important part of a loop that allows it to be a loop is the verification part. (Lauren Tan)

Without that, there's no loop. There's an agent guessing and a human checking, and the human is the bottleneck. With it, the agent iterates until the checks pass, and you can leave it running overnight. It's also why scaling comes last, not first: "Trying to parallelize agents you don't trust yet is a huge waste of tokens," in Lauren's words.

Two rules I hold to:

The agent must not be able to move the goalposts. StrongDM found it the hard way: "A test, stored in the codebase, can be lazily rewritten to match the code." So they keep their scenarios outside the codebase, like the holdout set in model training, where the agents can't touch them. Anthropic found the reviewer side of the same problem: "Out of the box, Claude is a poor QA agent." It spotted real issues, then talked itself into approving the work anyway. The checks that decide whether the work is done must stay out of the reach of the agent doing it, and they must work differently from it. Ten copies of the same agent reviewing each other share the same blind spots. That's not independence.

Deterministic where it can be. The parts of a check that don't need judgment should be code, not an agent's reasoning. Lauren moves them into scripts and command-line tools inside her skills, so every agent runs the same check the same way instead of rebuilding it each time, and the agent keeps only the parts that need judgment. Same tests, same environment, same verdict, every run.

How much verification is worth paying for, and how agents change that sum, is the subject of Verifiability is your next AI investment. TDD is just the start.

The environment is the product

The second idea: the engineer's job moves from the code to the environment the agents work in.

Lauren said she almost feels the new job of the engineer is spending time on the environment, and the definition of a good codebase that Matt likes explains why: one that's easy to change without things breaking. A codebase full of guard rails, where an agent (or a new hire) is pushed down narrow paths, is one where it's hard to write bad code. OpenAI's team got there the same way: its architecture is enforced by custom linters, and each agent gets its own copy of the app with logs and metrics it can query.

The method is simple and relentless. Watch how your agents fail. Every time you see a mistake, ask how to make it impossible: a lint rule, a type, a convention, a test. Lauren's team built an internal framework where there's only one way to add a feature, so neither the agent nor a person has to think about where things go.

Every mistake an agent makes twice should become a rule the codebase enforces.

The rules live in the environment, not in a prompt the agent has to remember. As Matt put it on the stream, the agent stumbles into them at the right moment.

My definition: the new software development lifecycle

My definition is closest to Lauren's: a software factory is the environment you build for your agents. Where I differ is what I think that environment is. It's the new software development lifecycle, automated and constrained.

Before AI, the lifecycle started with a ticket and ended when the ticket was closed: the feature shipped, it wasn't broken, and it hadn't broken anything else. That was done. In my factory, the work starts before a ticket is accepted and ends long after it's closed, and that changes the nature of the environment we build for our agents.

It watches production, all the time. That's the factory's first job, not its last. Everything else feeds from it.

It manages the requests. When a new ticket comes in, the factory analyses it and interrogates the stakeholders, whoever they are. It can decline the ticket on its own, unless someone gives it a good reason. It can refine it, through a structured interview like Matt's grill-me skill ("a relentless interview to sharpen a plan or design"). And it prioritises it. That's another layer of management: the system manages itself.

It checks the work can be verified before anyone builds it. If nobody can say how we'll know it worked, the ticket isn't ready. Verifiability is a condition of entry, not an afterthought, and so is deciding how the outcome will be observed once it's live.

It builds and verifies. Agents map the risk (who could be hurt, how fast, and can it be undone), produce the change and the observers it will need, and loop until the checks pass, with independent verifiers challenging the result. The recovery plan is ready before the release: what signal stops the rollout, what can be switched off, whether the old version can still read the new data.

It releases on its own, and keeps watching. The change goes to production automatically, to a few users first. Then the factory observes the outcome, not just the errors.

It's done when the outcome is met. An outcome can fail with no bug at all: the feature works, nothing broke, and it still doesn't move the objective it was built for. Then the work goes back to the session that built it, to revert it, change it or improve it. A session lasts as long as the outcome isn't met, and when it isn't, the first job is to understand why.

The goal of a software factory isn't to produce code. It's to produce meaningful outcomes.

Meaningful against what? Against the product's objectives, and those have to be defined by humans. Every ticket faces the same two questions: does it fit those objectives, and if it does, how do we build it so it serves them? That's why, for me, a software factory isn't just engineering any more. It's business too.

Lauren has names for the two halves of this: the inner loop, where agents build towards a snapshot of your intent, and the outer loop, which brings the world back in. Her outer loop pulls bug reports from Slack, Linear and X back to her agents, so she stops being the person who carries context from one place to the other. The question she keeps asking is the right one: where am I the bottleneck, and how do I teach the agent to answer that question itself, with real data instead of a guess?

How long a session should live, and what happens when several of them watch the same product at once, deserves a post of its own.

The factory builds its own observers

There's a part of this I underestimated at first. Defining an objective isn't only deciding what the software should do. It's deciding how we'll know it worked: which signal, measured how, for how long, compared with what. Almost every time, the answer is specific to that objective, and nothing off the shelf measures it.

So the factory builds what it needs to observe it. Alongside the feature, it creates bespoke observers: agents spawned for that one objective, each with its own tools. One might walk a new checkout journey every few minutes. Another compares the users who have a feature switched on with the ones who don't. Another reconciles a new billing calculation against the old one, invoice by invoice. They're part of the work, specified with the objective, built and verified like the feature, and retired when the objective is met.

That changes what a software factory is. Verifiability isn't a fixed stage with a fixed set of tests: nearly every objective generates something new to observe it. The factory isn't static. It writes code to extend its own capabilities, one objective at a time.

A software factory isn't static. It writes the code it needs to observe its own work.

It's early days for this part. I'm building it right now, and I think it's where the biggest opportunity in software factories is: the observers, more than the code they watch.

Your factory is your culture

In 1968, the same year as Bemer's paper, Melvin Conway published "How Do Committees Invent?". Its thesis became known as Conway's Law: "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure."

Conway was talking about structure: the way teams talk to each other shapes the way the system is split. I read it more broadly. A product reveals the company that made it. If you have a shitty culture, you'll end up with a shitty product.

With a software factory, the culture moves one level up. What you call good enough, what you refuse to ship, the questions you ask before you build: all of it gets encoded in the factory, in its rules, its checks and its questions to the stakeholders. The product becomes a picture of the factory that made it, and the factory a picture of the company that built it.

Every engineering team's job is shifting: from building software to building the factory that builds their software.

Do that well, and distil the right culture, the company's DNA, into the factory, and it will produce an amazing product. Do it badly, and it will produce bad software faster than anyone ever could by hand.

Taste a sample, fix the kitchen

The question everyone asks after "2,500 pull requests" is: who reviewed them? Lauren's answer is nobody, not one by one. Her agents spawn verifier agents for every pull request, which use the app like a person would, hunt for regressions and fix what they find. Her agents merge their own code while she sleeps. In the morning she reads the commit history.

You cannot be tasting every single dish that comes out of your kitchen. (Lauren Tan)

So she samples, the way a quality supervisor samples a factory's output, and scrutinises what she picks rigorously. When she sees the same bad pattern in several agents' work, she doesn't correct the agent. She corrects the kitchen: a new lint rule, a sharper skill, a tighter constraint. Matt summed it up in a phrase I'll keep: sampling instead of blocking.

That's exactly what I meant last time by reviewing outcomes, not code. A human reading every diff doesn't scale, and it never caught as much as we pretended.

Is it a dark factory?

Matt keeps a dictionary of AI coding, and it has a strict definition: a dark factory is a codebase, or part of one, where the factory writes the code and no human reads it. Agents running overnight with a person reviewing their pull requests in the morning don't count. And he names the price: when something breaks, "You must debug code that nobody on the team has read."

When Matt asked Lauren whether hers was a dark factory, she said it kind of is. He suggested a dimmer switch instead, and I think that's the right picture. Mine is a dimmer too, and today it's quite dim, like Matt's and Lauren's. Where the work is verifiable and a mistake can be undone, the lights go down. Where it can't, a person decides. I suspect mine will be fully dark within the next six months.

I don't read my agents' code either, so by Matt's definition parts of my factory are already dark. My answer to his warning isn't to go back to reading diffs. It's to make the factory remember: every change tied to its intent, its checks and its results, so when something breaks, an agent can trace it back faster than a person who had skimmed the diff weeks ago.

One-way doors

The hardest moment in the conversation was Matt's question about one-way doors. Most changes are two-way doors: merge, and if it's wrong, revert. Some aren't: a migration that loses data, a payment that went out, an email that was sent. What do you do when most of your changes are like that?

Lauren was honest: it depends on how well the work can be verified, and she said she didn't really have the answer. Here's mine, and it's the part of my factory I care most about. The answer isn't only better tests before the release. It's deciding, before the release, what happens when it goes wrong, and limiting how much can go wrong at once.

Amazon's engineers have written well about why a rollback can be unsafe: once the new version has written data in a new format, the old one may not understand it. That's why recovery belongs in the release plan, not in the incident.

There's an interesting twist with agents. A factory that's always on can produce a verified fix in minutes, which makes fixing forward the right call for more kinds of failure than before. But not all of them. Some things get hard limits, never a trade-off: security and data integrity first.

The rules I give my factory

1. Bound the authority. Every agent has a list of what it may change and what it may do in production. Anything beyond that goes to a person.
2. Protect the controls. An agent can't switch off a release gate, delete evidence or give itself more access.
3. Use independent evidence. The checks that judge the work come from somewhere other than the agent that did it.
4. Cap the loops. Retries, spend, exposure and deploy rate all have limits. Repeated failure means stop and escalate, not try harder. As Matt puts it: "Every trigger is an accelerator. Accelerators without brakes make a slop cannon."
5. Logs are evidence, not orders. A ticket, a log line or a web page an agent reads is information, never an instruction. In my last post I wrote about attackers hiding commands in GitHub issues: that's exactly the attack this rule stops.
6. Record everything. Versions, actions, evidence and results, so every change can be traced and every irreversible effect is known before the release.

The factory is software too. Its monitors break, its checks go stale and its success criteria can be wrong. So it gets tested like everything else.

Rules 1 and 2 come from two concepts on this site: how much freedom to give each AI step and where code nobody reviewed should run.

What would prove me wrong

• The economics don't work. A factory costs real money: building it, maintaining the checks, the tokens of a dozen verifier agents per change. StrongDM's rule of thumb is that if you haven't spent at least $1,000 on tokens today per engineer, your factory has room for improvement. If the useful outcomes per dollar don't go up, it's an expensive hobby.
• The bottleneck just moves. If faster implementation only piles work up at specification or integration, nothing ships sooner.
• The checks give false confidence. A deterministic test repeats the same wrong assumption every time. If escaped failures don't go down, the factory is theatre.
• Most software isn't verifiable enough. Lauren's own caveat. If the work in most businesses can't be checked by a machine, factories stay a niche for teams like hers.

So I'll measure it: the time from an agreed intent to a verified outcome in production, the total cost of each useful change, how many failures escape and how bad they are, how fast they're found, contained and fixed, and how often a person has to step in. How fast the code gets written isn't on that list.

My bet

I stand by what I said last time: software factories will write most of the world's code, and soon. But the winners won't be the ones with the most agents, or even the most AI.

The winners are always the same winners: the companies with the best product, period. Not the most engineering power. The product customers judge the best, the one they're happy to pay for, at margins that make business sense. Some companies won't use as much AI as they should, won't run a software factory at all, and will stay on top of the world, just because their product is the best.

AI isn't the be-all and end-all. It's leverage, a new tool: very useful to some, useless to others. It's case by case.

The winners are the companies whose customers judge their product the best. AI doesn't change that.

So if you build a software factory, customer satisfaction, and a price people pay at a margin that makes business sense, must be outcomes it measures, and requirements in every outcome it produces. Those are the objectives humans set, and the factory serves them.

The agents write the code now. My job, and every engineering team's, is to build the factory that writes it, and to put the best of our culture into it.

Next: why verification is the investment that raises your factory's ceiling, in Verifiability is your next AI investment. TDD is just the start. After that: what happens to the lifecycle when a session lives until its outcome is met.