Code Generation Got Cheap. An AI Software Factory Is How You Check It.
Code generation is cheap and verification is scarce, so an AI software factory is a pipeline of loops that share artifacts with a person at every gate.

Five eras, ten stages, and one rule: loops share artifacts, and a person stays at every human gate.
Engineers feel faster while delivery dashboards barely move, because cheap generated code is flooding a fixed human verification budget. A factory of loops, with a person at every gate, is how that budget gets spent on judgment instead of noise.
In this article: Coding agents can raise throughput and still leave delivery unstable. An AI software factory answers with nested eras, loops that stop, ten stages that share artifacts, and a person on every gate.
Ask any team that rolled out coding agents this year how it went, and you will hear two stories at once. Engineers say they are faster. Delivery dashboards say very little changed.
Both are true. DORA's 2025 research finds that AI now raises software delivery throughput but still increases delivery instability, and it calls AI an amplifier of an organization's existing strengths and weaknesses. Anthropic's playbook for an AI-native lifecycle names the mechanism: when agents multiply code output, either the review queue builds or code ships under-reviewed.
The controlled studies do not agree. METR's 2025 randomized trial gave 16 experienced open-source developers 246 tasks. Early-2025 AI tools made them about 19% slower, although they had forecast a 24% speedup and still believed afterward that they had been faster. METR's follow-up, with 57 developers, estimated smaller effects with confidence intervals that cross zero, and METR now calls its own data an unreliable signal. A 96-engineer randomized trial at Google found tasks about 21% faster.
The work moved to both sides of the build. Ahead of it, intents, specs, and acceptance criteria have to be approved fast enough to feed the agents. Behind it, the review queue grows, because review and security teams are sized for human output. Cheap generation hit a fixed verification budget. The answer is not a smarter chatbot. It is an AI software factory: people, agents, and tools planning, building, checking, releasing, supporting, and improving software by loop engineering.
Five eras, and a gate you do not remove
Five eras nest, and each stays inside the next: a factory chains loops, a loop runs inside a harness, a harness engineers context, and context still contains prompts. Each era moves one more rule from advice the model reads to behavior the system enforces. The figure's sixth layer is the human gate around that nest. Removing the gate is the failure the evidence does not yet support. It is not a sixth era.


The five eras each get one row, with four columns. Era gives the number and name. Unit of work says how big one piece of work is in that era. The engineer builds lists what a person has to make for that era to work. Failure that forces the next era names the problem the era cannot solve on its own.
- 1. Prompt engineering. The unit of work is one prompt and one answer, and the engineer builds instructions, examples, and output formats. It fails because the model knows only what fits in the prompt and cannot check its own answer.
- 2. Context engineering. The unit of work is what the model sees for one task. The engineer builds retrieval, rule files, progressive disclosure, skills, and notes. It fails because more context is not better context, and guidance is only advice.
- 3. Harness engineering. The unit of work is one agent session. The engineer builds hooks, tools, a sandbox, and subagents with their own context. It fails because a person still starts every task and watches every session.
- 4. Loop engineering. The unit of work is one repeating cycle. The engineer builds the trigger, goal, verification, stop rule, memory, budget, and escalation. It fails because loops run alone, and nothing chains them.
- 5. Software factory. The unit of work is one pipeline of loops. The engineer builds committed artifacts per stage, risk tiers, enforced gates, and provenance. Its failure would be removing the human gates, which the evidence does not yet support.
From era 1 to era 5, the unit of work grows from one prompt to a pipeline of loops.
Context is the spine: what should one model see at one moment? Accuracy falls as input grows, even on simple tasks, and a single distracting passage hurts. Models use the start and end of a long input better than the middle, and as the window nears full even the start fades. A task revealed in pieces over many turns does about 39% worse, on average, than the same task given whole. A model nearing what it believes is its context limit starts wrapping up early and skips steps. Curate the smallest set of current, consistent, high-signal tokens the task needs.
Harness engineering turns advice into enforcement. A rule file can tell an agent not to edit the tests. A hook that blocks the edit makes it true whatever the model decides. With the model held fixed, SWE-agent gained 10.7 points on SWE-bench from interface design alone. Loop engineering runs that harness on an event or a schedule, with nobody typing the next prompt. The ancestor is Geoffrey Huntley's "Ralph" loop: run the agent, check the result, run it again.
Each stage commits an artifact the next stage reads. Anthropic's playbook runs from an intent file through a spec, a plan, a reviewed pull request, and an incident record that becomes the next intent. OpenAI's factory, as reported, runs nine stages, from a person defining the outcome to agents responding to incidents. This reference design uses ten. Most teams today sit in the second or third era.
Human time per change falls in every era. The person writes every request, then curates context, then sets the boundary, then designs the checker, then only approves at the gates. Whoever approves the intent aims every later loop. Whoever writes the checker decides what every future change must prove.

A loop is five parts
A chat waits for you. A loop does not.

The skipped part is the check. An agent that grades its own work will usually pass itself. On grade-school math, GPT-4 fell from 95.5% to 91.5% when it revised its own answers, while an external verifier lifted its planning accuracy on one benchmark from 40% to 87%. Independence means a different session, different context, and often a different model, then a human who can say no. The stop protects the budget: a missing limit is not unlimited permission. Memory is why month two hurts. Agents remember nothing between runs, and an unwritten lesson comes back next week with the same confidence.
Ten stages, three groups, three concerns
A trigger, a human decision-maker, and a limit on each stage. Stages share artifacts. They do not call each other.

Shape gets a signed need. Plan's reviewer comments until the ticket is ready. Dispatch chooses the start, the workflow, and the budget. Schedule orders parallel work so agents do not collide. Build stays in a sandbox. Review uses independent checkers, and a person merges. Release decides exposure separately from that merge. Respond diagnoses read-only while people restore service. Maintain upgrades, migrates, and retires. Improve adopts a rule only after a test.
An incident becomes a ticket. A repeated review comment becomes a lint rule. A failed upgrade becomes a recorded fix. Forward-only repeats mistakes faster.
Evals are the stage's own check: tests, calibrated LLM judges, traces. Memory lives outside the agents. Deployment is the harness, the sandbox, and an identity of the stage's own. Named tools are examples, not requirements.
Three rules
Build opens a pull request and Review reacts. Nothing calls across that line. The artifact's contract is as strict as an API, including the spelling of ready. READY in one loop and Ready in the next stops the factory, and no model will notice. A handoff carries information, not permission. Silence is not approval.
People own intent and approve every merge. Agents draft, check, comment, and propose. Automating the 40% of the work that needs no judgment is already a large win, and accountability stays where auditors and customers expect it.
A stage recalls decisions, past fixes, and known failures before it acts, then writes what it learned. A chat will not store that. One option, not a requirement: WikiTicket as a git-native work log, and the OKF plugin family as a knowledge graph for decisions and architecture.
The fictional feature is a workspace export. The owner allows a workspace-only download and rules out search across other workspaces. Plan waits until access checks exist. Build stops when large exports need extra work and asks an engineer. Review catches a missing workspace filter. After release, too few trials means the honest answer is "we do not know yet."
Light, unless the evidence says otherwise
A dark factory lets people write specs and curate tests, and no human writes or reviews the code. A few teams have published charters on those lines. A light factory keeps the approval gates and lets agents do the work between them.

Meta's RADAR is the narrow evidence for removing a gate. It lands low-risk diffs chosen by a risk score: more than 331,000 landed, which are reverted at a third the rate of other diffs and cause production incidents at one fiftieth the rate. The threshold moved only as data accumulated. Review agents, on the best current benchmark, still miss more than 20% of the bugs human reviewers catch, and no operator publishes defect rates for a factory with no human gates.
Stay light for most organizations, until a change class earns otherwise. Review is where accountability lives, and regulated teams need a named person on each change. The strongest published results kept human review and made it cheaper, rather than removing it. Prompt injection through tickets, issues, and repository content is a real, exploited attack on CI-hosted agents, and a human gate does not depend on the model noticing it. One change class at a time, on evidence, never all at once.
Do this today
- Name your era. Most teams are still curating context or wrapping one session.
- Write the five parts of one agent you already run. A blank stop is not permission to continue.
- Write one artifact contract, including the exact spelling of ready. The next stage reads it. Nothing calls that stage.
- Name the person on intent and the person on the merge.
- Walk a fictional workspace export across those handoffs. An owner who cannot explain the need without the document has not approved it.
Verification is the budget
DORA's instability, METR's slowdown, and Google's speedup are one gap measured three ways: more output, the same human budget for checking it. An AI software factory spends that budget on shared artifacts, a check and an owner at every stage, memory outside the model, and a gate the evidence does not yet let you delete.
Which stage did you start with, and what broke first? A ticket reviewer and a ticket implementer from a loop engineering seminar are in the reliable-agentic-lab repository.
This is Part 1 of "AI Software Factory Series," a 17-part guide to building an AI software factory with human gates.