Agents Already Pay for Themselves on the Work Nobody Roadmaps

Maintenance is the safest first write access for an agent, because deterministic tools take the easy majority and people still merge.

Rick Hightower

Cover image for “Agents Already Pay for Themselves on the Work Nobody Roadmaps” by Rick Hightower

AI code migration works when scripts do the easy majority, agents take the long tail, and the same review gate still decides the merge.

The maintenance backlog never makes a roadmap slide, yet it is where published agent results are strongest. The pattern that holds is deterministic tooling for the easy majority, agents only for the long tail, and a person at the merge.

In this article: Published migration results are the strongest agent evidence in the factory. Scripts still do the easy majority, people still merge, and an expired timer still cannot retire a service.

Dependency upgrades, framework migrations, ten thousand deprecated call sites, merge conflicts on a long branch. Amazon, Google, Spotify, and Airbnb have all reported large migrations finished in a fraction of the estimated time, by small teams, with agents on the long tail.

Three properties make this the safest first write access, behind the same review and merge gate as any other change: a pass-or-fail signal, small repetitive diffs, and little product judgment. The other half of the stage is not an agent problem: when software should stop running.

What the stage is for

A timer, a dependency release, an advisory, a merge conflict, or a service that should close wakes the stage. The work is to upgrade, migrate, resolve conflicts, remove dead code, and prepare retirement evidence. Tests and a human merge are the check. Stop at maintained software, or at a recorded retirement.

Shrinks for the human: mechanical edits and failed upgrade pull requests. Grows for the human: which classes earn automatic merge, and whether a service lives.

The maintainer orders the upkeep. The app owner decides if the service continues. One campaign at a time. A framework upgrade, a test migration, and forty dependency bumps in the same week will clog review.

The migration pattern, deterministic first

Before LLMs, Spotify's deterministic scripts already handled about 70% of fleet-wide changes. Honk, its background coding agent, exists for the tail the scripts missed. Agents did not replace codemods. Call this "the migration pattern, deterministic first."

Deterministic discovery and codemods take the easy majority. An agent takes the long tail, and a person still merges.

Find the edit sites deterministically. Compiler, AST, or search. A model misses sites and invents others. A parser does not.

Let rules do the easy majority. A codemod is faster, cheaper, and identical every time.

Send the long tail to an agent. Unusual shapes, surrounding logic, and tests that need rewriting go one bounded change at a time, with the same checks and stop conditions as any build. Beads, one task-graph example, can give each edit site its own package so agents do not collide.

Check parity, not just green. Characterization tests pin behavior first. The same tests must pass after. New green tests are not enough.

Batch the pull requests by repository or module. Thousands of tiny ones bury reviewers. One giant one hides problems.

Google reports the clearest version of this: deterministic discovery plus model-generated edits, with validation gates, across 39 internal migrations.

The same gate as everything else

A minor logging bump is safe. A minor database-driver bump can change connection pooling and slow large exports. The feed makes them look the same.

An upgrade still walks tests, review, and a human merge. The agent only pushes a bounded fix when CI fails.

Same path as feature work: in-loop checks, Review, a human merge. Auth, crypto, the database layer, or the build stays high tier however small the diff.

Automatic merge waits for a narrow class watched for months: patch bumps with strong tests, formatting-only migrations, generated code. Spotify lets migration drivers approve their own PRs, with full validation before a PR is created. Driver self-approval is an end state. Months of clean merges, no reverts, and no incidents, written into policy as code, come first. Until then a person merges.

An agent may explain both sides of a conflict and may not pick one in silence. A person who understands both decides. An integration agent may merge branches, fix mechanical conflicts, and rerun checks only if it reports what merged, which conflicts it resolved, and what validation ran. A clash of intent still goes to a person. A fixer may return a branch to green within a capped number of attempts. The person sees a summary or a question, not every step.

Retirement is a business decision

Retirement is where automation does irreversible damage.

A sunset date prepares evidence. Only a recorded owner decision can retire the service, and late notifications cannot bring it back.

An expired timer alone cannot authorize deletion. The sunset date reminds someone to decide. The stage then gathers usage, dependents, data to keep or export, and contracts that name the service. The app owner decides. Deletion waits on a recorded decision.

Late notifications cannot restart a retired application. A delayed alert, a stale webhook, or a queued job must not bring it back. Retired is a final state, alongside completed and cancelled.

Agents can list every caller, dashboard, and runbook. They do not decide that the service can go.

Maintenance agents write the first fix records

A renamed key. A transitive dependency that drops an old runtime. A test helper whose timeout default changed. An hour the first time, a minute after, if the fix was written down.

Maintain covers upgrades, migrations, conflicts, retirement evidence, and fix records, with two different human decisions.

Recall past breakages before an upgrade. Record what broke and the fix, linked to the pull request and the ticket. Maintenance work repeats, so these records are the factory's first automatic fix notes.

OKF plugins, as examples: System Architecture Capture for libraries, runtimes, and shared clients; Data Engineering Knowledge Capture for schemas and pipelines; fix records with source and author. A person reviews those entries before other agents rely on them. A wrong record teaches the next upgrade the wrong lesson.

What the evidence says, and how to start

Maintain has the strongest evidence in the factory. It is still mostly self-reported, so check each figure against its source before you rely on it.

Table: Organization, Work, Reported result. Rows: Amazon; Google; Spotify; Airbnb.

Four organizations report results from agent-assisted maintenance at scale. The Organization column names the company. The Work column says what it changed and with which tools. The Reported result column gives the company's own figures, so treat each one as self-reported.

  • Amazon moved its internal portfolio from Java 8 and 11 to Java 17 with Amazon Q Developer. It reports over 30,000 production applications upgraded, about 4,500 developer-years of estimated effort saved, and 79% of generated code reviews applied without edits.
  • Google ran 39 internal migrations that pair deterministic discovery with model edits. A core team of three made 93,574 edits over 12 months, in roughly half the end-to-end time.
  • Spotify makes fleet-wide changes with its Honk agent on top of its existing migration scripts. It reports more than 650 merged agent pull requests a month and 60 to 90% time savings on migrations.
  • Airbnb migrated its test suite from one React testing library to another. About 3,500 test files moved, in a fraction of a 1.5-year manual estimate.

Every row follows the diagram: deterministic tooling where it works, agents for the tail, tests as the oracle, people merging.

Swappable examples: Renovate or Dependabot opens the PRs and an agent fixes failures; codemods and AST tools cover the majority; Claude Code on a scheduled GitHub Action runs the pass; worktrees isolate a batch of checkouts; Beads, or another task graph, hands out one package at a time.

Point the agent at dependency pull requests that already fail CI. It fixes the same branch, inside the usual budget and stop rules. A person still merges. Move on when engineers stop reworking those pull requests. With comment-only reviewers already running, this is the safest first place an agent writes code.

A fictional workspace export

A database-client upgrade changed connection pooling and slowed large exports. Maintain shipped that upgrade. The workspace export is a running example, not one of the four reported cases.

After the pooling failure, the fix takes the normal path, then Maintain records the lesson and raises the tier.

A test that reproduces the slow export lands a week later, on the normal path. The fix record says this client's minor release changed the default pool size, timeouts hit long-running queries, and an explicit pool setting fixed it. The next campaign recalls that before touching the client. The client joins auth and crypto on the always-high-tier list, so a patch will never auto-merge. The same implicit-default failure has now appeared twice in a quarter, and that repeated failure belongs with Improve.

Further reading

Do this today

  • Fix only dependency pull requests that already fail CI. A person merges.
  • Find edit sites with a compiler, an AST, or search, not a model.
  • Require the same characterization tests after the change.
  • Keep auth, crypto, the database layer, and the build high tier.
  • Let a sunset date remind the app owner. Only a recorded decision deletes.

The tail is the agent's job. The decision is not.

Give write access here before anywhere else.

An expired timer can remind you. It cannot delete the service.


This is Part 10 of "AI Software Factory Series," a 17-part guide to building an AI software factory with human gates.