A software factory that actually ships: the operator's checklist

Everyone I follow is building an AI software factory this month: markdown files, agent fleets, features assembling themselves overnight. I operate one, and the courses all stop at the same place - the moment the code exists. This is the other half: the operator's checklist for getting what the factory builds in front of strangers, on schedule, without babysitting it.

Published September 24, 2026 · By Dmitry Tolstoi · 8 min read

The factory everyone is building this month

In short: the current recipe is a handful of markdown files steering agents from a clean branch to a reviewed release, and the name is 57 years old.

The trigger is Greg Isenberg's video "Building a Software Factory that actually works (Full Course)" - my competitor tracker logged 133,100 views for it on September 21, growing at about nineteen thousand a day. In it, Ras Mic walks through his workflow. Isolate every feature in a fresh git worktree, build against house rules, prove the change with before-and-after evidence, then ship through an automated review that loops until the score is clean. Isenberg's own summary on X is honest about the packaging:

A software factory is 5-6 markdown files that sit next to your code and tell your agents how you like to work, so you can build high quality apps 24/7.

The name carries more history than the hype suggests. Hitachi opened a facility it literally ran as a software factory in 1969, and Michael Cusumano's 1991 book Japan's Software Factories documents that plant alongside Toshiba, NEC, Fujitsu and an earlier American attempt at System Development Corporation. Every generation rediscovers the assembly line. What each generation relearns more slowly is the loading dock - and that is the part this checklist covers.

The gap has a number: 240% more code, 30% more releases

In short: autonomous agents roughly triple coding activity while actual releases grow by less than a third, and every rule below exists to close that gap.

In May 2026 three researchers published NBER working paper 35275, "Writing Code vs. Shipping Code", built on more than 500,000 GitHub developers. Autocomplete tools raised coding activity by about 30 percent, interactive agents by 180, autonomous agents by 240. Then the authors follow the same work down the production chain, and the 240 percent shrinks to 80 percent measured in projects, and to 30 percent measured in actual releases. Their diagnosis is human bottlenecks between the keyboard and the release. They also note a sharp rise in new apps with no rise in total usage: more things exist, roughly the same number get used.

DORA, the research program behind the standard delivery metrics, defines change lead time as the time for a change to go from committed to version control to deployed in production. Notice where that clock starts: after the part the factory courses teach. Robert Laszczak put the working half plainly: "The biggest bottleneck of implementation quietly shifted from producing code to reading code." His caption for the diagram is shorter still: "Coding got faster. Review didn't." I have already written about the graveyard of vibe-coded projects; that graveyard is this chart, drawn as biographies.

"Shipped" means a stranger can reach it

In short: a deliverable sitting in a folder, an inbox, or a camera roll is inventory; shipped begins when a public URL answers.

A finished video of mine once sat in a phone's camera roll for a month with the status "delivered". Three separate progress reviews read "delivered" as "published" and went hunting for audience numbers that could not exist. Every word in the log was accurate. The pipeline simply had no vocabulary for the difference between an artifact that reached a device and an artifact a stranger could find.

So the first checklist item is a definition. A piece of work is shipped when its public address answers and someone has opened it from outside - a person or a probe, logged either way. Anything short of that gets a different word in my status files now: rendered, delivered, staged, queued. The factory can use whichever words it likes, as long as none of them can be mistaken for the last one.

Believe the artifact, not the log

In short: a success message describes the code path that ran, the artifact describes reality; open the output itself before trusting any green report.

One August morning my overnight article shift had exited with code zero, and the section it was supposed to fill was empty. The session had handed its work to background helpers; the headless runner killed them on exit; nothing failed loudly enough to change the exit code. The wrapper wrote "success" over a blank day. Another stage once announced "batch ready" for weeks while the library that physically writes image files was missing from the machine - every run reported done, and the output folder had never held a single file.

The pattern in both failures is that logs are the agent grading its own homework. The fix is mechanical. Every stage's definition of done names an artifact: a file with a nonzero size, a row in a table, a URL that responds. The check opens that artifact - counts it, measures it, renders it - and treats the log as commentary. Since adopting this rule I have caught silent failures in rendering, delivery and publishing that all wore a green exit code.

Keep the checklist in a file, not in the agent's memory

In short: an agent's internal to-do list bends under a long session's context pressure; a checkbox file with an artifact per line does not.

Late in a long run, an agent starts marking steps done by feel. Mine did, and so did I when the checklist lived in memory. The cure costs one file: every shift opens by copying a checklist template next to the work, every step is a checkbox, and a box closes only when the artifact it names exists on disk. The shift ends when a search for unchecked boxes returns zero, and a search does not negotiate.

This sounds like bureaucracy until the first time the grep catches a skipped verification at 4 a.m. It is the same discipline I described for running a team of AI agents as one person: the operator's edge is in what gets checked, because the doing is already delegated.

One door out - and it locks

In short: give the factory a single publish path, take a lock before entering it, and make the publish step safe to run twice.

Two of my sessions once deployed the same article a minute apart. Both test runs wrote into one shared test database and quietly erased each other's data, and the result looked exactly like a genuine regression in pagination and payments. A single run later that night passed 1,137 of 1,137 tests - my own count, from my own logs. The bug was never in the code. It was two workers in one doorway.

Three rules came out of that night. First, one door: everything reaches production through a single path, and that path takes a lock, so a second publisher waits or walks away. Second, the publish step is idempotent: it looks the piece up by its stable identifier and updates it, so a rerun after a dropped connection yields one copy, never two. Third, committed does not mean approved. My deploy once carried an unfinished internal page onto production because it shipped everything sitting near the approved article in the working tree. It now builds from a clean copy of one reviewed commit and, before going live, lists what a visitor will actually see changed. A commit proves code reached the repository, and nothing more than that.

A starving factory looks exactly like a working one

In short: an empty input queue throws no error, so schedule the refill like production or the line idles silently in perfect health.

The quietest failure available to a factory owner: every agent healthy, every check green, nothing produced, because the topic queue ran dry. An empty queue does not crash. Mine now holds at least six items ahead, refilled on a weekly schedule from measurement of what earned attention, and the refill is itself a scheduled job with its own artifact check. The overnight hardware has a queue of its own for the same reason - a paid box sitting idle is the same bug wearing a power cable.

Decide where you sit before the line starts

In short: write down the decisions only a human may take - money, credentials, first-time public output - and make the factory stop there and wake you.

A factory that needs me for everything is a job. A factory that needs me for nothing is a liability with my name on it. The working answer is a short written list of stop gates: anything that spends money, anything that touches accounts or keys, and anything that goes public under my name in a place it has not gone before. When a shift reaches one of those, it halts and writes down why, because an unfinished piece is recoverable in the morning and an unauthorized one is not.

The list is boring on purpose. It was written calmly, before the factory ran, which is the only time such lists are written well. Deciding your gates during an incident means the incident is deciding them.

The checklist on one screen

1. "Shipped" is defined as a public URL answering, confirmed from outside. Status words like rendered, delivered and staged stay distinct from it.
2. Every stage names its artifact. Checks open the artifact; the log is commentary.
3. The run's checklist is a file. A box closes only when its artifact exists; unchecked boxes block the finish.
4. One publish path, and it takes a lock.
5. Publishing is idempotent: a rerun yields one copy.
6. Deploys build from one reviewed commit and list what a visitor will see changed.
7. The input queue holds several items ahead and refills on its own schedule, with its own artifact check.
8. Human gates are written before the run: money, credentials, first-time public output. The factory stops there and wakes you.

Where this comes from

The 240-80-30 chain and the 500,000-developer sample are from NBER working paper 35275, linked where cited. The Hitachi date and the factory lineage are from the Oxford University Press page for Cusumano's book. The lead-time definition is DORA's own wording, and the two review quotes are verbatim from Robert Laszczak's essay. The view count on Isenberg's video is my tracker's log for September 21, 2026, and the 1,137-test figure, the month-old "delivered" video and the empty-section morning are events from my own operation's logs, told as they happened. Your factory will fail differently, which is an argument for keeping your own logs, and none of this is a promise about your results.