Work story · The software factory
Building a software factory around quality
Our five-engineer team produced 6,000 pull requests over 20 weeks while building enterprise AI software. That volume depended on a system that took planning, verification, and feedback seriously.
I built the team and contributed to the early planning. The team built and refined the factory itself.
6,000 PRs
from five engineers in 20 weeks, across two enterprise platforms
8,308 tests
7,897 unit and 411 end-to-end, green at the gate of one recent build loop
~64%
of code-change tickets on one program were corrections the audit loop found after planning
34% faster
median end-to-end run after telemetry-driven fixes, with every one of 411 tests kept
The system
One continuous loop
Quality isn’t a phase after the build. It runs during the build and feeds back into the next one.
- 01IntentProduct intent, Linear context, the codebase, and frontend prototypes.
- 02PlanWaves of tickets, each with an exhaustive build spec and its dependencies.
- 03BuildThe loop dispatches builder agents into isolated worktrees, several at once.
- 04VerifyRisk-classed gates, plus blind audit agents that never edit code.
- 05LandOne deliberate milestone merge under the full suite, then reflection.
↻ Observe · Distill · Inherit
Telemetry, retrospectives, and debug reports become gate rules, skills, and preflight checks that the next milestone starts with.
People, throughout
People author intent, review escalations, and hold the milestone boundary. They decide what the product should be.
01
Define the intended result
Our planning process brings together product intent, Linear context, the existing codebase, and frontend prototypes. Detailed specifications give builders and reviewers a shared reference for what should exist and how it should behave.
That preparation matters because agents can move quickly on an incomplete or mistaken understanding of the work. The tracker is the source of truth: every unit of work is a ticket, and the loop re-reads its state from the tracker each cycle, so a crashed run resumes without special recovery code.
02
Give agents an independent reference
Ask an agent to write tests for code it just wrote, and it will write tests that pass that code. The suite stays green while the product drifts. You need an oracle that isn’t the code.
Frontend
A runnable prototype sets expected structure, appearance, and interaction. Blind audits grade each surface on structure, style, responsiveness, interaction, and content.
Backend
Build specs and architectural constraints set expected behavior. Read-only audits check code quality, authorization and tenant isolation, migration safety, and API contract drift.
When code and the reference disagree, the reference wins.
03
Build verification into delivery
The factory combines automated checks, end-to-end testing, independent audits, and human review at milestone boundaries. A fast lane of lint, types, and changed-area tests keeps iteration quick; the full suite, including end-to-end and real-database tests, is the only gate that ships.
Merged pull requests don’t close a wave of work. A clean blind audit round does: zero new findings, with every gap filed as a ticket that builders fix and auditors never touch. After three rounds on one surface, a person steps in.
People remain responsible for product intent, escalations, and decisions that require judgment.
04
Use telemetry to improve the system
Agent sessions, CI runs, retries, and audit findings reveal how the factory actually behaves. Every coding session is traced, and CI reads the test runner’s own report rather than just the exit code: a test that passes on retry is a flake, invisible if all you look at is green.
The team uses that evidence to refine skills, instructions, preflight checks, and quality gates. After one overnight run, a retrospective on the session itself produced ten harness fixes. None were model problems. They were harness problems, and those are fixable in an evening.
The rulebook
Every gate rule is a compressed near-miss
The harness is versioned like a product. It holds roughly forty rules, and each traces back to something that almost shipped broken.
Example
A fast local check used an in-memory database and silently skipped every real-database suite for an entire build. The whole class went red only at the landing gate. Now the push gate runs those suites on real infrastructure whenever a change could break them.
What I learned
Operating a software factory requires ongoing engineering and organizational attention. The model is one part of the system; specifications, tools, verification, and feedback also shape the outcome.
Verification isn’t overhead on the real work. At agent speed, it is most of the real work, and it has to be budgeted that way.
Building the team and seeing that system develop reinforced the importance of giving engineers useful context, clear ownership, and visibility into their work. It also revealed an opportunity: other teams would need that visibility without building all the instrumentation themselves.