The pressure
Suppose your delivery cycle is six weeks and the business expects three. Output must keep scaling without headcount scaling at the same rate, and you run several systems that share state, where a change in one can quietly break another.
Scroll ↓
Dark factories,
engineered for trust
01Specification
02Planning
03Build
04Gates
05Review
06Release
Gates → BuildA failing gate returns the failure as the agent's next input.
Human reviewA named person decides high-risk changes.
Release → SpecificationCompound: failures, fixes and review comments are written back into specs, tests and harness rules.
The idea
Agents write. Gates verify.
Humans decide.
The risk
At the same time the risk list grows: evaluations, guardrails, data compliance, code provenance, and keeping a person accountable for what reaches production. Agents already write code in your organization. The question is whether your delivery system can absorb that speed safely.
How it works
The delivery loop
01Specification
02Planning
03Build
04Gates
05Review
06Release
Gates → BuildA failing gate returns the failure as the agent's next input.
Human reviewA named person decides high-risk changes.
Release → SpecificationCompound: failures, fixes and review comments are written back into specs, tests and harness rules.
Specification
What must happen, in which system, under which constraints. Quality is defined before any code exists.
Planning
Agents map which services and modules the change touches and what its blast radius is.
Build
Agents write the change, module by module, against the specification.
Gates
Your tests, contract checks, security and policy scans run automatically on every change.
Feedback
A failing gate sends the agent back with the failure as input, not a ticket to a human.
Review
What passes every gate reaches a person as one decision, not a thousand diffs. Which changes must wait for a named person is set by a risk policy agreed with you.
Release
Merge and deploy under your policy: manual, or automated by risk tier that you set.
Compound
Every failure, fix and review comment is written back into the specs, regression tests and harness rules. The next loop starts smarter than the last.
From hand-built software,
to engineered production.
Gates.
Compound.
Trust.
Safe at scale
When one change can break another system
Multi-system estates share state, and a change in one service can quietly alter another's behavior. The factory pattern answers with structure, not promises.
Your estate
BillingIdentityCore product
Contract tests on every boundary: producer and consumer must agree before anything merges.
Digital twin environment
Okta twinSlack twinJira twinDocs twin
Behavioral clones of the services you depend on, so agents can rehearse failure modes too dangerous or rate-limited for live services.
Staged rollout
Flagged→Small cohort→Wider
Explicit limits on error rate, latency and satisfaction: a breach halts the rollout; automatic rollback where reversible.
The six safeguards, in detail
- Contract and integration testing. Every boundary between systems gets contract tests: producer and consumer must agree on the shape of what passes between them before anything merges. Agents run integration suites across every system a change touches, not only the repo it lives in.
- Digital twin environments. Faithful behavioral clones of the services you depend on. StrongDM cloned Okta, Jira, Slack and Google Docs, so its agents validate thousands of scenarios per hour, including failure modes too dangerous or too rate-limited to test against live services.
- Checks and evals. Lint, unit tests, contract tests and security scans give near-reproducible answers for what they cover; coverage gaps and false negatives remain. Behavioral evals measure what share of scenario runs satisfy the user, because agent behavior is probabilistic. Both inform the release decision; neither replaces it.
- Staged rollout and recovery. Changes ship behind flags, to a small cohort first, with explicit limits on error rate, latency and satisfaction. Crossing a limit halts the rollout and, where the change is reversible, rolls back automatically. Destructive schema changes and external side effects need compatibility shims and recovery plans agreed in advance; gates limit blast radius but cannot make every change reversible.
- Human signoff where risk lives. A risk policy agreed with you, per system and change type, routes high-risk changes (data model, authentication, payments, destructive migrations, personal data) to a named person. The factory proposes; it never self-approves where the policy says a person decides.
- Data boundaries. The harness and agents run with least-privilege credentials and logged access. Where code, prompts and data flow, including which models see what, retention and training settings, subprocessors and residency, are decisions made with you before anything runs, not assumptions baked into the design.
Why it compounds
A failure becomes permanent knowledge
Compounding is what separates a factory from a pile of generated code. One example: a migration agent renames a field and breaks a downstream consumer. A contract test catches it before merge. Then the write-back splits in two directions.
Visible to every future agent
- The failing test joins the regression suite, runnable on every future change.
- The specification gains a constraint: field renames require a compatibility shim.
- The harness gains a rule: before planning a migration, enumerate every consumer of the touched schema.
Held out from every agent
- A separate set of evaluation scenarios, which agents never see, measures whether behavior generalizes rather than memorizes.
- StrongDM calls this the holdout: it is how you catch an agent teaching to the test.
Each cycle makes a repeat of the same class of mistake less likely: knowledge accumulates in the system, not only in people's heads.
Every, which publishes its compound engineering method, reports running five software products with each built mainly by one person, and credits this write-back loop. One team's account, worth reading directly.
The honest part
What the evidence says
DORA's 2025 research found that AI acts as an amplifier: it magnifies the strengths and the weaknesses of the organization it enters. METR's early-2025 randomized trial found experienced open-source developers 19% slower with AI assistance; their 2026 follow-up estimates late-2025 tools now speed developers up, while warning that selection effects make the true size hard to measure. The honest reading: returns depend on the system around the tools, and the cited studies do not establish superiority over well-managed alternatives. The credible test is your own baseline; the pilot's metrics are how you would find out.
19%
slower with unmanaged AI assistance: METR randomized trial, early 2025
Amplifier
AI magnifies the delivery system it enters: DORA, 2025
5 products
run mainly by one person each: Every's published compound engineering account
Two ways to work with Lyra
Ship features with the AI factory
Faster, with a person checking every result before it ships.
Implement factories for your teams
Your teams, set up to run the factory pattern with us.
Run your own dark-factory experiment
A free implementation guide: how to pick one task type, set your baseline, run the loop and read the results. It comes with a Markdown prompt your AI agents can use to draft a plan from your company context, or you can simply use the framework yourself.
PDF and Markdown. Synthetic example only: nothing here needs your real data.
The factory reports on itself
Cycle time
Specification to production, per change. Baseline set in the first pilot week; reviewed weekly.
Gate pass rate
Share of changes passing every gate without human repair. The trend matters more than any single week.
Lessons written back
Regression tests, spec constraints and harness rules added per cycle. The compounding, made visible.
Human decision load
Changes reaching a person for sign-off, by risk tier. This should concentrate on high-risk tiers over time.
Detection and recovery
Time to detect a bad release and time to a safe state. The numbers that make speed safe.
The engagement design
How Lyra would build this with you
- 01Strategy call. We map your delivery flow, your systems and your constraints, and pick the one task type where a factory pays back fastest.
- 02Scoped pilot. Forward-deployed engineers stand up the harness, specifications and gates for that one task type, configured to your toolchain, accounts and merge policy.
- 03Delivery cycles. The factory ships in short cycles with review and merge gates under your policy, and its metrics are visible to you from the first week.
- 04Enablement. Your engineers learn to run and extend the factory themselves, so the capability stays in-house. We stay as long as we are useful.
Decisions made with you before anything runs
01Models and routing+
Which models run, hosted or inside your own inference, and what code and data each may see.
02Retention and training+
Provider retention settings and no-training terms, verified per vendor.
03Subprocessors+
Every service in the path named and approved by you before it touches code or data.
04Residency+
Where processing and storage happen, matched to your compliance obligations.
05Access and approvals+
Least-privilege credentials, logged access, and who can grant what.
06Sign-off risk policy+
Which change types, per system, always wait for a named person.
Lyra delivers with forward-deployed engineers embedded in your team and a network of collaborating engineers who implement this way of working and enable teams on it.
Read more
The sources
Every claim on this page links to its source. Read them directly.
The research behind the methods
Three papers from this year that shape how the loop is designed. Expand each for what it shows and how it informs delivery.
How much can language models memorize?Morris et al., ICML 2026+
What it shows. GPT-style models store about 3.6 bits of information per parameter. Models memorize training data until that capacity fills; past it, unintended memorization of specific datapoints decreases as the model generalizes.
How it informs the method. Data boundaries are design decisions, not afterthoughts. Which models run, what code and documents each may see, and vendor retention and training settings are agreed with you before anything runs.
Read the paper ↗Towards Autonomous Mathematics ResearchGoogle DeepMind, 2026+
What it shows. Aletheia, a research agent, generates, verifies and revises its own work end-to-end. Evaluated on hundreds of open Erdos problems, it solved a small number fully autonomously, and the paper ships human-AI interaction cards documenting the autonomy behind each result.
How it informs the method. The generate, verify, revise loop is the same shape as the factory loop, and explicit autonomy documentation is how a person stays accountable for what ships. This is a mathematics result, not a software delivery claim.
Read the paper ↗Position: AI/ML Deepfake Research is Misaligned with AIG-NCIIQiwei et al., ICML 2026+
What it shows. Years of deepfake research optimized for detecting fakes and protecting viewers while missing the dominant harm: non-consensual imagery of the person depicted. Knowing an image is synthetic does not undo that harm.
How it informs the method. Measurement can miss the real risk. The factory's gates and evals start from your risk list, designed from the harm you actually fear rather than the metric that is easy to collect.
Read the paper ↗Shapiro, the dark factory essay (2026)Where the dark factory name and method come from: the index-card version, from the writer who coined the term for AI adoption levels.Willison on StrongDM's factory (2026)How code nobody reviews gets verified: holdout scenarios, satisfaction as a quality metric, and the Digital Twin Universe.StrongDM, Software Factories (2026)The primary source from the team itself: spec-driven, non-interactive development with two rules about human hands.Every, Compound Engineering (2025)The plan, work, assess, compound loop: one team's published account of the write-back economics.Anthropic, agent harnesses (2025)How agents stay reliable over long horizons: initialization, small increments, and clean mergeable state between sessions.METR, the 19% slower RCT (2025)The cautionary baseline: a randomized trial found experienced developers 19% slower with unmanaged AI assistance.METR, the 2026 follow-up (2026)The follow-up: late-2025 tools likely speed developers up, and an honest account of why the true size is hard to measure.DORA report (2025)Google Cloud's research program: AI acts as an amplifier, so returns come from the delivery system around the tools.EasySpecs, Dark Factory Delivery (2026)The factory patterns and spec-first quality model from the reference material behind this page.
See it on your own workload.
One call. We map your delivery flow and name the first task type worth a factory. You leave with a position on dark factories whether or not we ever work together.