Loop engineering is the meme. Outcome engineering is the job.

Skeptics hear "loop engineering" and see a scheduled job with better marketing, and on the mechanics they have a point. The while-loop is as old as programming, and businesses have trusted loops for decades for routine tasks like payroll runs and nightly backups that repeat pre-defined steps on a timer.

But those loops never decide anything. Deterministic code repeats exactly the steps a person wrote, and when something unexpected shows up, the job fails and waits for a human.

The new loops are non-deterministic. An AI agent reads the state of the work, decides the next step itself, takes it, and checks the result, hour after hour, with nobody prompting it. Whoever launched it can walk away and come back to finished work.

Setting up a loop keeps getting easier, because Claude Code and Codex are building the mechanics into the tools themselves.

Claude Code made workflow loops a native feature this May, and Anthropic has reported a single model run lasting 30 hours. 7

Likewise, an OpenAI engineer has described a single Codex run working for 25 hours, pausing at every milestone to fix what broke before moving on. 8 And I believe the AI labs will continue to make running loops easier and easier.

But, while any agent can run a loop, the question many companies are asking is, "Can I trust the agent running loops to make good decisions?" And often the answer is no.

Anthropic's engineers published the failure modes from their own long-running experiments with loop engineering. They watched agents wrap up work early as they approached what they believed was their context limit, mark features complete without testing them, and try to build everything in one pass, producing an app that looked finished but didn't respond to input. 9

They also watched agents identify real problems in their own output and then talk themselves into approving it anyway and "confidently praise" mediocre results. 9 Similarly, in a multi-university deployment study, six agents ran for two weeks and kept reporting tasks complete while the underlying system state contradicted them. 10

Agents are no better at fixing their own work than at grading it. In one benchmark across 14 open-source, non-reasoning models, they failed to correct 64.5% of the errors in their own output. 11

And reliability plummets if you stop watching altogether. On τ-bench retail customer-service tasks, GPT-4o passed 61% of tasks on a single try, and only about 25% when the same tasks ran eight times in a row. 12 Because a loop repeats the same kind of step hundreds of times, failures that look occasional in a demo become guaranteed somewhere in a long run.

Left alone, a loop under pressure to hit its goal will cheat. Practitioners running bare while-loop agents have observed loops deleting or skipping failing tests instead of fixing the code. 13

Machine-learning researchers call this specification gaming, "a behaviour that satisfies the literal specification of an objective without achieving the intended outcome." 14 The loop optimizes for the signal it's graded on, and gaming the signal is often cheaper or faster than doing the work.

The only thing that catches a gamed result is verification from outside the loop. But many teams are cutting this step because they can't keep up with how much code they're generating.

Faros AI's research across 22,000 developers found pull requests merged with no review rose 31% in a year, while individual output nearly doubled and organizational delivery stayed flat. The wait for review rose 441%. 15 A loop raises output further and leaves the checking further behind.

We can. At my company, loops run every day on research sweeps, competitive analysis, prototypes, and code, and they have made me and my team more productive.

But every failure in the previous sections originates from one principle. A loop cannot be the source of trust in its own output. The trust has to come from somewhere the loop can't make changes, whether that's a test the loop didn't write, a budget it can't raise, or a permission it can't grant itself.

Instead of keeping a "human in the loop" to approve every step, we shifted human judgment to the two ends of the run. Before the run, we write the rules. After each verified chunk of work, we judge results. In between, the guardrails are budgets, permissions, and a separate checker, so nothing depends on a person watching.

Building those outside anchors, and the decision rules they enforce, is a separate discipline from building the loop. I call it outcome engineering, and it's much more difficult to master.

The deliverable you build is a written specification with four required parts, the last of which is a way to verify results that the agents running the loop can't falsify. This is the key to running loops that are trustworthy.

The four parts of outcome engineering:

1. The result you want
2. The limits you won't cross to get there
3. The tie-breaker when two goals collide
4. The check that validates results from outside the loop

Every loop at Product.ai starts from an outcome file with those four parts. This is the one behind our competitive-research loop, lightly generalized:

RESULT      a brief on {topic} where every claim cites a primary source
LIMITS      public sources only; nothing behind a login; read-only access;
            a token budget the loop can't raise
TIE-BREAKER when completeness and verification collide, verification wins;
            the claim gets dropped
CHECK       a separate verifier agent reopens each cited source and fails
            any claim it can't confirm; a person reviews what gets flagged

The loop reads this file at the start of each cycle and can't edit it. The rest of this section walks through each part of outcome engineering, and how it works in the loops we run.

1. The result: Define what done means.

What it is. The result is the finished state you're asking for, written precisely enough that a stranger could tell whether or not you received it.

Why it's important. The result is the standard every later check grades against. If you leave it vague, the agent may misinterpret its goals and produce results you don't intend.

If you skip it. Imagine handing a loop a goal that sounds precise, like getting a page to load in under two seconds. On a complex site that page is one piece of a much larger system, and the loop could meet its target in ways that break other pages or that your stack can't support.

The result you want is the fast page without the disruption, and the file has to explain this. It might read: the page loads in under two seconds on real devices, no other page gets slower, and any infrastructure change needs a person's sign-off.

How we handle it. We believe a result is written well when a checker outside the loop could pass or fail it without asking you anything. Usually that means a measurable threshold or a checklist, because the loop needs a clear verdict to know when it's done.

"Write a good brief" fails that test because "good" is a judgment call, and two checkers could read the same brief and reach two different verdicts. So our research loop defines done as "every claim cites a primary source," a line any checker grades the same way.

2. The limits: Draw the lines the loop can't cross.

What it is. The limits are the actions the agent may not take, and they typically cover what it may spend, how long it may run, what it may read, and what it may change.

Why it's important. Limits keep agents behaving while a loop runs for hours unsupervised. Without clearly defined limits, a loop will drift, retry, and over-polish, producing busy work and devouring tokens at machine speed.

If you skip it. A loop with no hard stop has no reason to stop. It will spend whatever its retries demand, and it will keep spending long after the work has stopped improving. Seriously, don't skip this.

How we handle it. Our research loop's file limits it to viewing public sources only with read-only access, and specifies a token budget the loop agent can't raise.

Most of our limits involve one of the following hard stops: "a max iteration count, no-progress detection, and a token or dollar ceiling." 16 We enforce all three from outside the loop.

We cap budgets at the API key.

Every loop runs on a key with a hard spend limit, so the budget belongs to the key, and the loop can't modify it.

The obvious way around a capped key is to start more loops, and ours do this all the time as part of larger jobs. So we control the multiplication with inheritance. A child loop gets the same outcome, the same limits, and the same enforced stops as its parent, and it draws down the parent's budget instead of minting its own. Ten spawned loops can't multiply the cap.

Our harness also checks what's left of the budget before each expensive stage, and once the remainder drops below a floor, the loop stops taking on new work instead of starting something it can't afford to finish.

We fence permissions at the account.

An agent's account can read our production database and has no way to write to it. Loops work in sandboxes, and their changes reach production only through a review step the loop can't operate. Our longest-running loops write everything to a staging area and stop there. Only a person can move the work into our canonical library.

We assume a loop's own status reports can be wrong, because agents report success while systems break. So our monitoring reads the actual state of the system rather than the loop's account of it, and every run has a pause switch the agents can't disable.

Every outcome file also names when the loop must stop and call one of us, whether the budget runs out, progress stalls, or the work is about to touch something irreversible.

We never trust a prompt to enforce a limit.

The outcome file states our limits, but the enforcement never depends on the model reading them.

In one long-context benchmark, eleven of thirteen models dropped below half of their short-context performance once the input passed 32,000 tokens 17, part of a broader decay researchers have named context rot. 18 A rule defined in the context window is a rule the loop will eventually forget. A rule enforced at the permission layer is never forgotten, because it doesn't depend on the model remembering anything.

It's also why the loop reads the outcome file fresh at the start of every cycle. Each cycle begins with a short context and the file as its memory, so the rules are never more than one cycle old.

3. The tie-breaker: Decide which goal wins when two collide.

What it is. The tie-breaker names which goal wins when two or more outcomes defined in the spec collide mid-run.

Why it's important. If you write down your top five goals, you'll usually find two that can't both win. A loop that meets that conflict mid-run does whatever it thinks its instructions imply. This gives it a lot of leeway to make decisions that you don't agree with.

If you skip it. The loop still decides, just without you. Whichever goal is easier to score wins, and that choice repeats on every run until someone notices.

How we handle it. We find that resolving which goal wins for our agents helps us clarify our organizational values. Our research loop has two goals that can conflict: completeness and verification. A brief gets more complete with every claim it adds, but every added claim has to be confirmed, and some claims come from sources the verifier can't reopen. The loop can't deliver a brief that is both as complete as possible and fully verified.

We ultimately decided that we would rather ship a thinner brief than an unverified one. So, when completeness and verification collide, verification wins. In practice, the verifier fails any claim it can't confirm, and the loop deletes that claim instead of keeping it for coverage. The brief comes back shorter, and every line in it traces to a source.

The rules we settle this way join our grounding files, the documents we started writing in 2023 to tell our AI tools what we value, what we're trying to achieve, and what we will and won't do. A tie-breaker decided once keeps deciding.

We wrote a second tie-breaker into the same loop, between delivery and certainty. When a key evidence channel is down, the run still ships, but every finding comes out marked lower-confidence and stamped for re-verification once the channel returns. The loop can't remove the stamp, and the stamp travels with the work.

4. The check: Validate the results from outside the loop.

What it is. The check is how results get validated from outside the loop, by something the agents doing the work can't influence.

Why it's important. We spend most of our design time on verification, because the check is what makes the other three parts enforceable. Without it, you can't trust your results, and you can't be sure the agent will follow your documented limits and tie-breaker conditions.

If you skip it. The loop grades its own work, reports success, and you don't find out until you've shipped a broken app or released a false research report.

How we handle it. The check in our research loop has three legs: a verifier agent, a passing bar for claims, and a person.

The verifier has no stake in the work.

The verifier is a separate agent with read-only access that never saw the drafting session, so it has no stake in the brief it grades. It reopens each cited source, fails any claim it can't confirm, and flags the edge cases for a person. It costs a fraction of what the drafting loop costs.

We check every finished chunk of work.

The verifier grades after every completed unit of work, and the loop can restart from the last unit that passed. Waiting until the end of a long run lets one bad early step poison everything after it. Checkpoints a run can resume from are also most of the answer to state and recovery, the layer where long-running agents break most often: a loop that fails at hour nine restarts from hour eight instead of from zero.

A claim passes only when independent sources agree.

The drafting loop pulls evidence from several independent research channels. A claim passes only when at least three channels agree and their trails lead back to at least two independent sources. Three channels citing the same blog post count as one source. A claim with one channel behind it gets flagged instead of trusted, and when channels disagree, the brief presents the disagreement instead of averaging it away.

A person checks the biggest claims.

For the handful of claims a brief relies on the most, a person reads the primary source directly and compares it with what the loop reported. And every brief ends with a list of the gaps it couldn't close, so nobody mistakes silence for coverage.

The judge isn't always an agent.

Across our loops the judge is sometimes a script, sometimes a number in a system the agents can't edit, and often a person. Our prototype loop turns an idea into a clickable build the same day, and the judge is a person using the build rather than reading a progress report about it.

For a goal like page speed, the check might be an independent monitoring service confirming the load time on real devices, reviewed by someone who didn't build the loop. A sales team could judge an outreach loop by reply rate in a CRM the loop can't write to, with a person auditing a sample of what goes out.

For knowledge work with no mechanical test, nobody has fully solved the independent checker, including us. In the Berkeley survey, 74% of deployed agent systems relied primarily on human evaluation. 5 Our verifier narrows the human's job down to the flagged edge cases, but doesn't eliminate it completely.

If you don't master outcome engineering, you're less likely to get the results you want, and more likely to incinerate your budget.

On SWE-bench, two models within a point of each other on solve rate differed by roughly 10x in cost per suite, $376.95 against $36.64. 19 If you don't tell your agent which models to choose or enforce spending limits, costs can quickly get out of control.

Uber's engineers weren't even running loops when they burned the year's AI budget by April; nobody had tied the tokens to an outcome. 20 A loop doing busy work is a waste at any cap. A loop that produces a customer, a fix, or a decision delivers ROI.

And money is only the visible loss. Skip the check and gamed results get through, the untraced claim, the untested feature, and you learn about them from a customer instead of a verifier.

Before your next loop, or your first, stress-test one goal. Take something you'd like to hand to an agent team and write down the result, the limits, the tie-breaker, and how you'd validate the outcome from outside the loop.

If that's easy, build the loop around your document. If it's hard, you've uncovered your issues before burning a single token.

And you don't have to start with hours-long runs. Run loops that check in with you every few steps, and lengthen the leash as your outcomes and your checks improve. Ours run for hours without a check-in now, but that took three years of writing down what we value and where the lines are.

We'll keep running loops every day, and we'll keep spending most of our effort on the outcomes instead of the loops. The discipline of writing outcomes precisely enough that nobody can misread them keeps our agents safe, and it keeps our human team aligned too. Loop engineering won't work without it.

p.s., If you're running loops at your company, send me a note. I'd like to see what your outcome specs look like, and to hear what your loops keep getting wrong.

1. The practice predates the name. Researchers wired models into act-and-check cycles in 2022: Yao et al., "ReAct: Synergizing Reasoning and Acting in Language Models," arXiv:2210.03629. Working engineers have looped coding agents since July 14, 2025, when Geoffrey Huntley published Ralph: https://ghuntley.com/ralph/. The name "loop engineering" arrived in June 2026, almost a year after Ralph.
2. Boris Cherny, Acquired Unplugged live interview, June 2, 2026. Recap: https://workos.com/blog/boris-cherny-claude-code-acquired-interview-takeaways
3. Peter Steinberger, X post, June 7, 2026 (~2.2 million views). Context: Hiba Fathima, firecrawl.dev, June 11, 2026: https://www.firecrawl.dev/blog/loop-engineering
4. Sam Altman, X, February 15, 2026 ("Peter Steinberger is joining OpenAI to drive the next generation of personal agents"): https://x.com/sama/status/2023150230905159801 · TechCrunch, February 15, 2026: https://techcrunch.com/2026/02/15/openclaw-creator-peter-steinberger-joins-openai/
5. Pan et al., "Measuring Agents in Production," arXiv:2512.04123 (UC Berkeley-led; 20 case-study interviews and a survey of 86 deployed systems across 26 domains): https://arxiv.org/abs/2512.04123
6. Others have used the name for other things. An open-source project at outcome.engineering uses it for generating code from written specs, and an August 2025 HackerNoon essay applied it to writers handing work to AI agents: https://hackernoon.com/agentic-ai-and-the-rise-of-outcome-engineering. Neither describes the discipline in this essay, which is about trusting the work a loop hands back.
7. Anthropic, "Introducing dynamic workflows," claude.com blog, May 28, 2026: https://claude.com/blog/introducing-dynamic-workflows-in-claude-code · Anthropic, "Introducing Claude Sonnet 4.5" (the 30-hour autonomous run), September 29, 2025: https://www.anthropic.com/news/claude-sonnet-4-5
8. Derrick Choi, "Run Long-Horizon Tasks with Codex," OpenAI Developers blog, February 23, 2026: https://developers.openai.com/blog/run-long-horizon-tasks-with-codex
9. Prithvi Rajasekaran, "Harness design for long-running application development," Anthropic engineering, March 24, 2026: https://www.anthropic.com/engineering/harness-design-long-running-apps
10. Natalie Shapira et al., "Agents of Chaos," arXiv:2602.20021 — a multi-university red-team study; six primary agents in a two-week live deployment reported task completion while system state contradicted the reports: https://arxiv.org/abs/2602.20021
11. Ken Tsui, "Self-Correction Bench," arXiv:2507.02778 (NeurIPS 2025): https://arxiv.org/abs/2507.02778
12. Yao et al. (Sierra), "τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains," arXiv:2406.12045: https://arxiv.org/abs/2406.12045
13. Mathias Hansen, "Ship Features in Your Sleep with Ralph Loops," Geocodio, January 27, 2026: https://www.geocod.io/code-and-coordinates/2026-01-27-ralph-loops
14. Victoria Krakovna et al. (DeepMind), "Specification gaming: the flip side of AI ingenuity," April 2020: https://deepmind.google/blog/specification-gaming-the-flip-side-of-ai-ingenuity/
15. Faros AI, "AI Engineering Report 2026," April 2026 (~22,000 developers): https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways
16. Meghana Harishankara, "The 'AI loops' fight isn't about AI loops," Medium, June 12, 2026: https://medium.com/@meghanaharishankara/the-ai-loops-fight-isn-t-about-ai-loops-f87fe6d7e454
17. Modarressi et al., "NoLiMa: Long-Context Evaluation Beyond Literal Matching," arXiv:2502.05167 — at 32K tokens, 11 of 13 evaluated models drop below 50% of their short-length baselines: https://arxiv.org/abs/2502.05167
18. Kelly Hong, Anton Troynikov, Jeff Huber, "Context Rot: How Increasing Input Tokens Impacts LLM Performance," Chroma technical report, July 14, 2025: https://www.trychroma.com/research/context-rot
19. SWE-bench "Bash Only" leaderboard, runs of February 17, 2026: https://www.swebench.com/
20. Janakiram MSV, Forbes, May 17, 2026: https://www.forbes.com/sites/janakirammsv/2026/05/17/uber-burns-its-2026-ai-budget-in-four-months-on-claude-code/ · Lucia Auerbach, Inc., June 3, 2026: https://www.inc.com/lucia-auerbach/uber-blew-through-2026-ai-budget-in-four-months-now-it-is-capping-employee-use/91355199