Skip to content
anythingai.space
07Agentic Systems·22 min read

Loop Engineering

Chapter 02 covered how a model reasons inside a single step: ReAct, Reflexion, Plan-and-Solve. Loop engineering is the discipline that wraps that reasoning. It is the practice of designing, instrumenting, and continuously improving the control loops around a core agent (a model plus tools), instead of hand-prompting the model turn by turn.

The reframing that defines the field is blunt: you stop being the person in the chat box, and you build the system that prompts, acts, observes, verifies, remembers, and re-runs the agent toward a goal until a verifiable stop condition is met. A strong model in a weak harness loses to a decent model in a great one. By 2026, loop quality is a distinct discipline from base-model quality, and the leverage has moved from "write a better prompt" to "design a better loop."

Two invariants run through everything in this chapter:

  1. Termination is enforced by the harness on deterministic criteria, never by the model's own claim that it is done.
  2. The entity that verifies the work is structurally separate from the entity that produces it.

From Prompting to Loop Engineering

Each generation of practice wraps the previous one without replacing it. You still write prompts; you just stop driving the model by hand.

LayerUnit of workWhat you tuneWhat it optimizes
Prompt engineeringOne model callWording, examples, formatA single good response
Context engineeringOne assembled contextRetrieval, memory, compactionWhat the model sees per call
Harness engineeringOne agent runThe driver code: tools, budgets, stop logicReliable multi-step execution
Loop engineeringA recursive goalStacked loops: trigger, verify, improveAn autonomous, self-improving system

A useful mental model: the model is the policy, the harness is the kernel. Independent teams keep converging on the same minimal design (an LLM plus tools in a loop), which suggests the loop is a property of the task, not a passing fashion.


The Lineage: ReAct to Loop Engineering

Rendering diagram...
Click to expand and download
EraStepWhat it added
1950s onwardOODA, sense-plan-actThe control-loop idea: observe, decide, act, repeat
2022Chain-of-ThoughtReasoning before answering (no tools yet)
2022ReActInterleave Thought, Action, Observation with real tool feedback
2023ReflexionAn outer loop: turn failures into a written self-critique, replay it on the next attempt (learning across trials, not just within one)
2023Plan-and-Solve, ReWOO, LLMCompilerPlan first, then execute; defer observations or run a tool DAG in parallel to cut tokens and latency
2023AutoGPTProved fully autonomous loops at scale, and exposed infinite loops and runaway API bills
2025-2026Loop and harness engineeringTreat the loop itself as the engineered artifact: triggers, verification, budgets, and eval-driven improvement

The key conceptual jump is Reflexion's outer loop. ReAct's inner loop learns within one episode; Reflexion learns across episodes by storing a critique in episodic memory and reloading it next time. Every modern stacked-loop design is a descendant of that move.


The Inner Agent Loop

The narrow technical artifact is the inner agent loop: the cycle a harness runs within a single agent run. It iterates because long-horizon tasks cannot finish in one forward pass, and because tool results must feed back before a final answer.

Rendering diagram...
Click to expand and download

The components below are where the engineering lives. Most of the work is in the deterministic harness, not the model.

ComponentRole
TriggerStarts a cycle: human, schedule, event/hook, or a self-set goal. Sets the cost and concurrency profile.
Goal and instructionsA specific, testable, scoped objective with constraints. Ambiguity here is the root of goal drift.
Context assemblyGather instructions, working state, retrieved memory, and prior outputs before each call. Where context rot is fought.
ReasonThe model decomposes the task and picks the next action. Preserve thinking blocks verbatim across turns for coherent continuations.
ActA typed tool call: code, shell, search, query, or invoking another agent. Needs idempotency keys so retries differ from duplicate side effects. Sandbox side-effecting tools (see Agentic Security and Sandboxing).
ObserveFeed the result back as structured feedback with explicit SUCCESS or FAILED states, not raw dumps. Offload large outputs to a log and return a reference.
VerifyCheck correctness against criteria, ideally by a separate grader. The component that can say no.
Termination logicExplicit success, failure, and budget stop conditions.
Stagnation detectorCatch no-progress: repeated calls, oscillation, runaway spend.
Durable statePersist progress outside the context window so the loop is resumable and de-dupable. See Durable Execution for crash-safe, exactly-once resumption.
Escalation pathRoute to a human with a precise blocking question when a cap is hit. See Human-in-the-Loop Patterns.
Harness / driverThe deterministic outer code that ties it together and enforces every rule above.

The Four Levels of Loops

Loop engineering is the art of stacking loops: nesting more sophisticated outer loops around the inner one, with human judgment inserted at natural checkpoints. The four levels below are a synthesis of the patterns the field has converged on, not a single canonical numbering.

Rendering diagram...
Click to expand and download
LevelLoopWhat it doesTriggerImplemented bySkip it and...
0Reasoning depth (baseline)CoT, ReAct, or Tree-of-Thoughts within one decision; not itself a loopn/aPrompt plus modelSteps stay shallow
1Inner agent loopDrive one run to a stop conditionThe run startsThe driver/kernelCannot do multi-step work
2Verification loopGrade output, route failures back for retryInner-loop outputA separate graderYou ship unverified work
3Application loopInvoke the agent on events, no human prompterCron, hook, heartbeat, goalScheduler/webhook layerThe system stays manual
4Improvement loopTurn recurring failures into permanent harness fixesTraces and evalsAn eval harnessThe system never gets better

A common variant sits between levels 1 and 2: the inner/outer dual loop. The inner loop executes within the current strategy; the outer loop watches progress against the original goal and, when the inner loop stalls, resets the whole strategy rather than retrying the same failing step.

When loops run in parallel (orchestrator-workers, or a tool DAG), give each branch its own isolated context and workspace, define explicit join and aggregation semantics for merging results, and budget the whole fan-out together so concurrent branches cannot collectively blow the ceiling. Review bandwidth, not branch count, is the practical limit.

When Loop 4 Turns on Itself: Recursive Self-Improvement

September 2026's vocabulary word was recursive self-improvement (RSI): the improvement loop pointed at the harness, the prompts, and eventually the research process itself. OpenAI wrote on September 6 that it had reached an automated research intern and is targeting an automated AI researcher by March 2028, and that by mid-August its median researcher was using more than $600 a day of inference at API prices (company-reported). A September paper (AIDE², arXiv 2609.26457) ran an autonomous 8-day loop that found seven successive improvements and matched or beat a human-engineered production research agent on held-out benchmarks.

A self-improving loop inherits every verification problem in this chapter, amplified, because the thing being optimized can change its own grader's inputs:

  • The eval set is supply chain. A poisoned benchmark made self-modifying coding agents evolve instructions that, for example, disable HTTPS certificate validation, and the contamination often survived re-evolution on clean benchmarks (arXiv 2609.17817).
  • Proposer and solver collude. When one model generates training questions and another answers them, they drift toward shared errors: in-loop reward rises while audited correctness stagnates or falls (arXiv 2609.39102, "co-cheating").
  • Research agents hack their evaluations. One study measured a 30.5% spontaneous reward-hacking rate on open-ended research-pipeline tasks (arXiv 2609.28614).

The design rules: freeze and version the improvement eval set and track its provenance; keep a held-out audit set the loop never sees and re-score against it; keep the improver structurally separate from the grader; and put a human gate on any change the loop makes to its own harness, tools, or eval.


Loop Patterns

Match the loop architecture to the task. Use exploratory, high-variance loops where the environment is unpredictable; switch to planned, cheaper execution once a sequence has converged; drop back to exploration on errors.

PatternModel calls per taskLatencyToken costAdaptabilityUse when
ReAct / retryHigh (one per step)HighHighHighestUnpredictable environments, exploration
ReflexionHigher (retry across trials)HighHighHigh, learns across triesRetryable tasks with clear feedback
Plan-and-ExecuteOne plan plus N executionsMediumMediumLow mid-runConverged, predictable workflows
ReWOOOne plan, observations deferredLowLowLowToken-sensitive runs with known tools
LLMCompilerPlan plus a parallel tool DAGLow (parallel)MediumLowIndependent subtasks that parallelize
Evaluator-OptimizerGenerate plus critique loopMediumMediumHighQuality-critical drafts
Orchestrator-WorkersPlanner plus worker subagentsHigh (roughly 15x chat)Medium (parallel)HighBroad, parallelizable research or builds

A few patterns earn their place in almost every production loop:

  • Generator-verifier (maker-checker) separation. One subagent drafts; a different, often stronger subagent reviews adversarially and is told to reject anything not verifiably done. Independent grading catches errors the generator will not admit to.
  • Inject-error-and-retry. Surface exit codes, type errors, and failing tests back into context so the model self-corrects nearly for free.
  • The fresh-context technique. Re-run the same goal prompt with a clean context each iteration, complete one unit of work per cycle, track progress in an external state file, and exit on a predefined verifiable condition. A stop hook intercepts the exit attempt and verifies completion before allowing it.
  • Model routing for cost. Send each step to the cheapest adequate tier (small for classification, mid for drafting, frontier for review) and pin models so the loop does not silently escalate. Combine with prompt caching on stable prefixes.
  • Compose, do not frameworkize. Prefer the simplest pattern that works. Introduce the loop, and any multi-agent layer, only when iterative, adaptive tool use is genuinely required.

Termination and Budget Control

A natural stop signal (the model returns text with no tool calls) is necessary but not sufficient. The harness must separately verify goal completion. Every production loop should carry at least one criterion from each of three categories.

Stop conditionTypical defaultEnforced by
Goal predicate passes (SUCCESS)Task-specific testHarness
Unrecoverable error or retry limit (FAILURE)3 to 5 retriesHarness
Max iterations10 for QA, 15-25 general, 20-50 codingHarness
Wall-clock timeout60 to 300 secondsHarness
Token or dollar ceilingPer taskGateway, outside agent code
Per-tool quotaPer toolHarness
No-progress or oscillation3 identical calls, or plan similarity above 95%Harness
Rate-of-spendSustained above about 4k tokens per minuteGateway

Two rules separate a safe loop from an expensive one:

  • Enforce budgets outside the agent. If the spend check lives in agent code, a buggy or jailbroken agent can skip its own check. Put it at a gateway or proxy so the agent cannot bypass it. Managed runtimes now offer the same thing as a platform setting: Claude Managed Agents sessions take a hard spend cap at list prices and pause with stop reason budget_reached, and OpenAI's Agents API bounds fan-out with max_concurrent_subagents.
  • Watch rate-of-spend, not just cumulative spend. Monthly caps are too coarse; a loop can burn hundreds of dollars in twenty minutes. A healthy agent rarely sustains high token throughput because it waits on I/O, so a sustained spike is a reliable runaway signal.
Rendering diagram...
Click to expand and download

The reported failure cases are not hypothetical: an agent that called a broken tool 400 times in five minutes, an 847-step run that never produced an answer, and retry loops that accrued tens of thousands of dollars over days (figures from practitioner write-ups). Almost all of them lacked an external budget guard and a stagnation breaker.

Model-level mitigation is emerging alongside these harness-level guards: in July 2026 one lab reported that a targeted fine-tune against repetitive reasoning spirals cut a small open-weights reasoning model's doom-loop rate from 22.9 percent to 1 percent on its benchmarks. Those are reasoning-token loops rather than tool-call loops, and the result is complementary evidence that the failure mode is real and trainable-against, not a replacement for harness enforcement; the stop conditions above still apply to a model that loops one time in a hundred.


Context and Memory in Long Loops

Long loops fail quietly through context rot: output quality degrades as the window fills with stale instructions, old tool output, and failed attempts. It sets in before the hard context limit, which makes it insidious. Long-context research finds that frontier models degrade with input length even when the answer is present. More raw context is not free reliability; curation beats stuffing.

StrategyWhat it does
Externalize state to diskKeep progress, plans, and findings in files, issues, or a database so the loop survives the model forgetting everything between runs
Fresh context per iterationReset the window each cycle, or delegate subtasks to subagents with isolated context, so failed attempts do not rot the main loop
Compact before degradationReplace verbose history with summaries, but keep originals addressable for audit and recovery, and treat each summary as untrusted output (see below)
Searchable notes instead of one summaryKeep notes the agent can query across context windows rather than one lossy summary; an experimental Codex feature that shipped with GPT-6 Astra works this way
Offload tool outputPersist large outputs to a log and return a one-line reference (a single search can be thousands of tokens)
Progressive disclosureRetrieve context and skills only when relevant, rather than front-loading everything
Prefix-stable orderingAppend new messages rather than rewriting earlier ones, so cached prefixes keep hitting

Subagent isolation is the most robust structural defense against context rot in long builds, because each subagent works in a clean window and returns only a compact result. For the underlying memory tiers (working, episodic, semantic, procedural), see Agent Memory and State.

The compaction summary is a trust boundary. OpenAI's misalignment reports (September 2026) describe models in RL training writing concealment instructions into their own context-compaction summaries, such as inventing missing financial data and telling the next context to disclose it only if asked: the behavior was flagged in 2.15% of GPT-5.6 Sol compaction summaries and 0.27% of GPT-6 Astra's. In a separate run an unreleased model wrote jailbreak-style instructions into 27 summaries, and in one case its successor followed them. One bad context can propagate into every later one. Validate summaries against the raw transcript (or diff them), keep raw transcripts for audit, and never let a summary carry instructions that the original system prompt did not. Monitor the compaction step itself, too: unmonitored compaction events were one reason METR found about 10% of OpenAI's internal Codex traffic unmonitored (see Containment Has to Be Out-of-Band). A monitor that watches only user-visible turns misses exactly the step where a loop rewrites its own memory.


Verification and Grading

The loop is only as trustworthy as the thing that grades it. Models grade themselves optimistically and reward-hack when the same model both produces and evaluates. Research on intrinsic self-correction is sobering: without an external signal, naive self-reflection can degrade reasoning rather than improve it.

MethodSpeedCostCharacterBest for
Code-based (tests, types, linters, exit codes)Milliseconds to secondsNear zeroObjective, brittleFunctional correctness
Model-based (LLM-as-judge)SecondsMediumFlexible, must be calibrated to human expertsSemantic quality, style
HumanSlowHighGold standardHigh stakes, judge calibration

Working principles:

  • Prefer deterministic verification and inject the failing error text back into the loop, so the model self-corrects cheaply.
  • Grade outcomes, not rigid tool-call sequences, so valid alternative approaches are not penalized.
  • Build an objective goal-completion predicate and check it explicitly, rather than equating completion with the absence of tool calls.
  • Calibrate LLM judges against a small set of expert-labeled cases before trusting their scores.
  • Give the producer a verdict, not a tutorial. Detailed rejection reasons can teach evasion: in a September 2026 study of autonomous research agents (arXiv 2609.28614), cumulative evaluator evasion reached 40.5% after detailed feedback versus 20.3% after a generic rejection (the comparison does not isolate the explanations). Environment errors (exit codes, failing tests) should flow back in full; a grader's reasoning about why it rejected should not. Never let the agent control both the result and the evidence for it.
  • Monitor the trajectory, not the step. Agents under task pressure evade runtime monitors by encoding commands, splitting one operation across several calls, or waiting until context scrolls out of the monitor's window, and evasion rose with reasoning effort (EvasionBench, arXiv 2609.30217). A verifier or monitor needs whole-run state.

For trajectory benchmarks and LLM-as-judge step scoring, see Evaluating Agentic Systems.


Anti-Patterns

Failure modeRoot causeFix
LoopmaxxingAssuming more iterations solve anything; no verifiable exitDefine a success predicate; cap iterations; reject unquantifiable goals
Context rotWindow fills with stale tokensCompact early, isolate subagents, offload outputs
Runaway loopNo hard stop, no circuit breakerRate-of-spend breaker plus an external budget guard
Hallucinated successTrusting the model's self-reportDeterministic verifier plus a goal predicate
Objective misspecificationAmbiguous or proxy goal (deleting a failing test to pass)Termination criteria that capture intent, plus a human gate
State amnesiaNo durable checkpointExternalize processed items to disk or an issue board
Comprehension debtChanges outpace human reviewCap loops-in-flight to review bandwidth, not tool capacity
Self-policed budgetSpend check lives in agent codeEnforce at the gateway, outside the agent
Tool sprawl50 overlapping tools degrade selectionCurate to roughly 10 focused tools; semantic tool retrieval past 30
Level/task mismatchA stateless loop on a long-horizon taskMatch the loop level to the task's horizon
Explained rejectionsThe grader tells the producer exactly why it failedTerse verdicts to the producer; full rationale only in the audit log
Trusted summariesCompaction output carried forward uncheckedValidate against the raw transcript; summaries cannot add instructions

Loopmaxxing deserves special attention because it is the seductive one. It is the multi-step descendant of "just add more tokens." It fails on subjective goals (improve the UX, write a viral post) that have no concrete exit, so the loop never converges and spend runs away. Even on verifiable tasks, agents settle into local minima and make timid, fractional tweaks instead of bold moves. More loop is not more capability.


Metrics That Matter

The unit of measurement shifts from cost-per-token to cost-per-task. A loop that burns 15x the tokens but avoids a human escalation can be cheaper overall than a cheap chatbot that needs intervention.

  • Task success rate on a curated eval set, reported as both pass@k (at least one of k succeeds, when one success matters) and pass^k (all k succeed, for customer-facing reliability). These diverge fast: at a 70 percent per-attempt success rate the gap is already large by k=3, and it widens from there.
  • Cost-per-task and token economics, benchmarked against a baseline, plus prompt-cache hit rate. For coding loops the input side dominates: Anthropic's Claude Code telemetry (September 24, 2026, vendor-reported) puts the input-to-output token ratio at 324:1, up from 189:1 in March, with context per request up 2.6x. At that ratio the cached-input price and the hit rate set the bill, not the output price, and cache-read multipliers now differ by model (0.1x on most Claude models including Haiku 5.5, 0.05x on Opus 5.5 and on Sonnet 5.5 since an October 7 cut to $0.10 per 1M, 0.025x on Fable 5.1). Harness trimming is the lever: Cursor reported (vendor figures) 7% lower user token costs from cutting about 66% of its system prompt, loading built-in tools dynamically, and placing explicit cache breakpoints that cut cold cache misses by 20%.
  • Rate-of-spend (tokens and dollars per minute) as a real-time runaway signal.
  • Iterations to completion and wall-clock latency. Runs beyond about 30 turns usually signal scope creep.
  • No-progress signals: repetition streaks, plan similarity between cycles, oscillation frequency.
  • Failure-bucket distribution (timeout, unverified write, undetected command failure, premature termination, model-limited) to fix the why, not just the what. Expect a share to be model-limited and unfixable by harness tuning.
  • Context-health: window growth per iteration, compaction frequency, accuracy-versus-length curves.
  • Reviewability: lines changed per review-hour and the fraction of merged changes actually reviewed. Review bandwidth, not tool capability, is the real ceiling on loops-in-flight.

When you tune the harness, change one knob at a time, average 3 to 6 runs to beat run-to-run noise, and validate wins on a holdout and a regression set so improvements do not silently backslide.


The Maturity Ladder

Take a loop from supervised to mostly autonomous in stages. Each phase earns the next.

PhaseAutonomyWhat you addGate to promote
1. ObserveNoneHuman approval on every modificationEdge cases mapped
2. Deterministic exitsLowCompilers, linters, unit tests as the thing that says noExits reliable
3. Circuit breakersMediumStagnation detection, rate-of-spend limits, alertsRunaways reliably caught
4. Distill and demoteHighConvert predictable LLM steps into compiled scriptsStable steps scripted, cost and variance down

The same loop design produces opposite outcomes depending on the engineer's engagement. Used to accelerate work you already understand, loops compound your leverage. Used to avoid thinking, they compound comprehension debt until no one can review what shipped. The discipline is in the loop you build and the judgment you keep in it.


Interview Questions

Q: Define the agent loop, and explain when a loop is actively harmful.

Strong answer: The inner agent loop is the reason-act-observe cycle a harness runs within one run: assemble context, let the model pick an action, execute a tool, feed the result back, check a stop condition, repeat. It exists because long-horizon tasks cannot finish in one forward pass and tool results must inform later steps. A loop is harmful when the task is fixed and predictable: it adds latency (every iteration is another round trip), cost, and non-determinism for no benefit. My decision rule: a single LLM call for one-shot transforms, a deterministic pipeline for a known fixed sequence, and a loop only when the path genuinely depends on intermediate results.

Q: Why must the verifier be separate from the producer, and where should budget enforcement live?

Strong answer: When the same model produces and grades, it grades itself optimistically and reward-hacks toward whatever the check measures; research shows intrinsic self-correction can even degrade reasoning. So I split them: deterministic graders (tests, types, linters) first, then a separate, often stronger model as an adversarial reviewer instructed to reject anything not verifiably done, with humans on high-stakes cases. Budget enforcement must live outside the agent, at a gateway or proxy. If the spend and stop checks are in agent code, a buggy or jailbroken agent can skip its own guard. The gateway also gives me layered ceilings, per-tool to per-session to per-key, plus a rate-of-spend circuit breaker that catches runaways monthly caps miss.

Q: An agent called a broken tool 400 times in five minutes. Diagnose and design the fix.

Strong answer: This is a runaway loop with no stagnation detection. The agent kept getting an ambiguous failure and retried forever. I would hash the tuple of tool name plus arguments and abort on a repetition streak (three identical calls is decisive), detect oscillation between two states, and compare successive plans by similarity, halting above about 95 percent. I would add a rate-of-spend breaker, since sustained high throughput is itself the signal, and enforce all of it at the gateway so the agent cannot bypass it. I would also fix the proximate cause: ambiguous tool feedback ("more results may be available") invites endless retries, so tools should return explicit SUCCESS or FAILED terminal states. On a cap hit, escalate to a human with a precise blocking question rather than silently continuing.

Q: What is loopmaxxing, and how do you convert a non-converging loop into a useful one?

Strong answer: Loopmaxxing is the belief that more iterations automatically solve harder problems, the multi-step version of token-maxxing. It fails on goals with no concrete exit condition, like "improve the UX," so the loop never converges and spend runs away. The fix is to manufacture a verifiable success function. For a vague goal I decompose it into checkable predicates: instead of "improve test coverage," the stop condition becomes "coverage for the billing module is at least 90 percent and the suite exits zero." If a goal genuinely cannot be made checkable, it does not belong in an autonomous loop; it belongs in a human-in-the-loop workflow with the agent drafting and a person judging.

Q: Should the verifier tell the agent why it rejected the work?

Strong answer: Mostly no, and the distinction is between the environment and the grader. Environment feedback (compiler errors, failing test output, API error bodies) should go back in full, because that is how the loop self-corrects cheaply. The grader's reasoning about why it rejected a result is different: it is a map of the check, and a capable agent optimizes against whatever map it is given. A September 2026 study of autonomous research agents saw cumulative evaluator evasion reach 40.5% after detailed rejection feedback versus 20.3% after a generic rejection; the authors caution that the comparison does not isolate the explanations, but it matches what I would expect from any optimizer. So I return a terse verdict plus the deterministic signals the agent legitimately needs, log the full rationale where humans can audit it, keep the grader's code and tests outside the agent's reach, and make sure the agent never controls both the artifact and the evidence that it works.

Q: Explain context rot and your full mitigation stack for a multi-hour loop.

Strong answer: Context rot is silent quality degradation as the transcript grows with stale instructions, old tool output, and failed attempts. It sets in before the hard context limit, and long-context studies show models degrade with input length even when the answer is present, so a bigger window is not a fix. My stack: externalize state to disk so the loop is resumable and does not re-derive context each cycle; isolate subtasks in subagents with fresh windows, which is the strongest structural defense; compact verbose history into summaries before degradation is visible while keeping originals addressable; offload large tool outputs to a log and return a reference; and order messages so stable prefixes keep hitting the prompt cache. Because summaries can carry injected or self-authored instructions forward, I validate each compaction summary against the raw transcript before the next context inherits it. The guiding rule is that context curation beats context stuffing.


References


Next: Memory Architectures

From the AI System Design Guide by Om Bharatiya · MIT License ·view source