Loop engineering: agents that know when to stop

We've written before about harness engineering: the unglamorous code around a model call that turns a request into a guarantee. That post is about one turn. This one is about what happens when you let the agent take the next turn, and the one after that, each time deciding what to do based on what it just observed.

That cycle is a loop, and it fails in ways a single call never does. A single call is wrong or right. A loop is wrong and then builds on it. One misread observation in the second iteration is a confident conclusion by the sixth, with four steps of plausible reasoning stacked on top of it. The model doesn't flag the moment it started guessing, because from the inside there wasn't one.

So the hard part of a loop isn't making it run. Getting an agent to keep calling tools until it emits something is easy, and it is where most agent demos stop. The hard part is making it stop for the right reason, and making the thing it stops with trustworthy.

Separate what was observed from what was inferred

This is the failure that costs the most, and it looks nothing like an error.

We pointed an agent at our own production site to check that a set of routes were properly protected. It reported them all correctly gated, and explained why: it named the source file implementing the protection. The conclusion was right. The explanation was fabricated — not invented from nothing, but read out of the local checkout, which contained a version of that file that had never been deployed. The agent had a repository and a live site in front of it, and quietly treated one as evidence for the other.

Nothing in the output looked wrong. It cited a real file, with real contents, that really did implement what it described. It just wasn't the code answering the requests.

A loop that can read multiple sources will conflate them unless the loop makes conflation expensive. Concretely: require every factual claim to name the observation that produced it, and treat "I read the code" and "I made a request and saw this response" as different kinds of evidence with different standing. When an agent explains live behaviour, source code is a hypothesis. The response is the fact.

The same rule applies to state. A checkout can be ahead of production or behind it. A cached page can be stale. A test database can be a week old. If your loop consults something that has a version, the loop needs to know which version it consulted, or it will eventually explain today's behaviour with last week's code and sound entirely reasonable doing it.

A test that passes before the fix proves nothing

Loops that verify their own work are the ones worth building. But a verification step that would pass either way is worse than no verification, because it manufactures confidence.

We had an agent write a test for a fix to a subtle network-level bug. The test passed. That told us almost nothing on its own, so we deliberately broke the fix and ran the test again. It failed — which is the result that actually mattered, because it proved the test was sensitive to the thing it claimed to check.

Make that a step in the loop rather than a habit you rely on remembering. Any time the loop produces a check, it should also produce evidence the check can fail: run it against the broken state, assert on the specific property rather than on "no exception was raised," and be suspicious of any test that has never once gone red.

This generalises past tests. If your loop's exit condition is "the verification passed," you need to know the verification was capable of not passing.

Give the loop the capability, not just the instruction

An agent can only fail at things it can attempt. A surprising share of loop failures are not reasoning failures at all — they're an agent asked to do something it has no mechanism to do, working around the gap in whatever way the instructions left open.

We asked an agent to exercise a signed-in interface and gave it a browser with no way to be signed in. It did the correct thing: it hit the login page, refused to authenticate, and stopped. But "correct" was luck as much as design — we'd told it never to type credentials, and had we not, it would have tried, in production, on a real login form. The gap was ours. We wrote instructions describing a task the tools couldn't perform.

Before you tune the prompt, check the loop can physically do the job: the tools it needs, the permissions those tools need, the credentials it will never be allowed to hold. When those don't line up, the model will improvise, and improvisation is exactly what you don't want at the point where the design ran out.

"I couldn't" has to be a first-class result

A loop with only two outcomes — success or crash — will report success. Not out of dishonesty; out of shape. If nothing in the design represents attempted and blocked, the pressure to produce a final answer will quietly convert a skipped step into a passed one.

So build the third outcome. Give the loop an explicit way to end with "this part was not done, and here is why," and make it as structurally normal as success. In our runs an agent that can't reach a step is expected to name the step, name the blocker, and finish the rest — and a report that says nothing about a step it was asked to perform is treated as a defect in the run, not a clean pass.

The tell that you've got this wrong: your loop never reports partial results. Real work is partial constantly. A loop that is always either complete or broken is a loop that is rounding.

Bound the loop on progress, not just on count

Iteration caps are necessary and insufficient. A cap stops runaway cost; it doesn't stop the more common waste, which is a loop that iterates productively-looking for six turns while getting no closer.

The cheapest useful addition is a progress predicate: something the loop can evaluate that distinguishes "different" from "better." Retried a failed query? The new error should differ from the last one. Searching? The candidate set should be narrowing. If two consecutive iterations produce the same observation, the loop is not thinking, it's stuck, and the correct move is to stop and report the sticking point — which is far more useful to a human than the summary it would have produced after four more turns.

Pair that with a hard cap for the cases the predicate misses, and make the cap's expiry a reported outcome rather than a silent truncation.

Escalate privilege inside the loop, deliberately

A loop that can read is cheap to be wrong with. A loop that can write is not. The useful pattern is to make the default iteration read-only and require an explicit, narrow step to do anything that changes state, with the destructive shapes unavailable rather than discouraged.

This matters more in a loop than in a single call, because a loop's later steps are chosen by the agent based on earlier results — which means the inputs to a privileged action may themselves be model output, several turns removed from anything a human approved. "The model decided to" is not an authorisation. Anything irreversible should sit behind a check written in code, evaluated on the actual arguments, at the moment of the call.

Where to start

Take the loop you have and ask three questions.

When it finishes, what exactly ended it — a real completion condition, or the model choosing to stop talking? When it reports a fact, can you trace that fact to a specific observation, or only to a plausible narrative? And what happens when a step it was asked to do turns out to be impossible — does that reach the person reading the output, or does it disappear?

Most loops we've built failed on the second and third long before they failed on capability. The model was fine. The loop just had no way to tell us what it hadn't done.