Harness engineering: the part of an AI agent that isn't the model

Most demos of an AI agent are a model, a prompt, and a tool. Most production AI agents are a model, a prompt, a tool, and a few thousand lines of unglamorous code deciding what the model is allowed to see, what it's allowed to do, what happens when it's wrong, and how anyone would know. That surrounding code is the harness, and building it deliberately, rather than accreting it after each incident, is what we mean by harness engineering.

The distinction matters because of where teams spend their effort. When an agent misbehaves, the instinct is to reach for the prompt: add a rule, add an example, add a stern "NEVER do X." Prompts are worth tuning, but a prompt is a request. A harness is a guarantee. If your agent must never write to a production database, that fact belongs somewhere a model cannot talk its way past.

The harness is where your invariants live

Start by writing down what must be true regardless of what the model outputs. For an AI analyst pointed at a customer's live database, our list is short and non-negotiable: every query is a single read-only statement; no query returns raw card numbers or other sensitive values; no failure loops forever; no number reaches the user that the model computed in its head.

Each of those is enforced in code, not in the prompt.

Take read-only. The model is asked for a SELECT, and it usually produces one. But the guarantee comes from a validator that parses the generated SQL and rejects anything that isn't a single plain read: no INSERT, UPDATE, DELETE, DROP, ALTER, TRUNCATE, no stacked second statement smuggled in after a semicolon, no forbidden functions. The parse step matters. A regex that greps for "DROP" is a speed bump; walking the statement structure is a wall. And the same guard runs again on the inner SELECT of the one narrowly-shaped write we do support, so a permitted operation can't become a hole in the one that isn't.

This is the general pattern: constrain the action space, then let the model roam freely inside it. An agent with three carefully bounded capabilities is more useful in production than one with twenty that nobody can reason about. The bound is what lets you say yes.

Failure feedback, with a cap

A model that writes SQL against an unfamiliar schema will sometimes get a column name wrong. The naive harness surfaces the database error to the user. The naive-plus-one harness retries the same call and hopes. Neither is right.

What works is feeding the actual failure back as context: here is the SQL you produced, here is the error the database returned, correct it. Most first-attempt mistakes are exactly the kind a model can fix when it can see the error text: a misspelled column, a bad join, a dialect quirk.

The important half is the cap. We allow two regeneration attempts after the first failure, and that number is a deliberate ceiling, not a placeholder. An agent that retries until it succeeds is an agent with an unbounded bill and an unbounded latency tail, and a model that's wrong three times in a row is usually wrong in a way a fourth attempt won't fix. When the budget is spent, the harness stops and produces an honest failure, including how many attempts it made, rather than an answer.

We also separate kinds of failure, because they deserve different budgets. A query that errors is one thing. A query that runs perfectly and returns zero rows is another: usually a follow-up whose filter value doesn't exist in the data ("show me the Platinum tier" when the column says PLATINUM). That case gets its own single retry, with the profiled sample values for low-cardinality columns handed back so the model can pick a value that actually exists. And the second result is accepted even if it's also empty, because sometimes the true answer really is "none." Collapsing those two failure modes into one retry counter makes a genuinely-empty answer unreachable.

Context is assembled, not pasted

The cheapest way to build an agent is to dump everything into the prompt. It also stops working the moment the customer has 400 tables.

Context assembly is its own engineering surface: retrieving the schema subset relevant to this question, keeping tables from the prior turn sticky so a follow-up doesn't lose the thread, and layering business context ahead of raw structure so the model maps a question onto your vocabulary before it ever sees a column name. (We wrote about that ordering in how Knowledge Hub grounds answers.)

The mental model worth adopting: the context window is a working set you curate per call, not a bucket you fill once. Deciding what goes in is as much a part of the agent as the prompt telling it what to do.

Verify outside the model

Language models are unreliable arithmetic engines and always will be. So when a result needs an exact total, we don't ask the model to add up the rows. We issue a second query that computes the aggregate in the database, and attach the true value. The model narrates; the database counts.

Generalize that: for any output where a wrong answer is expensive, find a check that doesn't route through the model. Re-derive the number. Re-run the parse. Diff against the source. Verification you can run deterministically is worth more than any amount of "please double-check your work" in a system prompt.

The same logic drives redaction. Rather than trusting a model to never surface a card number, results are scanned on the way out and sensitive values are redacted, with a Luhn check so an order ID that happens to be sixteen digits doesn't get mangled. The model isn't in that loop at all, which is precisely the point.

Make the failure path a real output

Agents that can only succeed are the ones that fail worst, because their only way to handle "I don't know" is to invent something. A production harness treats the failure path as a designed output: an explicit "there isn't enough information here" when grounding comes up empty, a plain-language explanation when the retry budget runs out, and enough detail (the SQL it tried, the error it hit) that someone can actually debug it.

That last part is observability, and it's the difference between an agent you operate and an agent you pray over. Log the generated query, the validation verdict, the attempt count, the fallbacks taken. Not for the model's benefit. For yours, at 2am.

Where to start

If you're taking an agent from demo to production, the highest-leverage move isn't a better prompt or a bigger model. It's an afternoon spent writing down your invariants, then asking of each one: is this enforced, or merely requested? Every line you move from the prompt into the harness is a class of failure that stops being probabilistic.

The model is the part that gets better on its own. The harness is the part you own.