Four correct fixes, one symptom that didn't move

I worked through this with Claude, in the Claude desktop app's Claude Code tab. There is no transcript left of this one, so Claude's lines here are reconstructed from the note we wrote at the time, not quoted from a log.

This is the same pipeline as part 1, a few weeks on and a stage along. That piece is worth reading first, though this one stands on its own.

Three words this piece uses. The pipeline is a recruiter project of mine, built on n8n, a workflow-automation tool where each step is a "node" and a run is an "execution". It later became the project now called bart. A "queue" here is a database table of work waiting to be done, drained a few rows at a time.

The stage, and the symptom

The job was small. Take a company, research it, cache the result so nobody pays to research it twice.

It had never written a row. Not a wrong row, not a partial row. None, for the life of the project.

That is the whole difficulty in one line. "Nothing was produced" is the same output whether one thing is blocking it or four.

Two days, four fixes

The sequence, as it happened.

Fix 1, deployment drift. The running instance held an older copy of the code than the repository did. Redeployed, and verified: the node now emits the right fields. An hour later I ran the stage. Same failure, same item back at the head of the queue.

Fix 2, no error handling. One item's failure killed the whole run. We added per-item degrading and raised an iteration cap. Verified live. I ran it again. Failed identically.

Fix 3, an agent that couldn't finish. The research agent held web-search tools and a structured output parser. On this combination it never produced a final answer: it emitted tool calls until it hit the cap, then errored. We detached the parser and verified it against four output shapes.

That one produced the first rows ever written. Three of them. And then I noticed we were researching the same company over and over.

Fix 4, a queue that couldn't let go. Nothing moved a failed item out of the pending state, so the same row came back first on every run, forever.

Why the middle fixes looked like nothing

Here is the part worth keeping. None of those verifications was sloppy. Each one was true.

What was checked	What it proved	What it didn't
The deployed config, read back from the live instance	The deployed code matched the repository	That a run could finish
The full offline suite, 1,200+ cases green	No regression in the deterministic logic	Anything about the agent, which no test could run
The changed branch, against a captured real error	That the new branch handled that input	That the input ever occurred
Row counts and flags through the API	The change reached the instance	That the change was enough

Every check confirmed a property of the artefact. None confirmed a property of the outcome. An artefact-level check cannot see a missing fourth requirement.

Me: Who's looping through the companies? I don't see a loop node.

Claude: Nothing loops. The engine runs each node once per input item. So there is no iteration control to fix. What's missing is a terminal state.

That question was based on a wrong premise and was still the most useful thing asked in two days, because answering it precisely deleted a whole branch of the search.

The cap that was reached exactly

Fault 3 deserves its own paragraph, because the error message pointed the wrong way.

The node reported "max iterations reached". That reads like a budget problem, and it invites you to raise the budget. Raising the cap from 6 to 10 changed one number in the error: 30 model calls across 3 items is exactly 10 each.

The evidence was one level below the node. Every one of those 30 model turns returned empty text, while the tools returned correct data on the first call. The model had what it needed on turn one and could never finish.

A cap that is reached exactly is not a cap that is too low. It is a loop that never terminates.

The question that would have collapsed it

One query was available from the first hour and was not run until the third: how many rows are in the output table?

The answer was zero, and it had always been zero. That reframes everything. "This regressed" invites you to look for the one thing that changed. "This has never worked" invites you to inventory the requirements.

The fix here is not a tool. It is a question asked first, and it costs one query: has this ever worked?

What it cost

• About four hours of back-and-forth across two days, most of it waiting on runs that could not have succeeded.
• Three full runs of a paid model API that produced nothing storable.
• Trust. By the third round I was proposing mechanisms that didn't exist, because three fixes had landed and nothing had moved. That is the correct response to three failed fixes, and it is expensive to rebuild.
• No data was lost. Every fault was a failure to produce, not to corrupt, which is exactly why nothing alerted.

What changed

Safeguard	What it prevents	What it costs
A build stamp rendered into every deployed artefact, visible on the canvas	Fault 1. "Is the fix deployed?" becomes a glance.	One field, resolved at build time.
A failure-closing path with an attempt counter on the queue	Fault 4. A repeatedly failing item retires instead of pinning the head.	One migration, one node. A counter rather than instant retirement, so a transient outage doesn't bin real work.
Shape-based detection: "all items failed" throws even when the errors are unreadable	Silent no-op runs, while keeping per-item degrading	Nothing. Strictly more information than either extreme.
Invariant tests swept across every workflow, not just the one being fixed	A guard written for one workflow and nowhere else. The first sweep found a fourth instance of an already-fixed bug.	Small. Each sweep needs a control proving it can fail.
A rule: a tool-holding agent may not also hold an output parser	Fault 3, pinned as a test rather than a memory	Constrains a design option. The replacement is more code and more debuggable.

And one rule that isn't a safeguard, just a question asked earlier: query the thing the stage is supposed to produce, before forming any hypothesis.

Claude: For any stage whose purpose is to produce records, the status is the record count and its timestamp. Not the deployment.

The close

The counter-lesson matters too. The fail-loud design in fault 2 was not wrong when it was written. Its reasoning was sound and a test pinned it, with a comment explaining why. What decayed was an assumption about cost: "failing loudly costs a retry". It stopped being true when the failure moved to where it also blocked the work that would have fixed it.

Assumptions about cost decay quietly. The correctness reasoning around them does not, which is why it survives review and they don't.

One thing to keep straight about that table. It describes the project as it was fixed at the time. bart is a later fork of it, and not every fix travelled across.

At the time of writing, some of the planned safeguards are still in development. In bart today: there is no attempt counter to retire a failing item, no halt on authorization or quota errors, no restart: no on two services and no container labels, the panic script stops containers by project rather than by label, the intake path still makes one model call per document, and the storage-path setting accepts only one spelling.