Four-Model Claude Orchestrator: What Backfired on Terminal-Bench

A developer wired four Claude models into one Claude Code orchestrator, ran it on Terminal-Bench 2.1, and watched it place seventh — behind a single model working alone.

The test, run by a builder using the handle teamquesma, put four Claude models into distinct roles inside Claude Code: an orchestrator planning and delegating, an executor doing the actual edits and commands, a verifier checking the work, and a scout doing read-only reconnaissance. Terminal-Bench 2.1 drops an agent into a sandboxed terminal and scores it on 89 real command-line tasks — compiling projects, recovering passwords, blocking exploits — with five attempts allowed per task.

The build: four files, no framework

The entire orchestrator was four files and zero third-party libraries: one block appended to CLAUDE.md assigning roles, plus three files under .claude/agents/ — one each for executor, verifier, and scout — with a model and effort level pinned in each file's frontmatter. The setup pinned Opus 5 as executor, Sonnet 5 as verifier, and Haiku 4.5 as scout. Swapping the whole team out is a one-line edit to a model name.

That wiring lines up with how Claude Code's architecture actually works under the hood: a separate technical breakdown of the tool describes its tool surface as deliberately narrow — read, edit, bash, glob, grep, a task-management tool, and an MCP connector — with no purpose-built function for every job. Complex behavior comes from composing those primitives, not from adding new tools, which is exactly what the four-file role setup does: no new capability, just role assignment on top of the same primitives.

Backfire 1: delegation made Claude refuse the work

Three tasks — a leaked-secret hunt, an XSS filter break, and a password recovery — never resolved in five tries. Every attempt returned the same refusal message at zero cost. Run as a direct, non-delegated request to the same model, Opus 5 solved all three, six for six. The refusal wasn't a capability gap: identical tasks, identical model, different outcome depending on whether the request arrived as a human prompt or a stripped-down delegated subtask.

Backfire 2: optional verification cost 48 points

The orchestrator only called its verifier on 76% of runs, because the prompt said the verifier "reviews the work" rather than mandating review before shipping. Runs that got verified resolved at 91%; the ones that skipped it did not. That single gap — a 48-point spread on identical models and tasks — was the largest predictor of success in the whole benchmark. First place on the leaderboard sits at 83.8%; making verification mandatory would have put the run in range of it.

Backfire 3: the wrong model in the busy seat

Output tokens correlated with a trial's cost at r=0.93 — the number of delegations barely mattered. The executor, the highest-volume role, was staffed with Opus 5, the most expensive model, which ended up accounting for 58% of total spend. Haiku 4.5, the cheapest model, sat in the low-volume scout role and moved the success rate by one point. The final bill came to $1,178 for a 78% score — good for seventh place, at roughly twice the cost of the top single-model entry.

Backfire 4: six hand-offs sent it over a cliff

Delegation count mattered more than model choice. Trials with three to five hand-offs resolved 90% of the time. Beyond six hand-offs, the orchestrator started re-delegating the same work, burning nearly four times the tokens for half the success rate. The lesson: an orchestration prompt needs a hard cap on hand-offs, not just permission to delegate.

What actually held up

The run wasn't a wash. Any single attempt solved 78% of tasks, and across five attempts that climbed to 93% — the ceiling was reliability, not raw capability. Twenty-eight of the 89 tasks were flaky, solved on some attempts and missed on others with no change in model or task. Only four were genuinely out of reach in every attempt. And the resolve rate barely sagged from easy to hard tasks — 85%, 79%, 76% — even though hard tasks cost 2.4 times as much per trial, which is the opposite of what you'd expect if hand-off overhead compounded on harder problems.

Metric	Result
Final score	78% (89-task benchmark), 7th place
Total cost	$1,178 across 445 trials
Verified vs. unverified resolve rate	91% vs. roughly half that
Cost-to-output-tokens correlation	r=0.93
Optimal hand-off range	3-5 delegations (90% resolve rate)

Try it yourself

The whole setup is reproducible: append a role-assignment block to your project's CLAUDE.md, add three role files under .claude/agents/ pinning a model and effort per role, and you have the same four-model rig running inside Claude Code with no extra framework. Before you do, apply the fixes this run identified — make verification mandatory rather than optional, cap hand-offs at five, and put your cheapest capable model in the highest-volume seat, not your most expensive one. None of the four backfires traced back to model capability; all four traced back to how the orchestration was wired.