Claude Code vs eve: Same Models, eve Takes 7% Fewer Steps

Ora ran Claude Code against Vercel's eve on live websites and eve reached the goal in 7% fewer steps. Both harnesses drove the same models, so the gap is tooling, not intelligence.

What Ora actually measured

Ora, a platform that benchmarks AI agents on live websites, ran the two harnesses across hundreds of real user journeys. These were not coding tasks: Ora's agents attempt to sign up for products, integrate with them, and pay. They fail often — Ora estimates 99% of the web isn't agent-ready. One caveat to hold onto before reading the numbers: the results were published on Vercel's blog, and Vercel builds eve, so this is a vendor-published comparison rather than an independent one.

Metric	Result	What it measures
Steps to goal	7% fewer for eve	How much work the harness needs to finish a journey
Native success	2x for eve	Tasks finished on the customer's own site instead of falling back to web search
Valid endpoints	9% more for eve	Share of discovered endpoints that were actually callable

Same models, different results

Both harnesses ran Claude Fable 5 and Haiku 4.5. The only variable was the harness — the software that hands the model its tools and drives it step by step. That is the load-bearing detail in the whole write-up, and it is what makes the numbers worth reading at all: a model-versus-model comparison would tell you nothing about the scaffolding you actually control. The write-up's own framing is that harness matters more than model, with tool design and step-driving logic carrying the difference.

Which one to reach for

Claude Code is a coding agent, and the write-up is explicit that it excels at repo-level tasks: editing files, running tests, committing code. Ora's benchmark tests something else — navigating live websites, filling forms, making API calls — and that is where eve, built on Next.js, is the one the benchmark favors.

The practical read: if your agent's job ends at the repo boundary, this benchmark does not describe your workload and there is nothing here to act on. If your agent has to sign up, authenticate, and call a third party's live endpoints, the native-success number is the one that should move you, because a fallback to web search is a task the agent did not really complete. Analysis, not a source claim: a 7% step reduction is thin on its own, while doubling native success changes what you can ship.

Google's EnvHarness aims at the same blind spot

The evaluation side of this problem got its own release. Google's EnvHarness is a programmable layer that turns static agent environments into adaptive ones, adding plug-in components — Setup, Rule, Link — without modifying the original code, and reporting up to +9 points on held-out tasks. Setup initializes dynamic conditions, Rule injects constraints or changing objectives mid-trajectory, and Link connects across environment states or episodes.

Treat the +9 as unverified. Per the same report, the source does not disclose which benchmarks or agent architectures were used, nor the compute cost; there is no per-task breakdown, no baseline comparison, and no ablation of the three components. The control-condition figure was never given — only the headline delta — and the whole thing traces to a single social post with no repository link.

What to watch

Two concrete signals. First, whether anyone reproduces Ora's numbers outside Vercel's blog, ideally with the per-journey breakdown that a vendor post has no incentive to publish. Second, whether Google ships EnvHarness code and a paper — until then, +9 points is an announcement, not a result. In the meantime, the cheapest thing you can do is benchmark your own harness on one live-site journey you care about and count the fallbacks to web search.