Ora Measured 99% of the Web — and Found Your Site Is the Real Bottleneck

On August 21, 2026, Vercel published a case study about a benchmarking company called Ora. Ora sends agents onto live customer websites with a simple instruction: sign up for a product, integrate with it, and pay for it. They trace every step, count cost and latency, and run the same gauntlet against Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, and Vercel's own eve framework.

The line that should keep product teams up at night is at the bottom of the post: "By Ora's own measure, 99% of the web still can't handle an agent that shows up to sign up, integrate, and pay."

The easy read is that the agents aren't smart enough. The actual finding is more uncomfortable: the agents are fine. The world they land in isn't.

The harness is half the agent — and most of that half is already absorbed

For two years the AI engineering playbook has been: build a better harness. That was the right bet. Harness-Bench ran the same model over the same 106 tasks across different harnesses and got scores ranging from 52.4 to 76.2 — a 23.8-point spread with zero change to the model (Latent Space, "The Evolution of the Agent Harness", 2026-08-22). OpenAI did a similar thing on ARC-AGI-3: adding only retained reasoning and compaction, GPT-5.6 Sol's score tripled from 13.3% to 38.3%. Half the agent is the harness — that was true through 2024 and into early 2025.

Then something flipped. As Latent Space puts it, the model and the harness curves stopped running side by side and started braiding: models were trained inside the harness environment, started absorbing harness capabilities into their weights, and the harness shed the scaffold. Thariq Shihipar from Anthropic recently said the team deleted 80% of Claude Code's system prompt — capability unchanged, code cut. GPT-5.1-Codex-Max launched as "the first model natively trained to operate across multiple context windows through compaction." The loop is now train → absorb → shed → repeat.

When the harness is solved, the constraint migrates to the world the agent lands in. That's what Ora's 99% number is really telling us.

What Ora actually measures

Ora's setup is worth describing because it explains why their numbers carry weight. Co-founder Assaf Elovic ran Tavily (the agent-focused search engine, acquired by Nebius earlier this year). Search solved half the problem — an agent that can locate a product still has to actually use it. He and co-founder Liad Yosef started Ora to measure how ready the web is for agents, and to fix the parts that aren't.

Today the platform spawns agents against live customer sites from journey.ora.ai, runs a journey, and records the cost, latency, and number of steps an agent needs to finish a task. Each agent in the lineup — Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, eve — runs on the customer's website and gets watched through common workflows.

The engineering detail that matters: no two harnesses want the same infrastructure. Each expects its own environment and exposes its steps differently. Ora runs a separate runtime for every harness and traces every step. Engineering lead Ido Finder calls that side-by-side coverage "one of the most valuable things Ora brings to its customers." When an agent stalls in a signup flow, the customer sees which step and what it tried. Without the trace, the result is a score with nothing behind it.

Why they ended up picking eve

When Vercel launched eve, Ora gave it no special treatment. It went through the same benchmark, under the same conditions, as every other harness in the lineup. Ora works with Vercel Engineering as a design partner — Finder gave the eve team direct access to the platform to dig into the results — but the benchmark itself was blind.

The initial test put eve against Claude Code across hundreds of real journeys on multiple domains. Both harnesses ran the same models (Claude Fable 5 and Haiku 4.5), and every run gave the agent the same job: integrate with a product.

Three numbers came out of that comparison:

• 7% fewer steps to reach the goal
• 2× native success: twice as many tasks finished on the customer's own site instead of falling back to web search
• 9% more valid endpoints: more of the endpoints the agent found were ones it could actually call

The benchmark fed back into eve too. One run surfaced a prompt-caching issue, the team shipped a fix, and Ora's next round measured roughly 15% lower total cost. For a company that benchmarks every major harness for a living, this is not a casual choice. After those results, Ora builds on eve.

The feature that sealed it was the sandbox override. An agent framework like eve ships with its own sandbox — the isolated environment where the agent executes, runs tools, and touches files. That's a good default for most teams, because you get safe execution for free. But an agent in the framework's own sandbox runs outside the instrumented environment where Ora traces every step. The override lets the team swap that environment in, so eve agents get recorded like every other harness, with nothing new built. Because eve follows the Next.js paradigm, there was little to configure — tools, skills, and connectors took little code.

What "99%" actually breaks down to

The Ora number isn't a single failure mode. It's a pile of small assumptions that web developers made in the years before agents existed. A signup flow that requires a CAPTCHA before the API is reachable. A pricing page that loads a JavaScript bundle which an LLM can't reliably execute. A checkout that depends on a third-party widget which strips out headers an agent tries to set. A "confirm email" step that uses a tracking pixel the agent renders as a string and stops there.

Most of these aren't malicious. They're just not designed for an agent as a customer. The web was built on the assumption that a human is on the other end of the browser, with eyes, with patience, with the ability to interpret a dropdown that says "Other" when none of the listed options apply. An agent that hits a dropdown it can't reason through just gives up, and the score goes to zero.

This is the part of the case study that lands differently depending on your role:

• If you're building agents: the harness is solved. The customer is the problem.
• If you're running a SaaS: your signup funnel was instrumented for humans. It is now blocking your largest new class of customers.
• If you're an investor: the next billion-dollar agent company might not be a harness. It might be agent-readiness as a service — the equivalent of SEO shops for the agent era.

What this means for the next 12 months

The 2025 roadmap for AI startups looked like this: bigger context window, better tool calling, faster harness, longer memory. The 2026 roadmap is going to start looking like this: ship an MCP endpoint, expose your signup as an API, instrument your funnel for agents, accept a programmatic identity. Not because the agents need more capability — because the targets aren't ready.

Ora is one answer to that gap. They expose the failure mode to the customer in plain language. The customer fixes what they can, sees what they can't, and Ora helps with the rest. That business model only exists because the gap exists.

The Latent Space piece ends with the same kind of inversion: if every harness capability gets absorbed into the model, what does that leave us with? Their answer — a harness for human attention rather than for the model — is poetic, but it's not the whole story. The harness for human attention was built over twenty years of UI design. The harness for agents is what Ora and its peers are building right now, and the engineering surface area has barely been touched.

The open question

If 99% of the web is unreadable to agents, who builds the map?

Three plausible answers, none of them winning yet:

1. A benchmarking company (Ora's model) — find the failures, charge the customer to fix them. B2B, slow but durable.
2. An "agent-ready SaaS" tier — premium product lines where the site is certified for agent interaction. Catches the high end first; leaves the long tail.
3. A protocol layer — MCP-style site descriptions that agents read first, so they don't have to discover the failure mode. Standardizes the gap instead of fixing it.

The next time you read a benchmark that says a new agent "failed 99% of the time on real websites," remember: the agent isn't the variable. Your site is.