OpenAI says Jalapeño beats Blackwell on inference benchmarks

OpenAI's custom Jalapeño chip delivered 1.5 to 1.9 times more AI work per watt than the best previously recorded inference systems, the company said Tuesday.

The figures landed at the Hot Chips conference, where OpenAI shared what TechCrunch described as "the first batch of benchmark results for the new system." They are the first hard numbers attached to a chip the company has been building with Broadcom, and they arrive with a deployment timeline that matters more than the benchmark itself.

What was measured

OpenAI ran Jalapeño on InferenceX, a benchmarking platform from SemiAnalysis that measures how well AI systems handle inference — the process of running a trained model to complete a task or deploy an agent. Per The Verge, the test compared Jalapeño against the best results recorded at the time, which came from Nvidia's GB200 or GB300 superchips.

Two numbers came out of it, both reported by The Verge:

Metric	Jalapeño vs. comparison systems	Models tested
AI work per watt	1.5–1.9× more	GPT-OSS 120B, DeepSeek R1, Kimi K2.5 1T
End-to-end latency	1.7–3.6× lower	same three models

The spread across three models of very different sizes is the useful part. A gain that holds from a 120B open-weight model up to a trillion-parameter one is harder to dismiss as a single-workload artifact than a single headline multiplier would be.

OpenAI also published a chart measuring time between tokens, or TBT — the time it takes to deliver a response. That is the metric users feel directly in an agent loop, and it is the one the latency claim maps onto.

What OpenAI's hardware chief said

"The bottom line is that the results show a very, very significant performance advance over state of the art," Richard Ho, OpenAI's head of hardware, said in a press call reported by TechCrunch. "Jalapeño can serve more AI work per unit of power, while also returning responses more quickly. It's very efficient to serve a lot of customers, but it can also be very low latency."

In a separate briefing with reporters covered by The Verge, Ho framed it as the "best of both worlds" — lower latency and higher throughput at once — noting that AI systems typically "have to make a trade-off between the two." That trade-off is the real claim here. Serving more concurrent users usually means batching harder, and batching harder usually means each individual user waits longer.

The architectural bet: keep the data still

Jalapeño is an Application-Specific Integrated Circuit built in partnership with Broadcom, designed specifically for inference rather than training. TechCrunch reports OpenAI's own models assisted in the development process, and that the company plans to make Jalapeño a multigenerational platform, with AI products, models, chips, and memory developed in concert.

That full-stack framing points at where the gains come from. Jalapeño is designed to minimize delays during the prefill and communication phases of processing, which OpenAI says often act as bottlenecks.

"We designed Jalapeño to minimize data movement and communication delays," the company said in a blog post presenting the results. "This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase."

Read that as a bet on placement rather than raw arithmetic. Keeping the KV cache local and activating different mixes of compute, memory, and networking per phase attacks the part of inference that general-purpose accelerators handle generically — which is exactly the kind of specialization an ASIC exists to buy.

The asterisk: you cannot buy this yet

TechCrunch flags the caveat plainly: the comparison is against an Nvidia Blackwell system, but by the time Jalapeño reaches full deployment, the competition may have advanced significantly.

The timeline makes that concrete. Ho estimated Jalapeño would deploy at the end of 2026 "in very small volumes," with more significant deployment coming in 2027. He told reporters OpenAI will "ramp the volume up" into 2027, and the company did not say how many chips it plans to deploy next year.

So the benchmark is a 2026 result against 2026 hardware, for a chip that reaches meaningful volume in 2027. Nvidia will have shipped its next generation into that gap. Anyone modeling inference costs on these multipliers is modeling a comparison that will have moved by the time the silicon is available at scale.

This is not a replacement for Nvidia

Ho was explicit that OpenAI does not expect to replace its entire chip lineup with Jalapeño, saying its overall compute strategy includes "very good partners," like Nvidia. The company will continue developing the second and third generations of the chip.

That is the sober reading of the announcement. A custom inference ASIC that is meaningfully more power-efficient on OpenAI's own dominant workloads is worth building even if it never displaces the general-purpose fleet — it absorbs the traffic whose shape you know in advance, and leaves everything else on hardware you buy. The multigenerational language and the confirmed second and third generations say OpenAI intends to keep pulling that lever.

What to watch

Three things will tell you whether these numbers survive contact with production. First, whether OpenAI ever publishes volume figures — the refusal to say how many chips deploy in 2027 is the largest gap in the disclosure. Second, whether the efficiency gain shows up as lower API prices or faster responses for developers, or gets absorbed entirely into serving capacity. Third, whether independent InferenceX results appear from someone other than the chip's owner; every number above comes from OpenAI's own benchmark run and its own blog post.

For developers, nothing changes this quarter. If you are planning inference capacity for 2027, the signal worth taking is narrower than the headline: OpenAI now has a credible reason to keep pushing custom silicon at its own traffic, and a stated intent to do it across multiple generations.