OpenAI Escaped Nvidia’s Hardware Stack. The Price Was 100 Engineers.

After Jalapeño arrived from the foundry, OpenAI’s first custom AI accelerator was performing at 0.31% of its theoretical ceiling. Forty hours later, after OpenAI’s AI models ran through the software optimization layer, it was at 88.94%. That jump — not the chip itself, but the speed of the jump — is what the Jalapeño project is actually about.

The headline number will be the 3.6x latency improvement over Nvidia’s GB300 on inference workloads. That is real, and it matters for OpenAI’s economics. But the benchmark that should be making chip designers nervous is the software optimization run: a task that typically takes months of manual tuning by small teams of specialized engineers, completed by AI in less than two days. No human guidance on what to tune. No direction on where to start. The models found the ceiling by themselves.

What the Chip Is

Jalapeño is OpenAI’s debut AI accelerator — purpose-built for the inference workloads that generate its revenue. The hardware spec, as OpenAI disclosed to IEEE Spectrum: 13.4 petaflops of 4-bit compute, 232GB of HBM4 memory accessible at 15.4 terabytes per second of bandwidth. On OpenAI’s own inference benchmarks, it delivers 3.6x lower end-to-end latency than a comparable Nvidia GB300 configuration. The design went from initial architecture to first silicon in under 20 months. Nine of those months separated the first RTL code from tape-out — a timeline the chip industry typically measures in years, not months, for an accelerator of this complexity and performance tier.

The comparison chip is worth understanding carefully. Nvidia’s HGX B300 — the Blackwell Ultra generation — delivers 144 petaflops of FP4 compute (sparse) across an eight-GPU node, with 2.1 terabytes of total memory and NVLink bandwidth of 1.8 terabytes per second per GPU. It is shipping now and is what Nvidia positions for demanding inference workloads. Nvidia’s next generation, the Rubin NVL8, claims ten times the token throughput of the B200 — its arrival will reset the comparison baseline. But Rubin is not what data centers are deploying today. Jalapeño’s 3.6x advantage is against what’s in the rack right now.

The latency claim needs its scope stated clearly: this is on OpenAI’s own workloads, against a GB300 system running OpenAI’s inference patterns. Latency, unlike raw throughput, is highly sensitive to memory access patterns, attention kernel design, and batch size — all of which OpenAI has optimized precisely for Jalapeño. A purpose-built chip running purpose-built software, benchmarked on the workload it was designed for, is supposed to beat the general-purpose alternative. The margin of 3.6x suggests the gap is substantial rather than marginal, but it does not imply Jalapeño would beat the GB300 at someone else’s workload. That is a separate claim OpenAI has not made.

How They Built It

The methodology starts with Google’s open-source XLS toolchain (Accelerated Hardware Synthesis), which lets engineers describe hardware in DSLX — a domain-specific language designed to feel like Rust — rather than traditional hardware description languages. The significance of that choice is not aesthetic. For most of the chip industry’s history, the cultural barrier between hardware and software engineering has been the hardware description language itself: VHDL and Verilog operate on abstractions foreign to software developers, requiring years of specialized training to use effectively. DSLX changes that at the margin. It does not make hardware design easy, but it makes it legible to a category of engineers who know Rust — and to AI models trained on software corpora.

Because DSLX code looks like software, OpenAI’s AI models — fine-tuned internally for chip design tasks, and not released publicly — could operate on it directly. The models could read the design, generate candidate modifications, evaluate tradeoffs, and iterate through optimization paths in a domain where AI systems are already strong: code that looks like code, rather than a hardware specification language designed by and for electrical engineers over four decades. This is a narrow but decisive foothold.

The XLS toolchain itself is worth noting as context. The project’s GitHub has 1,900 stars and carries a disclaimer: it is not an officially supported Google product, it breaks DSLX backward compatibility between releases, and it is explicitly experimental. That OpenAI bet a production chip on experimental tooling reflects either unusual confidence in the tool, unusual confidence in their ability to work around its rough edges, or both. Chris Leary, OpenAI’s member of technical staff on the hardware team, offered the rationale: “XLS in some ways looks like software, so it got that benefit.”

AI’s contribution to the design divided cleanly along architectural lines. On the frontend — high-level synthesis, RTL generation, code-level optimization — AI was active and measurably effective: a 10% reduction in the area of the matrix multiplication units through automated optimization. On the backend — routing, clock trees, power distribution, the physical implementation that determines whether a design actually functions as silicon at yield — Broadcom handled the work through conventional workflows. Not AI-assisted. The traditional approach by engineers with decades of physical design experience. This partition is not a limitation that future models will easily overcome; physical design requires optimization against constraints that are simultaneously electrical, thermal, and spatial, in ways that current AI architectures have not demonstrated the ability to handle at production quality.

What AI Actually Did — and Didn’t

The 0.31% to 88.94% story is software optimization, not chip design. The distinction is not a technicality. After tape-out, every chip goes through a process of software bringup: benchmark kernels, memory access patterns, kernel scheduling, and the dozens of tuning decisions that determine whether the hardware’s theoretical capability shows up in actual workload performance. On a conventional timeline, this process runs for months, with human engineers iterating manually through a narrow set of configurations. OpenAI’s AI models completed the equivalent run in approximately 40 hours, with no predetermined direction. The ceiling was already in the silicon; the question was how quickly to reach it, and the answer was: forty hours after the chip arrived.

The prominent critique from the chip design community is that the framing conflates AI-assisted software optimization with AI chip design — and that the conflation overstates AI’s role in the harder part of the problem. Physical design (routing, power planning, the physical area and timing analysis that determines whether a design yields at the foundry) remains beyond what current models can do at production quality. Broadcom’s engineers did that work. Richard Ho, OpenAI’s VP Hardware, is deliberate about the scope: “We’re not saying anyone can just build frontier chips using our tools…we’re focusing on small teams and fast timelines.” The emphasis on small teams is a precision about the claim’s reach, not a modesty disclaimer.

A second critique from chip designers goes deeper: the manufacturing bottleneck is real regardless of how fast the design cycle runs. Foundry capacity at TSMC and its peers is allocated years in advance, and securing a leading-node process slot requires either long-term purchase commitments or strategic partnerships of the kind that take years to build. Designing faster does not compress the foundry queue. OpenAI, as a company with the revenue and relationships to negotiate those allocations, can secure the slot. The question of who else can is the question of who else this method actually helps.

The Hardware Feedback Loop

Strip away the caveats and the strategic picture is still striking. OpenAI is now vertically integrated across the inference stack: it controls the hardware, the kernel software, and the model weights in a way that no other AI lab does at comparable commercial scale. The 3.6x latency advantage on OpenAI’s workloads is a consequence of that vertical control — a chip designed for one set of models, tuned by those same models, deployed exclusively for serving those models. That loop is closed in a way it was not 20 months ago.

The feedback mechanism that Jalapeño represents is not metaphorical. The AI models that optimized Jalapeño’s software stack in 40 hours are the same class of models that Jalapeño will now run at scale. The hardware that the models helped design is the hardware the models will run on next — and eventually, the hardware that will train the next iteration of the models. This has been discussed as a future possibility in AI research. OpenAI ran it in production. The question is not whether it worked. The question is whether AI can take on more of the backend in the next design cycle, and whether the 9-month timeline compresses further.

How many labs can close this loop? The cost of Jalapeño was approximately 100 engineers (excluding Broadcom partners), a sustained Broadcom partnership, a TSMC process allocation, and 20 months. Google has Tensor Processing Units, purpose-designed for its own models and deployed at scale for years. Amazon has Trainium and Inferentia, running inference for AWS customers. Microsoft’s custom hardware work is less public but ongoing. Everyone else — including every sovereign AI program that has spent billions on national compute — is running on Nvidia hardware that OpenAI now outperforms on the workloads Nvidia is supposed to be best at. That gap is deployed and serving users today.

The 88.94% figure will not be the ceiling for long. The next software optimization run will start where this one ended. Richard Ho described the models as giving engineers superpowers, letting them explore many more paths. A feedback loop sounds exactly like that from the inside: more paths, faster convergence, each cycle starting from a better position. The question is what Nvidia does about it. The company’s answer is the Rubin NVL8, but Rubin is a bet on Nvidia’s ability to stay ahead on general-purpose capability. OpenAI has just demonstrated that general-purpose capability is not the only race worth running.

AI-generated editorial illustration · TemperatureZero · September 19, 2026