OpenAI Unveils First Jalapeño Custom Silicon Benchmarks for High-Efficiency AI Inference

OpenAI has published the first performance benchmarks for Jalapeño, its custom first-party inference chip and integrated system architecture. Evaluated across diverse model architectures—including GPT-OSS 120B, DeepSeek R1, and Kimi K2.5 1T—the silicon demonstrated 1.5x to 1.9x higher throughput per watt at peak load and achieved 1.7x to 3.6x lower end-to-end latency compared to baseline systems. For highly interactive tasks, Jalapeño delivered a 2.1x to 4.1x performance increase. OpenAI confirmed that previous-generation models assisted in the physical layout and bring-up of the silicon, establishing a recursive development cycle for a multi-generational platform. For cloud platform architects, DevOps engineers, and MLOps teams, inference unit economics now dictate the viability of large-scale agentic and reasoning systems. Standard data center GPUs typically require engineering teams to navigate a severe trade-off between maximizing batch throughput (to keep token costs manageable) and minimizing per-token latency (to satisfy interactive user experience requirements). Jalapeño’s benchmarks show that custom silicon optimized specifically for transformer attention mechanisms and dynamic memory movement can provide low latency and high concurrency concurrently without prohibitive power dissipation. This release fits into the broader architectural transformation across cloud computing, where major AI providers are deploying dedicated ASICs to alleviate power grid constraints and bypass the supply bottlenecks of general-purpose GPUs. While hyperscalers have deployed platforms like Google TPUs and AWS Inferentia, OpenAI's demonstration specifically validates custom silicon efficiency against both proprietary weights and independent third-party foundation models. Furthermore, it exemplifies the growing trend of full-stack vertical integration, where the serving engine, runtime kernels, networking fabric, and silicon architecture are engineered simultaneously based on real-world production telemetry. In practice, infrastructure practitioners should anticipate a widening divergence between training and inference environments. While general-purpose GPU clusters remain essential for flexible, multi-tenant model training, production inference is aggressively consolidating around specialized accelerators. Engineering teams should audit their serving stacks to ensure portability through hardware-agnostic compilation frameworks and runtime abstractions. Decoupling model serving logic from proprietary accelerator APIs will be essential to leverage multi-vendor inference capacity and capitalize on emerging efficiency gains without necessitating deep code refactoring.