Jalapeño首批结果显示AI推理速度与效率行业领先

自发布OpenAI首款定制推理芯片Jalapeño以来，我们一直在测试该芯片及其配套构建的系统。结果显示其性能显著提升：Jalapeño在单位功耗下可处理更多AI工作负载，同时响应速度更快。Jalapeño以单一架构同时实现更高吞吐量和更低延迟，而现有硬件系统往往需要在两者之间做出取舍。

对客户而言，这意味着更快的响应、更灵敏的智能体，以及随着需求增长更可靠的访问体验。我们的使命是确保通用人工智能惠及全人类。这些进步将有助于让日益强大的AI变得更加经济实惠、更广泛可用。

OpenAI的模型也加速了Jalapeño的开发进程。早期模型帮助团队设计和启动芯片，而我们最新的模型正在加速芯片的优化和编程工作。Jalapeño的性能覆盖GPT‑OSS 120B、DeepSeek R1和Kimi K2.5 1T，表明该架构适用于OpenAI内外开发的各类模型。在这三个模型上，Jalapeño在峰值吞吐量下每瓦特可完成的AI工作量是对比系统的1.5至1.9倍，端到端延迟则低1.7至3.6倍。对于高交互性工作负载，其性能提升达2.1至4.1倍。

Jalapeño也证明了更广泛的全栈优势。OpenAI能够协同设计模型、产品、服务软件、芯片、内存、网络和系统，利用从真实工作负载中获得的经验来改进技术栈的每一层。Jalapeño是经过实测验证的自研硅片，也是多代际平台的起点。未来数月内，我们将逐步扩大Jalapeño的部署，为客户提供更快、更强、更高效的产品。

我们如何衡量Jalapeño的性能

我们在匹配的用户体验条件下评估性能，衡量每个系统在满足客户和交互式智能体所需延迟的同时，单位功耗可完成的有效AI工作量。这对智能体尤为重要，因为它们需要按顺序完成多个步骤，延迟会在整个任务中不断累积。

为了解Jalapeño在实际中的表现，我们在InferenceX上进行了测试——这是SemiAnalysis推出的公开基准，用于衡量AI请求服务的完整流程。我们将Jalapeño与市面上领先的商用AI系统在测试运行范围内进行了比较，涵盖高吞吐量服务到高交互性、低延迟使用场景。Jalapeño在吞吐量、功耗效率和延迟方面实现了更优组合。虽然性能有时按单芯片报告，但我们认为更有用的标准是单位功耗的性能。

Jalapeño在先前最佳TBT水平上扩大了领先优势

Jalapeño为每位用户提供更多token

Jalapeño每千瓦时提供更高吞吐量

在全部三个公开模型上，Jalapeño在测试运行范围内实现了每瓦性能与延迟的更优组合，处于帕累托前沿。为保持比较的一致性，我们使用各加速器公布的芯片功耗额定值对结果进行了归一化处理。Jalapeño额定功率为700瓦，但在测试工作负载下其持续实测功耗保持在550瓦或以下。

Jalapeño在GPT‑OSS 120B、DeepSeek R1 670B和Kimi K2.5 1T上均表现强劲。在我们测试的最大公开模型Kimi上，其每瓦峰值性能约为对比系统的1.5倍，端到端延迟低3.4倍。在我们的内部测试中，Jalapeño在OpenAI前沿模型上的优势进一步扩大，表明随着工作负载规模更大、要求更高，该架构的价值也愈发凸显。

在单芯片内实现速度与效率的架构设计

Jalapeño从设计之初就围绕一个问题展开：如果硬件的主要任务是服务现代和未来的语言模型，尤其是交互式智能体，我们会构建什么样的硬件？Jalapeño的优势源于围绕真实语言模型工作负载，将芯片、内存、网络、软件和机架级系统进行协同设计。语言模型推理会经历多个瓶颈各异的阶段。预填充阶段（系统处理提示词时）为计算密集型，而解码阶段（系统逐token生成响应时）更多受内存带宽限制。当数据需要在核心和芯片之间移动时，通信也会增加延迟，导致部分处理单元在等待时闲置。一个在某阶段表现出色的系统，可能在等待数据或在不同资源间移动模型状态时丧失优势。

我们设计Jalapeño的目标是最大限度减少数据移动和通信延迟。这意味着模型状态（包括生成响应时使用的KV缓存）可以被显式放置并保持本地化，同时系统为每个推理阶段激活合适的计算、内存和网络组合。网络是架构的有机组成部分。其大域（large domain）使整个工作负载能够保持在单一互联系统内，最大限度减少数据移动，确保完整请求从始至终保持快速高效。最终成果是一款均衡且可灵活调配的加速器，能够支持不断变化的模型架构，在预填充和解码阶段均表现出色，并能适应两者之间平衡的变化——这正是智能体工作负载的典型特征。

我们用AI设计芯片，并让芯片便于AI编程

AI在Jalapeño的开发中发挥了直接作用，使团队能够在九个月内从初始设计推进到流片，通过探索实现方案、缩短设计、测量和验证循环，并基于模型工作负载持续迭代。AI还帮助优化了芯片的算术电路，使团队能够按时在芯片中集成更多计算性能。

Jalapeño被设计为对人类和AI都清晰、可预测的编程目标。工程师可以通过局部张量、显式通信和可预测的同步来描述工作。AI随后可以优化这些工作在系统中的映射、放置、调度和协调方式。这种清晰、可预测的结构为AI提供了一条可行的路径，来解决传统上棘手的并行编程问题。

支持每一个新模型家族仍然需要新的内核和针对模型的优化。通过使用 Codex 与 GPT‑Astra，团队在两个月内将三个未纳入 Jalapeño 原始生产计划的开权重模型提升到了高性能水平。这既展示了架构的灵活性，也体现了 AI 帮助我们编程的速度。对于选定的 GPT‑OSS 注意力机制和混合专家模块，AI 生成的实现比现有的人类专家编写的实现运行速度快 1.5 到 1.8 倍。这些数据适用于选定的模块，而非整个模型，但它们指向了一个强大的新开发循环。

高效、超快推理的前进之路

AI 基础设施的价值在于它能够实现有用的现实世界工作。通过从相同的功耗和硬件中产生更多有用的工作，Jalapeño 可以帮助我们满足更多需求，并降低交付成功结果的成本。对于 OpenAI 来说，这可以通过让有用工作和收入的增长速度超过服务成本来提高运营杠杆。它还可以支持更广泛的采用，并持续投资于更好的模型、产品和基础设施。更快的推理可以实现更快的迭代和新的用例。

Jalapeño 扩展了高效、低延迟推理的可能性：

• 以以前仅在快速模式下可用的效率进行超快速模式推理
• 以以前仅在批处理模式下可用的效率进行快速模式推理
• 批处理模式推理的更高效率

我们计划在今年年底前开始在 OpenAI 的计算基础设施内部署 Jalapeño。这是一个多代路线图的第一代：Gen 2 正在深入开发中，Gen 3 正在成形。每一代都将建立在我们所学到的基础上，并进一步推动效率和速度的提升。

满足日益增长的 AI 需求将需要从所有可用来源获得更多计算资源。我们将继续广泛部署来自 NVIDIA 和其他合作伙伴的加速器，用于训练和推理工作负载。我们的使命是确保通用人工智能惠及全人类。

在准备部署的过程中，我们正在继续进行生产资格认证、完善软件、准备大规模运营 Jalapeño，并在更多模型上验证性能。迄今为止的结果展示了当我们共同设计整个系统时可能实现的目标：以更高的效率向更多人提供更灵敏、更有能力、更具代理性的 AI。

附录

Jalapeño 在 GPT‑OSS 120B 上处于帕累托前沿

每千瓦吞吐量前沿

InferenceX · GPT‑OSS‑120B · 标称 8k/1k · STP · 封装 TDP：Jalapeño 700 W；GB200 1,200 W

Jalapeño 在 GPT‑OSS 各运行点上领先

峰值和匹配吞吐量

InferenceX · GPT‑OSS‑120B · 标称 8k/1k · STP · 封装 TDP：Jalapeño 700 W；GB200 1,200 W

更高的峰值混合 TPS / kW

≈1.9×

85,448 vs. 44,960 混合 / kW

更低的端到端延迟

≈1.7×

1.03 秒 vs. 1.80 秒

更低的 TBT 最小值

≈2.7×

0.69 vs. 1.87 毫秒（1,459 vs. 535 令牌/秒/用户）

在先前 TBT 下更高的吞吐量

≈53.7×

22,935 vs. 427 混合 / kW（在 535.28 令牌/秒/用户时）

Jalapeño 在 DeepSeek R1 670B 上处于帕累托前沿

每千瓦吞吐量前沿

InferenceX · DeepSeek R1 MXFP4 · 标称 8k/1k · STP · 封装 TDP：Jalapeño 700 W；GB300 1,400 W

Jalapeño 在 DeepSeek R1 各运行点上领先

峰值和匹配吞吐量

InferenceX · DeepSeek R1 MXFP4 · 标称 8k/1k · STP · 封装 TDP：Jalapeño 700 W；GB300 1,400 W

更高的峰值混合 TPS / kW

≈1.7×

19,641 vs. 11,781 混合 / kW

更低的端到端延迟

≈3.6×

1.65 秒 vs. 5.99 秒

更低的 TBT 最小值

≈4.1×

1.43 vs. 5.90 毫秒（700 vs. 169 令牌/秒/用户）

在先前 TBT 下更高的吞吐量

≈104.3×

12,258 vs. 118 混合 / kW（在 169.41 令牌/秒/用户时）

Jalapeño 在 Kimi K2.5 1T 上处于帕累托前沿

每千瓦吞吐量前沿

InferenceX · Kimi K2.5 MXFP4 · 标称 8k/1k · STP · 封装 TDP：Jalapeño 700 W；GB300 1,400 W

Jalapeño 在 Kimi K2.5 各运行点上领先

峰值和匹配吞吐量

InferenceX · Kimi K2.5 MXFP4 · 标称 8k/1k · STP · 封装 TDP：Jalapeño 700 W；GB300 1,400 W

更高的峰值混合 TPS / kW

≈1.5×

18,195 vs. 11,862 混合 / kW

更低的端到端延迟

≈3.4×

1.56 秒 vs. 5.31 秒

更低的 TBT 最小值

≈3.8×

1.44 vs. 5.48 毫秒（694 vs. 182 令牌/秒/用户）

在先前 TBT 下更高的吞吐量

≈56.1×

6,744 vs. 120 混合 / kW（在 182.46 令牌/秒/用户时）

Since announcing Jalapeño, OpenAI’s first custom inference chip, we have been testing the chip and the system built around it. The results show a significant performance advance: Jalapeño can serve more AI work per unit of power while also returning responses more quickly. Jalapeño delivers both higher throughput and lower latency with one architecture, where existing hardware systems often have to make a tradeoff between the two.

For customers, that can mean faster responses, more responsive agents, and more reliable access as demand grows. Our mission is to ensure that artificial general intelligence benefits all of humanity. These gains will help make increasingly capable AI more affordable and more broadly available.

OpenAI models also accelerated Jalapeño’s development. Earlier generations helped the team design and bring up the chip, while our latest models are accelerating how we optimize and program it. Jalapeño’s performance extends across GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T, showing that the architecture works across models developed both inside and outside OpenAI. Across all three, Jalapeño delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the comparison systems. For highly interactive workloads, it delivered 2.1 to 4.1 times higher performance.

Jalapeño is also evidence of a broader full-stack advantage. OpenAI can design models, products, serving software, chips, memory, networking, and systems together, using what we learn from real workloads to improve every layer of the stack. Jalapeño is working first-party silicon with measured results, and it is the beginning of a multigenerational platform. In the months ahead, we will ramp Jalapeño to deliver faster, more capable, and more efficient products for our customers.

How we measured Jalapeño’s performance

We evaluate performance at a matched user experience, measuring how much useful AI work each system can complete per unit of power while meeting the latency customers and interactive agents require. This matters especially for agents, which need to complete many steps in sequence, so delays can compound across an entire task.

To understand how Jalapeño performs in practice, we tested it on InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. We compared Jalapeño with leading commercially available AI systems across the tested operating range, from high-throughput serving to highly interactive, low-latency use. Jalapeño delivered a better combination of throughput, power efficiency, and latency. Although performance is sometimes reported per chip, we believe the more useful standard is performance per unit of power.

Jalapeño widens the lead at previous-best TBT

Jalapeño delivers more tokens per user

Jalapeño delivers more throughput per kilowatt

Across all three public models, Jalapeño delivered a better combination of performance per watt and latency across the tested operating range, placing it on the Pareto frontier. To compare the systems consistently, we normalized the results using each accelerator’s published chip power rating. Jalapeño is rated at 700 watts, although its measured sustained power remained at or below 550 watts on the workloads tested.

Jalapeño performed strongly across GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. On Kimi, the largest public model we tested, it delivered approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system. In our internal testing, Jalapeño’s advantage widened further on frontier OpenAI models, suggesting that the architecture becomes more valuable as workloads grow larger and more demanding.

Architecting for speed and efficiency within a single chip

Jalapeño was designed from the start by asking: what hardware would we build if its primary job were serving modern and future language models, especially interactive agents? Jalapeño’s gains come from designing the chip, memory, network, software, and rack-scale system together around real language-model workloads. Language-model inference moves through several distinct phases with different bottlenecks. Prefill, when the system processes a prompt, is compute-intensive, while decode, when the system generates the response token by token, is constrained more by memory bandwidth. Communication can also add latency when data must move between cores and chips, leaving some processing units idle while they wait. A system that excels at one phase can lose that advantage while waiting for data or moving model state between different resources.

We designed Jalapeño to minimize data movement and communication delays. This means that model state, including the KV cache used while generating a response, can be explicitly placed and kept local while the system activates the right combination of compute, memory, and networking for each inference phase. The network is integral to the architecture. Its large domain allows the entire workload to remain within one connected system, minimizing data movement and helping the complete request stay fast and efficient from beginning to end. The result is a balanced and fungible accelerator that can support changing model architectures, excel at both prefill and decode, and adapt as the balance between them changes, a defining feature of agentic workloads.

We used AI to design the chip, and designed the chip so AI could program it

AI played a direct role in Jalapeño’s development, enabling the team to move from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, and continuously iterating on model workloads. AI also helped optimize the chip’s arithmetic circuits, allowing the team to fit more compute performance into the chip on schedule.

Jalapeño was designed as a clear, predictable programming target for both humans and AI. Engineers can describe work through local tensors, explicit communication, and predictable synchronization. AI can then optimize how that work is mapped, placed, scheduled, and coordinated across the system. That clear, predictable structure gives AI a tractable way to tackle the traditionally difficult problem of parallel programming.

Supporting each new model family still requires new kernels and model-specific optimizations. Using Codex with GPT‑Astra, the team brought three open-weight models that were not part of Jalapeño’s original production plan to high performance within two months. This demonstrated both the flexibility of the architecture and the speed at which AI can help us program it. For selected GPT‑OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than the existing human-expert-written implementations. Those figures apply to the selected blocks, not the full model, but they point toward a powerful new development loop.

The path ahead for efficient, ultra-fast inference

AI infrastructure is valuable because of the useful real-world work it enables. By producing more useful work from the same power and hardware, Jalapeño can help us serve more demand and lower the cost of delivering a successful result. For OpenAI, that can improve operating leverage by allowing useful work and revenue to grow faster than the cost to serve. It can also support broader adoption and continued investment in better models, products, and infrastructure. Faster inference can enable faster iteration and new use cases.

Jalapeño expands what is possible for efficient, low-latency inference:

• Ultra-fast-mode inference at efficiencies previously available only in fast mode
• Fast-mode inference at efficiencies previously available only in batched mode
• Higher efficiency for batched-mode inference

We plan to begin deploying Jalapeño within OpenAI’s compute infrastructure by the end of the year. It is the first generation of a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape. Each generation will build on what we learn and further advance both efficiency and speed.

Meeting growing demand for AI will require more compute from every available source. We will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference workloads. Our mission is to ensure that artificial general intelligence benefits all of humanity.

As we prepare for deployment, we are continuing production qualification, maturing the software, preparing to operate Jalapeño at scale, and validating performance across more models. The results so far show what is possible when we design the full system together: more responsive, capable, and agentic AI delivered more efficiently to more people.

Appendix

Jalapeño is pareto frontier at GPT‑OSS 120B

THROUGHPUT-per-kW FRONTIER

InferenceX · GPT‑OSS‑120B · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB200 1,200 W

Jalapeño leads across GPT‑OSS operating points

PEAK AND MATCHED THROUGHPUT

InferenceX · GPT‑OSS‑120B · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB200 1,200 W

Higher peak mixed TPS / kW

≈1.9×

85,448 vs. 44,960 mixed / kW

Lower end-to-end latency

≈1.7×

1.03 s vs. 1.80 s

Lower min TBT

≈2.7×

0.69 vs. 1.87 ms (1,459 vs. 535 tok/s/user)

More throughput at previous TBT

≈53.7×

22,935 vs. 427 mixed / kW (at 535.28 tok/s/user)

Jalapeño is pareto frontier at DeepSeek R1 670B

THROUGHPUT-per-kW FRONTIER

InferenceX · DeepSeek R1 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W

Jalapeño leads across DeepSeek R1 operating points

PEAK AND MATCHED THROUGHPUT

InferenceX · DeepSeek R1 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W

Higher peak mixed TPS / kW

≈1.7×

19,641 vs. 11,781 mixed / kW

Lower end-to-end latency

≈3.6×

1.65 s vs. 5.99 s

Lower min TBT

≈4.1×

1.43 vs. 5.90 ms (700 vs. 169 tok/s/user)

More throughput at previous TBT

≈104.3×

12,258 vs. 118 mixed / kW (at 169.41 tok/s/user)

Jalapeño is pareto frontier at Kimi K2.5 1T

THROUGHPUT-per-kW FRONTIER

InferenceX · Kimi K2.5 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W

Jalapeño leads across Kimi K2.5 operating points

PEAK AND MATCHED THROUGHPUT

InferenceX · Kimi K2.5 MXFP4 · nominal 8k/1k · STP · package TDP: Jalapeño 700 W; GB300 1,400 W

Higher peak mixed TPS / kW

≈1.5×

18,195 vs. 11,862 mixed / kW

Lower end-to-end latency

≈3.4×

1.56 s vs. 5.31 s

Lower min TBT

≈3.8×

1.44 vs. 5.48 ms (694 vs. 182 tok/s/user)

More throughput at previous TBT

≈56.1×

6,744 vs. 120 mixed / kW (at 182.46 tok/s/user)