OpenAI unveils Jalapeño custom AI inference chip to cut ChatGPT costs

OpenAI and Broadcom have introduced Jalapeño, the company's first custom-built application‑specific integrated circuit (ASIC) designed solely for large‑language‑model (LLM) inference. Unlike the general‑purpose graphics processing units (GPUs) that dominate AI hardware, Jalapeño is optimized for the data‑movement bottleneck of token generation, pairing a reticle‑sized compute chiplet with high‑bandwidth memory (HBM) placed directly beside the compute dies. Early internal testing reports substantially better performance‑per‑watt than leading GPUs, which could lower the electricity and silicon costs of running ChatGPT and similar services.

The chip was co‑designed by OpenAI and Broadcom and moved from concept to tape‑out in roughly nine months, an unusually fast development cycle. While OpenAI will continue to rely on Nvidia GPUs for model training, Jalapeño targets the inference stage—answering user prompts—so the service can respond faster, support more users during peak demand, and reduce the risk of outages caused by hardware shortages. The announcement marks a step toward greater vertical integration for OpenAI, giving it more control over the hardware stack that powers its AI products.