OpenAI Unveils Jalapeño AI Accelerator Chip, Designed with Assistance from Its Own LLMs

Jalapeño Chip Specifications and Performance

OpenAI officially unveiled Jalapeño, its first AI accelerator chip, on August 25. The chip is capable of up to 13.4 petaflops of 4-bit compute and features 232 gigabytes of advanced memory, with a bandwidth of 15.4 terabytes per second.

Benchmarks provided by OpenAI indicate that Jalapeño can reduce end-to-end latency by up to 3.6 times compared to Nvidia’s GB300, while also consuming less power. These performance claims are for its use in OpenAI's inference fleet.

LLM-Accelerated Design Process

A notable aspect of Jalapeño's development is the use of OpenAI's own large language models (LLMs) in its design. This approach contributed to a rapid development cycle, moving from initial architectural concept to first silicon in under 20 months.

The time from the first register-transfer level (RTL) code to tape-out, when the design is sent for manufacturing, was nine months. Richard Ho, OpenAI's vice president of hardware, stated that LLMs provided "superpowers" to engineers, allowing for faster exploration of design paths.

Small Design Team and Partnership

The team responsible for designing Jalapeño averaged fewer than 100 people throughout the project. This team handled the end-to-end system design, including the inference accelerator, memory hierarchy, and networking.

OpenAI partnered with Broadcom, which was responsible for the physical design aspects of the chip, starting from the gates. This division of labor contributed to the project's timeline.