● Covered by 1 source · 1 reportMedium impact1 neutral
Meta Introduces MTIA 300 Custom Accelerator for AI Recommendation Models
Key points
- MTIA 300 is Meta's first in-house accelerator for recommendation models.
- It integrates networking and collective communication directly into the chip.
- Features 12 custom 800 Gbps RDMA NICs for 1.2 TB/s I/O bandwidth.
- Includes 16 dedicated message engines for concurrent communication and compute.
MTIA 300: A New Accelerator for Recommendation Models
Meta has announced MTIA 300, its custom-designed accelerator specifically optimized for training ranking and recommendation models. This new hardware represents Meta's expansion of its custom silicon strategy beyond general compute into specialized networking capabilities for AI workloads.
Addressing Communication Bottlenecks
Unlike large language model training, recommendation models frequently involve extensive communication between accelerators, largely due to embedding tables that can comprise over 99% of a model's parameters. These models generate frequent AllReduce, AllToAll, and AllGather operations, making network efficiency crucial. MTIA 300 tackles this by integrating networking directly into the chip.
Integrated Networking Architecture
The MTIA 300 accelerator includes two network chiplets, each containing six custom 800 Gbps RDMA NICs, providing a total of 1.2 TB/s of I/O bandwidth without relying on a PCIe bus. These twelve NICs support both scale-up communication within a rack and scale-out traffic between racks, offering flexibility in bandwidth allocation.
Dedicated Communication Engines
To prevent communication from consuming compute resources, MTIA 300 features 16 dedicated message engines. These engines handle collective communication independently of the main compute grid, including near-memory hardware for reduction operations. Meta reports that this separation allows large matrix operations and collective communication to run concurrently with less than 0.5% degradation in compute throughput, compared to over 20% degradation observed on conventional GPU architectures.
Co-design with HCCL
The hardware was co-designed with HCCL, Meta’s collective-communication library. HCCL compiles collective operations into subgraphs that MTIA 300’s message engines can execute autonomously, reducing the host CPU's involvement in communication orchestration. In production, HCCL achieves up to 940 GB/s of communication bandwidth within a rack. For a 150-billion-parameter recommendation model across 40 accelerators, MTIA 300 reduced total communication time by 3.9x compared to previous methods.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Reporting from
Meta has unveiled MTIA 300, its first in-house accelerator designed for training ranking and recommendation models, which integrates networking and collective communication directly into the chip. This development addresses the communication bottlenecks prevalent in recommendation model training, where network performance can be as critical as computational throughput.