DeepX Tapes Out DX-M2 AI Accelerator for Edge Devices, Claims 100B Parameter Inference at Under 5W

Second-Generation Edge AI Accelerator

DeepX, a South Korean edge AI chip startup, has taped out its DX-M2, a second-generation AI accelerator. This chip is intended to bring generative AI capabilities to edge devices such as robots and drones. The DX-M2 is designed to perform inference for models with up to 100 billion parameters while operating at less than 5 watts of power.

Performance and Efficiency Improvements

The DX-M2 aims to enable local AI on devices with strict power and thermal limitations. It can run models ranging from 20 billion to 100 billion parameters (with 3 billion to 20 billion active parameters) at speeds of 10 to 30 tokens per second, maintaining the same 5-watt power budget as its predecessor, the DX-M1. Key improvements include a tripling of memory bandwidth from 50 GB/s to 150 GB/s. The NPU's power efficiency has doubled due to design enhancements, and a move to a 2-nm process node further doubled efficiency, allowing the chip to achieve 80 TOPS.

Processor-in-Memory Integration

A significant feature of the DX-M2 is its integration with Samsung's custom LPDDR5x-PIM (processor-in-memory) technology. PIM schemes address the memory bandwidth bottleneck in AI inference, particularly for general matrix-vector multiplication (GEMV) operations, by performing computations directly within the memory chip. This increases effective memory bandwidth and reduces memory access times. DeepX has collaborated with Samsung Memory for 1.5 years on this integration and is currently testing LPDDR5x PIM samples.

Support for Advanced AI Models

The DX-M2 is capable of handling large mixture-of-experts (MoE) large language models (LLMs). It also supports newer techniques such as speculative decoding and TurboQuant. These capabilities, combined with its low power consumption, are intended to facilitate the deployment of sophisticated AI in a wide range of battery-powered and edge devices.