● Covered by 1 source · 1 reportLow impact1 neutral
Maple-Preview: Ternary 20B MoE AI model runs at 120 tokens/second on iPhone
Key points
- Maple-Preview is a ternary 20B MoE model.
- It achieves 120 tokens/second on an iPhone.
- The model is designed for efficient mobile AI inference.
Model Performance on Mobile
The Maple-Preview model, a ternary 20B Mixture-of-Experts (MoE) architecture, has been shown to operate at a speed of 120 tokens per second when running on an iPhone. This performance metric highlights its capability for local inference on consumer mobile hardware.
Technical Specifications
Maple-Preview utilizes a ternary quantization scheme, which reduces the precision of the model's weights to three values. This approach, combined with the Mixture-of-Experts design, contributes to its efficiency and ability to run on devices with limited computational resources like smartphones.
Implications for On-Device AI
The ability to run a 20-billion parameter model at high speeds on an iPhone suggests advancements in optimizing large language models for edge devices. This could lead to more powerful AI applications that operate without constant cloud connectivity, offering benefits in privacy, latency, and offline functionality.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Reporting from
A new AI model, Maple-Preview, a ternary 20B Mixture-of-Experts (MoE) model, has demonstrated performance of 120 tokens per second on an iPhone. This development indicates progress in running large language models efficiently on mobile devices.