● Covered by 1 source · 1 reportMedium impact1 neutral
MicroGPT in pure C achieves 10M tokens/sec inference on Apple M5
Key points
- MicroGPT is a character-level transformer written in pure C.
- It includes forward pass, backprop, Adam, and sampling in a single C file.
- Achieved 10.1M tokens/sec inference speed on Apple M5 with NEON.
- The model has 4192 parameters and generalizes well on name generation.
Pure C Implementation of MicroGPT
MicroGPT is a character-level transformer developed in pure C, designed for training and inference without external dependencies beyond the standard C library (libc). The implementation includes core components such as forward pass, backpropagation, the Adam optimizer, and sampling capabilities within a single C file.
Performance Benchmarks
The MicroGPT model demonstrated significant performance, achieving an inference speed of 10,168,430 tokens per second (10.1M tps) on an Apple M5 chip, utilizing NEON optimizations. It is compatible with macOS, Linux, and Windows (MSYS2), supporting ARM64 with NEON and x86-64 with AVX2.
Training and Generalization
The model, which has 4192 parameters, trains on approximately 32,000 names in a few seconds and can generate new names. When trained on 20,000 names from a dataset of 32,033, it achieved a loss of 2.2054 nats per character on seen data and 2.2039 on unseen data. This performance indicates that the model generalizes effectively rather than memorizing the training data, outperforming an interpolated trigram model with nearly five times more parameters.
Technical Details
Training and inference utilize separate forward passes. The `gpt_forward` function stores activations for backpropagation, while `gpt_forward_infer` is a specialized single-token path. The logits from both paths match within fp32 rounding precision. Documentation on performance details and limitations is available in `docs/PERFORMANCE.md`.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub cmc_internal/api GitHub github/collect GitHub vixhal-baraiya/microgpt-c GitHub _private/browser GitHub get-started/accessibility GitHub open-source/sponsorsReporting from
A character-level transformer, MicroGPT, implemented in pure C with no external dependencies beyond libc, achieved over 10 million tokens per second inference speed on an Apple M5 chip. This implementation demonstrates efficient GPT training and inference in a minimal C environment, highlighting performance capabilities on modern hardware.