← All stories
● Covered by 1 source · 1 reportLow impact1 neutral

DeepSeek-V3 Architecture Implemented in 190 Lines of Pure JAX

🔄 Updated 7d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • DeepSeek-V3 architecture implemented in 190 lines of JAX.
  • No external frameworks like Flax, Equinox, or PyTorch used.
  • Pretrained on a MacBook Air with public-domain detective fiction.
  • Demonstrates expert balancing with validation loss of 3.921 at step 2000.

Pure JAX Implementation of DeepSeek-V3

The DeepSeek-V3 architecture has been re-implemented in approximately 190 lines of pure JAX code. This implementation avoids reliance on higher-level frameworks such as Flax, Equinox, or PyTorch, presenting the model's components as functions operating on plain Python dictionaries of arrays.

Pretraining and Validation

The model was pretrained from scratch on a MacBook Air M5 (24 GB) using a dataset of public-domain detective fiction. During pretraining, the validation loss at step 2000 was 3.921 with expert balancing enabled, and 3.923 without, confirming the paper's claim that bias evens out experts without additional cost.

Architectural Demonstration

This project serves as an architectural demonstration rather than a production-ready language model. It highlights the core mechanics of DeepSeek-V3, including its expert balancing mechanism. A v1.0 release provides the exact checkpoint and data used in the published run for replication.

Setup and Requirements

To run the implementation, users need an Apple Silicon Mac, Python 3.12, and the `uv` package manager. The published run consumed 24 GB of memory. Specific instructions are provided for installation, checking the setup, and building the dataset, which involves downloading and processing approximately 120 MB of text data.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Reporting from

A developer has implemented the DeepSeek-V3 architecture in 190 lines of pure JAX, without using frameworks like Flax or PyTorch. The implementation was pretrained from scratch on a MacBook Air using public-domain detective fiction to demonstrate the architecture's core principles, including expert balancing.