← All stories
● Covered by 1 source · 2 reportsLow impact2 neutral

Homelabber Runs 2.8-Trillion-Parameter AI Model on CPU with Discontinued Optane Memory

🔄 Updated 9h ago — new reporting from Tom's Hardware
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • Kimi K3 AI model (2.8T parameters) run on CPU with Optane PMem 200.
  • Used 40-core Xeon CPU, no GPU, achieving 0.85 tokens/second.
  • Optane memory provides necessary capacity at lower cost than DRAM.
  • Requires active cooling due to high heat output from Optane DIMMs.
  • Added two RTX 3090 GPUs to the build.
  • Achieved 3.11 tokens/second for generation and 16.8 tokens/second for prompt evaluation.
  • Nearly half the model (414 GB out of 865 GB) resides on a RAM drive.
  • RAM drive increased generation speed by 41% and prompt evaluation by 26%.
  • Without the RAM drive, generation speed was 2.20 tokens/second and prompt evaluation 13.3 tokens/second.

Running Large AI Models on Homelab Hardware

A homelab enthusiast on Reddit has managed to run Moonshot AI’s Kimi K3, a 2.8-trillion-parameter AI model, using only a 40-core 3rd-gen Xeon Scalable CPU. This was achieved by utilizing discontinued second-generation Intel Optane PMem 200 memory sticks, bypassing the need for a GPU for inference. The project is a follow-up to an earlier build that used first-generation Optane memory.

Optane Memory for Capacity and Cost Savings

The Kimi K3 model is designed for data center inference engines and requires at least 610GB of RAM for even the smallest community quantizations. With rising DRAM prices, using secondhand Optane memory provides a cost-effective way to achieve the necessary memory capacity at home. Intel's Optane Persistent Memory 200 Series (PMem 200), launched in 2020, offers capacities up to 512GB per stick and can deliver higher memory bandwidth than its predecessor.

Performance and Configuration Details

The setup achieved an inference speed of 0.85 tokens per second with 1-bit quantization, which dropped to 0.75 tokens per second with 3-bit quantization. This is considerably slower than typical inference speeds, with a 50,000-token task estimated to take 13 hours. The build used Optane in App Direct Mode, which exposes it as a separate, persistent region, giving more control over tensor placement. The previous build, which used a smaller model and an RTX 3060 GPU, achieved around 4 tokens per second.

Thermal Management Challenges

A significant challenge encountered was the heat generated by the Optane DIMMs. The builder emphasized the necessity of active DIMM cooling, employing dual 12W blower fans directed at the modules to maintain acceptable operating temperatures.

Updates

🕒 2026-11-21 · new reporting from Tom's Hardware
  • Added two RTX 3090 GPUs to the build.
  • Achieved 3.11 tokens/second for generation and 16.8 tokens/second for prompt evaluation.
  • Nearly half the model (414 GB out of 865 GB) resides on a RAM drive.
  • RAM drive increased generation speed by 41% and prompt evaluation by 26%.
  • Without the RAM drive, generation speed was 2.20 tokens/second and prompt evaluation 13.3 tokens/second.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

How outlets covered it

A homelab enthusiast improved their AI build by adding two RTX 3090 GPUs and a RAM drive to run the 2.8-trillion-parameter Moonshot Kimi K3 model. This configuration achieved 3.11 tokens per second for generation and 16.8 tokens per second for prompt evaluation, with nearly half the model residing on a RAM drive for speed gains.

A Reddit user successfully ran Moonshot AI's 2.8-trillion-parameter Kimi K3 model on a 40-core Xeon CPU using second-gen Intel Optane PMem 200 memory, without a GPU. This project demonstrates a method for running large AI models on consumer-grade hardware by utilizing high-capacity, secondhand persistent memory to overcome RAM limitations, despite slow inference speeds.