● Covered by 1 source · 1 reportMedium impact1 neutral
Janas-LLM enables running large language models exceeding RAM by streaming experts from SSD
Key points
- Janas-LLM runs LLMs larger than system RAM by streaming experts from SSD.
- Achieves 37 tok/s for Qwen3-Next-80B on a laptop with 32 GB RAM.
- Outperforms llama.cpp in token generation on tested models and hardware.
- Available for Linux x86-64 and Windows 10/11 x86-64.
Janas-LLM Introduces SSD Streaming for Large LLMs
Janas-LLM is a new tool designed to run large language models (LLMs) that exceed a computer's available RAM. It achieves this by keeping model weights on disk and streaming necessary 'experts' from an NVMe SSD as each token is generated. Frequently used experts are cached in memory to improve performance.
Performance Benchmarks
On a laptop with 32 GB of memory, Janas-LLM achieved 37 tokens per second (tok/s) when running Qwen3-Next-80B (a 48 GB model). This performance significantly surpasses llama.cpp, which managed 2.8 tok/s for the same model on the same hardware. For prompt reading, Janas-LLM reached 62 tok/s compared to llama.cpp's 16 tok/s.
The tool demonstrated faster performance than llama.cpp across all models tested on a Debian 13 Linux system with an Intel Core Ultra 9 185H processor and integrated Arc GPU. Similar performance was observed on Windows 11, with Qwen3-4B achieving 25.6 tok/s and Qwen3-30B-A3B reaching 34.2 tok/s for writing.
Availability and Implementation
Janas-LLM is written in C and has no dependencies beyond the standard C library. It is available for Linux x86-64 (Ubuntu 22.04, Debian 12 and later) and Windows 10 (1903 or later) and 11, x86-64. Models are pre-converted from Hugging Face for direct use.
The project is described as an experiment rather than a product, developed by Maurizio "camauri" Cammalleri.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub cmc_internal/api GitHub github/collect GitHub prabanta-dev/janas GitHub _private/browser GitHub get-started/accessibility GitHub open-source/sponsorsReporting from
Janas-LLM, a new tool written in C, allows large language models (LLMs) like Qwen3-Next-80B to run on consumer-grade hardware with limited RAM by streaming model experts from an NVMe SSD. This method achieves significantly faster token generation rates compared to llama.cpp on the same hardware, making larger LLMs accessible on ordinary computers.