● Covered by 1 source · 1 reportMedium impact1 neutral
Configuration and Patches for DeepSeek V4 Flash on AMD MI300X Released
Key points
- Repository enables DeepSeek-V4-Flash-0731 on single AMD MI300X.
- Includes Docker Compose, file overlays, and tuning tables.
- Addresses MI300X-specific FP8, MoE, and kernel issues.
- Model runs without additional weight quantization or offload.
Enabling DeepSeek V4 Flash on AMD MI300X
A new repository provides the necessary configuration and patches to deploy the deepseek-ai/DeepSeek-V4-Flash-0731 model on a single AMD MI300X GPU for production use. This includes a Docker Compose stack, SHA-256-pinned file overlays, reference diffs against upstream, and tuning tables. The model checkpoint runs as shipped, without requiring additional weight quantization or offload.
Addressing MI300X Specific Challenges
The official vLLM recipe primarily targets NVIDIA and newer AMD hardware, not fully supporting the MI300X. Running the model reliably on the MI300X required specific fixes for its FP8 format, MoE routing at high concurrency, causal speculative verification, CPU-KV synchronization, and several untuned kernel shapes. This repository collects these fixes and pins the versions used in production.
MI300X Capabilities and Model Fit
The MI300X features 192 GB of HBM3 and 5.3 TB/s of memory bandwidth, offering 2.4 times the HBM capacity of an H100 SXM5. This capacity allows for a simple single-GPU deployment of the 304B-parameter checkpoint, fitting the entire model in HBM without PCIe weight streaming or layer offload. It also provides room for a 20 GB GPU KV pool and a 96 GiB CPU tier for evicted prefix-cache entries, handling 2–8 typical concurrent streams and bursts of up to 64 streams.
FP8 Compatibility and Performance Tuning
The MI300X (CDNA3) implements the AMD/Graphcore fnuz variant of E4M3 for FP8, which differs from the OCP-standard FP8 used by newer MI325X and MI355X GPUs. Kernels assuming OCP semantics on MI300X can lead to incorrect results. Correctness for this specific FP8 implementation was prioritized, followed by performance tuning. The repository includes correctness overlays for the pinned ROCm nightly, a validated serving configuration with probabilistic DSpark drafting, and AITER GEMM tuning tables for recurring gfx942 shapes.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub cmc_internal/api GitHub github/collect GitHub ryanzhou/deepseek-v4-flash-mi300x GitHub _private/browser GitHub get-started/accessibility GitHub open-source/sponsorsReporting from
A repository has been released containing configurations and patches to run the DeepSeek-V4-Flash-0731 model on a single AMD MI300X GPU in a production environment. This provides solutions for FP8 format compatibility, MoE routing, and kernel shape tuning specific to the MI300X, which are not fully addressed by the official vLLM recipe.