← All stories
● Covered by 1 source · 1 reportMedium impact1 neutral

ggml v1.0.0 Released with API Pruning, FP8 Support, and Broad Performance Improvements

🔄 Updated 7d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • Deprecated API and legacy DIAG_MASK ops removed.
  • Added configurable YaRN truncation and FP8 support.
  • Introduced few-row quantized matmuls and MoE fusion on CUDA/Metal.
  • Vulkan prefill FlashAttention path and SYCL device-side MoE scheduler added.

API and Core Changes

ggml v1.0.0 prunes deprecated public APIs and removes legacy DIAG_MASK_INF/DIAG_MASK_ZERO operations, resulting in an RPC protocol bump to reject incompatible peers. New API functions, ggml_rope_set_truncate and ggml_rope_get_truncate, were added to control YaRN correction-bound rounding. The backward pass for TRI was implemented, and MUL_MAT src1 precision tests were added.

Core scheduler improvements include skipping uploads for unread inputs and prefetching only inputs produced by earlier splits. The MoE cache can now compute a share of cache misses on the CPU and asynchronously copy results back. The Meta backend also raises the compute view limit for buffers with fewer tensors.

CUDA Backend Enhancements

CUDA received several performance updates, including few-row MMQ for batch sizes 2-24 and MMVF for BF16 few-row matmuls. OCP FP8 support was added, and NVFP4 gate/up matmuls were fused with GLU in MMQ. FlashAttention was extended with new head-size configurations, and graph handling was improved to skip compatibility checks on reuse and re-instantiate on cluster-dimension changes. MoE optimizations include skipping empty MMQ stream-k tiles and linearizing the MoE layout.

Metal and Other Backend Improvements

The Metal backend now fuses MoE routing and keeps fusable node groups within a single command buffer. Other backend updates include a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, decomposed Adreno FlashAttention in OpenCL, and new WebGPU kernels. Hexagon op coverage was expanded, and scheduler improvements were implemented to avoid uploading unused inputs.

Impact and Significance

These updates collectively aim to improve the performance and efficiency of ggml across various hardware platforms and use cases, particularly for operations involving quantized models and Mixture-of-Experts (MoE) architectures. The API pruning simplifies the codebase, while new features like FP8 support enable broader compatibility and potential for reduced memory footprint and faster computation in AI inference workloads.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Reporting from

ggml v1.0.0 has been released, pruning deprecated APIs, adding configurable YaRN truncation and FP8 support, and introducing significant performance enhancements across various backends. These updates improve efficiency for quantized matrix multiplications, MoE fusion, and FlashAttention paths on platforms like CUDA, Metal, Vulkan, and OpenCL.