ggml v1.0.0 prunes deprecated public APIs and removes legacy DIAG_MASK_INF/DIAG_MASK_ZERO operations, resulting in an RPC protocol bump to reject incompatible peers. New API functions, ggml_rope_set_truncate and ggml_rope_get_truncate, were added to control YaRN correction-bound rounding. The backward pass for TRI was implemented, and MUL_MAT src1 precision tests were added.
Core scheduler improvements include skipping uploads for unread inputs and prefetching only inputs produced by earlier splits. The MoE cache can now compute a share of cache misses on the CPU and asynchronously copy results back. The Meta backend also raises the compute view limit for buffers with fewer tensors.
CUDA received several performance updates, including few-row MMQ for batch sizes 2-24 and MMVF for BF16 few-row matmuls. OCP FP8 support was added, and NVFP4 gate/up matmuls were fused with GLU in MMQ. FlashAttention was extended with new head-size configurations, and graph handling was improved to skip compatibility checks on reuse and re-instantiate on cluster-dimension changes. MoE optimizations include skipping empty MMQ stream-k tiles and linearizing the MoE layout.
The Metal backend now fuses MoE routing and keeps fusable node groups within a single command buffer. Other backend updates include a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, decomposed Adreno FlashAttention in OpenCL, and new WebGPU kernels. Hexagon op coverage was expanded, and scheduler improvements were implemented to avoid uploading unused inputs.
These updates collectively aim to improve the performance and efficiency of ggml across various hardware platforms and use cases, particularly for operations involving quantized models and Mixture-of-Experts (MoE) architectures. The API pruning simplifies the codebase, while new features like FP8 support enable broader compatibility and potential for reduced memory footprint and faster computation in AI inference workloads.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
ggml v1.0.0 has been released, pruning deprecated APIs, adding configurable YaRN truncation and FP8 support, and introducing significant performance enhancements across various backends. These updates improve efficiency for quantized matrix multiplications, MoE fusion, and FlashAttention paths on platforms like CUDA, Metal, Vulkan, and OpenCL.