● Covered by 1 source · 1 reportMedium impact1 neutral
Netflix Replaces In-House Batch Job System with Open-Source Kueue
Key points
- Netflix moved batch workloads from CMB to Kueue.
- Kueue is an open-source, Kubernetes-native solution.
- CMB was an in-house system developed in 2018.
- Migration aimed for transparency to users and easy rollback.
Migration to Kueue
Netflix has transitioned the majority of its batch processing tasks to Kueue, an open-source, cloud-native batch job execution system. This move replaces Netflix's internally developed solution, Compute Managed Batch (CMB), which was part of its Titus container platform.
Reasons for the Change
The decision to adopt Kueue stemmed from the observation that many features initially developed for CMB had become standard in open-source Kubernetes projects. Furthermore, developing new functionalities within CMB became increasingly difficult due to its less integrated nature with Kubernetes. Kueue offered extensive capabilities, flexibility, and a faster pace of innovation compared to the aging in-house system.
Kueue's Capabilities
Kueue is a Kubernetes-native solution designed for batch job management. It provides features such as priority-based job queuing, advanced resource management, multi-cluster and topology-aware scheduling, and comprehensive observability. These functionalities either matched or exceeded those of CMB, with the added benefit of being part of a widely adopted open-source ecosystem.
Migration Process and Impact
The migration process was designed to be transparent to existing CMB users, ensuring that the new system could handle the required container launch rate and throughput from the start. Netflix engineers mapped internal tenant configurations from CMB to Kueue's Cohorts, ClusterQueue, and LocalQueue resources. The company is currently managing millions of batch workloads in production using Kueue, with the migration still ongoing.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Reporting from
Netflix has migrated most of its batch workloads from its proprietary Compute Managed Batch (CMB) system to Kueue, an open-source cloud-native job queueing system. This shift was driven by Kueue's advanced capabilities and better integration with the Kubernetes ecosystem, which CMB lacked as it aged.