Uber Introduces Normalized Compute Units for Consistent CPU Allocation Across Heterogeneous Fleet

Addressing Heterogeneous Server Performance

Uber has introduced Normalized Compute Units (NCUs) as part of its Odin stateful orchestration platform. This system aims to standardize CPU allocation across Uber's extensive server fleet, which includes approximately 5 million CPU cores spanning on-premises data centers and public clouds, with various generations of x86 and ARM processors.

The Challenge of Inconsistent CPU Performance

Uber identified that single-threaded CPU performance can vary by nearly a factor of two between its fastest and slowest cores. This disparity means that the same workload might require significantly different numbers of physical CPU cores depending on the server it runs on. This creates a dilemma: allocating resources based on the slowest hardware wastes capacity on faster machines, while allocating based on average usage risks under-provisioning workloads on slower servers, particularly critical for stateful services where workload migration is often not feasible.

Normalized Compute Units Solution

The NCU solution separates a workload's required compute performance from the physical cores providing it. Odin assigns a performance factor to each server type, using an older server generation as a baseline of 100. Some newer hardware can have factors approaching 190. These factors are initially set during hardware qualification and then refined using data from production workloads.

Dynamic Core Allocation

CPU usage is converted into normalized units using these performance factors. Odin then determines the required compute capacity for a workload and translates that into a specific number of physical cores based on the hosting server. This means a workload on faster hardware might receive fewer cores, while the same workload on slower hardware receives more, ensuring comparable computational capacity rather than identical core counts.

Refining Performance Factors with Production Data

Uber also observed that performance factors vary across different technologies. For instance, Kafka measurements showed distinct performance characteristics between ARM, older on-premises hardware, and newer x86 systems. Consequently, Uber updates these factors using actual production measurements, rather than relying solely on synthetic benchmarks, to ensure accuracy.