Kubernetes v1.37 Enables Workload Scaling to Zero with HorizontalPodAutoscaler

Scaling to Zero in Kubernetes v1.37

Kubernetes v1.37 now includes Beta API support for the HorizontalPodAutoscaler (HPA) to scale workloads down to zero replicas. This functionality is enabled by default, allowing HPAs using suitable object or external metrics to scale a workload completely down and then back up based on metric changes.

Previous Implementations and Core Integration

Prior to v1.37, achieving scale-to-zero required add-ons, external components, or enabling an Alpha feature gate. The capability is now integrated into core Kubernetes, simplifying its adoption and use for managing resource consumption.

Benefits and Trade-offs

Scaling to zero offers significant cost savings by eliminating idle Pods, especially for workloads like queue consumers and batch processors that reserve expensive resources such as dedicated CPUs or GPUs. The primary trade-off is an increased cold-start time, as the HPA must observe the metric, schedule a Pod, and start the application. This approach is best suited for workloads where processing can tolerate a delay, such as tasks waiting in a durable queue.

It is important to note that Kubernetes Services do not buffer requests when no Pods are ready, meaning HTTP and other request-driven workloads require a separate buffering layer to handle requests during scale-to-zero periods.

Metric Considerations for Scaling

Traditional HPA metrics like CPU or memory usage are derived from running Pods, making them unsuitable for scaling back up from zero replicas. Object and external metrics, such as queue length, do not have this limitation as they exist independently of the worker Pods. This allows the HPA to continuously monitor the metric and initiate scaling when needed, even if no Pods are currently running.