Kubernetes v1.37 Introduces Scheduler Preemption for In-Place Pod Resize (Alpha)

Addressing Resource Scheduling Gaps

Kubernetes v1.37 introduces scheduler preemption for in-place Pod resize as an alpha feature, enabled by the InPlacePodVerticalScalingSchedulerPreemption feature gate. This mechanism allows the Kubernetes scheduler to actively free up capacity on a fully-utilized node by preempting lower-priority workloads. This ensures that pending in-place resizes of critical, higher-priority applications can succeed.

Background on In-Place Pod Resize

The core in-place Pod resize feature, which reached General Availability in v1.35, allows dynamic adjustment of CPU and memory allocations for running containers without requiring restarts. However, a limitation existed: if a Pod requested a resource scale-up that exceeded the host node's allocatable headroom, the Kubelet would mark the request as 'Deferred'. The Pod would then remain in this state indefinitely, waiting for resources to become available on the node.

Understanding Deferred Resize Requests

When a user or controller updates a container's resource requests, the Kubelet checks for sufficient spare capacity on the node. If the node is fully utilized and cannot satisfy the new limits, the Kubelet sets the container's resizeStatus to 'Deferred'. Unlike 'Infeasible' requests, which are immediately rejected, a 'Deferred' status indicates a valid request that is temporarily blocked due to lack of node capacity.

Impact of Preemption

Before this preemption mechanism, a Pod's in-place resize scale-up request could become permanently blocked on heavily utilized nodes. This posed a problem for critical applications, such as in-memory databases, that might require more memory to prevent out-of-memory crashes but could not obtain it due to resource constraints. The new preemption feature aims to resolve this by prioritizing critical application resource needs.