Kubernetes v1.37 Introduces etcd RangeStream to Reduce Memory for Large List Reads

Introduction of etcd RangeStream

Kubernetes v1.37 includes the graduation of etcd RangeStream to beta. This feature, when used with etcd v3.7, aims to decrease the memory footprint of the API server and etcd during operations involving large list reads. It also makes peak memory usage more predictable.

Addressing Memory Consumption Issues

The Kubernetes API server typically serves list and watch requests from its in-memory watch cache. Populating this cache involves reading the full state of a resource from etcd, which can be memory-intensive for resources with many or large objects, like Pods. Previously, the API server paginated these reads by key count, but this method did not account for object size, leading to potentially large pages that could trigger out-of-memory (OOM) errors.

How RangeStream Works

etcd v3.7 introduces the RangeStream RPC, a streaming version of the read operation. Instead of building an entire response before sending it, etcd splits the data into adaptively sized chunks and streams them. This allows memory to be freed as the stream progresses, rather than being held until a complete page is assembled. The API server decodes each chunk upon arrival and releases it before requesting the next, ensuring neither side holds the full collection in memory.

Feature Activation and Compatibility

RangeStream is utilized when the EtcdRangeStream feature gate is enabled on the kube-apiserver. This feature gate is in beta and is enabled by default in Kubernetes v1.37. It requires etcd v3.7 or later. The API server checks for etcd's support at startup and can fall back to older pagination methods at runtime if an older etcd version is detected or if a call returns 'Unimplemented'.