Kubernetes 1.36 Restores Consistency Group Guarantee for Database Backups

Addressing Inconsistent Backups in Kubernetes

Kubernetes 1.36 introduces a critical feature that restores the consistency group guarantee for database backups. Previously, stateful workloads like PostgreSQL clusters, which often spread data across multiple PersistentVolumeClaims (PVCs) for I/O isolation, faced a risk of inconsistent backups. This inconsistency arose because individual volume snapshots were taken sequentially, not simultaneously, leading to a mismatch in data states between volumes at the time of backup.

The Lost Consistency Group

Traditional enterprise storage arrays provided consistency groups, allowing administrators to snapshot multiple LUNs belonging to the same application at the exact same instant. This ensured that all volumes captured the same point in time, making restores coherent. The Container Storage Interface (CSI) in Kubernetes, however, standardized snapshots around a single VolumeSnapshot per PVC, losing this multi-volume synchronization capability. This gap primarily manifested during restore operations after an incident, when it was too late to correct the issue.

Impact on Stateful Workloads

For applications that distribute their state across several volumes, such as databases with separate data and write-ahead log disks or sharded datastores, the per-PVC snapshot model created a vulnerability. Backup tools would snapshot each PVC sequentially, meaning changes could occur on one volume between the snapshot of another, resulting in an inconsistent backup. Kubernetes 1.36 resolves this by reintroducing the ability to ensure all relevant volumes are captured at a single, consistent point in time.