Google Cloud Storage SDKs Enable End-to-End Checksums by Default for Data Integrity

Default End-to-End Checksumming

Google Cloud Storage SDKs now enable end-to-end checksumming by default. This feature applies to both data uploads and downloads, aiming to maintain data durability and integrity within Cloud Storage.

Previously, developers had to manually calculate and provide checksums for full end-to-end data integrity. Cloud Storage always stored a checksum for every object, but client-side checksums were not universally adopted, leaving data vulnerable during transfer.

Addressing Data Transfer Vulnerabilities

The update addresses a gap where data uploads without client-provided checksums were susceptible to bit flips while in transit. The SDKs now internally calculate and pass checksums to Cloud Storage if the application does not provide one.

For downloads, the SDKs also support verifying the object's checksum, ensuring the integrity of retrieved data.

gRPC API Integration for Range Reads

For applications performing partial object downloads (range reads) using Cloud Storage SDKs with the gRPC API, the SDKs utilize gRPC's built-in end-to-end range checksum. This verifies the integrity of the specific data ranges received.

Impact on Developers

Developers are recommended to update to the latest SDK versions to benefit from these integrity features. This change reduces the manual effort required to ensure data integrity during transfers to and from Cloud Storage.