● Covered by 3 sources · 5 reportsMedium impact2 neutral3 positive
Cloudflare prototypes Zstandard compression in Pingora to save petabytes of cache storage
Key points
- Cloudflare prototyped Cache Transcoding using Zstandard (zstd) in Pingora.
- The system compresses eligible assets to one-third of their original size.
- It reduces storage needs and inter-data center bandwidth.
- The trade-off is a small increase in CPU usage for encoding.
- Cache Transcoding compresses uncompressed text content like HTML, JSON, CSS, and JavaScript.
- The approach could provide petabytes of additional effective cache capacity.
- Compression reduced eligible content size by about 2.8 times.
- Cloudflare optimized its pingora-ketama library.
- The optimization reduced over 100TB of RAM usage across Cloudflare's network.
- The optimization was for Cloudflare's internal load-balancing service.
- The DNS team shed 100TB of memory last month.
- Ivan filed a ticket about excessive memory usage from pingora-ketama.
- Cloudflare reduced server hashes from 100,000 to 10,000 per machine.
- The optimization addressed memory bloat from excessive hash entries.
- Cloudflare redesigned the in-memory representation of its 1.1.1.1 DNS cache.
- The optimization reduced the per-entry footprint by 56%.
- Cache insertion throughput increased by 43%.
- Lookup latency decreased by 19%.
- The changes were made to Big Pineapple, Cloudflare's DNS platform.
- Big Pineapple stores over 250 billion DNS cache entries.
- Cloudflare replaced Vec and String with Box<[T]> and Box<str>.
- This saved 64 bytes per entry and over 15 TB fleetwide.
- Cloudflare combined answer, authority, and additional records into one list.
- Cloudflare packed Booleans into bitflags.
- Cloudflare omitted owner names matching the queried domain.
Cache Transcoding with Zstandard
Cloudflare has prototyped a system called Cache Transcoding, which integrates Zstandard (zstd) compression into its Pingora proxy. This system encodes eligible assets with zstd before writing them to disk, keeping them compressed while in cache and during transfer between data centers via Tiered Cache. Assets are then decoded before being served to clients.
Storage and Bandwidth Savings
Initial testing showed that this encoding shrunk eligible assets to one-third of their original on-disk size. This provides Cloudflare with petabytes of effective cache capacity and reduces data transferred between data centers. The encoding cost is paid once when an asset enters the cache, with storage and bandwidth savings realized every time the asset is reused.
CPU Trade-off
The implementation involves a minor increase in CPU cost in the origin-facing proxy for the compression process. Zstandard, developed by Yann Collet at Facebook and open-sourced in 2016, is a lossless compression algorithm designed to balance compression ratio with speed. The prototype uses zstd level 3 to achieve significant compression benefits without creating a CPU bottleneck during cache fills.
Impact on Cloudflare Operations
With increasing memory and storage costs, making efficient use of deployed memory is critical for Cloudflare's distributed storage products, including its CDN. This Cache Transcoding system allows Cloudflare to expand its effective cache capacity and continue serving customers efficiently by optimizing storage and bandwidth usage.
Updates
- Cloudflare redesigned the in-memory representation of its 1.1.1.1 DNS cache.
- The optimization reduced the per-entry footprint by 56%.
- Cache insertion throughput increased by 43%.
- Lookup latency decreased by 19%.
- The changes were made to Big Pineapple, Cloudflare's DNS platform.
- Big Pineapple stores over 250 billion DNS cache entries.
- Cloudflare replaced Vec and String with Box<[T]> and Box<str>.
- This saved 64 bytes per entry and over 15 TB fleetwide.
- Cloudflare combined answer, authority, and additional records into one list.
- Cloudflare packed Booleans into bitflags.
- Cloudflare omitted owner names matching the queried domain.
- Cloudflare reduced server hashes from 100,000 to 10,000 per machine.
- The optimization addressed memory bloat from excessive hash entries.
- Cloudflare optimized its pingora-ketama library.
- The optimization reduced over 100TB of RAM usage across Cloudflare's network.
- The optimization was for Cloudflare's internal load-balancing service.
- The DNS team shed 100TB of memory last month.
- Ivan filed a ticket about excessive memory usage from pingora-ketama.
- Cache Transcoding compresses uncompressed text content like HTML, JSON, CSS, and JavaScript.
- The approach could provide petabytes of additional effective cache capacity.
- Compression reduced eligible content size by about 2.8 times.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
GitHub facebook/zstd CVE CVE-2023-503877.5 HIGH GitHub cloudflare/pingora GitHub nginx/nginx GitHub RJ/ketama GitHub rust-lang/rustHow outlets covered it
Cloudflare redesigned the in-memory representation of its 1.1.1.1 DNS cache, reducing the per-entry footprint by 56% and freeing approximately 100 TB of memory across its fleet. This optimization also increased cache insertion throughput by 43% and reduced lookup latency by 19%.
Cloudflare optimized its Ketama-based hash-mapping algorithm within its Pingora framework, reducing the number of server hashes from 100,000 to 10,000 per machine. This optimization led to a 100TB reduction in RAM usage across its network. The change improves efficiency in caching and routing by addressing memory bloat caused by excessive hash entries.
Cloudflare optimized its `pingora-ketama` library, used in the Pingora Backend Router for consistent hashing, resulting in a reduction of over 100TB of RAM usage across its global network. This optimization improves resource efficiency for Cloudflare's internal load-balancing service, which distributes cacheable requests to servers.
Cloudflare is testing a prototype called Cache Transcoding that compresses uncompressed text content like HTML, JSON, CSS, and JavaScript using Zstandard before storing it in cache. This approach could provide petabytes of additional effective cache capacity and reduce data transfer between data centers by compressing content once upon entry and decompressing it upon serving.
Cloudflare prototyped a Cache Transcoding system using Zstandard compression within its Pingora proxy, reducing cached asset sizes by two-thirds on average. This system aims to expand effective cache capacity and decrease cross-data center bandwidth usage by trading a minor CPU increase for significant storage savings.