● Covered by 2 sources · 2 reportsMedium impact2 neutral
Cloudflare Halves DNS Build Delay and Cuts Memory Use by 85% for Large Zones
Key points
- DNS record changes increased from 250 to 700 per second.
- New planner and scheduler reduce build delays by 50%.
- Peak memory usage cut by 85% in the Zone Builder.
- Zone setting changes no longer trigger full zone rebuilds.
- Cloudflare now processes 800 zone changes per second.
Addressing Growing DNS Zone Complexity
Cloudflare's Zone Builder now handles approximately 700 DNS record changes per second, up from 250 in 2022, alongside about 100 other events like zone setting changes. The primary challenge was not the volume of events but the work required for certain event types, particularly how that work scaled with zone size. Some zones currently contain over 20 million records, with projections for growth into hundreds of millions.
Optimizing Build Processes
A 'build' is the process of converting a DNS change into data usable by authoritative DNS servers, involving reading the database state, converting it to an edge format, and writing changes to Quicksilver. Previously, per-record builds were significantly faster than full builds. The goal was to apply similar optimizations to zone setting changes, which often triggered full zone rebuilds even for minor adjustments.
Introducing Targeted Build Planning
Cloudflare developed a targeted build planner that identifies the minimal safe build required for a given change. This was crucial because zone events, unlike record events, often lack specific details about what changed and can affect data shared across multiple zones. An accompanying new scheduler combines work for the same zone, preventing redundant processing.
Performance Improvements
The implementation of the targeted build planner and new scheduler has led to substantial improvements. Zone setting changes no longer necessitate rebuilding zones with tens of millions of records. This optimization resulted in an 85% reduction in peak memory usage. The increased memory headroom allowed Cloudflare to safely decrease pipeline wait times, effectively halving the delay from a DNS change to its data being written to Quicksilver.
DNS Change Propagation Pipeline
Cloudflare's DNS propagation pipeline involves several stages: a customer or service initiates a DNS change, which is stored in a database. This database change or another system generates a Kafka event. The DNS Kafka Aggregator then consolidates events from various Kafka topics before sending them for further processing.
Updates
- Cloudflare now processes 800 zone changes per second.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
The daily brief
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Today's brief
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's brief
New every morning, and the back catalogue is archived by date.
Primary sources
CVE CVE-2023-503877.5 HIGHHow outlets covered it
Cloudflare engineers optimized their DNS build services, resulting in a 50% reduction in DNS zone rebuild times and an 85% decrease in memory usage. This improvement addresses the increasing volume of zone changes and the complexity of maintaining consistency across their large DNS fleet.
Cloudflare implemented a targeted build planner and a new scheduler in its DNS propagation pipeline, reducing DNS build delays by 50% and peak memory usage by 85%. These changes address the growing complexity of large DNS zones, some exceeding 20 million records, by processing zone setting changes more efficiently.