Cloudflare Redesigns D1 Control Plane for Millions of Serverless SQL Databases

D1 Control Plane Redesign

Cloudflare redesigned the control plane for its D1 serverless SQL database to support millions of databases. The previous architecture faced potential bottlenecks with increasing scale. The new design focuses on avoiding centralized bottlenecks, implementing retries and idempotency, and using eventual consistency to manage distributed operations.

Architectural Components

The D1 control plane architecture now uses Postgres for transactional metadata, ClickHouse for analytics, and Workers KV for globally replicated routing data. This combination allows for specialized handling of different data types and operations. Query processing remains at the edge, distinct from centralized management operations, to maintain performance and reduce latency.

Scalability and Resilience Mechanisms

To improve scalability and recover from failures, the system incorporates distributed metadata caching, connection pooling, asynchronous workflows, and periodic health checks. Cloudflare uses 30-second heartbeats with per-database jitter to reconcile each D1 Durable Object with Postgres, which acts as the source of truth for database existence and account limits. This mechanism ensures self-correction for failed creation or deletion operations, repairing orphaned objects and Workers KV routing metadata if best-effort updates fail.

Separation of Data and Control Planes

D1 queries are routed directly from an Edge API Worker through an Eyeball Worker to the relevant D1 Durable Object, keeping query traffic off the centralized Postgres database. The Edge API Worker handles authentication, authorization, rate limiting, and request validation. The Eyeball Worker trusts database-specific authorization encoded in D1 Bindings, preventing redundant checks in the query path. Workers KV is used as a durable datastore for routing, ensuring that reads can be served from KV's own storage even if edge caches are purged, thus keeping Postgres out of the data plane.

Impact of Growth

Cloudflare D1, announced in 2022, is a serverless SQL database built on SQLite that runs on Cloudflare’s global network. Over the past year, the number of daily active databases on D1 has grown 18-fold, necessitating the control plane redesign to accommodate this rapid expansion and ensure continued performance and reliability.