Perplexity Replaces DynamoDB with Custom CobbleDB to Reduce Latency and Storage Costs

Migration to CobbleDB

Perplexity transitioned its core search serving infrastructure from Amazon DynamoDB to CobbleDB, a custom-built distributed key-value store written in Rust. This move was driven by challenges with latency and cost associated with serving large document batches to language models under high query volumes.

Addressing Latency and Cost Bottlenecks

The previous DynamoDB setup resulted in severe latency issues due to its black-box nature, which concealed internal partition placement and caching policies. This led to tail-latency spikes from uncached reads and cross-zone networking. Additionally, DynamoDB's usage-based pricing became financially unsustainable at over 200,000 requests per second, as every transferred byte incurred costs for the multi-kilobyte document batches required by language models.

Reprocessing jobs for updated chunking algorithms or embedding models also caused high-volume writes directly into DynamoDB, creating contention with live user requests.

Architectural Solution

To resolve these issues, Perplexity re-architected its storage into three specialized systems: Pillar for durable state management, Lorry for batch aggregation, and CobbleDB for low-latency serving. This separation allowed for optimized handling of different data access patterns.

Pillar uses YTsaurus on high-capacity mechanical drives for versioned storage of web page metadata, passages, and vector representations. Lorry acts as a stateless consumer, grouping Pillar exports into partition-aligned batch files, storing them in Amazon S3, and notifying CobbleDB. CobbleDB worker nodes then independently ingest these S3 batches, isolating the hot serving path.

Impact of the Transition

The migration to CobbleDB and the new storage architecture resulted in a fivefold reduction in batch-read latencies. Furthermore, overall storage expenses were lowered by at least 20%. This improvement was achieved by decoupling durable document storage from the hot-tier retrieval system, which is critical for the specific read patterns of AI answer engines.