Uber has re-engineered its translation cache, built in Go, to achieve a significant reduction in memory usage. The initial memory footprint has decreased by more than 90%, while critical performance metrics, specifically p99 cache miss latency, remain below one millisecond. This optimization is crucial for a system that processes millions of translations per second and maintains a cache hit rate exceeding 99.9%.
The primary motivation for this redesign stemmed from memory profiling across Uber's Go services. It was found that garbage collection (GC) accounted for over 20% of CPU utilization in some services, with translation objects making up approximately 70% of allocated memory. These inefficiencies, while seemingly small, accumulate to substantial resource consumption at Uber's operational scale.
An initial step involved replacing Go's `sync.Map` with a standard `map[string]string` protected by an `RWMutex`. This change reduced object allocations and improved CPU utilization by over 30% in affected services, as the workload was predominantly read-heavy. However, this did not fully resolve the overall memory consumption, especially during cache refreshes where both old and new caches temporarily resided in memory, sometimes leading to out-of-memory errors.
To tackle the remaining memory issues, Uber implemented a hybrid storage design. This approach keeps all translation keys in memory for rapid lookups, while the corresponding translation values are stored in disk-backed files. This design preserves the existing `Get()` and `Put()` API, minimizing integration effort. The system is sharded into 256 parts, each with its own mutex, in-memory entries, and disk file. Six-byte hash-based keys are used to reduce metadata overhead, with a fallback mechanism for collisions. A time-based global Least Recently Used (LRU) policy manages the in-memory entries.
This hybrid architecture effectively addresses the memory footprint problem without compromising latency. By intelligently managing where data resides, Uber can handle large translation datasets and frequent cache refreshes more efficiently, reducing resource contention and improving overall service stability. During cache refreshes, the system uses access history to preload frequently used translations, further optimizing performance.
✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →
One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.
One email a day. Unsubscribe in one click, any time.
Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.
▶ Play today's briefNew every morning, and the back catalogue is archived by date.
Uber redesigned its Go-based translation cache, reducing its initial memory footprint by over 90% while maintaining p99 cache miss latency below one millisecond. The new hybrid architecture stores translation keys and frequently accessed values in memory, with other values on disk, addressing high garbage collection CPU utilization and memory consumption issues at scale.