← All stories
● Covered by 1 source · 1 reportMedium impact1 neutral

Uber Reduces Translation Cache Memory by 90% with Hybrid Go Architecture

🔄 Updated 1d ago
New to BrevFeed? We gather this story from every outlet covering it into one summary — ranked by real-world impact, not just the latest headline — so you never miss what matters. What is BrevFeed? →

Key points

  • Memory footprint reduced by over 90%.
  • P99 latency remains under 1ms.
  • Hybrid architecture uses in-memory keys, disk-backed values.
  • Addresses high GC CPU usage and memory consumption.

Cache Redesign for Efficiency

Uber has re-engineered its translation cache, built in Go, to achieve a significant reduction in memory usage. The initial memory footprint has decreased by more than 90%, while critical performance metrics, specifically p99 cache miss latency, remain below one millisecond. This optimization is crucial for a system that processes millions of translations per second and maintains a cache hit rate exceeding 99.9%.

Addressing Memory and CPU Overhead

The primary motivation for this redesign stemmed from memory profiling across Uber's Go services. It was found that garbage collection (GC) accounted for over 20% of CPU utilization in some services, with translation objects making up approximately 70% of allocated memory. These inefficiencies, while seemingly small, accumulate to substantial resource consumption at Uber's operational scale.

Initial Optimizations and Challenges

An initial step involved replacing Go's `sync.Map` with a standard `map[string]string` protected by an `RWMutex`. This change reduced object allocations and improved CPU utilization by over 30% in affected services, as the workload was predominantly read-heavy. However, this did not fully resolve the overall memory consumption, especially during cache refreshes where both old and new caches temporarily resided in memory, sometimes leading to out-of-memory errors.

Hybrid Storage Solution

To tackle the remaining memory issues, Uber implemented a hybrid storage design. This approach keeps all translation keys in memory for rapid lookups, while the corresponding translation values are stored in disk-backed files. This design preserves the existing `Get()` and `Put()` API, minimizing integration effort. The system is sharded into 256 parts, each with its own mutex, in-memory entries, and disk file. Six-byte hash-based keys are used to reduce metadata overhead, with a fallback mechanism for collisions. A time-based global Least Recently Used (LRU) policy manages the in-memory entries.

Impact and Benefits

This hybrid architecture effectively addresses the memory footprint problem without compromising latency. By intelligently managing where data resides, Uber can handle large translation datasets and frequent cache refreshes more efficiently, reducing resource contention and improving overall service stability. During cache refreshes, the system uses access history to preload frequently used translations, further optimizing performance.

✨ This summary was generated by AI from the outlets' reporting listed below. It is not independently verified and may contain errors — check the original sources. How BrevFeed works →

The daily brief

One email each morning: the day's tech stories, clustered across outlets and summarized. No account needed.

One email a day. Unsubscribe in one click, any time.

Today's brief

Spend a few minutes, get the whole day. Every topic's top stories in one hands-free rundown — listen, watch, or read the transcript.

~4 min · 3 stories · Nov 21

▶ Play today's brief Listen on Spotify

New every morning, and the back catalogue is archived by date.

Reporting from

Uber redesigned its Go-based translation cache, reducing its initial memory footprint by over 90% while maintaining p99 cache miss latency below one millisecond. The new hybrid architecture stores translation keys and frequently accessed values in memory, with other values on disk, addressing high garbage collection CPU utilization and memory consumption issues at scale.