Grab Migrates Counter Service to Aerospike, Achieves 50% Lower P99 Latency

Storage Backend Migration

Grab has moved the storage backend for its Counter Service from a wide-column database to Aerospike. This migration involved a redesign of the data model and a staged traffic migration process to prevent service downtime. The Counter Service is a critical component of Grab’s anti-fraud platform, processing tens of thousands of queries per second and around one billion requests daily.

Performance and Cost Improvements

The migration resulted in significant performance and cost benefits. Grab reported a roughly 50% reduction in production P99 read latency. On-disk data storage decreased from approximately 3 TB to 1 TB. Additionally, the cost per node was lowered by 45% to 50%.

Staged Migration Process

To ensure a smooth transition, Grab implemented a staged migration. They separated storage access from the service's business logic using a storage facade that supported legacy, Aerospike, and mock implementations. Traffic was gradually shifted, starting with shadow reads at 5%, then increasing to 20%, 50%, and 100%, with parity validated through existing metrics before live traffic was directed to Aerospike.

Data Model Redesign

A key aspect of the migration was a new data model. Instead of storing 15-minute, hourly, and daily counts as separate rows, which was the previous approach, Grab consolidated all buckets for a counter and granularity into a single record containing a timestamp-ordered map. This redesign significantly reduced the record count and contributed to the lower index and disk footprint by addressing Aerospike's primary index metadata overhead per record.