Catalogue / uber-2024-gulfstream-ledgerstore-migration

Uber migrates over a trillion payment ledger entries from DynamoDB and TerraBlob to LedgerStore over two years with zero downtime

Uber, 2024. Gulfstream payment platform ledger storage.

Organisation
Uber
Evidence
Reported from production
Source
Migrating a Trillion Entries of Uber’s Ledger Data from DynamoDB to LedgerStore

Uber Blog, published 2024-04-11

Records

Decision#d1

Consolidate Gulfstream ledger storage from DynamoDB, TerraBlob, and LSG onto LedgerStore alone

Decision#d2

Combine bounded live shadow validation (99.99% to 99.9999%) with Spark offline validation and iterative backfill

Decision#d3

Run backfills incrementally with AIMD dynamic rate control under a safety upper bound

Decision#d4

Backfill guidance: roughly 1 GB dump files, bad records dumped separately, rate-limited logs

Decision#d5

Roll out LSG reads over weeks with DynamoDB fallback, fallback log audits, and a one-month retention window

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.