Decision#d1
Catalogue / uber-2024-job-counting-pinot
Uber moves job-tenure counting to an Apache Pinot hybrid table tuned with sorted segments, Bloom filters and 32 daily segments
Uber, 2024. User Tenure Store (job-counting service on Apache Pinot).
Records
Failure#f1
Full-dataset production traffic acted like full table scans on Pinot, maxing out SSDs and breaching 10-second broker timeouts
Decision#d2
Prune off-heap MMAP Pinot segments with on-heap Bloom filters, 32 daily segments and a 30-minute consumer cache
Failure#f2
Periodic 10-second rate-limiter bursts from upstream breached Pinot's 10-second broker timeout
Decision#d3
Cover upstream trip data up to a week late with dynamic T-7d backfills and T-1d Hive COUNT(*) checks
Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.