Catalogue

Cards
165
Records
486
Workloads
19
Corpus version
74847648e25a
Workload
Kind
Setup

20 of 486 records match.

KindRecordOrganisationYearSetup
Failure

Users who follow hundreds of thousands of accounts made hot shards that slowed the timelines sharing them

bluesky-2025-lossy-timelines#f1peak of 100,000 requests/s or more

Bluesky2025Sharded
Decision

Drop timeline writes with a probability set by how many accounts a user follows, accepting imperfect timelines

bluesky-2025-lossy-timelines#d1peak of 100,000 requests/s or more

Bluesky2025Sharded
Decision

Cache high-follow accounts in a Redis sorted set loaded into memory every 30 seconds instead of reading follow counts per write

bluesky-2025-lossy-timelines#d2peak of 100,000 requests/s or more

Bluesky2025Sharded
Decision

Build an S3 and Spark data lake for update-heavy block data while keeping Snowflake and Fivetran for simpler workloads

notion-2024-data-lake#d1

Notion2024Sharded
Decision

Use Debezium-to-Kafka incremental CDC with one topic per table and RDS export-to-S3 only for bootstrapping

notion-2024-data-lake#d2

Notion2024Sharded
Decision

Choose Apache Hudi over Iceberg and Delta Lake and align S3 partitioning with Postgres's 480 shards

notion-2024-data-lake#d3

Notion2024Sharded
Decision

Land raw CDC data in S3 before processing and handle skewed shards differently in Scala Spark

notion-2024-data-lake#d4

Notion2024Sharded
Decision

Replace paginated job lookups with a dedicated Apache Pinot hybrid table for arbitrary-window job tenure counts

uber-2024-job-counting-pinot#d1

Uber2024Sharded
Failure

Full-dataset production traffic acted like full table scans on Pinot, maxing out SSDs and breaching 10-second broker timeouts

uber-2024-job-counting-pinot#f1

Uber2024Sharded
Decision

Prune off-heap MMAP Pinot segments with on-heap Bloom filters, 32 daily segments and a 30-minute consumer cache

uber-2024-job-counting-pinot#d2

Uber2024Sharded
Failure

Periodic 10-second rate-limiter bursts from upstream breached Pinot's 10-second broker timeout

uber-2024-job-counting-pinot#f2

Uber2024Sharded
Decision

Cover upstream trip data up to a week late with dynamic T-7d backfills and T-1d Hive COUNT(*) checks

uber-2024-job-counting-pinot#d3

Uber2024Sharded
Decision

Replace ad-hoc multi-hop legacy pipelines with one open-source Kafka and Flink platform

doordash-2022-iguazu#d1peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Put a customized async Confluent Kafka Rest Proxy in front of Kafka instead of direct clients in every service

doordash-2022-iguazu#d2peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Use replication factor 2, leader-only acknowledgments and 50ms to 100ms linger for non-transactional events

doordash-2022-iguazu#d3peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Register Protobuf event schemas in Confluent Schema Registry at build time in CI/CD instead of at runtime

doordash-2022-iguazu#d4peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Stage Kafka events to S3 Parquet via per-event Flink jobs before loading Snowflake with Snowpipe

doordash-2022-iguazu#d5peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Automate multi-repo Terraform event onboarding with a GitHub App, Cadence workflows and Retool UIs

doordash-2022-iguazu#d6peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Build a schema-less, index-free distributed column store for raw unaggregated spans

honeycomb-2024-distributed-column-store#d1

HoneycombSharded
Decision

Partition column files into timestamp-bounded segments across workers instead of maintaining secondary indexes

honeycomb-2024-distributed-column-store#d2

HoneycombSharded