Catalogue

Cards
165
Records
486
Workloads
19
Corpus version
74847648e25a
Workload
Kind
Setup

21 of 486 records match; the first 20 are shown.

KindRecordOrganisationYearSetup
Decision

Build an S3 and Spark data lake for update-heavy block data while keeping Snowflake and Fivetran for simpler workloads

notion-2024-data-lake#d1

Notion2024Sharded
Decision

Use Debezium-to-Kafka incremental CDC with one topic per table and RDS export-to-S3 only for bootstrapping

notion-2024-data-lake#d2

Notion2024Sharded
Decision

Choose Apache Hudi over Iceberg and Delta Lake and align S3 partitioning with Postgres's 480 shards

notion-2024-data-lake#d3

Notion2024Sharded
Decision

Land raw CDC data in S3 before processing and handle skewed shards differently in Scala Spark

notion-2024-data-lake#d4

Notion2024Sharded
Decision

Replace paginated job lookups with a dedicated Apache Pinot hybrid table for arbitrary-window job tenure counts

uber-2024-job-counting-pinot#d1

Uber2024Sharded
Failure

Full-dataset production traffic acted like full table scans on Pinot, maxing out SSDs and breaching 10-second broker timeouts

uber-2024-job-counting-pinot#f1

Uber2024Sharded
Decision

Prune off-heap MMAP Pinot segments with on-heap Bloom filters, 32 daily segments and a 30-minute consumer cache

uber-2024-job-counting-pinot#d2

Uber2024Sharded
Failure

Periodic 10-second rate-limiter bursts from upstream breached Pinot's 10-second broker timeout

uber-2024-job-counting-pinot#f2

Uber2024Sharded
Decision

Cover upstream trip data up to a week late with dynamic T-7d backfills and T-1d Hive COUNT(*) checks

uber-2024-job-counting-pinot#d3

Uber2024Sharded
Decision

Replace ad-hoc multi-hop legacy pipelines with one open-source Kafka and Flink platform

doordash-2022-iguazu#d1peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Put a customized async Confluent Kafka Rest Proxy in front of Kafka instead of direct clients in every service

doordash-2022-iguazu#d2peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Use replication factor 2, leader-only acknowledgments and 50ms to 100ms linger for non-transactional events

doordash-2022-iguazu#d3peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Register Protobuf event schemas in Confluent Schema Registry at build time in CI/CD instead of at runtime

doordash-2022-iguazu#d4peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Stage Kafka events to S3 Parquet via per-event Flink jobs before loading Snowflake with Snowpipe

doordash-2022-iguazu#d5peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Automate multi-repo Terraform event onboarding with a GitHub App, Cadence workflows and Retool UIs

doordash-2022-iguazu#d6peak of 100,000 requests/s or more

DoorDash2022Sharded
Decision

Serve every Q&A site from one multi-tenant application and keep web servers overprovisioned

stackoverflow-2016-architecture#d1peak under 10,000 requests/s

Stack Overflow2016Sharded
Decision

Use Elasticsearch for search instead of SQL Server full-text search, because SQL CPUs are expensive

stackoverflow-2016-architecture#d2peak under 10,000 requests/s

Stack Overflow2016Sharded
Decision

Cache in two layers, a web-server HTTP cache and Redis, and clear the first layer through Redis pub/sub

stackoverflow-2016-architecture#d3peak under 10,000 requests/s

Stack Overflow2016Sharded
Decision

Push updates to browsers over websockets, which are more efficient than polling, at the cost of load balancer port exhaustion

stackoverflow-2016-architecture#d4peak under 10,000 requests/s

Stack Overflow2016Sharded
Decision

Build a schema-less, index-free distributed column store for raw unaggregated spans

honeycomb-2024-distributed-column-store#d1

HoneycombSharded

1 more records. Sign in to see all 21.