Catalogue
- Cards
- 165
- Records
- 486
- Workloads
- 19
- Corpus version
- 74847648e25a
33 of 486 records match; the first 20 are shown.
| Kind | Record | Organisation | Year | Setup |
|---|---|---|---|---|
| Decision | Own the webhook pipeline after customers could not see what happened to a webhook, leaving a hosted queue | Marble | 2026 | |
| Decision | Workers atomically claim a delivery before sending so overlapping workers or retried queue messages do not send duplicates | Marble | 2026 | |
| Failure | One customer's endpoint outage made deliveries time out, filled the shared worker pool and delayed Directory Sync webhooks for everyone | WorkOS | 2025 | |
| Failure | Users who follow hundreds of thousands of accounts made hot shards that slowed the timelines sharing them | Bluesky | 2025 | Sharded |
| Decision | Drop timeline writes with a probability set by how many accounts a user follows, accepting imperfect timelines | Bluesky | 2025 | Sharded |
| Decision | Cache high-follow accounts in a Redis sorted set loaded into memory every 30 seconds instead of reading follow counts per write | Bluesky | 2025 | Sharded |
| Failure | A surge of duplicate webhooks and the retry setup filled one queue's provider quota, so new events were not delivered for hours | Onfido | 2024 | |
| Decision | Deploy unicast PTPv2 with custom Go server ptp4u, PPS-disciplined NICs, and switch transparent clocks | Meta | 2022 | Multi-region |
| Decision | Expose TAI time as a 6-9s Window of Uncertainty via fbclock with parallel linearizability cross-checks | Meta | 2022 | Multi-region |
| Decision | Add client-side Priority3 tiebreaking to ptp4l BMCA so clients distribute load across identical Time Appliances | Meta | 2022 | Multi-region |
| Failure | Widely deployed NIC's 128-packet timestamp buffer overflowed and its driver silently substituted software timestamps | Meta | 2022 | Multi-region |
| Decision | Adopt Openark Orchestrator for automated MySQL primary failover across GTID and non-GTID clusters | 2022 | ||
| Failure | Recurring huge rogue transactions stalled shared MySQL primaries and triggered unwanted failovers | 2022 | ||
| Decision | Measure absolute replication lag with heartbeat writes and guard promotions with FailMasterpromotionOnLagMinutes | 2022 | ||
| Decision | Register each cluster's dedicated backup/ETL replica as a prefer failover candidate rather than Must | 2022 | ||
| Decision | Throttle Orchestrator failovers within a two-minute window via a pre-failover hook, skipped during maintenance | 2022 | ||
| Failure | Consul streaming channel contention and BoltDB freelist write amplification stall the sole coordination cluster | Roblox | 2021 | |
| Failure | Cold-start cache deployment failures prolong recovery after Consul stabilises | Roblox | 2021 | |
| Decision | Keep performance- and latency-critical core and edge workloads on-prem while using public cloud for burst, DevOps and analytics | Roblox | 2021 | |
| Decision | Per-customer SQS queue and Lambda handler replace one shared RabbitMQ queue so slow or heavy customers stop delaying others | Dwolla | 2019 |
13 more records. Sign in to see all 33.