Catalogue
- Cards
- 165
- Records
- 486
- Workloads
- 19
- Corpus version
- 74847648e25a
20 of 486 records match.
| Kind | Record | Organisation | Year | Setup |
|---|---|---|---|---|
| Failure | Daily peak event burst saturated shared database primary, and with no automatic circuit breaker the throttling was manual | GitHub | 2026 | |
| Failure | Brief pod reduction during routine rollout saturated remaining sites, OOMed service-mesh sidecars, and triggered invalid-job runner retries | GitHub | 2026 | Multi-region |
| Failure | Recurring Amazon EBS latency spikes and I/O stalls trigger daily database partial failures and correlated single-AZ events | PlanetScale | 2025 | |
| Failure | Recurring incidents where errors on the Pushes MySQL cluster early in RepositoryPushJob kept unrelated pull requests from synchronizing | GitHub | 2024 | |
| Failure | Recurring 2020 CI/CD cascading failures when growth peaks overloaded Git, Checkpoint queues, and CI search | Slack | 2022 | |
| Failure | A bulk deletion script fed site IDs instead of app IDs deleted 883 customer sites with no warning | Atlassian | 2022 | Multi-region |
| Failure | Restoring hundreds of sites out of shared multi-tenant data stores had no batch tooling and took up to two weeks | Atlassian | 2022 | Multi-region |
| Failure | Recurring huge rogue transactions stalled shared MySQL primaries and triggered unwanted failovers | 2022 | ||
| Failure | A bug in per-tenant limit enforcement let one tenant overload a shared cluster, and error handling made the overload spread | Grafana Labs | 2021 | Multi-region |
| Failure | A viral customer's pageviews backlog the shared queue for all sites while the database runs out of disk IOPS | Fathom Analytics | 2020 | One database |
| Failure | Unindexed backlog tables and a text-typed site ID kept the database slow after the disk was fixed | Fathom Analytics | 2020 | One database |
| Failure | Outage of one InfluxDB cluster dropped metrics from direct relays and left datacenters out of sync | Hulu | 2019 | Multi-region |
| Failure | Millions of message tombstones in one channel triggered 10-second Cassandra stop-the-world GC pauses | Discord | 2017 | Sharded |
| Failure | Database contention filled Redis memory, and a full Redis could neither accept nor release jobs | Slack | 2017 | Sharded |
| Failure | A hundred checker Lambdas each opened a database connection at once, and Postgres refused further connections | OnlineOrNot | One database | |
| Failure | Unbalanced queue_id DocStore partitioning and unscaled entity-queue hosts caused errors, rate limiting, and timeouts | Uber | ||
| Failure | Unrelated connection leak failed Cadence inventory workflows, recovered by touching database timestamps | DoorDash | ||
| Failure | Recurring noisy-neighbor bursts and expensive queries starved bare-metal Postgres clusters and blocked PgBouncer pools | Cloudflare | Multi-region | |
| Failure | Controller rate-limiting, top-of-hour bursts, and sidecar startup caused CronJobs to miss runs and get stuck | Lyft | ||
| Failure | A long-lived transaction pinned dead rows and degraded a Postgres job queue | Not named in the source |