Catalogue / stripe-2019-dns-vpc-limit

Hourly Hadoop reverse DNS lookups, amplified 7x by retries, apparently reach AWS VPC resolver's 1,024 pps limit and bring SERVFAILs

Stripe. Internal DNS resolver cluster (Unbound on AWS VPC).

Organisation
Stripe
Evidence
Incident report
Source
The secret life of DNS packets: investigating complex networks

Stripe Blog, published 2019-05-21

Records

Failure#f1

Hourly Hadoop reverse DNS burst amplifies 7x via layered retries and apparently reaches AWS VPC resolver 1,024 pps cap

Decision#d1

Bypass central DNS servers for reverse lookups and isolate private and public PTR forwarding rules

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.