Decision#d1
Catalogue / meta-2026-data-ingestion-migration
Meta migrates tens of thousands of MySQL-to-warehouse CDC ingestion jobs via automated shadow and reverse-shadow phases
Meta, 2026. Social graph MySQL-to-data-warehouse CDC ingestion system.
Records
Decision#d2
Migrate jobs through three gated phases—shadow, reverse shadow and cleanup—verified on row counts, checksums, latency and resources
Decision#d3
Contain CDC bad-data propagation via paired reverse-shadow backfills and metadata bad-partition flags
Decision#d4
Batch shadow migrations under limited capacity, holding back jobs with known bugs and reusing legacy snapshots
Decision#d5
Automate lifecycle promotion from Scuba signals and run hourly mismatch row-level debugging
Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.