Decision#d1
Catalogue / lyft-2023-verity-data-quality
Lyft builds Verity, an SQS- and DynamoDB-backed asynchronous platform running 140,000 weekly Hive data quality checks
Lyft, 2023. Verity data quality platform for Hive Data Warehouse.
Records
Decision#d2
Isolate traffic classes with dedicated SQS Check Queue and Executor pairs
Decision#d3
Store Check Results in DynamoDB for key-and-time lookups without database maintenance
Decision#d4
Run scheduled checks in Verity's own scheduler and gate Airflow/Flyte ETLs with thin SDK-backed blocking operators
Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.