Catalogue / meta-2023-async-tier

Meta decouples its Async tier into tailers, a push scheduler, and a delay service for multi-source workloads

Meta. Async tier serverless asynchronous computing platform.

Organisation
Meta
Evidence
Reported from production
Source
Asynchronous computing at Meta: Overview and learnings

Engineering at Meta, published 2023-01-31

Records

Decision#d1

Split the monolithic Dispatcher into per-source Tailer adapters and a generic Scheduler service

Decision#d2

Switch workload transport from pull to RPC push from Tailers to the Scheduler

Decision#d3

Support both FOQS queues and Scribe streams as first-class storage backends for async jobs

Decision#d4

Offload failed stream items to a controlled-delay service to keep At-Least-Once retries from clogging checkpoints

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.