Catalogue / linkedin-2022-kafka-topicgc

LinkedIn builds TopicGC to safely garbage-collect unused Kafka topic metadata via a state machine

LinkedIn. TopicGC Kafka metadata garbage collection service.

Organisation
LinkedIn
Evidence
Reported from production
Source
TopicGC: How LinkedIn cleans up unused metadata for its Kafka clusters

LinkedIn Engineering Blog, published 2022-11-29

Records

Decision#d1

Build a state-machine Kafka topic garbage collector with access sealing, wildcard-consumer detach, and throttled batch deletes

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.