Catalogue / discord-2017-message-storage

Discord moves chat history from MongoDB to a 12-node Cassandra cluster with only 4 backend engineers in the whole company

Discord, 2017. Chat message storage.

Organisation
Discord
Evidence
Reported from production
Source
How Discord Stores Billions of Messages

Discord Blog, published 2017-01-13

Records

Decision#d1

Migrate chat history from a single MongoDB replica set to Cassandra once data and index outgrew RAM

Decision#d2

Bound Cassandra partition sizes with 10-day time buckets and Snowflake clustering keys

Decision#d3

Delete corrupt edit/delete race rows when author_id is null and omit null columns on Cassandra writes

Failure#f1

Millions of message tombstones in one channel triggered 10-second Cassandra stop-the-world GC pauses

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.