Catalogue / hotstar-2020-social-feed-emojis

Hotstar replaces a third-party emoji service with an async Go, Kafka (Knol), Spark Streaming and PubSub pipeline

Hotstar, 2020. Social Feed emoji and live audience reaction pipeline.

Organisation
Hotstar
Evidence
Reported from production
Source
Capturing A Billion Emo(j)i-ons

Hotstar (JioHotstar) tech blog, published 2020-04-08

Records

Decision#d1

Build an in-house decomposed Go, Knol (Kafka) and Spark Streaming emoji pipeline to replace a third-party service

Decision#d2

Acknowledge emoji HTTP submissions asynchronously from a local Go channel buffer before Kafka flush

Decision#d3

Aggregate emoji streams in 2-second micro-batches with Spark Streaming instead of Flink, Storm, or Kafka Streams

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.