Catalogue / shopify-2024-realtime-embeddings

Shopify tunes Google Cloud Dataflow worker threads, model loading and batching for near real-time GPU embedding pipelines

Shopify, 2024. Real-time ML embeddings streaming pipelines.

Organisation
Shopify
Evidence
Reported from production
Source
How Shopify improved consumer search intent with real-time ML

Shopify Engineering, published 2024-10-18

Records

Decision#d1

Build near real-time Dataflow streaming pipelines for merchant embeddings

Decision#d2

Reduce Dataflow worker harness threads from 12 to 4 per process to stop image-pipeline OOMs without high-memory VMs

Decision#d3

Keep one embedding model instance per Dataflow worker process instead of sharing one model across processes

Decision#d4

Keep in-bundle batching rather than stateful cross-bundle batching despite bursty single-element bundles

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.