Catalogue / shopify-2026-catalog-clustering

Shopify clusters billions of products across millions of stores with a Liquid singleton filter, FAISS pre-chunking, and two-stage structured LLMs

Shopify, 2026. Shopify Catalog intra-store product clustering pipeline.

Organisation
Shopify
Evidence
Reported from production
Source
Clustering billions of products for agentic commerce with Catalog API

Shopify Engineering, published 2026-06-17

Records

Decision#d1

Partition catalog clustering to per-shop tasks and filter standalone stores deterministically via Liquid themes before calling LLMs

Decision#d2

Replace naive linkage-tree clustering with FAISS HNSW plus sparse UPGMA average linkage to form coherent chunks up to 200 products

Decision#d3

Use a two-stage propose-and-critique LLM flow with dynamic per-chunk Structured Output schemas and normalized product IDs

Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.