Decision#d1
Catalogue / ebay-2023-nugraph-export
eBay exports 15-billion-vertex NuGraph graphs to HDFS from a FoundationDB DR cluster via Spark and JVM fixes
eBay, 2023. NuGraph graph export ETL pipeline.
Records
Decision#d2
Build a Spark plugin with resumable 5-second iterators, vertex-aligned splits and supernode partition grouping
Decision#d3
Replace JanusGraph's Guava LRU VertexCache with Caffeine to eliminate over-10% Spark GC overhead
Decision#d4
Remove unnecessary Spark shuffles in TinkerPop CloneVertexProgram and shrink ObjectWritable.toString() to stop OOMs
Sign in to read what was chosen, what was rejected and what it cost, each with its place in the source. An account is free.