Subject
2 entries
Dataflow
Bookmarks
Pathways: Asynchronous Distributed Dataflow for ML
Google's Pathways is a single-controller orchestration layer for ML accelerators that runs sharded asynchronous dataflow across thousands of TPUs while matching SPMD performance. Designed to break the MPI-style lockstep model so heterogeneous workloads like MoE, pipelining, and foundation-model multiplexing become first-class.
Timely Dataflow: An Introduction
An introduction to Timely Dataflow, the distributed computation model developed at Microsoft Research that unified batch and streaming processing through a novel timestamp-based progress tracking system. A technically significant but underappreciated alternative to the Spark/Storm paradigm.