Pathways: Asynchronous Distributed Dataflow for ML

One controller, thousands of accelerators — parallel dispatch makes single-controller ML as fast as SPMD.

Barham et al. · MLSys 2022 · Systems. Read the paper ↗

A free, interactive, animated visual explainer of Pathways: Asynchronous Distributed Dataflow for ML — every exhibit computed from the real formulas, with verbatim quotes from the source.

Questions

What is Pathways: Asynchronous Distributed Dataflow for ML?

One controller, thousands of accelerators — parallel dispatch makes single-controller ML as fast as SPMD.

Who published Pathways: Asynchronous Distributed Dataflow for ML, and where?

Barham et al. — MLSys 2022 (arXiv:2203.12533).

Where can I find a visual explainer of Pathways: Asynchronous Distributed Dataflow for ML?

Right here — a free, interactive, animated walkthrough of the whole paper, with exhibits computed from the real formulas and verbatim quotes from the source.