GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
Split a giant model across chips and pipeline micro-batches to keep them all busy
Huang et al. · NeurIPS 2019 · Parallelism. Read the paper ↗
A free, interactive, animated visual explainer of GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism — every exhibit computed from the real formulas, with verbatim quotes from the source.
Questions
- What is GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism?
- Split a giant model across chips and pipeline micro-batches to keep them all busy
- Who published GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism, and where?
- Huang et al. — NeurIPS 2019 (arXiv:1811.06965).
- Where can I find a visual explainer of GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism?
- Right here — a free, interactive, animated walkthrough of the whole paper, with exhibits computed from the real formulas and verbatim quotes from the source.