Differential Transformer
Two softmax maps subtracted — attention noise cancels like a differential amplifier.
Ye et al. · ICLR 2025 · Model Architectures. Read the paper ↗
A free, interactive, animated visual explainer of Differential Transformer — every exhibit computed from the real formulas, with verbatim quotes from the source.
Questions
- What is Differential Transformer?
- Two softmax maps subtracted — attention noise cancels like a differential amplifier.
- Who published Differential Transformer, and where?
- Ye et al. — ICLR 2025 (arXiv:2410.05258).
- Where can I find a visual explainer of Differential Transformer?
- Right here — a free, interactive, animated walkthrough of the whole paper, with exhibits computed from the real formulas and verbatim quotes from the source.