RWKV: Reinventing RNNs for the Transformer Era

An RNN trained like a Transformer — constant-state inference at GPT scale.

Peng et al. · EMNLP 2023 Findings · Model Architectures. Read the paper ↗

A free, interactive, animated visual explainer of RWKV: Reinventing RNNs for the Transformer Era — every exhibit computed from the real formulas, with verbatim quotes from the source.

Questions

What is RWKV: Reinventing RNNs for the Transformer Era?

An RNN trained like a Transformer — constant-state inference at GPT scale.

Who published RWKV: Reinventing RNNs for the Transformer Era, and where?

Peng et al. — EMNLP 2023 Findings (arXiv:2305.13048).

Where can I find a visual explainer of RWKV: Reinventing RNNs for the Transformer Era?

Right here — a free, interactive, animated walkthrough of the whole paper, with exhibits computed from the real formulas and verbatim quotes from the source.

Related explainers

• DeepSeek-V3
• Qwen3
• OLMo 2
• MiniMax-01
• Gemma 4
• Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
• Scalable Diffusion Models with Transformers