Kimi Linear: An Expressive, Efficient Attention Architecture

Kimi Delta Attention, interleaved 3:1 with full attention — the first linear hybrid to beat softmax.

Kimi Team · arXiv 2025 · Model Architectures. Read the paper ↗

A free, interactive, animated visual explainer of Kimi Linear: An Expressive, Efficient Attention Architecture — every exhibit computed from the real formulas, with verbatim quotes from the source.

Questions

What is Kimi Linear: An Expressive, Efficient Attention Architecture?

Kimi Delta Attention, interleaved 3:1 with full attention — the first linear hybrid to beat softmax.

Who published Kimi Linear: An Expressive, Efficient Attention Architecture, and where?

Kimi Team — arXiv 2025 (arXiv:2510.26692).

Where can I find a visual explainer of Kimi Linear: An Expressive, Efficient Attention Architecture?

Right here — a free, interactive, animated walkthrough of the whole paper, with exhibits computed from the real formulas and verbatim quotes from the source.

Related explainers

• PagedAttention (vLLM)
• DeepSeek-V3
• Qwen3
• OLMo 2
• MiniMax-01
• Gemma 4
• Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model