Open model architectures

Read the herd and tech reports of the strongest open models side by side — the attention, MoE, data, and training choices that separate them — and see the shared template emerge.

• The Llama 3 Herd of Models
• Mistral 7B
• Qwen3
• Phi-4 Technical Report
• OLMo 2
• Gemma 4
• MiniMax-01
• Mixtral of Experts
• DeepSeek-V3
• Kimi K2: Open Agentic Intelligence
• LongCat-Flash Technical Report