Dense vs Mixture-of-Experts

Activate every parameter for every token, or route each token to a few of many experts.

A clear, side-by-side comparison with examples — part of Rudrite Research.