Mixture of Experts
Also called MoE
A mixture-of-experts model uses multiple internal expert networks and a routing mechanism. In a sparse language-model implementation, the router sends each token through only a subset of those experts.
[Hugging Face]In practice · hypothetical example
A model’s router selects two internal expert networks to process a token while other experts remain inactive for that token.
[Hugging Face]A little deeper
The experts are neural-network components, not independently prompted assistants. Sparse activation separates the total parameter count from the parameters used for a particular token. [Hugging Face]
A common mix-up
Each expert is a separate agent with its own tools.
Experts here are components inside the model. [Hugging Face]
What does a sparse MoE router select?
Sources & editorial notes
Evidence: supported. Primary-source support for this scoped entry; publication approved by the project owner.
- Mixture of Experts Explained ↗ (opens in new tab)Hugging Face · Publication date unknown
Relevant section: What is a Mixture of Experts?; What does an expert learn?
Last editorial review: 2026-09-13 by project-owner.
First observed in this corpus: Unknown.
Revision history
Revision 2 · Created 2026-09-13 · Updated 2026-09-13
Project owner approved the current content for publication. Existing evidence scope and limitations remain applicable.