Knowledge Maxxing← Back to topic library
AI MaxxingAdvanced90 sec
Expert routing
Expert routing is the mechanism that decides which expert components in a Mixture-of-Experts model process each token or example.
Prerequisite: Mixture-of-Experts
What matters
The router itself is a small learned component, trained alongside the experts, so which expert handles which input emerges from training rather than being fixed by a person.
Example
A router might learn to send tokens that look like code to one expert and conversational tokens to another, without anyone having labeled the tokens that way.
Common misconception
Expert routing does not guarantee balanced use of every expert; without additional balancing techniques, a router can overuse a few experts and leave others rarely used.
Recall question
Who or what decides which expert handles a given token in a Mixture-of-Experts model?
Keywords and related concepts
expert routingroutergating networkmixture-of-experts