Mixture-of-experts layer mathematical notation

Mixture-of-experts layer is a recurring research-paper notation family. Routes each token to a small subset of specialized feed-forward expert networks and combines their outputs.

Mixture-of-experts layer: Routes each token to a small subset of specialized feed-forward expert networks and combines their outputs. Example: Send x to chosen experts and blend their transformed outputs.