Expert Parallelism & Expert Placement

A router returns logical expert IDs. A GPU can execute those assignments only after the engine knows which weights it owns. With tensor parallelism each device holds a slice of every expert; with expert parallelism each device holds complete experts and receives the tokens assigned to them.

In this lesson

• The fourth axis
• A per-device plan
• From global IDs to local slots
• Requested placement and actual placement
• Practice: revise the placement sheet

Continue learning

Pro unlocks the complete lesson, video, practice, and code where available.