Medusa, EAGLE & Multi-Token Prediction

The two drafters this module has met were both free of the model they serve: a rule that searches text the request has already produced holds no parameters at all, and a second model holds a full set of its own. The three lines of work here take the opposite trade. The drafter moves inside the target model, and the…

In this lesson

• Heads on the last hidden state
• Medusa's two acceptance rules
• Drafting at the feature level
• Heads trained with the trunk
• What each family asks for

Continue learning

Pro unlocks the complete lesson, video, practice, and code where available.