AWQ, GPTQ & SmoothQuant

A number format is a container. It fixes how many bits a value gets and how those bits are read, and it decides nothing about which value goes in. Store a matrix at four bits per entry when it was written at sixteen, and most of its values no longer have a slot of their own. Move every one to…

In this lesson

• Activation outliers
• GPTQ's compensation step
• The channels AWQ protects
• Migrating the difficulty
• The three methods compared

Continue learning

Pro unlocks the complete lesson, video, practice, and code where available.