Training & Hyperparameters

How to Fine-Tune Models

A glimpse of Training & Hyperparameters

An optimizer update uses gradients from training examples to change the model's trainable parameters. The learning rate controls the size of that change. Before searching for a good learning rate, make sure the gradient represents the examples you intended to average.

This becomes easy to get wrong when a batch does not fit in memory.

Continue learning

Pro unlocks guided learning, complete notes, practical walkthroughs, exercises, and every AI Research lesson.