---
title: "Training & Hyperparameters"
description: "Before tuning the learning rate, check that each update averages the examples you meant, across microbatches, gradient accumulation and ignored targets."
canonical_url: "https://fanout.sh/ai/lessons/fine-tuning-training-loop"
md_url: "https://fanout.sh/ai/lessons/fine-tuning-training-loop.md"
access: "pro-preview"
---

# Training & Hyperparameters

Before tuning the learning rate, check that each update averages the examples you meant, across microbatches, gradient accumulation and ignored targets.

- Access: Fanout Pro (public preview)
- Module: How to Fine-Tune Models

## A glimpse of Training & Hyperparameters

An optimizer update uses gradients from training examples to change the model's trainable parameters. The learning rate controls the size of that change. Before searching for a good learning rate, make sure the gradient represents the examples you intended to average.

This becomes easy to get wrong when a batch does not fit in memory.

## Continue learning

Pro unlocks guided learning, complete notes, practical walkthroughs, exercises, and every AI Research lesson.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md