---
title: "Adam parameter update"
description: "Updates each parameter using a smoothed gradient divided by an adaptive scale derived from recent squared gradients."
canonical_url: "https://fanout.sh/labs/math-decoder/formula/adam-bias-corrected-update"
md_url: "https://fanout.sh/labs/math-decoder/formula/adam-bias-corrected-update.md"
last_updated: "2026-08-09"
access: "public"
---

# Adam parameter update

Updates each parameter using a smoothed gradient divided by an adaptive scale derived from recent squared gradients.

## Public overview

Theta at step t equals theta at the previous step minus alpha times bias-corrected first moment m hat t divided by the square root of corrected second moment v hat t plus epsilon.

Move in the smoothed gradient direction, but take smaller coordinate steps where recent gradients have been large.

The equals sign defines an iterative update; the fraction is the adaptive coordinate-wise preconditioner.

This line is only the final Adam update; the paper's algorithm also defines how m, v, and their bias corrections evolve.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md