---
title: "RMSNorm"
description: "Rescale a vector by the root mean square of its entries, then apply a learned scale. Work [3, 4] by hand, pick the axis, and compare with LayerNorm."
canonical_url: "https://fanout.sh/ai/lessons/neural-network-from-scratch-rmsnorm"
md_url: "https://fanout.sh/ai/lessons/neural-network-from-scratch-rmsnorm.md"
access: "pro-preview"
---

# RMSNorm

Rescale a vector by the root mean square of its entries, then apply a learned scale. Work [3, 4] by hand, pick the axis, and compare with LayerNorm.

- Access: Fanout Pro (public preview)
- Module: Neural Network from Scratch

## A glimpse of RMSNorm

A network layer may receive vectors whose overall magnitude varies. RMSNorm rescales a vector using the root mean square of its entries, then applies a learned scale to each feature.

Start with [3, 4]. Square the entries to get [9, 16], average them to get 12.5, then take the square root: approximately 3.536. Dividing both entries by that number gives approximately [0.849, 1.131].

## Continue learning

Pro unlocks guided learning, complete notes, practical walkthroughs, exercises, and every AI Research lesson.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md