---
title: "State value under a policy"
description: "Defines how valuable a state is when future actions are selected by policy pi."
canonical_url: "https://fanout.sh/labs/math-decoder/formula/reinforcement-learning-state-value"
md_url: "https://fanout.sh/labs/math-decoder/formula/reinforcement-learning-state-value.md"
last_updated: "2026-08-09"
access: "public"
---

# State value under a policy

Defines how valuable a state is when future actions are selected by policy pi.

## Public overview

V sub pi of s equals the expected return G at time t, under policy pi, given that the state at time t is s.

The value of state s is the average return you should expect after arriving there and continuing with policy pi.

The expectation is the main verb: average future returns over uncertainty while holding the current state fixed.

This is a definition, not an update rule. A paper may later introduce Bellman equations or estimators for calculating it.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md