---
title: "Reasoning models with human feedback"
description: "From next-token predictor to assistant: supervised fine-tuning teaches the format, then reinforcement learning from human feedback (RLHF) — a reward model"
canonical_url: "https://fanout.sh/ai-engineering/lessons/reasoning-models-with-human-feedback"
md_url: "https://fanout.sh/ai-engineering/lessons/reasoning-models-with-human-feedback.md"
access: "public"
---

# Reasoning models with human feedback

From next-token predictor to assistant: supervised fine-tuning teaches the format, then reinforcement learning from human feedback (RLHF) — a reward model

## Public overview

From next-token predictor to assistant: supervised fine-tuning teaches the format, then reinforcement learning from human feedback (RLHF) — a reward model trained on human preferences steering PPO — teaches judgment. And the modern turn: RL against verifiable rewards (math checks, unit tests) is what produced the o1/R1 class of reasoning models.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md