---
title: "Ring all-reduce communication volume"
description: "Estimates ideal bytes sent by each rank when an N-element tensor is all-reduced using a bandwidth-optimal reduce-scatter followed by all-gather ring."
canonical_url: "https://fanout.sh/labs/math-decoder/formula/ring-all-reduce-bytes-per-rank"
md_url: "https://fanout.sh/labs/math-decoder/formula/ring-all-reduce-bytes-per-rank.md"
last_updated: "2026-08-09"
access: "public"
---

# Ring all-reduce communication volume

Estimates ideal bytes sent by each rank when an N-element tensor is all-reduced using a bandwidth-optimal reduce-scatter followed by all-gather ring.

## Public overview

Ring communication volume per rank equals two times p minus one divided by p, times N elements, times b bytes per element.

In each of two ring phases, every rank sends almost one full tensor, split into equal chunks across its peers.

Multiplication combines phase count, ring traversal fraction, tensor element count, and element width into a byte estimate.

This is a communication-volume estimate, not a duration model; link bandwidth, contention, latency, topology, and algorithm selection determine elapsed time.

---
This representation contains public Fanout content only. Protected Pro lessons, account data, billing, checkout, and pricing are not included.

Browse the public content map: https://fanout.sh/sitemap.md