Reduce-scatter collective mathematical notation
Reduce-scatter collective is a recurring research-paper notation family. Reduces equal contributions across ranks and leaves each rank with a distinct equal-sized shard of the reduced result.
Reduce-scatter collective: Reduces equal contributions across ranks and leaves each rank with a distinct equal-sized shard of the reduced result. Example: Combine contributions and distribute different pieces rather than copying the whole result.