Tensor-parallel weight shard mathematical notation

Tensor-parallel weight shard is a recurring research-paper notation family. Names the rank-local slice of a weight tensor produced by partitioning a layer across parallel devices.

Tensor-parallel weight shard: Names the rank-local slice of a weight tensor produced by partitioning a layer across parallel devices. Example: Store only one partition of the layer weights on each device.