Gradient accumulation mathematical notation

Gradient accumulation is a recurring research-paper notation family. Combines gradients from several smaller microbatches before applying one optimizer update to emulate a larger effective batch.

Gradient accumulation: Combines gradients from several smaller microbatches before applying one optimizer update to emulate a larger effective batch. Example: Process small batches separately, then average their gradients for one update.