Activation function mathematical notation

Activation function is a recurring research-paper notation family. Transforms pre-activation values, usually elementwise, to introduce nonlinear behavior between learned affine mappings.

Activation function: Transforms pre-activation values, usually elementwise, to introduce nonlinear behavior between learned affine mappings. Example: Apply a nonlinear response after the affine layer.