Categorical cross-entropy from logits

Converts unnormalized class logits into probabilities and penalizes low probability assigned to the observed target class.

Cross-entropy loss of logits z and target y equals the negative sum over classes i of target y i times the log of softmax z at class i.

Turn scores into probabilities, keep the log probability relevant to the target distribution, and make confident mistakes expensive.

The summation is the outer reduction, while softmax and logarithm transform each class score before aggregation.

This is an objective definition. Implementations usually fuse softmax and log-sum-exp for numerical stability rather than computing probabilities first.