Direct preference optimization loss mathematical notation

Direct preference optimization loss is a recurring research-paper notation family. Fits a policy directly to preference pairs through a logistic comparison of learned-to-reference sequence log-probability ratios.

Direct preference optimization loss: Fits a policy directly to preference pairs through a logistic comparison of learned-to-reference sequence log-probability ratios. Example: Penalize the policy unless the preferred response gains the larger reference-relative score.