Policy KL penalty mathematical notation

Policy KL penalty is a recurring research-paper notation family. Discourages a trained policy from moving too far from a reference distribution by charging scaled KL divergence.

Policy KL penalty: Discourages a trained policy from moving too far from a reference distribution by charging scaled KL divergence. Example: Trade higher reward against staying close to baseline behavior.