PPO clipped surrogate objective

Limits the optimization incentive from policy probability ratios that move too far from the data-collecting policy.

PPO clipped objective of theta equals the expectation over time of the minimum between policy ratio r t times advantage A hat t and the clipped policy ratio times the same advantage.

Reward useful policy changes, but stop giving extra objective improvement once the probability ratio moves beyond the trusted interval.

Minimum is the conservative gate inside an outer expectation; its effect depends on the advantage sign.

This is a surrogate training objective, not a guarantee that the final policy ratio remains inside the interval for every sample.