On Calibration of Modern Neural Networks

A classifier can become more accurate while its probabilities become less honest. Follow reliability diagrams, expected calibration error, and the one learned temperature that softens confidence without changing a single predicted class.

Reading focus: Why accuracy and calibrated confidence answer different questions, and how a reliability diagram exposes the gap. How one validation-fitted temperature rescales every logit while preserving the model's class ranking and accuracy. What the paper's cross-dataset results establish, and why binning, distribution shift, and newer architectures limit the recipe.

ICML 2017. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 13 min read, easy difficulty.