A Mathematical Theory of Communication

Shannon's information theory paper is hard but worth anchoring early in the calendar. Entropy, bits, uncertainty, and channel capacity sit underneath compression, coding, language modeling, and evaluation intuition.

Reading focus: How entropy gives uncertainty a quantitative unit. Why communication can be separated into source, channel, noise, and code. How information-theoretic language appears later in ML objectives.

Bell System Technical Journal 1948. Claude Shannon. 65 min read, hard difficulty.