Lectures on probability, information and large scale behaviour

Abstract

This book stems from lecture notes prepared by the author over decades of teaching. It is divided into two parts. The first part focuses on classical probability, with the aim of helping students develop a solid intuition about random phenomena. It guides them in translating real-world problems into probabilistic language and equips them with the tools needed to derive quantitative insights. The second part investigates the typical behaviours that emerge in asymptotic regimes -- such as the law of large numbers, limit theorems, and large deviations -- and elucidates their connection to information theory. This latter part offers a unifying perspective, based on principles of information theory and statistical mechanics, for understanding large-scale phenomena like phase transitions that appear across disciplines such as statistical physics, inference, coding theory, and computer science.