theory and information theory. The intent was to develop the tools of ergodic theory of potential use to information theory and to demonstrate their use by proving Shannon coding theorems …

theory and information theory. The intent was to develop the tools of ergodic theory of potential use to information theory and to demonstrate their use by proving Shannon coding theorems …

This site provides the current version of the first edition of the book Entropy and Information Theory by R.M. Gray in the Adobe portable document format (PDF).

Conditional Rate Distortion Theory, Scanned copy of Information Systems Laboratory Technical Report 6502-2, October 1972, an early work on conditional rate distortion functions and related …

Errata for Entropy and Information Theory: Second Edition Robert M. Gray Formatted June 4, 2023

ideas of modeling, coding, and process distance measures common to ergodic theory and information theory.

Shannon theory well known and highly developed. High rate theory less well known in information theory and signal processing theory, but classic results ubiquitous in practice e.g., 6 dB/bit …

This talk emphasizes one of them: the transportation distance (Monge, Kantorovich, Wasserstein/Vasershtein, Ornstein, d-bar, Mallows, earth mover’s, etc.) from the viewpoint of …

Proof of lower bound to l: Simple application of Kraft inequality and Divergence inequality, which follows from de ̄nition of divergence (relative entropy) and Jensen's inequality.

The essential prerequisites are a knowledge of matrix theory, an en-gineer’s knowledge of Fourier series and random processes, and calculus (Riemann integration).