Mathematical Foundations of Information Theory

Comprehensive, rigorous introduction to work of Shannon, McMillan, Feinstein and Khinchin. Translated by R. A. Silverman and M. D. Friedman.

The Entropy Concept In Probability Theory
1. Entropy of Finite Schemes
2. The Uniqueness Theorem
3. Entropy of Markov chains
4. Fundamental Theorems
5. Application to Coding Theory
On the Fundamental Theorems of Information Theory
INTRODUCTION
CHAPTER I. Elementary Inequalities
  1. Two generalizations of Shannon's inequality
  2. Three inequalities of Feinstein
CHAPTER II. Ergodic Sources
  3. Concept of a source. Stationarity. Entropy
  4. Ergodic Sources
  5. The E property. McMillan's theorem.
  6. The martingale concept. Doob's theorem.
  7. Auxillary propositions
  8. Proof of McMillan's theorem.
CHAPTER III. Channels and the sources driving them
  9. Concept of channel. Noise. Stationarity. Anticipation and memory
  10. Connection of the channel to the source
  11. The ergodic case
CHAPTER IV. Feinstein's Fundamental Lemma
  12. Formulation of the problem
  13. Proof of the lemma
CHAPTER V. Shannon's Theorems
  14. Coding
  15. The first Shannon theorem
  16. The second Shannon theorem
CONCLUSION
REFERENCES