Discrete Stochastic Processes and Applications

This unique text for beginning graduate students gives a self-contained introduction to the mathematical properties of stochastics and presents their applications to Markov processes, coding theory, population dynamics, and search engine design. The book is ideal for a newly designed course in an introduction to probability and information theory. Prerequisites include working knowledge of linear algebra, calculus, and probability theory. The first part of the text focuses on the rigorous theory of Markov processes on countable spaces (Markov chains) and provides the basis to developing solid probabilistic intuition without the need for a course in measure theory. The approach taken is gradual beginning with the case of discrete time and moving on to that of continuous time. The second part of this text is more applied its core introduces various uses of convexity in probability and presents a nice treatment of entropy.

Preface.- I. Markov processes.- 1. Discrete time, countable space.- 2. Linear algebra and search engines.- 3. The Poisson process.- 4. Continuous time, discrete space.- 5. Examples.- II. Entropy and applications.- 6. Prelude: a users guide to convexity.- 7. The basic quantities of information theory.- 8. An example of application: binary coding.- A. Some useful facts from calculus.- B. Some useful facts from probability.- C. Some useful facts from linear algebra.- D. An arithmetical lemma.- E. Table of exponential families.- References.- Index.