Introduction to Stochastic Processes with R

An introduction to stochastic processes through the use of R

Introduction to Stochastic Processes with R is an accessible and well–balanced presentation of the theory of stochastic processes, with an emphasis on real–world applications of probability theory in the natural and social sciences. The use of simulation, by means of the popular statistical freeware R, makes theoretical results come alive with practical, hands–on demonstrations.

Written by a highly–qualified expert in the field, the author presents numerous examples from a wide array of disciplines, which are used to illustrate concepts and highlight computational and theoretical results. Developing readers problem–solving skills and mathematical maturity, Introduction to Stochastic Processes with R features:

Over 200 examples and 600 end–of–chapter exercises A tutorial for getting started with R, and appendices that contain review material in probability and matrix algebra Discussions of many timely and interesting supplemental topics including Markov chain Monte Carlo, random walk on graphs, card shuffling, Black–Scholes options pricing, applications in biology and genetics, cryptography, martingales, and stochastic calculus Introductions to mathematics as needed in order to suit readers at many mathematical levels A companion website that includes relevant data files as well as all R code and scripts used throughout the book

Introduction to Stochastic Processes with R is an ideal textbook for an introductory course in stochastic processes. The book is aimed at undergraduate and beginning graduate–level students in the science, technology, engineering, and mathematics disciplines. The book is also an excellent reference for applied mathematicians and statisticians who are interested in a review of the topic.

Robert P. Dobrow, PhD, is Professor of Mathematics and Statistics at Carleton College. He has taught probability and stochastic processes for over 15 years and has authored numerous research papers in Markov chains, probability theory and statistics.

 

 

Preface xi

Acknowledgments xv

List of Symbols and Notation xvii

1 Introduction and Review 1

1.1 Deterministic and Stochastic Models 1

1.2 What is a Stochastic Process? 6

1.3 Monte Carlo Simulation 10

1.4 Conditional Probability 11

1.5 Conditional Expectation 19

Exercises 36

2 Markov Chains: First Steps 41

2.1 Introduction 41

2.2 Markov Chain Cornucopia 43

2.3 Basic Computations 53

2.4 LongTerm Behavior the Numerical Evidence 61

2.5 Simulation 67

2.6 Mathematical Induction∗ 71

Exercises 73

3 Markov Chains for the Long Term 79

3.1 Limiting Distribution 79

3.2 Stationary Distribution 83

3.3 Can You Find the Way to State a? 98

3.4 Irreducible Markov Chains 108

3.5 Periodicity 111

3.6 Ergodic Markov Chains 114

3.7 Time Reversibility 119

3.8 Absorbing Chains 124

3.9 Regeneration and the Strong Markov Property∗ 140

3.10 Proofs of Limit Theorems∗ 141

Exercises 152

4 Branching Processes 167

4.1 Introduction 167

4.2 Mean Generation Size 169

4.3 Probability Generating Functions 174

4.4 Extinction is Forever 178

Exercises 185

5 Markov Chain Monte Carlo 191

5.1 Introduction 191

5.2 MetropolisHastings Algorithm 197

5.3 Gibbs Sampler 208

5.4 Perfect Sampling∗ 216

5.5 Rate of Convergence: the Eigenvalue Connection∗ 222

5.6 Card Shuffling and Total Variation Distance∗ 224

Exercises 231

6 Poisson Process 235

6.1 Introduction 235

6.2 Arrival, Interarrival Times 239

6.3 Infinitesimal Probabilities 246

6.4 Thinning, Superposition 250

6.5 Uniform Distribution 256

6.6 Spatial Poisson Process 261

6.7 NonHomogeneous Poisson Process 265

6.8 Parting Paradox 267

Exercises 271

7 ContinuousTime

Markov Chains 277

7.1 Introduction 277

7.2 Alarm Clocks and Transition Rates 283

7.3 Infinitesimal Generator 286

7.4 LongTerm Behavior 297

7.5 Time Reversibility 308

7.6 Queueing Theory 316

7.7 Poisson Subordination 322

Exercises 329

8 Brownian Motion 337

8.1 Introduction 337

8.2 Brownian Motion and Random Walk 343

8.3 Gaussian Process 347

8.4 Transformations and Properties 351

8.5 Variations and Applications 363

8.6 Martingales 375

Exercises 386

9 A Gentle Introduction to Stochastic Calculus∗ 393

9.1 Introduction 393

9.2 Ito Integral 401

9.3 Stochastic Differential Equations 407

Exercises 420

Appendices 422

A Getting Started with R 423

B Probability Review 445

B.1 Discrete Random Variables 446

B.2 Joint Distribution 448

B.3 Continuous Random Variables 451

B.4 Common Probability Distributions 452

B.5 Limit Theorems 463

B.6 MomentGenerating Functions 464

C Summary of Probability Distributions 467

D Matrix Algebra Review 469

Problem Solutions 481

References 497

Index 501