Introduction to Stochastic Processes With R

An introduction to stochastic processes through the use of R

Introduction to Stochastic Processes with R is an accessible and well-balanced presentation of the theory of stochastic processes, with an emphasis on real-world applications of probability theory in the natural and social sciences. The use of simulation, by means of the popular statistical software R, makes theoretical results come alive with practical, hands-on demonstrations.

Written by a highly-qualified expert in the field, the author presents numerous examples from a wide array of disciplines, which are used to illustrate concepts and highlight computational and theoretical results. Developing readers’ problem-solving skills and mathematical maturity, Introduction to Stochastic Processes with R features:

• More than 200 examples and 600 end-of-chapter exercises
• A tutorial for getting started with R, and appendices that contain review material in probability and matrix algebra
• Discussions of many timely and stimulating topics including Markov chain Monte Carlo, random walk on graphs, card shuffling, Black–Scholes options pricing, applications in biology and genetics, cryptography, martingales, and stochastic calculus
• Introductions to mathematics as needed in order to suit readers at many mathematical levels
• A companion web site that includes relevant data files as well as all R code and scripts used throughout the book

Introduction to Stochastic Processes with R is an ideal textbook for an introductory course in stochastic processes. The book is aimed at undergraduate and beginning graduate-level students in the science, technology, engineering, and mathematics disciplines. The book is also an excellent reference for applied mathematicians and statisticians who are interested in a review of the topic.

Preface xi

Acknowledgments xv

List of Symbols and Notation xvii

About the Companion Website xxi

1 Introduction and Review 1

1.1 Deterministic and Stochastic Models 1

1.2 What is a Stochastic Process? 5

1.3 Monte Carlo Simulation 9

1.4 Conditional Probability 10

1.5 Conditional Expectation 18

Exercises 34

2 Markov Chains: First Steps 40

2.1 Introduction 40

2.2 Markov Chain Cornucopia 42

2.3 Basic Computations 52

2.4 Long-Term Behavior—the Numerical Evidence 59

2.5 Simulation 65

2.6 Mathematical Induction* 68

Exercises 70

3 Markov Chains for the Long Term 76

3.1 Limiting Distribution 76

3.2 Stationary Distribution 80

3.3 Can you Find the Way to State a? 94

3.4 Irreducible Markov Chains 103

3.5 Periodicity 106

3.6 Ergodic Markov Chains 109

3.7 Time Reversibility 114

3.8 Absorbing Chains 119

3.9 Regeneration and the Strong Markov Property* 133

3.10 Proofs of Limit Theorems* 135

Exercises 144

4 Branching Processes 158

4.1 Introduction 158

4.2 Mean Generation Size 160

4.3 Probability Generating Functions 164

4.4 Extinction is Forever 168

Exercises 175

5 Markov Chain Monte Carlo 181

5.1 Introduction 181

5.2 Metropolis–Hastings Algorithm 187

5.3 Gibbs Sampler 197

5.4 Perfect Sampling* 205

5.5 Rate of Convergence: the Eigenvalue Connection* 210

5.6 Card Shuffling and Total Variation Distance* 212

Exercises 219

6 Poisson Process 223

6.1 Introduction 223

6.2 Arrival, Interarrival Times 227

6.3 Infinitesimal Probabilities 234

6.4 Thinning, Superposition 238

6.5 Uniform Distribution 243

6.6 Spatial Poisson Process 249

6.7 Nonhomogeneous Poisson Process 253

6.8 Parting Paradox 255

Exercises 258

7 Continuous-Time Markov Chains 265

7.1 Introduction 265

7.2 Alarm Clocks and Transition Rates 270

7.3 Infinitesimal Generator 273

7.4 Long-Term Behavior 283

7.5 Time Reversibility 294

7.6 Queueing Theory 301

7.7 Poisson Subordination 306

Exercises 313

8 Brownian Motion 320

8.1 Introduction 320

8.2 Brownian Motion and Random Walk 326

8.3 Gaussian Process 330

8.4 Transformations and Properties 334

8.5 Variations and Applications 345

8.6 Martingales 356

Exercises 366

9 A Gentle Introduction to Stochastic Calculus* 372

9.1 Introduction 372

9.2 Ito Integral 378

9.3 Stochastic Differential Equations 385

Exercises 397

A Getting Started with R 400

B Probability Review 421

B.1 Discrete Random Variables 422

B.2 Joint Distribution 424

B.3 Continuous Random Variables 426

B.4 Common Probability Distributions 428

B.5 Limit Theorems 439

B.6 Moment-Generating Functions 440

C Summary of Common Probability Distributions 443

D Matrix Algebra Review 445

D.1 Basic Operations 445

D.2 Linear System 447

D.3 Matrix Multiplication 448

D.4 Diagonal, Identity Matrix, Polynomials 448

D.5 Transpose 449

D.6 Invertibility 449

D.7 Block Matrices 449

D.8 Linear Independence and Span 450

D.9 Basis 451

D.10 Vector Length 451

D.11 Orthogonality 452

D.12 Eigenvalue, Eigenvector 452

D.13 Diagonalization 453

Answers to Selected Odd-Numbered Exercises 455

References 470

Index 475

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.