Applied Probability Models With Optimization Applications

Concise advanced-level introduction to stochastic processes that frequently arise in applied probability. Largely self-contained text covers Poisson process, renewal theory, Markov chains, inventory theory, Brownian motion and continuous time optimization models, much more. Problems and references at chapter ends. "excellent introduction."-Jrnl. of the American Statistical Association. Bibliography. 1970 edition.

1. INTRODUCTION TO STOCHASTIC PROCESSES
  1.1. Random Variables and Probability Theory
  1.2. Conditional Expectation
  1.3. Stochatic Processes
    Problems
2. THE POISSON RPCESS
  2.1 Introduction and Definitions
  2.2 Interarrival and Waiting Time Distributions
  2.3 Conditional Distribution of the Arrival Times
  2.4 Compound and Nonhomogenous Poisson Processes
  2.5 Stationary Point Processes
    Problems
    References
3. RENEWAL THEORY
  3.1 Introduction and Preliminaries
  3.2 Renewal Equation and Generalizations
  3.3 Limit Theorems
  3.4 Wald's Equation
  3.5 Back to Renewal Theory
  3.6 Excess Life and Age Distribution
  3.7 Delayed Renewal Processes
  3.8 Counter Models
  3.9 Renewal Reward Process
  3.10 Nonterminating versus Terminating Renewal Processes
  3.11 Age Dependent Branching Processes
    Problems
    References
4. MARKOV CHAINS
  4.1 Preliminaries and Examples
  4.2 Classification of States
  4.3 Limit Theorems
  4.4 Transitions Among Classes
  4.5 Branching Processes
  4.6 Transient States
    Problems
    References
5. "SEMI-MARKOV, MARKOV RENEWAL AND REGERNERATIVE PROCESSES"
  5.1 Introduction and Preliminaries
  5.2 Classification of States
  5.3 Some Simple Relationships
  5.4 Regenerative Processes
  5.5 A Queueing Application
  5.6 Back to Markov Renewal Processes-Limiting Probabilities
  5.7 Limiting Distributions of the Markov Renewal Process
  5.8 Continuous Time Markov Chains
  5.9 Birth and Death Processes
    Problems
    References
6. MARKOV DECISION PROCESSES
  6.1 Introduction
  6.2 Expected Discounted Cost
  6.3 Some Examples
  6.4 "Positive Costs, No Discounting"
  6.5 Applications: Optimal Stopping and Sequential Analysis
  6.6 Expected Average Cost Criterion-Introduction and Counter examples
  6.7 Expected Average Cost Criterion
  6.8 Finite State Space-Computational Approaches
    Problems
    References
7. SEMI-MARKOV DECISION PROCESSES
  7.1 Introduction
  7.2 Discounted Cost Criterion
  7.3 Average Cost-Preliminaries and Equality of Criteria
  7.4 Average Cost-Results
  7.5 Some Examples
    Problems
    References
8. INVENTORY THEORY
  8.1 Introduction
  8.2 A Single Period Model
  8.3 Multi-Period Models
  8.4 A Multi-Period Stationary Optimal Policy
  8.5 Inventory Issuing Policies
    Problems
    References
9. BROWNIAN MOTION AND CONTINUOUS TIME OPTIMIZATION MODELS
  9.1 Introduction and Preliminaries
  9.2 Maximum of the Wiener Process
  9.3 The Wiener Process and Optimization
  9.4 The Maximum Variable-A Renewal Application
  9.5 Optimal Dispatching of a Poisson Process
  9.6 Infinitesimal Look-Ahead Stopping Rules
    Problems
    Reference
  APPENDICES
  INDEX