Numerical Methods for Controlled Stochastic

The Markov chain approximation methods are widely used for the numerical solution of nonlinear stochastic control problems in continuous time. This book extends the methods to stochastic systems with delays. Because such problems are infinite-dimensional, many new issues arise in getting good numerical approximations and in the convergence proofs. Useful forms of numerical algorithms and system approximations are developed in this work, and the convergence proofs are given. All of the usual cost functions are treated as well as singular and impulsive controls. A major concern is on representations and approximations that use minimal memory.Features and topics include:* Surveys properties of the most important stochastic dynamical models, including singular control, and those for diffusion and reflected diffusion models.* Gives approximations to the dynamical models that simplify the numerical problem, but have only small effects on the behavior.* Develops an ergodic theory for reflected diffusions with delays, as well as model simplifications useful for numerical approximations for average cost per unit time problems.* Provides numerical algorithms for models with delays in the path, or path and control, with reduced memory requirements.* Develops transformations of the problem that yield more efficient approximations when the control, driving Wiener process, and/or reflection processes might be delayed, as well as the path.* Presents examples with applications to control and modern communications systems.The book is the first on the subject and will be of interest to all those who work with stochastic delay equations and whose main interest is in either the use of the algorithms or the underlying mathematics. An excellent resource for graduate students, researchers, and practitioners, the work may be used as a graduate-level textbook for a special topics course or seminar on numerical methods in stochastic control.

Preface	p. xi
Examples and Introduction	p. 1
Outline of the Chapter	p. 1
An Introductory Example: Controlling the Temperature of a Fluid Flow	p. 1
An Example from Internet Regulation	p. 5
Models With Finite-Dimensional Equivalences	p. 10
Weak Convergence and Martingales	p. 13
Outline of the Chapter	p. 13
Weak Convergence	p. 14
Basic Theorems of Weak Convergence	p. 15
The Function Spaces D(S;I)	p. 17
Martingales and the Martingale Method	p. 18
Martingales	p. 18
Verifying That a Process Is a Martingale	p. 20
Stochastic Delay Equations: Models	p. 23
Outline of the Chapter	p. 23
The System Model: Boundary Absorption	p. 25
Reflecting Diffusions	p. 30
The Reflected Diffusion	p. 30
Delayed Control, Reflection Term, and/or Wiener Process	p. 35
Neutral Equations	p. 37
Controlled Variance and Jumps	p. 37
The Girsanov Transformation	p. 38
Cost Functions	p. 40
Existence of an Optimal Control	p. 43
Reflecting or Absorbing Boundary: Discounted Cost	p. 43
Singular and Impulsive Controls	p. 50
Singular Controls	p. 50
Definition of and Existence of Solutions	p. 51
Existence of an Optimal Control	p. 56
Impulsive Controls	p. 58
Approximations to the Dynamical Models	p. 61
Outline of the Chapter	p. 61
Approximations of the Dynamical Systems	p. 63
A Basic Approximation	p. 63
Approximations by Time-Varying Delays	p. 69
Discretized Delays	p. 69
Periodic Delays	p. 71
Randomly Varying Delays	p. 73
Periodic-Erlang Delays	p. 75
Convergence of Costs and Existence of Optimal Controls	p. 77
Differential Operator for the Periodic-Erlang Approximation	p. 78
Simulations Illustrating the Model Approximations	p. 78
Simulations Based on the Periodic Approximation	p. 78
Simulations Based on the Periodic-Erlang Approximation	p. 82
Approximations: Path and Control Delayed	p. 84
Singular Controls	p. 90
Rapidly Varying Delays	p. 92
The Ergodic Cost Problem	p. 97
Outline of the Chapter	p. 97
The Basic Model	p. 98
Relaxed Feedback Controls	p. 98
Density Properties and Preliminary Results	p. 101
The Doeblin Condition	p. 105
Approximations of the Models	p. 107
Approximations with Periodic Delays	p. 110
Limit and Approximation Results for Periodic Delays	p. 110
Smoothed Nearly Optimal Controls	p. 115
Delays in the Variance Term	p. 117
The Periodic-Erlang Approximation	p. 118
Markov Chain Approximations: Introduction	p. 125
Outline of the Chapter	p. 125
The System Model	p. 126
Approximating Chains and Local Consistency	p. 127
Continuous-Time Interpolations	p. 131
The Continuous-Time Interpolation	p. 131
A Markov Continuous-Time Interpolation	p. 133
The "Explicit" Approximation Procedure	p. 137
The "Implicit" Approximating Processes	p. 140
The General Implicit Approximation Method	p. 142
Continuous-Time Interpolations	p. 144
Representations of the Cost Function	p. 146
Asymptotic Equivalence of the Timescales	p. 147
Convergence	p. 149
Singular and Impulsive Controls	p. 149
Singular Controls	p. 149
Impulsive Control	p. 153
The Ergodic Cost Function	p. 153
Introduction	p. 153
The Markov Chain Approximation Method	p. 154
Markov Chain Approximations: Path and Control Delayed	p. 159
Outline of the Chapter	p. 159
The Model and Local Consistency	p. 161
The Models	p. 161
Delay in Path Only: Local Consistency and Interpolations	p. 162
Delay in the Path and Control	p. 167
Absorbing Boundaries and Other Cost Functions	p. 171
Approximations to the Memory Segments	p. 172
Computational Procedures	p. 175
Delay in Path Only: State Representations and the Bellman Equation	p. 175
Delay in the Both Path and Control	p. 178
A Comment on Higher-Dimensional Problems	p. 179
The Implicit Numerical Approximation: Path Delayed	p. 180
Local Consistency and the Memory Segment	p. 180
The Cost Function and Bellman Equation	p. 185
The Use of Averaging in Constructing the Path Memory Approximation	p. 186
Timescales	p. 187
Convergence Theorems	p. 188
The Implicit Approximation Procedure and the Random Delay Model	p. 189
Path and Control Delayed: Continued	p. 193
Outline of the Chapter	p. 193
Periodic Approximations to the Delay: Path Delayed	p. 194
A Periodic-Erlang Model	p. 196
The Number of Points in the State Space: Path Only Delayed	p. 200
The Implicit and Periodic-Erlang Approximation Methods: Reduced Memory	p. 200
Control and Path Delayed	p. 203
A Periodic Approximating Memory Segment	p. 204
A Periodic-Erlang Approximation	p. 207
Proofs of Convergence	p. 213
Proofs of Theorems from Chapter 7	p. 213
Proof of Theorem 4.1	p. 218
Singular Controls	p. 219
Neutral Equations	p. 220
The Ergodic Cost Problem	p. 222
A Wave Equation Approach	p. 227
Outline of the Chapter	p. 227
The Model and Assumptions	p. 228
A Key Representation of x (.)	p. 230
A Representation of the Solution	p. 230
Comments on the Dimension and the System State	p. 231
Proof of the Representation	p. 232
Extensions	p. 235
A Discrete-Time Approximation	p. 236
The Markov Chain Approximation	p. 240
Preliminaries and Boundaries	p. 242
Transition Probabilities and Local Consistency: An Implicit Approximation Procedure	p. 242
Dynamical Representations, the Cost Function and Bellman Equation	p. 248
Size of the State Space for the Approximating Chain	p. 250
Proof of Convergence: Preliminaries	p. 252
The Randomization Errors	p. 252
Continuous Time Interpolations	p. 255
Convergence of the Numerical Algorithm	p. 259
Alternatives: Periodic and Periodic-Erlang Approximations	p. 261
A Periodic Approximation	p. 261
The Effective Delay and Numerical Procedures	p. 265
Singular and Impulsive Controls	p. 265
References	p. 267
Index	p. 273
Symbol Index	p. 277
Table of Contents provided by Ingram. All Rights Reserved.

What is included with this book?

The New copy of this book will include any supplemental materials advertised. Please check the title of the book to determine if it should include any access cards, study guides, lab manuals, CDs, etc.

The Used, Rental and eBook copies of this book are not guaranteed to include any supplemental materials. Typically, only the book itself is included. This is true even if the title states it includes any access cards, study guides, lab manuals, CDs, etc.

FREE SHIPPING

Amazon no longer offers textbook rentals. We do!

Numerical Methods for Controlled Stochastic Delay Systems

Summary

Table of Contents

Supplemental Materials