Numerical Methods for Stochastic Control Problems in Continuous Time

1. Front Matter

   Pages i-xii

   PDF

2. Introduction

   • Harold J. Kushner, Paul Dupuis

   Pages 1-6

3. Review of Continuous Time Models

   • Harold J. Kushner, Paul Dupuis

   Pages 7-34

4. Controlled Markov Chains

   • Harold J. Kushner, Paul Dupuis

   Pages 35-52

5. Dynamic Programming Equations

   • Harold J. Kushner, Paul Dupuis

   Pages 53-66

6. The Markov Chain Approximation Method: Introduction

   • Harold J. Kushner, Paul Dupuis

   Pages 67-88

7. Construction of the Approximating Markov Chain

   • Harold J. Kushner, Paul Dupuis

   Pages 89-151

8. Computational Methods for Controlled Markov Chains

   • Harold J. Kushner, Paul Dupuis

   Pages 153-189

9. The Ergodic Cost Problem: Formulation and Algorithms

   • Harold J. Kushner, Paul Dupuis

   Pages 191-214

10. Heavy Traffic and Singular Control Problems: Examples and Markov Chain Approximations

    • Harold J. Kushner, Paul Dupuis

    Pages 215-244

11. Weak Convergence and the Characterization of Processes

    • Harold J. Kushner, Paul Dupuis

    Pages 245-265

12. Convergence Proofs

    • Harold J. Kushner, Paul Dupuis

    Pages 267-299

13. Convergence for Reflecting Boundaries, Singular Control and Ergodic Cost Problems

    • Harold J. Kushner, Paul Dupuis

    Pages 301-323

14. Finite Time Problems and Nonlinear Filtering

    • Harold J. Kushner, Paul Dupuis

    Pages 325-345

15. Controlled Variance and Jumps

    • Harold J. Kushner, Paul Dupuis

    Pages 347-366

16. Problems from the Calculus of Variations: Finite Time Horizon

    • Harold J. Kushner, Paul Dupuis

    Pages 367-400

17. Problems from the Calculus of Variations: Infinite Time Horizon

    • Harold J. Kushner, Paul Dupuis

    Pages 401-442

18. The Viscosity Solution Approach to Proving Convergence of Numerical Schemes

    • Harold J. Kushner, Paul Dupuis

    Pages 443-453

19. Back Matter

    Pages 455-476

    PDF

Changes in the second edition. The second edition differs from the first in that there is a full development of problems where the variance of the diffusion term and the jump distribution can be controlled. Also, a great deal of new material concerning deterministic problems has been added, including very efficient algorithms for a class of problems of wide current interest. This book is concerned with numerical methods for stochastic control and optimal stochastic control problems. The random process models of the controlled or uncontrolled stochastic systems are either diffusions or jump diffusions. Stochastic control is a very active area of research and new problem formulations and sometimes surprising applications appear regu­ larly. We have chosen forms of the models which cover the great bulk of the formulations of the continuous time stochastic control problems which have appeared to date. The standard formats are covered, but much emphasis is given to the newer and less well known formulations. The controlled process might be either stopped or absorbed on leaving a constraint set or upon first hitting a target set, or it might be reflected or "projected" from the boundary of a constraining set. In some of the more recent applications of the reflecting boundary problem, for example the so-called heavy traffic approximation problems, the directions of reflection are actually discontin­ uous. In general, the control might be representable as a bounded function or it might be of the so-called impulsive or singular control types.

• Calculus of Variations
• Markov chain
• Stochastic processes
• Variance
• algorithms
• filtering problem
• stochastic process

"The second edition of this acclaimed book from Springer-Verlag has the latest theoretical and practical information on solving stochastic control problems. Including proofs and algorithms using diffusion, jump-diffusion, and other process models, the authors help make randomness a little less scary."
Amazon.com Delivers Mathematics and Statistics e-bulletin, July 2001

• Division of Applied Mathematics, Brown University, Providence, USA

  Harold J. Kushner, Paul Dupuis