Discrete-Time Markov Chains

Two-Time-Scale Methods and Applications

Authors:

ISBN: 978-0-387-21948-6 (Print) 978-0-387-26871-2 (Online)

Table of contents (13 chapters)

  1. Front Matter

    Pages i-xix

  2. Prologue and Preliminaries

    1. No Access

      Chapter

      Pages 3-23

      Introduction, Overview, and Examples

    2. No Access

      Chapter

      Pages 25-39

      Mathematical Preliminaries

  3. Asymptotic Properties

    1. No Access

      Chapter

      Pages 43-68

      Asymptotic Expansions

    2. No Access

      Chapter

      Pages 69-102

      Occupation Measures

    3. No Access

      Chapter

      Pages 103-118

      Exponential Bounds

    4. No Access

      Chapter

      Pages 119-129

      Interim Summary and Extensions

  4. Applications

    1. No Access

      Chapter

      Pages 133-158

      Stability of Dynamic Systems

    2. No Access

      Chapter

      Pages 159-193

      Filtering

    3. No Access

      Chapter

      Pages 195-223

      Markov Decision Processes

    4. No Access

      Chapter

      Pages 225-250

      LQ Controls

    5. No Access

      Chapter

      Pages 251-270

      Mean-Variance Controls

    6. No Access

      Chapter

      Pages 271-284

      Production Planning

    7. No Access

      Chapter

      Pages 285-313

      Stochastic Approximation

  5. Back Matter

    Pages 315-347