Continuous-Time Markov Chains and Applications

A Two-Time-Scale Approach

Authors:

ISBN: 978-1-4614-4345-2 (Print) 978-1-4614-4346-9 (Online)

Table of contents (10 chapters)

  1. Front Matter

    Pages i-xxi

  2. Prologue and Preliminaries

    1. Front Matter

      Pages 1-1

    2. No Access

      Book Chapter

      Pages 3-16

      Introduction and Overview

    3. No Access

      Book Chapter

      Pages 17-29

      Mathematical Preliminaries

    4. No Access

      Book Chapter

      Pages 31-56

      Markovian Models

  3. Two-Time-Scale Markov Chains

    1. Front Matter

      Pages 57-57

    2. No Access

      Book Chapter

      Pages 59-140

      Asymptotic Expansions of Solutions for Forward Equations

    3. No Access

      Book Chapter

      Pages 141-234

      Occupation Measures: Asymptotic Properties and Ramification

    4. No Access

      Book Chapter

      Pages 235-257

      Asymptotic Expansions of Solutions for Backward Equations

  4. Applications: MDPs, Near-optimal Controls, Numerical Methods, and LQG with Switching

    1. Front Matter

      Pages 259-259

    2. No Access

      Book Chapter

      Pages 261-283

      Markov Decision Problems

    3. No Access

      Book Chapter

      Pages 285-318

      Stochastic Control of Dynamical Systems

    4. No Access

      Book Chapter

      Pages 319-339

      Numerical Methods for Control and Optimization

    5. No Access

      Book Chapter

      Pages 341-372

      Hybrid LQG Problems

  5. Back Matter

    Pages 373-427