Continuous-Time Markov Chains and Applications

A Two-Time-Scale Approach

Authors:

ISBN: 978-1-4614-4345-2 (Print) 978-1-4614-4346-9 (Online)

Table of contents (10 chapters)

  1. Front Matter

    Pages i-xxi

    Download PDF (329KB)

  2. Prologue and Preliminaries

    1. Front Matter

      Pages 1-1

      Download PDF (14KB)

    2. No Access

      Book Chapter

      Pages 3-16

      Introduction and Overview

      Look Inside Get Access

    3. No Access

      Book Chapter

      Pages 17-29

      Mathematical Preliminaries

      Look Inside Get Access

    4. No Access

      Book Chapter

      Pages 31-56

      Markovian Models

      Look Inside Get Access

  3. Two-Time-Scale Markov Chains

    1. Front Matter

      Pages 57-57

      Download PDF (15KB)

    2. No Access

      Book Chapter

      Pages 59-140

      Asymptotic Expansions of Solutions for Forward Equations

      Look Inside Get Access

    3. No Access

      Book Chapter

      Pages 141-234

      Occupation Measures: Asymptotic Properties and Ramification

      Look Inside Get Access

    4. No Access

      Book Chapter

      Pages 235-257

      Asymptotic Expansions of Solutions for Backward Equations

      Look Inside Get Access

  4. Applications: MDPs, Near-optimal Controls, Numerical Methods, and LQG with Switching

    1. Front Matter

      Pages 259-259

      Download PDF (15KB)

    2. No Access

      Book Chapter

      Pages 261-283

      Markov Decision Problems

      Look Inside Get Access

    3. No Access

      Book Chapter

      Pages 285-318

      Stochastic Control of Dynamical Systems

      Look Inside Get Access

    4. No Access

      Book Chapter

      Pages 319-339

      Numerical Methods for Control and Optimization

      Look Inside Get Access

    5. No Access

      Book Chapter

      Pages 341-372

      Hybrid LQG Problems

      Look Inside Get Access

  5. Back Matter

    Pages 373-427

    Download PDF (627KB)