Discrete-Time Markov Chains

Two-Time-Scale Methods and Applications

  • G. George Yin
  • Qing Zhang

Part of the Stochastic Modelling and Applied Probability book series (SMAP, volume 55)

Table of contents

  1. Front Matter
    Pages i-xix
  2. Prologue and Preliminaries

  3. Asymptotic Properties

  4. Applications

    1. Pages 159-193
    2. Pages 225-250
    3. Pages 251-270
    4. Pages 271-284
    5. Pages 285-313
  5. Back Matter
    Pages 315-347

About this book


Focusing on discrete-time-scale Markov chains, the contents of this book are an outgrowth of some of the authors' recent research.  The motivation stems from existing and emerging applications in optimization and control of complex hybrid Markovian systems in manufacturing, wireless communication, and financial engineering.  Much effort in this book is devoted to designing system models arising from these applications, analyzing them via analytic and probabilistic techniques, and developing feasible computational algorithms so as to reduce the inherent complexity.

This book presents results including asymptotic expansions of probability vectors, structural properties of occupation measures, exponential bounds, aggregation and decomposition and associated limit processes, and interface of discrete-time and continuous-time systems.  One of the salient features is that it contains a diverse range of applications on filtering, estimation, control, optimization, and Markov decision processes, and financial engineering.  This book will be an important reference for researchers in the areas of applied probability, control theory, operations research, as well as for practitioners who use optimization techniques.  Part of the book can also be used in a graduate course of applied probability, stochastic processes, and applications.


Markov Chain Markov Chains Measure Stochastic Approximation Stochastic Processes algorithms operations research optimization

Authors and affiliations

  • G. George Yin
    • 1
  • Qing Zhang
    • 2
  1. 1.Department of MathematicsWayne State UniversityDetroitUSA
  2. 2.Department of MathematicsUniversity of GeorgiaAthensUSA

Bibliographic information

  • DOI https://doi.org/10.1007/b138226
  • Copyright Information Springer Science+Business Media, Inc. 2005
  • Publisher Name Springer, New York, NY
  • eBook Packages Mathematics and Statistics
  • Print ISBN 978-0-387-21948-6
  • Online ISBN 978-0-387-26871-2
  • Series Print ISSN 0172-4568
  • About this book