Skip to main content

Markov Chains

  • 828 Accesses

Abstract

In the previous chapter, we presented the Generalized Semi-Markov Process (GSMP) framework as a means of modeling stochastic DES. By allowing event clocks to tick at varying speeds, we also provided an extension to the basic GSMP. In addition, we introduced the Poisson process as a basic building block for a class of stochastic DES which possess the Markov (memoryless) property. Thus, we obtained the class of stochastic processes known as Markov chains, which we will study in some detail in this chapter.

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-72274-6_7
  • Chapter length: 60 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   119.00
Price excludes VAT (USA)
  • ISBN: 978-3-030-72274-6
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Hardcover Book
USD   159.99
Price excludes VAT (USA)
Figure 7.1:
Figure 7.2:
Figure 7.3:
Figure 7.4:
Figure 7.5:
Figure 7.6:
Figure 7.7:
Figure 7.8:
Figure 7.9:
Figure 7.10:
Figure 7.11:
Figure 7.12:
Figure 7.13:
Figure 7.14:
Figure 7.15:
Figure 7.16:
Figure 7.17:
Figure 7.18:
Figure 7.19:
Figure 7.20:
Figure 7.21:
Figure 7.22:
Figure 7.23:
Figure 7.24:

Notes

  1. 1.

    A supervisory controller S of the type considered in Chap. 3 could be synthesized, if necessary, to ensure that the controlled DES S/G (now modeled as a Markov chain) satisfies these requirements. One would rely upon the notions of marked states and nonblocking supervisor for this purpose.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Christos G. Cassandras .

Rights and permissions

Reprints and Permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this chapter

Verify currency and authenticity via CrossMark

Cite this chapter

Cassandras, C.G., Lafortune, S. (2021). Markov Chains. In: Introduction to Discrete Event Systems. Springer, Cham. https://doi.org/10.1007/978-3-030-72274-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-72274-6_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72272-2

  • Online ISBN: 978-3-030-72274-6

  • eBook Packages: Computer ScienceComputer Science (R0)