Skip to main content
SpringerLink
Log in

Performance Modelling and Markov Chains

  • Chapter
Book cover Formal Methods for Performance Evaluation (SFM 2007)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 4486))

Abstract

Markov chains have become an accepted technique for modeling a great variety of situations. They have been in use since the early 1900’s, but it is only in recent years with the advent of high speed computers and cheap memory that they have begun to be applied to large-scale modeling projects. This paper outlines the events that have lead to the present state-of-the-art in the numerical approach to Markov chain performance modeling and describes current solution methods and ongoing research efforts.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ajmone Marsan, M., Balba, G., Conte, G.: A class of generalized stochastic Petri nets for the performance evaluation of multiprocessor systems. ACM Trans. Comput. Systems 2(2), 93–122 (1994)

    Article  Google Scholar 

  2. Basharin, G.P., Naoumov, V.A.: The life and work of A.A. Markov. In: Langville, A.N., Stewart, W.J. (eds.) Proceedings for the Fourth International Conference on the Numerical Solution of Markov Chains, Urbana, Champaign, pp. 1–10 (2003)

    Google Scholar 

  3. Allmaier, S.C., Horton, G.: Parallel shared-memory state-space exploration in stochastic modelling. In: Lüling, R., Bilardi, G., Ferreira, A., Rolim, J.D.P. (eds.) IRREGULAR 1997. LNCS, vol. 1253, Springer, Heidelberg (1997)

    Google Scholar 

  4. Benoit, A., Plateau, B., Stewart, W.J.: Memory-efficient Kronecker algorithms with applications to modelling parallel systems. Future Generation Computer Systems (FGCS) (Special Issue on System Performance Analysis and Evaluation) 22(7), 838–847 (2006)

    Article  Google Scholar 

  5. Benzi, M.: A direct projection method for Markov chains. In: Langville, A.N., Stewart, W.J. (eds.) Proceedings for the Fourth International Conference on the Numerical Solution of Markov Chains, Urbana, Champaign, pp. 11–30 (2003)

    Google Scholar 

  6. Berson, S., de Souza e Silva, E., Muntz, R.R.: A methodology for the specification and generation of Markov models. In: Stewart, W.J. (ed.) Numerical Solution of Markov Chains, pp. 11–36. Marcel Dekker Inc., New York (1991)

    Google Scholar 

  7. Berson, S., Muntz, R.R.: Detecting block GI/M/1 and block M/G/1 matrices from model specifications. In: Stewart, W.J. (ed.) Computations with Markov Chains, pp. 1–19. Kluwer International Publishers, Boston (1995)

    Google Scholar 

  8. Bini, D., Meini, B.: On cyclic reduction applied to a class of Toeplitz matrices arising in queueing problems. In: Stewart, W.J. (ed.) Computations with Markov Chains, pp. 21–38. Kluwer Academic Publishers, Boston (1995)

    Google Scholar 

  9. Buchholz, P., Dayar, T.: Block SOR for Kronecker structured representations. In: Langville, A.N., Stewart, W.J. (eds.) Proceedings for the Fourth International Conference on the Numerical Solution of Markov Chains, Urbana, Champaign, pp. 121–144 (2003)

    Google Scholar 

  10. Buchholz, P.: Equivalence relations for stochastic automata networks. In: Stewart, W.J. (ed.) Computations with Markov Chains, pp. 197–215. Kluwer International Publishers, Boston (1995)

    Google Scholar 

  11. Buchholz, P.: An aggregation-disaggregation algorithm for stochastic automata networks. Prob. in the Eng and Inf. Sci. 11, 229–254 (1997)

    MATH  MathSciNet  Google Scholar 

  12. Caselli, S., Conte, G., Marenzoni, P.: Parallel state space exploration for GSPN models. In: DeMichelis, G., Díaz, M. (eds.) ICATPN 1995. LNCS, vol. 935, Springer, Heidelberg (1995)

    Google Scholar 

  13. Cao, W.-L., Stewart, W.J.: Iterative aggregation/disaggregation techniques for nearly uncoupled Markov chains. Journal of the ACM 32, 702–719 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  14. Chakravarthy, S.R., Alfa, A.S. (eds.): Advances in matrix-analytic methods for stochastic models. Lecture Notes in Pure and Applied Mathematics (1998)

    Google Scholar 

  15. Ciardo, G., Muppala, J., Trivedi, K.S.: SPNP: stochastic Petri net package. In: Proceedings of the Third International Workshop on Petri Nets and Performance Models (PNPM89), Kyoto, Japan (1989)

    Google Scholar 

  16. Ciardo, G., Tilgner, M.: On the use of Kronecker operators for the solution of generalized stochastic Petri nets. ICASE Report 96–35, Hampton, VA (May 1996)

    Google Scholar 

  17. Ciardo, G., Miner, A.S.: Storage alternatives for large structured state spaces. In: Marie, R., Plateau, B., Calzarossa, M.C., Rubino, G.J. (eds.) TOOLS 1997. LNCS, vol. 1245, pp. 44–57. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  18. Courtois, P.J.: Decomposability. Academic Press, New York (1977)

    MATH  Google Scholar 

  19. Clarotti, C.A.: The Markov approach to calculating system reliability: Computational Problems. In: Serra, A., Barlow, R.E. (eds.) International School of Physics “Enrico Fermi”, Varenna, Italy, pp. 54–66. North-Holland Physics Publishing, Amsterdam (1984)

    Google Scholar 

  20. Daigle, J.N., Lucantoni, D.M.: Queueing systems having phase-dependent arrival and service rates. In: Stewart, W.J. (ed.) Numerical Solution of Markov Chains, pp. 161–202. Marcel Dekker Inc., New York (1991)

    Google Scholar 

  21. Donatelli, S.: Superposed stochastic automata: A class of stochastic Petri nets with parallel solution and distributed state space. Performance Evaluation 18, 21–36 (1993)

    Article  MATH  MathSciNet  Google Scholar 

  22. Elwalid, A.I., Mitra, D., Stern, T.E.: Theory of statistical multiplexing of Markovian sources: Spectral expansions and algorithms. In: Stewart, W.J. (ed.) Numerical Solution of Markov Chains, pp. 223–238. Marcel Dekker Inc., New York (1991)

    Google Scholar 

  23. Fernandes, P., Plateau, B., Stewart, W.J.: Efficient descriptor-vector multiplication in stochastic automata networks. Journal of the ACM 45(3) (1998)

    Google Scholar 

  24. Fourneau, J.-M., Lecoz, M., Quessette, F.: Algorithms for an irreducible and lumpable strong stochastic bound. In: Langville, A.N., Stewart, W.J. (eds.) Proceedings for the Fourth International Conference on the Numerical Solution of Markov Chains, Urbana, Champaign, pp. 191–206 (2003)

    Google Scholar 

  25. Fourneau, J.-M., Quessette, F.: Graphs and stochastic automata networks. In: Stewart, W.J. (ed.) Computations with Markov Chains, Kluwer International Publishers, Boston (1995)

    Google Scholar 

  26. Franceschinis, G., Muntz, R.: Computing bounds for the performance indices of quasi-lumpable stochastic well-formed nets. In: Proceedings of the 5th International Workshop on Petri Nets and Performance Models, Toulouse, France, October 1993, pp. 148–157. IEEE Press, Los Alamitos (1993)

    Chapter  Google Scholar 

  27. Gershwin, S.B., Schick, I.C.: Modelling and analysis of three-stage transfer lines with unreliable machines and finite buffers. Operations Research 31(2), 354–380 (1983)

    Article  MATH  Google Scholar 

  28. Golub, G.H., Van Loan, C.F.: Matrix Computations, 2nd edn. The Johns Hopkins University Press, Baltimore (1989)

    MATH  Google Scholar 

  29. Grassmann, W.: Transient solutions in Markovian queueing systems. Comput. Opns. Res. 4, 47–56 (1977)

    Article  Google Scholar 

  30. Grassmann, W., Taksar, M.I., Heyman, D.P.: Regenerative analysis and steady state distributions. Operations Research 33, 1107–1116 (1985)

    MATH  MathSciNet  Google Scholar 

  31. Gross, D., Miller, D.R.: The randomization technique as modelling tool and solution procedure for transient Markov processes. Operations Research 32(2), 343–361 (1984)

    MATH  MathSciNet  Google Scholar 

  32. Hillston, J.: Computational Markovian modelling using a process algebra. In: Stewart, W.J. (ed.) Computations with Markov Chains, Kluwer International Publishers, Boston (1995)

    Google Scholar 

  33. Jafari, M.A., Shanthikumar, J.G.: Finite state spacially non-homogeneous quasi birth-death processes. Working Paper #85-009, Dept. of Industrial Engineering and Operations Research, Syracuse University, Syracuse, New York.

    Google Scholar 

  34. Kemper, P.: Closing the gap between classical and tensor based iteration techniques. In: Stewart, W.J. (ed.) Computations with Markov Chains, Kluwer International Publishers, Boston (1995)

    Google Scholar 

  35. Klevans, R., Stewart, W.J.: From queueing networks to Markov chains: The XMarca interface. Performance Evaluation 24, 23–45 (1995)

    Article  MATH  Google Scholar 

  36. Krieger, U., Muller-Clostermann, B., Sczittnick, M.: Modeling and analysis of communication systems based on computational methods for Markov chains. IEEE Journ. on Selec. Ar. in Comm. 8, 1630–1648 (1990)

    Article  Google Scholar 

  37. Knottenbelt, W.J.: Generalized Markovian analysis of timed transition systems. Master’s thesis, University of Capetown (1995)

    Google Scholar 

  38. Knottenbelt, W.J., Mestern, M., Harrison, P.G., Kritzinger, P.S.: Probability, Parallelism and the State Space Exploration Problem. In: Puigjaner, R., Savino, N.N., Serra, B. (eds.) TOOLS 1998. LNCS, vol. 1469, p. 165. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  39. Latouche, G., Ramaswami, Y.: A logarithmic reduction algorithm for quasi birth and death processes. Technical Report, Bellcore TM-TSV-021374 and Université Libre de Bruxelles, Sémin. Théor. Prob., Rapp. Tech. 92/3 (1992)

    Google Scholar 

  40. Li, S.Q., Mark, J.W.: Performance of voice/data integration on a TDM system. IEEE Trans. Communications 33(12), 1265–1273 (1985)

    Google Scholar 

  41. Lindemann, C.: Exploiting isomorphisms and special structures in the analysis of Markov regenerative stochastic Petri nets. In: Stewart, W.J. (ed.) Computations with Markov Chains, pp. 383–402. Kluwer International Publishers, Boston (1995)

    Google Scholar 

  42. Malhotra, M., Trivedi, K.S.: Higher-order methods for transient analysis of stiff Markov chains. Duke University Technical Report DUKE-CCSR-91, Center for Computer Systems Research, Durham, USA (1991)

    Google Scholar 

  43. Marek, I., Szyld, D.B.: Algebraic Schwartz methods for the numerical solution of Markov chains. In: Langville, A.N., Stewart, W.J. (eds.) Proceedings for the Fourth International Conference on the Numerical Solution of Markov Chains, Urbana, Champaign, pp. 45–56 (2003)

    Google Scholar 

  44. McAllister, D.F., Stewart, G.W., Stewart, W.J.: On a Rayleigh-Ritz refinement technique for nearly uncoupled stochastic matrices. Journal of Linear Algebra and Its Applications 60, 1–25 (1984)

    Article  MATH  MathSciNet  Google Scholar 

  45. Meyer, C.D.: Stochastic complementation, uncoupling Markov chains and the theory of nearly reducible systems. SIAM Rev. 31, 240–272 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  46. Meini, B.: New convergence results on functional techniques for the numerical solution of M/G/1 type Markov chains. Numer. Math 78, 39–58 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  47. Moler, C.B., Van Loan, C.F.: Nineteen dubious ways to compute the exponential of a matrix. SIAM Review 20(4), 801–836 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  48. Muth, E.J., Yeralan, S.: Effect of buffer size on productivity of work stations that are subject to breakdowns. In: The 20th IEEE Conference on Decision and Control, pp. 643–648 (1981)

    Google Scholar 

  49. Neuts, M.F.: Matrix Geometric Solutions in Stochastic Models – An Algorithmic Approach. The Johns Hopkins University Press, Baltimore (1981)

    MATH  Google Scholar 

  50. Neuts, M.F.: Structured Stochastic Matrices of M/G/1 Type and Their Applications. Marcel Dekker, New York (1989)

    MATH  Google Scholar 

  51. Philippe, B., Saad, Y., Stewart, W.J.: Numerical methods in Markov chains modeling. Operations Research 40(6) (1992)

    Google Scholar 

  52. Philippe, B., Sidje, R.B.: Transient solutions of Markov processes by Krylov subspaces. In: Stewart, W.J. (ed.) Computations with Markov Chains, pp. 95–119. Kluwer International Publishers, Boston (1995)

    Google Scholar 

  53. Plateau, B.: On the stochastic structure of parallelism and synchronization models for distributed algorithms. In: Proc. ACM Sigmetrics Conference on Measurement and Modelling of Computer Systems, Austin, Texas (August 1985)

    Google Scholar 

  54. Plateau, B., Atif, K.: Stochastic automata network for modelling parallel systems. IEEE Trans. on Software Engineering 17(10), 1093–1108 (1991)

    Article  MathSciNet  Google Scholar 

  55. Plateau, B., Fourneau, J.M., Lee, K.H.: PEPS: A package for solving complex Markov models of parallel systems. In: Puigjaner, R., Potier, D. (eds.) Modelling Techniques and Tools for Computer Performance Evaluation, Spain (September 1988)

    Google Scholar 

  56. Veran, M., Potier, D.: QNAP2: A portable environment for queueing systems modelling. In: Potier, D. (ed.) Modelling and Tools for Performance Analysis, pp. 25–63. Elsevier Science Publishers, Amsterdam (1985)

    Google Scholar 

  57. Ramaswami, V.: A stable recursion for the steady state vector in Markov chains of M/G/1 type. Commun. Statist. Stochastic Models 4, 183–188 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  58. Ramaswami, V.: Nonlinear matrix equations in applied probability — Solution techniques and open problems. SIAM Review 30(2), 256–263 (1988)

    Article  MATH  MathSciNet  Google Scholar 

  59. Ramaswami, V., Neuts, M.F.: Some explicit formulas and computational methods for infinite server queues with phase type arrivals. J. Appl. Probab. 17, 498–514 (1980)

    Article  MATH  MathSciNet  Google Scholar 

  60. Ramaswami, V., Lucantoni, D.M.: Stationary waiting time distributions in queues with phase-type service and in quasi-birth-and-death processes. Comm. Statist. Stochastic Models 1, 125–136 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  61. Reibman, A., Trivedi, K.: Transient analysis of cumulative measures of Markov model behavior. Commu. Statist.-Stochastic Models 5(4), 683–710 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  62. Saad, Y.: Iterative solution of sparse linear systems. PWS Publishing, New York (1996)

    Google Scholar 

  63. Saad, Y.: Numerical methods for large eigenvalue problems. John Wiley & Sons, Chichester (1992)

    MATH  Google Scholar 

  64. Sbeity, I., Plateau, B., Brenner, L., Stewart, W.J.: Phase-Type Distributions in Stochastic Automata Networks. To appear in European Journal of Operations Research.

    Google Scholar 

  65. Schweitzer, P.J.: Aggregation methods for large Markov chains. In: Iazeolla, G., Courtois, P.J., Hordijk, A. (eds.) Mathematical Computer Performance and Reliability, pp. 275–286. North-Holland, Amsterdam (1984)

    Google Scholar 

  66. Semal, P.: Two bounding schemes for the steady state solution of Markov chains. In: Stewart, W.J. (ed.) Computations with Markov Chains, pp. 307–320. Kluwer International Publishers, Boston (1995)

    Google Scholar 

  67. Seneta, E.: MArkov and the Creation of Markov Chains. In: MAM-2006, MArkov Anniversary Meeting: An International Conference to Celebrate the 150th Anniversary of the Birth of A.A. Markov, pp. 1–20. Boson Books, Raleigh (2006)

    Google Scholar 

  68. Sidje, R.B.: Parallel algorithms for large sparse matrix exponentials: Application to numerical transient analysis of Markov processes. PhD thesis, University of Rennes (1 July 1994)

    Google Scholar 

  69. Sidje, R.B.: Expokit. A software package for computing matrix exponentials. Accepted for publication in ACM-Transactions of Mathematical Software (1997)

    Google Scholar 

  70. Squillante, M.: MAGIC: A computer performance modeling tool based on matrix geometric techniques. In: Balbo, G., Serazzi, G. (eds.) Computer Performance Evaluation: Modeling Techniques and Tools, pp. 411–425. North-Holland Publishers, Amsterdam (1992)

    Google Scholar 

  71. Stewart, W.J.: An Introduction to the Numerical Solution of Markov Chains. Princeton University Press, Princeton (1994)

    Google Scholar 

  72. Stewart, W.J., Atif, K., Plateau, B.: The numerical solution of stochastic automata networks. European Journal of Operations Research 86(3), 503–525 (1995)

    Article  MATH  Google Scholar 

  73. Stewart, W.J.: Marca: Markov chain analyzer. IEEE Computer Repository No. R76 232 (1976), See the URL: http://www.csc.ncsu.edu/faculty/WStewart/

  74. School of Mathematics and Statistics University of St. Andrews, Scotland (December 1996), http://www-history.mcs.st-andrews.ac.uk/history/Mathematicians/Markov.html

  75. Vincent, J.-M., Marchand, C.: On the exact simulation of functional of stationary Markov chains. In: Langville, A.N., Stewart, W.J. (eds.) Proceedings for the Fourth International Conference on the Numerical Solution of Markov Chains, Urbana, Champaign, pp. 77–98 (2003)

    Google Scholar 

  76. Von Hilgers, P., Langville, A.N.: The five greatest applications of Markov Chains. In: MAM-2006, MArkov Anniversary Meeting: An International Conference to Celebrate the 150th Anniversary of the Birth of A.A. Markov, pp. 155–168. Boson Books, Raleigh (2006)

    Google Scholar 

  77. Wallace, V.I., Rosenberg, R.S.: RQA-1, The recursive queue analyzer. Technical report No.2, Systems Engineering Laboratory, University of Michigan, Ann Arbor (Feb. 1966)

    Google Scholar 

  78. Wallace, V.: The solution of quasi birth and death processes arising from multiple access computer systems. Ph.D. Dissertation, Systems Engineering Laboratory, University of Michigan, Tech. Report No. 07742-6-T (1969)

    Google Scholar 

  79. Wong, B., Giffin, W., Disney, R.L.: Two finite M/M/1 queues in tandem: A matrix solution for the steady state. OPSEARCH 14(1), 1–18 (1977)

    MathSciNet  Google Scholar 

  80. Yang, T., Posner, M.J.M., Templeton, J.G.C.: A generalized recursive technique for finite Markov processes. In: Stewart, W.J. (ed.) Numerical Solution of Markov Chains, pp. 203–221. Marcel Dekker Inc., New York (1991)

    Google Scholar 

  81. Ye, J., Li, S.Q.: Analysis of multi-media traffic queues with finite buffer and overload control — Part I: Algorithms. In: Proc. of INFOCOM ’91, pp. 1464–1474 (1991)

    Google Scholar 

  82. Ye, J., Li, S.Q.: Analysis of multi-media traffic queues with finite buffer and overload control — Part II: Applications. In: Proc. of INFOCOM ’92, pp. 848–859 (1992)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, North Carolina State University, Raleigh, NC 27695, USA

    William J. Stewart

Authors
  1. William J. Stewart
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Marco Bernardo Jane Hillston

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer Berlin Heidelberg

About this chapter

Cite this chapter

Stewart, W.J. (2007). Performance Modelling and Markov Chains. In: Bernardo, M., Hillston, J. (eds) Formal Methods for Performance Evaluation. SFM 2007. Lecture Notes in Computer Science, vol 4486. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72522-0_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-72522-0_1

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-72482-7

  • Online ISBN: 978-3-540-72522-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation