A Fast EM Algorithm for Fitting Marked Markovian Arrival Processes with a New Special Structure

  • Gábor Horváth
  • Hiroyuki Okamura
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8168)


This paper presents an EM algorithm for fitting traces with Markovian arrival processes (MAPs). The proposed algorithm operates on a special subclass of MAPs. This special structure enables the efficient implementation of the EM algorithm; it is more orders of magnitudes faster than methods operating on the general MAP class while providing similar or better likelihood values. An other important feature of the algorithm is that it is able to fit multi-class traces with marked Markovian arrival processes as well. Several numerical examples demonstrate the efficiency of the procedure.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, T.W., Goodman, L.A.: Statistical inference about Markov chains. The Annals of Mathematical Statistics, 89–110 (1957)Google Scholar
  2. 2.
    Asmussen, S., Nerman, O., Olsson, M.: Fitting phase-type distributions via the EM algorithm. Scandinavian Journal of Statistics, 419–441 (1996)Google Scholar
  3. 3.
    Bause, F., Horváth, G.: Fitting Markovian Arrival Processes by Incorporating Correlation into Phase Type Renewal Processes. In: 2010 Seventh International Conference on the Quantitative Evaluation of Systems (QEST), pp. 97–106. IEEE (2010)Google Scholar
  4. 4.
    Breuer, L.: An EM algorithm for batch Markovian arrival processes and its comparison to a simpler estimation procedure. Annals of Operations Research 112(1-4), 123–138 (2002)MathSciNetMATHCrossRefGoogle Scholar
  5. 5.
    Buchholz, P.: An EM-algorithm for MAP fitting from real traffic data. In: Kemper, P., Sanders, W.H. (eds.) TOOLS 2003. LNCS, vol. 2794, pp. 218–236. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  6. 6.
    Buchholz, P., Kemper, P., Kriege, J.: Multi-class Markovian arrival processes and their parameter fitting. Performance Evaluation 67(11), 1092–1106 (2010)CrossRefGoogle Scholar
  7. 7.
    Buchholz, P., Kriege, J.: A heuristic approach for fitting MAPs to moments and joint moments. In: Sixth International Conference on the Quantitative Evaluation of Systems, QEST 2009, pp. 53–62. IEEE (2009)Google Scholar
  8. 8.
    Casale, G., Zhang, E.Z., Smirni, E.: KPC-toolbox: Simple yet effective trace fitting using Markovian arrival processes. In: Fifth International Conference on Quantitative Evaluation of Systems, QEST 2008, pp. 83–92. IEEE (2008)Google Scholar
  9. 9.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 1–38 (1977)Google Scholar
  10. 10.
    El Abdouni Khayari, R., Sadre, R., Haverkort, B.R.: Fitting world-wide web request traces with the EM-algorithm. Performance Evaluation 52(2), 175–191 (2003)CrossRefGoogle Scholar
  11. 11.
    He, Q., Neuts, M.F.: Markov chains with marked transitions. Stochastic Processes and their Applications 74, 37–52 (1998)MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Horváth, A., Horváth, G., Telek, M.: A traffic based decomposition of two-class queueing networks with priority service. Computer Networks 53(8), 1235–1248 (2009)MATHCrossRefGoogle Scholar
  13. 13.
    Horváth, G., Buchholz, P., Telek, M.: A MAP fitting approach with independent approximation of the inter-arrival time distribution and the lag correlation. In: Second International Conference on the Quantitative Evaluation of Systems, pp. 124–133. IEEE (2005)Google Scholar
  14. 14.
    Klemm, A., Lindemann, C., Lohmann, M.: Modeling IP traffic using the batch Markovian arrival process. Performance Evaluation 54(2), 149–173 (2003)CrossRefGoogle Scholar
  15. 15.
    Latouche, G., Ramaswami, V.: Introduction to matrix analytic methods in stochastic modeling. Society for Industrial and Applied Mathematics, vol. 5 (1987)Google Scholar
  16. 16.
    Okamura, H., Dohi, T.: Faster maximum likelihood estimation algorithms for Markovian arrival processes. In: Sixth International Conference on the Quantitative Evaluation of Systems, QEST 2009, pp. 73–82. IEEE (2009)Google Scholar
  17. 17.
    Okamura, H., Dohi, T., Trivedi, K.S.: A refined EM algorithm for PH distributions. Performance Evaluation 68(10), 938–954 (2011)CrossRefGoogle Scholar
  18. 18.
    Rydén, T.: An EM algorithm for estimation in Markov-modulated Poisson processes. Computational Statistics & Data Analysis 21(4), 431–447 (1996)MathSciNetMATHCrossRefGoogle Scholar
  19. 19.
    Telek, M., Horváth, G.: A minimal representation of Markov arrival processes and a moments matching method. Performance Evaluation 64(9), 1153–1168 (2007)CrossRefGoogle Scholar
  20. 20.
    Thummler, A., Buchholz, P., Telek, M.: A novel approach for phase-type fitting with the EM algorithm. IEEE Transactions on Dependable and Secure Computing 3(3), 245–258 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Gábor Horváth
    • 1
    • 2
    • 3
  • Hiroyuki Okamura
    • 4
  1. 1.Department of Networked Systems and ServicesBudapest University of Technology and EconomicsHungary
  2. 2.MTA-BME Information Systems Research GroupHungary
  3. 3.Inter-University Center of Telecommunications and InformaticsDebrecenHungary
  4. 4.Department of Information Engineering, Graduate School of EngineeringHiroshima UniversityHigashiJapan

Personalised recommendations