A Critical Evaluation Study of Model-Log Metrics in Process Discovery

  • Jochen De Weerdt
  • Manu De Backer
  • Jan Vanthienen
  • Bart Baesens
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 66)


The development of a well-defined evaluation framework for process discovery techniques is definitely one of the most important challenges within this subdomain of process mining. Any researcher in the field will acknowledge that such a framework is vital. With this paper, we aim to provide a tangible analysis of the currently available model-log evaluation metrics for mined control-flow models. Also, we will indicate strengths and weaknesses of the existing metrics and propose a number of opportunities for future research.


process discovery evaluation metrics machine learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tan, P.N., Steinbach, M., Kumar, V.: Introduction to Data Mining. Addison-Wesley, Reading (2005)Google Scholar
  2. 2.
    Weske, M.: Business Process Management: Concepts, Languages, Architectures. Springer, Heidelberg (2007)Google Scholar
  3. 3.
    Rozinat, A., Veloso, M., van der Aalst, W.M.P.: Using hidden markov models to evaluate the quality of discovered process models. BPM Center Report BPM-08-10 (2008)Google Scholar
  4. 4.
    Weijters, A.J.M.M., van der Aalst, W.M.P., Alves de Medeiros, A.K.: Process mining with the heuristicsminer algorithm. BETA Working Paper Series 166. Eindhoven University of Technology (2006)Google Scholar
  5. 5.
    Greco, G., Guzzo, A., Pontieri, L., Saccà, D.: Discovering expressive process models by clustering log traces. IEEE Trans. Knowl. Data Eng. 18(8), 1010–1027 (2006)CrossRefGoogle Scholar
  6. 6.
    Rozinat, A., van der Aalst, W.M.P.: Conformance checking of processes based on monitoring real behavior. Information Systems 33(1), 64–95 (2008)CrossRefGoogle Scholar
  7. 7.
    Alves de Medeiros, A.K.: Genetic Process Mining. PhD thesis, TU Eindhoven (2006)Google Scholar
  8. 8.
    Goedertier, S., Martens, D., Vanthienen, J., Baesens, B.: Robust process discovery with artificial negative events. Journal of Machine Learning Research 10, 1305–1340 (2009)MathSciNetMATHGoogle Scholar
  9. 9.
    Alves de Medeiros, A.K., Weijters, A.J.M.M., van der Aalst, W.M.P.: Genetic process mining: an experimental evaluation. Data Mining and Knowledge Discovery 14(2), 245–304 (2007)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Ferreira, H., Ferreira, D.: An integrated life cycle for workflow management based on learning and planning. International Journal of Cooperative Information Systems 15(4), 485–505 (2006)CrossRefGoogle Scholar
  11. 11.
    van der Aalst, W.M.P., Weijters, A.J.M.M., Maruster, L.: Workflow mining: Discovering process models from event logs. IEEE Trans. Knowl. Data Eng. 16(9), 1128–1142 (2004)CrossRefGoogle Scholar
  12. 12.
    Rozinat, A., de Medeiros, A.K.A., Günther, C.W., Weijters, A.J.M.M., van der Aalst, W.M.P.: The need for a process mining evaluation framework in research and practice. In: Business Process Management Workshops, pp. 84–89 (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Jochen De Weerdt
    • 1
  • Manu De Backer
    • 1
    • 2
    • 3
  • Jan Vanthienen
    • 1
  • Bart Baesens
    • 1
  1. 1.Department of Decision Sciences and Information ManagementKatholieke Universiteit LeuvenLeuvenBelgium
  2. 2.Department of HABE, Hogeschool GentUniversiteit GentGhentBelgium
  3. 3.Department of Management Information SystemsUniversity of AntwerpAntwerpBelgium

Personalised recommendations