Information Systems and e-Business Management

, Volume 13, Issue 1, pp 37–67 | Cite as

Measuring precision of modeled behavior

  • A. Adriansyah
  • J. Munoz-Gama
  • J. Carmona
  • B. F. van Dongen
  • W. M. P. van der Aalst
Original Article

Abstract

Conformance checking techniques compare observed behavior (i.e., event logs) with modeled behavior for a variety of reasons. For example, discrepancies between a normative process model and recorded behavior may point to fraud or inefficiencies. The resulting diagnostics can be used for auditing and compliance management. Conformance checking can also be used to judge a process model automatically discovered from an event log. Models discovered using different process discovery techniques need to be compared objectively. These examples illustrate just a few of the many use cases for aligning observed and modeled behavior. Thus far, most conformance checking techniques focused on replay fitness, i.e., the ability to reproduce the event log. However, it is easy to construct models that allow for lots of behavior (including the observed behavior) without being precise. In this paper, we propose a method to measure precision of process models, given their event logs by first aligning the logs to the models. This way, the measurement is not sensitive to non-fitting executions and more accurate values can be obtained for non-fitting logs. Furthermore, we introduce several variants of the technique to deal better with incomplete logs and reduce possible bias due to behavioral property of process models. The approach has been implemented in the ProM 6 framework and tested against both artificial and real-life cases. Experiments show that the approach is robust to noise and applicable to handle logs and models of real-life complexity.

Keywords

Precision measurement Log-model alignment Conformance checking Process mining 

References

  1. Adriansyah A, van Dongen B, van der Aalst W (2011a) Conformance checking using cost-based fitness analysis. In: Proceedings of the 2011 IEEE 15th international enterprise distributed object computing conference (EDOC ’11), Helsinki, Finland. IEEE Computer Society, pp 55–64Google Scholar
  2. Adriansyah A, Sidorova N, van Dongen B (2011b) Cost-based fitness in conformance checking. In: 11th International conference on application of concurrency to system design (ACSD), pp 57–66Google Scholar
  3. Adriansyah A, van Dongen B, van der Aalst W (2013a) Memory-efficient alignment of observed and modeled behavior. Technical report BPM-03-03, BPMcenter.orgGoogle Scholar
  4. Adriansyah A, Munoz-Gama J, Carmona J, van Dongen B, van der Aalst W (2013b) Alignment based precision checking. In: Rosa ML, Soffer P (eds) Business process management workshops. Lecture notes in business information processing, vol 132. Springer, Berlin, pp 137–149Google Scholar
  5. Banescu S, Zannone N (2011) Measuring privacy compliance with process specifications. In: Proceedings of the 2011 third international workshop on security measurements and metrics (METRISEC ’11). IEEE Computer Society, pp 41–50Google Scholar
  6. Buijs J, van Dongen B, van der Aalst W (2012a) On the role of fitness, precision, generalization and simplicity in process discovery. In: Meersman R, Panetto H, Dillon T, Rinderle-Ma S, Dadam P, Zhou X, Pearson S, Ferscha A, Bergamaschi S, Cruz I (eds) On the move to meaningful internet systems (OTM 2012). Lecture notes in computer science, vol 7565. Springer, Berlin, pp 305–322Google Scholar
  7. Buijs J, van Dongen B, van der Aalst W (2012b) Towards cross-organizational process mining in collections of process models and their executions. In: Daniel F, Barkaoui K, Dustdar S (eds) Business process management workshops. Lecture notes in business information processing, vol 100. Springer, Berlin, pp 2–13Google Scholar
  8. Cook J, Wolf A (1999) Software process validation: quantitatively measuring the correspondence of a process to a model. ACM Trans Softw Eng Methodol (TOSEM) 8:147–176CrossRefGoogle Scholar
  9. de Weerdt J, de Backer M, Vanthienen J, Baesens B (2011a) A critical evaluation study of model-log metrics in process discovery. In: Muehlen M, Su J (eds) Business process management workshops. Lecture notes in business information processing, vol 66. Springer, Berlin, pp 158–169Google Scholar
  10. de Weerdt J, de Backer M, Vanthienen J, Baesens B (2011b) A robust F-measure for evaluating discovered process models. In: IEEE symposium series on computational intelligence. IEEE, pp 148–155Google Scholar
  11. Dijkman R, Dumas M, Ouyang C (2008) Semantics and analysis of business process models in BPMN. Inf Softw Technol 50(12):1281–1294CrossRefGoogle Scholar
  12. Dumas M, Rosa ML, Mendling J, Reijers H (2013) Fundamentals of business process management. Springer, BerlinCrossRefGoogle Scholar
  13. Gerke K, Cardoso J, Claus A (2009) Measuring the compliance of processes with reference models. In: Proceedings of the confederated international conferences, CoopIS, DOA, IS, and ODBASE 2009 on. On the move to meaningful internet systems: part I (OTM ’09). Springer, Berlin, pp 76–93Google Scholar
  14. Greco G, Guzzo A, Pontieri L, Sacca D (2006) Discovering expressive process models by clustering log traces. IEEE Trans Knowl Data Eng 18:1010–1027CrossRefGoogle Scholar
  15. Hofstede A, van der Aalst W, Adams M, Russell N (2010) Modern business process automation. Springer, BerlinCrossRefGoogle Scholar
  16. Kristensen L, Schmidt K, Valmari A (2006) Question-guided stubborn set methods for state properties. Formal Methods Syst Des 29(3):215–251CrossRefGoogle Scholar
  17. Munoz-Gama J, Carmona J (2010) A fresh look at precision in process conformance. In: Proceedings of the 8th international conference on business process management (BPM ’10). Springer, Berlin, pp 211–226Google Scholar
  18. Munoz-Gama J, Carmona J (2011) Enhancing precision in process conformance: stability, confidence and severity. In: Proceedings of the IEEE symposium on computational intelligence and data mining (CIDM 2011), April 11–15, Paris, France. IEEE, pp 184–191Google Scholar
  19. Munoz-Gama J, Carmona J (2012) A general framework for precision checking. Int J Innov Comput Inf Control (IJICIC) 8(7B):5317–5339Google Scholar
  20. Murata T (1989) Petri nets: properties, analysis and applications. Proc IEEE 77(4):541–580CrossRefGoogle Scholar
  21. Pande P, Neuman R, Cavanagh R (2000) The six sigma way: how GE, Motorola, and other top companies are honing their performance. McGraw-Hill, New YorkGoogle Scholar
  22. Petkovic M, Prandi D, Zannone N (2011) Purpose control: did you process the data for the intended purpose? In: Jonker W, Petkovic M (eds) Secure data management. Lecture notes in computer science, vol 6933. Springer, Berlin, pp 145–168Google Scholar
  23. Porter L, Parker A (1993) Total quality managementthe critical success factors. Total Qual Manag 4(1):13–22CrossRefGoogle Scholar
  24. Rozinat A, van der Aalst W (2008) Conformance checking of processes based on monitoring real behavior. Inf Syst 33:64–95CrossRefGoogle Scholar
  25. Rozinat A, de Jong I, Günther C, van der Aalst W (2009) Process Mining applied to the test process of wafer steppers in ASML. IEEE Trans Syst Man Cybern Part C 39:474–479CrossRefGoogle Scholar
  26. Schmidt K (1999) Stubborn sets for standard properties. In: Proceedings of the 20th international conference on application and theory of petri nets. Springer, London, pp 46–65Google Scholar
  27. van der Aalst W (2011a) Business process configuration in the cloud: how to support and analyze multi-tenant processes? In: Zavattaro G, Schreier U, Pautasso C (eds) Proceedings of the 9th IEEE European conference on web services (ECOWS 2011). IEEE Computer Society Press, pp 3–10Google Scholar
  28. van der Aalst W (2011b) Process mining: discovery, conformance and enhancement of business processes. Springer, New YorkCrossRefGoogle Scholar
  29. van der Aalst W, van Hee K (2004) Workflow management: models, methods, and systems. MIT Press, Cambridge, MAGoogle Scholar
  30. van der Aalst W, Rubin V, Verbeek H, van Dongen B, Kindler E, Gunther C (2010) Process mining: a two-step approach to balance between underfitting and overfitting. Softw Syst Model 9(1):87–111CrossRefGoogle Scholar
  31. van der Aalst W, Adriansyah A, van Dongen B (2011) Causal nets: a modeling language tailored towards process discovery. In: Proceedings of the 22nd international conference on concurrency theory (CONCUR ’11). Springer, Berlin, pp 28–42Google Scholar
  32. van der Aalst W, Adriansyah A, van Dongen B (2012) Replaying history on process models for conformance checking and performance analysis. Wiley Interdiscip Rev Data Min Knowl Discov 2(2):182–192CrossRefGoogle Scholar
  33. vanden Broucke S, de Weerdt J, Baesens B, Vanthienen J (2012) Improved artificial negative event generation to enhance process event logs. In: Ralyt J, Franch X, Brinkkemper S, Wrycza S (eds) Advanced information systems engineering. Lecture notes in computer science, vol 7328. Springer, Heidelberg, pp 254–269Google Scholar
  34. vanden Broucke S, de Weerdt J, Vanthienen J, Baesens B (2013) A comprehensive benchmarking framework (CoBeFra) for conformance analysis between procedural process models and event logs in ProM. In: Proceedings of the IEEE symposium on computational intelligence and data mining (CIDM 2013), part of the IEEE symposium series on computational intelligence 2013 (SSCI 2013)Google Scholar
  35. Weidlich M, Polyvyanyy A, Desai N, Mendling J (2010) Process compliance measurement based on behavioural profiles. In: Proceedings of the 22nd international conference on Advanced information systems engineering (CAiSE ’10). Springer, Berlin, pp 499–514Google Scholar
  36. Weijters A, van der Aalst W, de Medeiros AA (2006) Process mining with the heuristics miner-algorithm. BETA working paper series, WP 166. Technical report, Eindhoven University of Technology, EindhovenGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • A. Adriansyah
    • 1
  • J. Munoz-Gama
    • 2
  • J. Carmona
    • 2
  • B. F. van Dongen
    • 1
  • W. M. P. van der Aalst
    • 1
  1. 1.Department of Mathematics and Computer ScienceEindhoven University of TechnologyEindhovenThe Netherlands
  2. 2.Universitat Politecnica de CatalunyaBarcelonaSpain

Personalised recommendations