Abstract
One of the goals of process discovery is to construct, from a given event log, a process model which correctly represents the underlying system. As with any abstraction, one does not necessarily want to represent all possible behavior, but only the significant behavior. While various discovery algorithms support this use case of discovering the significant process behavior, proper evaluation measures for this use case appear to be missing.
Therefore, this paper presents a new precision metric that quantifies to what extent the discovered model contains significant system behavior. Besides being a metric with a clear and intuitive interpretation, the metric distinguishes itself in two other areas. Firstly, it introduces the concept of \(\alpha \)-significance, which only measures precision with respect to significant behavior. Secondly, it is designed as a system measure and estimates the precision with respect to the underlying system rather than the observed log. This work introduces a new precision measure and a statistical estimation method. Additionally, an empirical demonstration and evaluation of the metric are provided, which creates initial insights and knowledge about the performance and characteristics of the new measure. The results show that the \(\alpha \)-precision measure provides a solid foundation for future work on developing quality measures for this particular use case.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
In the specific case that \(\gamma = 1\), \(\textbf{D}^0\) equals the identity matrix \(\textbf{I}\), and thus \(\mathbf {o^TD}^{\gamma - 1}\textbf{f} = \mathbf {o^Tf}\), which is the number of activities that are both valid start and end activities. This is indeed equal to the number of valid sequences of length one.
References
Badouel, E.: On the \({\alpha }\)-reconstructibility of workflow nets. In: Haddad, S., Pomello, L. (eds.) PETRI NETS 2012. LNCS, vol. 7347, pp. 128–147. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31131-4_8
Brons, D., Scheepens, R., Fahland, D.: Striking a new balance in accuracy and simplicity with the probabilistic inductive miner. In: ICPM, pp. 32–39. IEEE (2021)
Buijs, J.C.A.M., van Dongen, B.F., van der Aalst, W.M.P.: Quality dimensions in process discovery: the importance of fitness, precision, generalization and simplicity. Int. J. Cooper. Inf. Syst. 23(1) (2014)
Burke, A., Leemans, S.J.J., Wynn, M.T.: Stochastic process discovery by weight estimation. In: Leemans, S., Leopold, H. (eds.) ICPM 2020. LNBIP, vol. 406, pp. 260–272. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72693-5_20
van Dongen, B.: BPI challenge 2012, April 2012. https://doi.org/10.4121/uuid:3926db30-f712-4394-aebc-75976070e91f
Gelman, A., Carlin, J.B., Stern, H.S., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian Data Analysis. Taylor & Francis Ltd, Boca Raton (2013)
Günther, C.W., van der Aalst, W.M.P.: Fuzzy mining – adaptive process simplification based on multi-perspective metrics. In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 328–343. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-75183-0_24
Günther, C.W., Rozinat, A.: Disco: discover your processes. In: BPM. CEUR Workshop Proceedings, vol. 940, pp. 40–44. CEUR-WS.org (2012)
Janssenswillen, G., Depaire, B.: Towards confirmatory process discovery: making assertions about the underlying system. Bus. Inf. Syst. Eng. 61(6), 713–728 (2019)
Janssenswillen, G., Depaire, B., Faes, C.: Enhancing discovered process models using Bayesian inference and MCMC. In: Del Río Ortega, A., Leopold, H., Santoro, F.M. (eds.) BPM 2020. LNBIP, vol. 397, pp. 295–307. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-66498-5_22
Janssenswillen, G., Jouck, T., Creemers, M., Depaire, B.: Measuring the quality of models with respect to the underlying system: an empirical study. In: La Rosa, M., Loos, P., Pastor, O. (eds.) BPM 2016. LNCS, vol. 9850, pp. 73–89. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-45348-4_5
Kotz, S., Balakrishnan, N., Johnson, N.L.: Continuous Multivariate Distributions: Models and Applications, vol. 1. Wiley, Hoboken (2004)
Leemans, S.J.J., van der Aalst, W.M.P., Brockhoff, T., Polyvyanyy, A.: Stochastic process mining: earth movers’ stochastic conformance. Inf. Syst. 102, 101724 (2021)
Leemans, S.J.J., Fahland, D.: Information-preserving abstractions of event data in process mining. Knowl. Inf. Syst. 62(3), 1143–1197 (2019). https://doi.org/10.1007/s10115-019-01376-9
Leemans, S.J.J., Fahland, D., van der Aalst, W.M.P.: Discovering block-structured process models from event logs containing infrequent behaviour. In: Lohmann, N., Song, M., Wohed, P. (eds.) BPM 2013. LNBIP, vol. 171, pp. 66–78. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06257-0_6
Leemans, S.J.J., Fahland, D., van der Aalst, W.M.P.: Discovering block-structured process models from incomplete event logs. In: Ciardo, G., Kindler, E. (eds.) PETRI NETS 2014. LNCS, vol. 8489, pp. 91–110. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07734-5_6
Leemans, S.J.J., Polyvyanyy, A.: Stochastic-aware conformance checking: an entropy-based approach. In: Dustdar, S., Yu, E., Salinesi, C., Rieu, D., Pant, V. (eds.) CAiSE 2020. LNCS, vol. 12127, pp. 217–233. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-49435-3_14
Leemans, S.J.J., Poppe, E., Wynn, M.T.: Directly follows-based process mining: exploration & a case study. In: ICPM, pp. 25–32. IEEE (2019)
Polyvyanyy, A., Moffat, A., García-Bañuelos, L.: An entropic relevance measure for stochastic conformance checking in process mining. In: ICPM, pp. 97–104. IEEE (2020)
Syring, A.F., Tax, N., van der Aalst, W.M.P.: Evaluating conformance measures in process mining using conformance propositions. Trans. Petri Nets Other Model. Concurr. 14, 192–221 (2019)
Tax, N., Lu, X., Sidorova, N., Fahland, D., van der Aalst, W.M.P.: The imprecisions of precision measures in process mining. Inf. Process. Lett. 135, 1–8 (2018)
Weijters, A., van Der Aalst, W.M., De Medeiros, A.A.: Process mining with the heuristics miner-algorithm. Technical report WP 166, Technische Universiteit Eindhoven, pp. 1–34 (2006)
van Zelst, S.J., van Dongen, B.F., van der Aalst, W.M.P.: Avoiding over-fitting in ILP-based process discovery. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 163–171. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23063-4_10
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Depaire, B., Janssenswillen, G., Leemans, S.J.J. (2022). Alpha Precision: Estimating the Significant System Behavior in a Model. In: Di Ciccio, C., Dijkman, R., del Río Ortega, A., Rinderle-Ma, S. (eds) Business Process Management Forum. BPM 2022. Lecture Notes in Business Information Processing, vol 458. Springer, Cham. https://doi.org/10.1007/978-3-031-16171-1_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-16171-1_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16170-4
Online ISBN: 978-3-031-16171-1
eBook Packages: Computer ScienceComputer Science (R0)