Advertisement

Towards Reliable Predictive Process Monitoring

  • Christopher Klinkmüller
  • Nick R. T. P. van Beest
  • Ingo Weber
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 317)

Abstract

Predictive process monitoring is concerned with anticipating the future behavior of running process instances. Prior work primarily focused on the performance of monitoring approaches and spent little effort on understanding other aspects such as reliability. This limits the potential to reuse the approaches across scenarios. From this starting point, we discuss how synthetic data can facilitate a better understanding of approaches and then use synthetic data in two experiments. We focus on prediction as classification of process instances during execution, solely considering the discrete event behavior. First, we compare different feature representations and reveal that sub-trace occurrence can cover a broader variety of relationships in the data than other representations. Second, we present evidence that the popular strategy of cutting traces to certain prefix lengths to learn prediction models for ongoing instances is prone to yield unreliable models and that the underlying problem can be avoided by using approaches that learn from complete traces. Our experiments provide a basis for future research and highlight that an evaluation solely targeting performance incurs the risk of incorrectly assessing benefits and limitations.

Keywords

Behavioral classification Predictive process monitoring Process mining Machine learning 

References

  1. 1.
    van der Aalst, W.M.P.: Process Mining: Discovery, Conformance and Enhancement of Business Processes. Springer, Heidelberg (2011).  https://doi.org/10.1007/978-3-642-19345-3CrossRefMATHGoogle Scholar
  2. 2.
    Rozinat, A., van der Aalst, W.M.: Conformance checking of processes based on monitoring real behavior. Inf. Syst. 33(1), 64–95 (2008)CrossRefGoogle Scholar
  3. 3.
    García-Bañuelos, L., van Beest, N.R.T.P., Dumas, M., La Rosa, M., Mertens, W.: Complete and interpretable conformance checking of business processes. IEEE Trans. Softw. Eng. 44(3), 262–290 (2018)CrossRefGoogle Scholar
  4. 4.
    Dumas, M., Maggi, F.M.: Enabling process innovation via deviance mining and predictive monitoring. In: vom Brocke, J., Schmiedel, T. (eds.) BPM - Driving Innovation in a Digital World. MP, pp. 145–154. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-14430-6_10CrossRefGoogle Scholar
  5. 5.
    Rogge-Solti, A., Weske, M.: Prediction of remaining service execution time using stochastic petri nets with arbitrary firing delays. In: Basu, S., Pautasso, C., Zhang, L., Fu, X. (eds.) ICSOC 2013. LNCS, vol. 8274, pp. 389–403. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-45005-1_27CrossRefGoogle Scholar
  6. 6.
    Tax, N., Verenich, I., La Rosa, M., Dumas, M.: Predictive business process monitoring with LSTM neural networks. In: Dubois, E., Pohl, K. (eds.) CAiSE 2017. LNCS, vol. 10253, pp. 477–492. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59536-8_30CrossRefGoogle Scholar
  7. 7.
    Evermann, J., Rehse, J.R., Fettke, P.: Predicting process behaviour using deep learning. Decis. Support Syst. 100(August), 129–140 (2017)CrossRefGoogle Scholar
  8. 8.
    Di Francescomarino, C., Ghidini, C., Maggi, F.M., Petrucci, G., Yeshchenko, A.: An eye into the future: leveraging a-priori knowledge in predictive business process monitoring. In: Carmona, J., Engels, G., Kumar, A. (eds.) BPM 2017. LNCS, vol. 10445, pp. 252–268. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65000-5_15CrossRefGoogle Scholar
  9. 9.
    Conforti, R., de Leoni, M., La Rosa, M., van der Aalst, W.M.P.: Supporting risk-informed decisions during business process execution. In: Salinesi, C., Norrie, M.C., Pastor, Ó. (eds.) CAiSE 2013. LNCS, vol. 7908, pp. 116–132. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-38709-8_8CrossRefGoogle Scholar
  10. 10.
    Conforti, R., Fink, S., Manderscheid, J., Röglinger, M.: PRISM – a predictive risk monitoring approach for business processes. In: La Rosa, M., Loos, P., Pastor, O. (eds.) BPM 2016. LNCS, vol. 9850, pp. 383–400. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-45348-4_22CrossRefGoogle Scholar
  11. 11.
    Maggi, F.M., Di Francescomarino, C., Dumas, M., Ghidini, C.: Predictive monitoring of business processes. In: Jarke, M., Mylopoulos, J., Quix, C., Rolland, C., Manolopoulos, Y., Mouratidis, H., Horkoff, J. (eds.) CAiSE 2014. LNCS, vol. 8484, pp. 457–472. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-07881-6_31CrossRefGoogle Scholar
  12. 12.
    van Beest, N.R.T.P., Weber, I.: Behavioral classification of business process executions at runtime. In: Dumas, M., Fantinato, M. (eds.) BPM 2016. LNBIP, vol. 281, pp. 339–353. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-58457-7_25CrossRefGoogle Scholar
  13. 13.
    Leontjeva, A., Conforti, R., Di Francescomarino, C., Dumas, M., Maggi, F.M.: Complex symbolic sequence encodings for predictive monitoring of business processes. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 297–313. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23063-4_21CrossRefGoogle Scholar
  14. 14.
    Teinemaa, I., Dumas, M., Maggi, F.M., Di Francescomarino, C.: Predictive business process monitoring with structured and unstructured data. In: La Rosa, M., Loos, P., Pastor, O. (eds.) BPM 2016. LNCS, vol. 9850, pp. 401–417. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-45348-4_23CrossRefGoogle Scholar
  15. 15.
    Metzger, A., Föcker, F.: Predictive business process monitoring considering reliability estimates. In: Dubois, E., Pohl, K. (eds.) CAiSE 2017. LNCS, vol. 10253, pp. 445–460. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-59536-8_28CrossRefGoogle Scholar
  16. 16.
    Aha, D.W.: Generalizing from case studies: a case study. In: ICML (1992)Google Scholar
  17. 17.
    Cohen, P.R., Jensen, D.: Overfitting explained. In: AISTATS (1997)Google Scholar
  18. 18.
    Salzberg, S.L.: On comparing classifiers: Pitfalls to avoid and a recommended approach. Data Min. Knowl. Discov. 1(3), 317–328 (1997)CrossRefGoogle Scholar
  19. 19.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. 7, 1–30 (2006)MathSciNetMATHGoogle Scholar
  20. 20.
    Jo, J., Bengio, Y.: Measuring the tendency of CNNs to learn surface statistical regularities. CoRR abs/1711.11561 (2017)Google Scholar
  21. 21.
    van Beest, N.R.T.P., Dumas, M., García-Bañuelos, L., La Rosa, M.: Log delta analysis: interpretable differencing of business process event logs. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 386–405. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23063-4_26CrossRefGoogle Scholar
  22. 22.
    Maaradji, A., Dumas, M., La Rosa, M., Ostovar, A.: Fast and accurate business process drift detection. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 406–422. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-23063-4_27CrossRefGoogle Scholar
  23. 23.
    van Zelst, S.J., Bolt, A., Hassani, M., van Dongen, B.F., van der Aalst, W.M.: Online conformance checking: relating event streams to process models using prefix-alignments. Int. J. Data Sci. Anal. 1–16 (2017).  https://doi.org/10.1007/s41060-017-0078-6
  24. 24.
    Burattin, A.: Online conformance checking for petri nets and event streams. In: BPM 2017 Demo Track (2017)Google Scholar
  25. 25.
    Weber, I., Rogge-Solti, A., Li, C., Mendling, J.: CCaaS: online conformance checking as a service. In: BPM, Demo Track (2015)Google Scholar
  26. 26.
    van der Aalst, W., Schonenberg, M., Song, M.: Time prediction based on process mining. Inf. Syst. 36(2), 450–475 (2011)CrossRefGoogle Scholar
  27. 27.
    Metzger, A., Leitner, P., Ivanovic, D., Schmieders, E., Franklin, R., Carro, M., Dustdar, S., Pohl, K.: Comparing and combining predictive business process monitoring techniques. IEEE Trans. SCM Syst. 45(2), 276–290 (2015)Google Scholar
  28. 28.
    Senderovich, A., Di Francescomarino, C., Ghidini, C., Jorbina, K., Maggi, F.M.: Intra and inter-case features in predictive process monitoring: a tale of two dimensions. In: Carmona, J., Engels, G., Kumar, A. (eds.) BPM 2017. LNCS, vol. 10445, pp. 306–323. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65000-5_18CrossRefGoogle Scholar
  29. 29.
    Pika, A., van der Aalst, W.M.P., Fidge, C.J., ter Hofstede, A.H.M., Wynn, M.T.: Predicting deadline transgressions using event logs. In: La Rosa, M., Soffer, P. (eds.) BPM 2012. LNBIP, vol. 132, pp. 211–216. Springer, Heidelberg (2013).  https://doi.org/10.1007/978-3-642-36285-9_22CrossRefGoogle Scholar
  30. 30.
    Di Francescomarino, C., Dumas, M., Federici, M., Ghidini, C., Maggi, F.M., Rizzi, W.: Predictive business process monitoring framework with hyperparameter optimization. In: Nurcan, S., Soffer, P., Bajec, M., Eder, J. (eds.) CAiSE 2016. LNCS, vol. 9694, pp. 361–376. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-39696-5_22CrossRefGoogle Scholar
  31. 31.
    Di Francescomarino, C., Dumas, M., Maggi, F.M., Teinemaa, I.: Clustering-based predictive process monitoring. IEEE Trans. Serv. Comput. (2016, in press)Google Scholar
  32. 32.
    Maggi, F.M., Mooij, A.J., van der Aalst, W.M.P.: User-guided discovery of declarative process models. In: CIDM (2011)Google Scholar
  33. 33.
    Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE TSE 37(5), 649–678 (2011)Google Scholar
  34. 34.
    Zaki, M.J.: SPADE: an efficient algorithm for mining frequent sequences. Mach. Learn. 42(1), 31–60 (2001)CrossRefGoogle Scholar
  35. 35.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefGoogle Scholar
  36. 36.
    Hastie, T., Tibshirani, R., Friedman, J.: Random forests. The Elements of Statistical Learning. SSS, pp. 587–604. Springer, New York (2009).  https://doi.org/10.1007/978-0-387-84858-7_15CrossRefMATHGoogle Scholar
  37. 37.
    Cohen, W.W.: Fast effective rule induction. In: ICML (1995)CrossRefGoogle Scholar
  38. 38.
    Quinlan, J.: Learning logical definitions from relations. Mach. Learn. 5(3), 239–266 (1990)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Christopher Klinkmüller
    • 1
  • Nick R. T. P. van Beest
    • 2
  • Ingo Weber
    • 1
  1. 1.Data61, CSIROSydneyAustralia
  2. 2.Data61, CSIROBrisbaneAustralia

Personalised recommendations