Skip to main content
SpringerLink
Log in

Using Process Mining for Face Validity Assessment in Agent-Based Simulation Models: An Exploratory Case Study

  • Conference paper
  • First Online:
Cooperative Information Systems (CoopIS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14353))

Included in the following conference series:

  • 407 Accesses

Abstract

In the field of simulation, the key objective of a system designer is to develop a model that performs a specific task and accurately represents real-world systems or processes. A valid simulation model allows for a better understanding of the system’s behavior and improved decision-making in the real world. Face validity is a subjective measure that assesses the extent to which a simulation model and its outcomes appear reasonable to an expert based on a superficial examination of the simulator’s realism. Process mining techniques, which are novel data-driven methods for obtaining real-life insights into processes based on event logs, show promise when combined with effective visualization techniques. These techniques can augment the face validity assessment of simulation models in reflecting real-life behavior and play a key role in supporting humans conducting such assessments. In this paper, we present an approach that utilizes process mining techniques to assess the face validity of agent-based simulation models. To illustrate our approach, we use the Schelling model of segregation. We demonstrate how graphical representation, immersive assessment, and sensitivity analysis can be used to assess face validity based on event logs produced by the simulation model. Our study shows that process mining in combination with visualization can strongly support humans in assessing face validity of agent-based simulation models.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. van der Aalst, W.M.P.: Process Mining: Discovery, Conformance and Enhancement of Business Processes. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-19345-3

    Book  MATH  Google Scholar 

  2. Belhadi, A., Djenouri, Y., Diaz, V.G., Houssein, E.H., Lin, J.C.W.: Hybrid intelligent framework for automated medical learning. Expert. Syst. 39(6), e12737 (2022). https://doi.org/10.1111/exsy.12737

    Article  Google Scholar 

  3. Bemthuis, R., Lazarova-Molnar, S.: An approach for face validity assessment of agent-based simulation models through outlier detection with process mining. In: Enterprise Design, Operations, and Computing (in press)

    Google Scholar 

  4. Bemthuis, R., Mes, M., Iacob, M.E., Havinga, P.: Using agent-based simulation for emergent behavior detection in cyber-physical systems. In: 2020 Winter Simulation Conference (WSC), pp. 230–241. IEEE (2020). https://doi.org/10.1109/WSC48552.2020.9383956

  5. Bemthuis, R.H., Koot, M., Mes, M.R., Bukhsh, F.A., Iacob, M.E., Meratnia, N.: An agent-based process mining architecture for emergent behavior analysis. In: 2019 IEEE 23rd International Enterprise Distributed Object Computing Workshop (EDOCW), pp. 54–64. IEEE (2019). https://doi.org/10.1109/EDOCW.2019.00022

  6. Bemthuis, R.H., Lazarova-Molnar, S.: Discovering agent models using process mining: initial approach and a case study. In: 2022 IEEE International Conference on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom), pp. 163–172 (2022). https://doi.org/10.1109/ISPA-BDCloud-SocialCom-SustainCom57177.2022.00028

  7. Berti, A., van Zelst, S.J., van der Aalst, W.M.P.: Process mining for python (PM4Py): bridging the gap between process-and data science. arXiv preprint arXiv:1905.06169 (2019)

  8. Burattin, A.: Streaming process mining. In: van der Aalst, W.M.P., Carmona, J. (eds.) Process Mining Handbook. LNBIP, vol. 448, pp. 349–372. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-08848-3_11

    Chapter  Google Scholar 

  9. Cabac, L., Knaak, N., Moldt, D., Rölke, H.: Analysis of multi-agent interactions with process mining techniques. In: Fischer, K., Timm, I.J., André, E., Zhong, N. (eds.) MATES 2006. LNCS (LNAI), vol. 4196, pp. 12–23. Springer, Heidelberg (2006). https://doi.org/10.1007/11872283_2

    Chapter  Google Scholar 

  10. Carter, F., et al.: Consensus guidelines for validation of virtual reality surgical simulators. Surv. Methodol. 19, 1523–1532 (2005). https://doi.org/10.1007/s00464-005-0384-2

    Article  Google Scholar 

  11. Clark, W.A., Fossett, M.: Understanding the social context of the Schelling segregation model. Proc. Natl. Acad. Sci. 105(11), 4109–4114 (2008). https://doi.org/10.1073/pnas.0708155105

    Article  Google Scholar 

  12. Ferreira, D.R., Szimanski, F., Ralha, C.G.: Mining the low-level behaviour of agents in high-level business processes. Int. J. Bus. Process Integr. Manag. 8 6(2), 146–166 (2013). https://doi.org/10.1504/IJBPIM.2013.054678

    Article  Google Scholar 

  13. Foramitti, J.: AgentPy: a package for agent-based modeling in Python. J. Open Source Softw. 6(62), 3065 (2021). https://doi.org/10.21105/joss.03065

    Article  Google Scholar 

  14. Halaška, M., Šperka, R.: Advantages of application of process mining and agent-based systems in business domain. In: Jezic, G., Chen-Burger, Y.-H.J., Howlett, R.J., Jain, L.C., Vlacic, L., Šperka, R. (eds.) KES-AMSTA-18 2018. SIST, vol. 96, pp. 177–186. Springer, Cham (2019). https://doi.org/10.1007/978-3-319-92031-3_17

    Chapter  Google Scholar 

  15. Hardesty, D.M., Bearden, W.O.: The use of expert judges in scale development: implications for improving face validity of measures of unobservable constructs. J. Bus. Res. 57(2), 98–107 (2004). https://doi.org/10.1016/S0148-2963(01)00295-8

    Article  Google Scholar 

  16. Henry, A.D., Prałat, P., Zhang, C.Q.: Emergence of segregation in evolving social networks. Proc. Natl. Acad. Sci. 108(21), 8605–8610 (2011). https://doi.org/10.1073/pnas.1014486108

    Article  MathSciNet  MATH  Google Scholar 

  17. Hill, A.L., Rand, D.G., Nowak, M.A., Christakis, N.A.: Infectious disease modeling of social contagion in networks. PLoS Comput. Biol. 6(11), e1000968 (2010). https://doi.org/10.1371/journal.pcbi.1000968

    Article  MathSciNet  Google Scholar 

  18. Hüllen, G., Zhai, J., Kim, S.H., Sinha, A., Realff, M.J., Boukouvala, F.: Managing uncertainty in data-driven simulation-based optimization. Comput. Chem. Eng. 136, 106519 (2020). https://doi.org/10.1016/j.compchemeng.2019.106519

    Article  Google Scholar 

  19. Ito, S., Vymětal, D., Šperka, R., Halaška, M.: Process mining of a multi-agent business simulator. Comput. Math. Organ. Theory 24(4), 500–531 (2018). https://doi.org/10.1007/s10588-018-9268-6

    Article  Google Scholar 

  20. Klügl, F.: A validation methodology for agent-based simulations. In: Proceedings of the 2008 ACM Symposium on Applied Computing, pp. 39–43 (2008). https://doi.org/10.1145/1363686.1363696

  21. Law, A.M., Kelton, W.D., Kelton, W.D.: Simulation Modeling and Analysis, vol. 3. McGraw-Hill, New York (2007)

    MATH  Google Scholar 

  22. Liu, Z., Li, X., Khojandi, A., Lazarova-Molnar, S.: On the extension of Schelling’s segregation model. In: 2019 Winter Simulation Conference (WSC), pp. 285–296. IEEE (2019). https://doi.org/10.1109/WSC40007.2019.9004848

  23. Mourtzis, D.: Simulation in the design and operation of manufacturing systems: state of the art and new trends. Int. J. Prod. Res. 58(7), 1927–1949 (2020). https://doi.org/10.1080/00207543.2019.1636321

    Article  Google Scholar 

  24. Negahban, A., Smith, J.S.: Simulation for manufacturing system design and operation: literature review and analysis. J. Manuf. Syst. 33(2), 241–261 (2014). https://doi.org/10.1016/j.jmsy.2013.12.007

    Article  Google Scholar 

  25. Royal, K.: “Face validity’’ is not a legitimate type of validity evidence! Am. J. Surg. 212(5), 1026–1027 (2016). https://doi.org/10.1016/j.amjsurg.2016.02.018

    Article  Google Scholar 

  26. Rozinat, A., van der Aalst, W.M.P.: Conformance checking of processes based on monitoring real behavior. Inf. Syst. 33(1), 64–95 (2008). https://doi.org/10.1016/j.is.2007.07.001

    Article  Google Scholar 

  27. Sargent, R.G.: Validation and verification of simulation models. In: Proceedings of the 24th Conference on Winter Simulation, pp. 104–114 (1992)

    Google Scholar 

  28. Sargent, R.G.: Verification and validation of simulation models. In: Proceedings of the 2010 Winter Simulation Conference, pp. 166–183. IEEE (2010). https://doi.org/10.1109/WSC.2010.5679166

  29. Sargent, R.G.: Verification and validation of simulation models: an advanced tutorial. In: 2020 Winter Simulation Conference (WSC), pp. 16–29 (2020)

    Google Scholar 

  30. Schelling, T.C.: Models of segregation. Am. Econ. Rev. 59(2), 488–493 (1969)

    Google Scholar 

  31. Schelling, T.C.: Dynamic models of segregation. J. Math. Sociol. 1(2), 143–186 (1971)

    Article  MATH  Google Scholar 

  32. Sert, E., Bar-Yam, Y., Morales, A.J.: Segregation dynamics with reinforcement learning and agent based modeling. Sci. Rep. 10(1), 11771 (2020). https://doi.org/10.1038/s41598-020-68447-8

    Article  Google Scholar 

  33. Shannon, R.: Introduction to the art and science of simulation. In: 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274), vol. 1, pp. 7–14 (1998). https://doi.org/10.1109/WSC.1998.744892

  34. Singh, A., Vainchtein, D., Weiss, H.: Schelling’s segregation model: parameters, scaling, and aggregation. Demogr. Res. 21, 341–366 (2009). https://doi.org/10.4054/DemRes.2009.21.12

    Article  Google Scholar 

  35. Šperka, R., Spišák, M., Slaninová, K., Martinovič, J., Dráždilová, P.: Control loop model of virtual company in BPM simulation. In: Snášel, V., Abraham, A., Corchado, E. (eds.) Soft Computing Models in Industrial and Environmental Applications. AIS, pp. 515–524. Springer, Cham (2013). https://doi.org/10.1007/978-3-642-32922-7_53

    Chapter  Google Scholar 

  36. Sulis, E., Taveter, K.: Beyond process simulation. In: Sulis, E., Taveter, K. (eds.) Agent-Based Business Process Simulation, pp. 175–182. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-98816-6_9

    Chapter  Google Scholar 

  37. Tour, A., Polyvyanyy, A., Kalenkova, A.: Agent system mining: vision, benefits, and challenges. IEEE Access 9, 99480–99494 (2021). https://doi.org/10.1109/ACCESS.2021.3095464

    Article  Google Scholar 

  38. Weijters, A.J.M.M., van der Aalst, W.M.P., de Medeiros, A.K.A.: Process mining with the HeuristicsMiner algorithm (2006)

    Google Scholar 

Download references

Acknowledgements

We acknowledge the Helmholtz Information & Data Science Academy (HIDA) for providing financial support enabling a short-term research stay at Karlsruhe Institute of Technology (KIT), Germany.

Author information

Authors and Affiliations

  1. University of Twente, Drienerlolaan 5, 7522 NB, Enschede, The Netherlands

    Rob Bemthuis & Ruben Govers

  2. Karlsruhe Institute of Technology, Kaiserstraße 89, 76133, Karlsruhe, Germany

    Rob Bemthuis & Sanja Lazarova-Molnar

Authors
  1. Rob Bemthuis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruben Govers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sanja Lazarova-Molnar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rob Bemthuis .

Editor information

Editors and Affiliations

  1. Telecom SudParis, Evry, France

    Mohamed Sellami

  2. Leibniz University Hannover, Hannover, Germany

    Maria-Esther Vidal

  3. Eindhoven University of Technology, Eindhoven, The Netherlands

    Boudewijn van Dongen

  4. Telecom SudParis, Evry, France

    Walid Gaaloul

  5. University of Lorraine, Vandoeuvre-les-Nancy, France

    Hervé Panetto

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bemthuis, R., Govers, R., Lazarova-Molnar, S. (2024). Using Process Mining for Face Validity Assessment in Agent-Based Simulation Models: An Exploratory Case Study. In: Sellami, M., Vidal, ME., van Dongen, B., Gaaloul, W., Panetto, H. (eds) Cooperative Information Systems. CoopIS 2023. Lecture Notes in Computer Science, vol 14353. Springer, Cham. https://doi.org/10.1007/978-3-031-46846-9_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-46846-9_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-46845-2

  • Online ISBN: 978-3-031-46846-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation