Skip to main content
SpringerLink
Log in

A Case for Business Process-Specific Foundation Models

  • Conference paper
  • First Online:
Business Process Management Workshops (BPM 2023)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 492))

Included in the following conference series:

  • 395 Accesses

Abstract

The inception of large language models has helped advance the state-of-the-art on numerous natural language tasks. This has also opened the door for the development of foundation models for other domains and data modalities (e.g., images and code). In this paper, we argue that business process data has unique characteristics that warrant the creation of a new class of foundation models to handle tasks like activity prediction, process optimization, and decision making. These models should also tackle the challenges of applying AI to business processes which include data scarcity, multi-modal representations, domain specific terminology, and privacy concerns. To support our claim, we show the effectiveness of few-shot learning and transfer learning in next activity prediction, crucial properties for the success of foundation models.

His contributions were completed while he was an intern at IBM Research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.gartner.com/smarterwithgartner/the-disruptive-power-of-artificial-intelligence.

  2. 2.

    https://www.marketwatch.com/press-release/business-process-management-market-size-growth-with-top-leading-players-growth-key-factors-global-trends-industry-share-and-forecast-2022-2031-2022-08-18.

  3. 3.

    https://www.gartner.com/en/newsroom/press-releases/2021-09-29-gartner-finds-33-percent-of-technology-providers-plan-to-invest-1-million-or-more-in-ai-within-two-years.

  4. 4.

    To avoid confusion with business process tasks or activities, we will use “downstream tasks” to refer to foundation model specific prediction tasks.

References

  1. Van der Aalst, W.M., Bichler, M., Heinzl, A.: Robotic process automation (2018)

    Google Scholar 

  2. Alsentzer, E., et al.: Publicly available clinical BERT embeddings. arXiv preprint arXiv:1904.03323 (2019)

  3. Arlbjørn, J.S., Haug, A.: Business Process Optimization. Academica (2010)

    Google Scholar 

  4. Bach, S.H., et al.: PromptSource: an integrated development environment and repository for natural language prompts. arXiv preprint arXiv:2202.01279 (2022)

  5. Bernhart, W., Winterhoff, M.: Autonomous driving: disruptive innovation that promises to change the automotive industry as we know it. In: Langheim, J. (ed.) Energy Consumption and Autonomous Driving. LNM, pp. 3–10. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-19818-7_1

    Chapter  Google Scholar 

  6. Blodgett, S.L., Madaio, M.: Risks of AI foundation models in education. arXiv preprint arXiv:2110.10024 (2021)

  7. Bommasani, R., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021)

  8. Brown, T., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)

    Google Scholar 

  9. Camargo, M., Dumas, M., González-Rojas, O.: Learning accurate LSTM models of business processes. In: Hildebrandt, T., van Dongen, B.F., Röglinger, M., Mendling, J. (eds.) BPM 2019. LNCS, vol. 11675, pp. 286–302. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-26619-6_19

    Chapter  Google Scholar 

  10. Chakraborti, T., et al.: From robotic process automation to intelligent process automation. In: Asatiani, A., et al. (eds.) BPM 2020. LNBIP, vol. 393, pp. 215–228. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58779-6_15

    Chapter  Google Scholar 

  11. Chen, H., Fang, X., Fang, H.: Multi-task prediction method of business process based on BERT and transfer learning. Knowl.-Based Syst. 254, 109603 (2022)

    Article  Google Scholar 

  12. Chui, M., Henke, N., Miremadi, M.: Most of AI business uses will be in two areas. Harv. Bus. Rev. 20 (2018)

    Google Scholar 

  13. van Dongen, B.B.: BPI challenge 2015 (2015)

    Google Scholar 

  14. van Dongen, B., Borchert, F.F.: BPI challenge 2018 (2018)

    Google Scholar 

  15. Dunzer, S., Stierle, M., Matzner, M., Baier, S.: Conformance checking: a state-of-the-art literature review. In: 11th International Conference on Subject-Oriented Business Process Management (2019)

    Google Scholar 

  16. Finn, C., Abbeel, P., Levine, S.: Model-agnostic meta-learning for fast adaptation of deep networks. In: International Conference on Machine Learning, pp. 1126–1135. PMLR (2017)

    Google Scholar 

  17. Geyer-Klingeberg, J., Nakladal, J., Baldauf, F., Veit, F.: Process mining and robotic process automation: a perfect match. In: BPM (2018)

    Google Scholar 

  18. Grosskopf, A., Decker, G., Weske, M.: The Process: Business Process Modeling Using BPMN. Meghan Kiffer Press (2009)

    Google Scholar 

  19. Huo, S., Völzer, H., Reddy, P., Agarwal, P., Isahagian, V., Muthusamy, V.: Graph autoencoders for business process anomaly detection. In: Polyvyanyy, A., Wynn, M.T., Van Looy, A., Reichert, M. (eds.) BPM 2021. LNCS, vol. 12875, pp. 417–433. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85469-0_26

    Chapter  Google Scholar 

  20. Jia, C., et al.: Scaling up visual and vision-language representation learning with noisy text supervision. In: ICML, pp. 4904–4916. PMLR (2021)

    Google Scholar 

  21. Kecht, C., Egger, A., Kratsch, W., Röglinger, M.: Quantifying chatbots ability to learn business processes. Inf. Syst. 113, 102176 (2023)

    Google Scholar 

  22. Kenton, J.D., Chang, M.W., Toutanova, L.K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the NAACL-HLT (2019)

    Google Scholar 

  23. Kratsch, W., Manderscheid, J., Röglinger, M., Seyfried, J.: Machine learning in business process monitoring: a comparison of deep learning and classical approaches used for outcome prediction. Bus. Inf. Syst. Eng. 63, 261–276 (2021)

    Article  Google Scholar 

  24. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. arXiv preprint arXiv:2107.13586 (2021)

  25. Maaradji, A., Dumas, M., La Rosa, M., Ostovar, A.: Fast and accurate business process drift detection. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) BPM 2015. LNCS, vol. 9253, pp. 406–422. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23063-4_27

    Chapter  Google Scholar 

  26. Maaradji, A., Dumas, M., La Rosa, M., Ostovar, A.: Detecting sudden and gradual drifts in business processes from execution traces. IEEE Trans. Knowl. Data Eng. 29(10), 2140–2154 (2017)

    Article  Google Scholar 

  27. McKendrick, J.: AI adoption skyrocketed over the last 18 months. Harv. Bus. Rev. (2021)

    Google Scholar 

  28. Mehdiyev, N., Evermann, J., Fettke, P.: A novel business process prediction model using a deep learning method. Bus. Inf. Syst. Eng. 62, 143–157 (2020)

    Article  Google Scholar 

  29. Mendling, J.: Advancing business process science via the co-evolution of substantive and methodological knowledge. In: Di Ciccio, C., Dijkman, R., del Río Ortega, A., Rinderle-Ma, S. (eds.) BPM 2022. LNCS, vol. 13420, pp. 3–18. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16103-2_1

    Chapter  Google Scholar 

  30. Min, S., et al.: Rethinking the role of demonstrations: what makes in-context learning work? arXiv preprint arXiv:2202.12837 (2022)

  31. Nguyen, P., Isahagian, V., Muthusamy, V., Slominski, A.: Summarizing process traces for analysis tasks: an intuitive and user-controlled approach. In: International Joint Conference on Artificial Intelligence (2022)

    Google Scholar 

  32. Park, G., Song, M.: Predicting performances in business processes using deep neural networks. Decis. Support Syst. 129, 113191 (2020)

    Article  Google Scholar 

  33. Pettey, C., van der Meulen, R.: Gartner says global artificial intelligence business value to reach \$1.2 trillion in 2018 (2018)

    Google Scholar 

  34. Poesia, G., et al.: Synchromesh: reliable code generation from pre-trained language models. arXiv preprint arXiv:2201.11227 (2022)

  35. Radford, A., et al.: Language models are unsupervised multitask learners. OpenAI Blog 1(8), 9 (2019)

    Google Scholar 

  36. Radford, A., et al.: Learning transferable visual models from natural language supervision. In: ICML, pp. 8748–8763. PMLR (2021)

    Google Scholar 

  37. Rama-Maneiro, E., Vidal, J., Lama, M.: Deep learning for predictive business process monitoring: review and benchmark. IEEE Trans. Serv. Comput. (2021)

    Google Scholar 

  38. Rizk, Y., Venkateswaran, P., Isahagian, V., Muthusamy, V., Talamadupula, K.: Can you teach robotic process automation bots new tricks? In: Marrella, A., et al. (eds.) BPM 2022. LNBIP, vol. 459, pp. 246–259. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-16168-1_16

    Chapter  Google Scholar 

  39. Schneider, F.: How users reciprocate to Alexa. In: Stephanidis, C., Antona, M., Ntoa, S. (eds.) HCII 2020. CCIS, vol. 1293, pp. 376–383. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60700-5_48

    Chapter  Google Scholar 

  40. Senderovich, A., Di Francescomarino, C., Ghidini, C., Jorbina, K., Maggi, F.M.: Intra and inter-case features in predictive process monitoring: a tale of two dimensions. In: Carmona, J., Engels, G., Kumar, A. (eds.) BPM 2017. LNCS, vol. 10445, pp. 306–323. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-65000-5_18

    Chapter  Google Scholar 

  41. Tian, X., Pavur, R., Han, H., Zhang, L.: A machine learning-based human resources recruitment system for business process management: using LSA, BERT and SVM. Bus. Process. Manag. J. 29(1), 202–222 (2022)

    Article  Google Scholar 

  42. Van Der Aalst, W.: Process mining: overview and opportunities. ACM Trans. Manage. Inf. Syst. 3(2), 1–17 (2012)

    Article  Google Scholar 

  43. Venkateswaran, P., Isahagian, V., Muthusamy, V., Venkatasubramanian, N.: FedGen: generalizable federated learning for sequential data. In: IEEE International Conference on Cloud Computing (2023)

    Google Scholar 

  44. Venkateswaran, P., Muthusamy, V., Isahagian, V., Venkatasubramanian, N.: Environment agnostic invariant risk minimization for classification of sequential datasets. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 1615–1624 (2021)

    Google Scholar 

  45. Venkateswaran, P., Muthusamy, V., Isahagian, V., Venkatasubramanian, N.: Robust and generalizable predictive models for business processes. In: Polyvyanyy, A., Wynn, M.T., Van Looy, A., Reichert, M. (eds.) BPM 2021. LNCS, vol. 12875, pp. 105–122. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85469-0_9

    Chapter  Google Scholar 

  46. Wang, W., Bao, H., Dong, L., Wei, F.: VLMo: unified vision-language pre-training with mixture-of-modality-experts. arXiv preprint arXiv:2111.02358 (2021)

  47. Wang, W., et al.: Image as a foreign language: BEiT pretraining for all vision and vision-language tasks. arXiv preprint arXiv:2208.10442 (2022)

  48. Weske, Mathias: Business process management methodology. In: Weske, M. (ed.) Business Process Management, pp. 373–388. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-28616-2_8

    Chapter  Google Scholar 

  49. White, S.A.: Introduction to BPMN. IBM Coop. 2 (2004)

    Google Scholar 

  50. Wiggins, W.F., Tejani, A.S.: On the opportunities and risks of foundation models for natural language processing in radiology. Radiol.: Artif. Intell. 4(4), e220119 (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IBM Research, Cambridge, USA

    Yara Rizk, Praveen Venkateswaran, Vatche Isahagian & Vinod Muthusamy

  2. Stanford University, Stanford, USA

    Austin Narcomey

Authors
  1. Yara Rizk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Praveen Venkateswaran
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vatche Isahagian
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Austin Narcomey
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vinod Muthusamy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yara Rizk .

Editor information

Editors and Affiliations

  1. KU Leuven, Leuven, Belgium

    Jochen De Weerdt

  2. TUM School of Computation, Information and Technology, Heilbronn, Germany

    Luise Pufahl

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rizk, Y., Venkateswaran, P., Isahagian, V., Narcomey, A., Muthusamy, V. (2024). A Case for Business Process-Specific Foundation Models. In: De Weerdt, J., Pufahl, L. (eds) Business Process Management Workshops. BPM 2023. Lecture Notes in Business Information Processing, vol 492. Springer, Cham. https://doi.org/10.1007/978-3-031-50974-2_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-50974-2_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-50973-5

  • Online ISBN: 978-3-031-50974-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation