Abstract
Process design artifacts have been increasingly used to guide the modeling of business processes. To support users in designing and understanding process models, different process artifacts have been combined in several ways leading to the emergence of the so-called “hybrid process artifacts”. While many hybrid artifacts have been proposed in the literature, little is known about how they can actually support users in practice. To address this gap, this work investigates the way users engage with hybrid process artifacts during comprehension tasks. In particular, we focus on a hybrid representation of DCR Graphs (DCR-HR) combining a process model, textual annotations and an interactive simulation. Following a qualitative approach, we conduct a multi-granular analysis exploiting process mining, eye-tracking techniques, and verbal data analysis to scrutinize the reading patterns and the strategies adopted by users when being confronted with DCR-HR. The findings of the coarse-grained analysis provide important insights about the behavior of domain experts and IT specialists and show how user’s background and task type change the use of hybrid process artifacts. As for the fine-grained analysis, user’s behavior was classified into goal-directed and exploratory and different strategies of using the interactive simulation were identified. In addition, a progressive switch from an exploratory behavior to a goal-directed behavior was observed. These insights pave the way for an improved development of hybrid process artifacts and delineate several directions for future work.
Similar content being viewed by others
Notes
Atlas.ti, a qualitative data analysis tool. See https://atlasti.com.
Quote translated from Danish.
References
Hevner Von Alan, R., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28(1), 75–105 (2004)
Dengler, F., Vrandečić, D.: Wiki-Based Maturing of Process Descriptions, pp. 313–328 (2011)
Pinggera, J., Porcham, T., Zugal, S., Weber, B.: LiProMo-Literate process modeling. In: CEUR Workshop Proceedings, vol. 855, pp. 163–170 (2012)
Zugal, S., Pinggera, J., Weber, B.: Creating declarative process models using test driven modeling suite. In: Nurcan, S. (ed.) IS Olympics: Information Systems in a Diverse World, pp. 16–32. Springer, Berlin (2012)
Zugal, S., Pinggera, J., Weber, B.: Toward enhanced life-cycle support for declarative processes. J. Softw. Evol. Process 24(3), 285–302 (2012)
De Smedt, J., De Weerdt, J., Serral, E., Vanthienen, J.: Improving understandability of declarative process models by revealing hidden dependencies. In: Nurcan, S., Soffer, P., Bajec, M., Eder, J. (eds.) Advanced Information Systems Engineering, pp. 83–98. Springer, Cham (2016)
De Smedt, J., De Weerdt, J., Serral, E., Vanthienen, J.: Discovering hidden dependencies in constraint-based declarative process models for improving understandability. Inf. Syst. 74, 40–52 (2018)
López, H.A., Debois, S., Hildebrandt, T.T., Marquard, M.: The process highlighter: from texts to declarative processes and back. In: BPM (Dissertation/Demos/Industry), pp. 66–70 (2018)
Andaloussi, A.A., Buch-Lorentsen, J., López, H.A., Slaats, T., Weber, B.: Exploring the modeling of declarative processes using a hybrid approach. In: Laender, A.H.F., Pernici, B., Lim, E.-P., de Oliveira, J.P.M. (eds.) Conceptual Modeling, pp. 162–170. Springer, Cham (2019)
Andaloussi, A.A., Burattin, A., Slaats, T., Petersen, A.C.M., Hildebrandt, T.T., Weber, B.: Exploring the understandability of a hybrid process design artifact based on DCR graphs. In: Reinhartz-Berger, I., Zdravkovic, J., Gulden, J., Schmidt, R. (eds.) Enterprise, Business-Process and Information Systems Modeling, pp. 69–84. Springer, Cham (2019)
Andaloussi, A.A., Burattin, A., Slaats, T., Kindler, E., Weber, B.: On the declarative paradigm in hybrid business process representations: a conceptual framework and a systematic literature study. Inf. Syst. 91, 101505 (2020)
Reichert, M., Weber, B.: Enabling Flexibility in Process-Aware Information Systems. Springer, Berlin (2012)
Zugal, S., Pinggera, J., Weber, B.: The impact of testcases on the maintainability of declarative process models. In: Halpin, T., Nurcan, S., Krogstie, J., Soffer, P., Proper, E., Schmidt, R., Bider, I. (eds.) Enterprise, Business-Process and Information Systems Modeling, pp. 163–177. Springer, Berlin (2011)
De Smedt, J., De Weerdt, J., Vanthienen, J., Poels, G.: Mixed-paradigm process modeling with intertwined state spaces. Bus. Inf. Syst. Eng. 58(1), 19–29 (2016)
Pesic, M., Schonenberg, H., van der Aalst, W.M.P.: DECLARE: full support for loosely-structured processes. In: 11th IEEE International Enterprise Distributed Object Computing Conference (EDOC 2007), pp. 287–287. IEEE (2007)
Debois, S., Hildebrandt, T., Marquard, M., Slaats, T.: Hybrid process technologies in the financial sector. In: Proceedings of the Industry Track at the 13th International Conference on Business Process Management 2015, vol. 4, pp. 107–119 (2015)
Hildebrandt, T.T., Mukkamala, R.R.: Declarative event-based workflow as distributed dynamic condition response graphs. Electron. Proc. Theor. Comput. Sci. 69, 59–73 (2011)
Marquard, M., Shahzad, M., Slaats, T.: Web-based modelling and collaborative simulation of declarative processes. In: Motahari-Nezhad, H.R., Recker, J., Weidlich, M. (eds.) Business Process Management, pp. 209–225. Springer, Cham (2016)
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., van de Weijer, J.: Eye Tracking: A Comprehensive Guide to Methods and Measures. OUP, Oxford (2011)
Bergstrom, J.R., Schall, A.: Eye Tracking in User Experience Design. Elsevier, Amsterdam (2014)
Buscher, G., Dengel, A., Biedert, R., Elst, L.V.: Attentive documents: eye tracking as implicit feedback for information retrieval and beyond. ACM Trans. Interact. Intell. Syst. (TiiS) 1(2), 9 (2012)
Van Der Aalst, W.M.P.: Process Mining: Discovery, Conformance, and Enhancement of Business Processes. Springer, Berlin (2010)
Blascheck, T., Kurzhals, K., Raschke, M., Burch, M., Weiskopf, D., Ertl, T.: Visualization of eye tracking data: a taxonomy and survey. In: Computer Graphics Forum, vol. 36, pages 260–284. Wiley Online Library, London (2017)
Janiszewski, C.: The influence of display characteristics on visual exploratory search behavior. J. Consum. Res. 25(3), 290–301 (1998)
Pfeiffer, J., Meißner, M., Prosiegel, J., Pfeiffer, T.: Classification of goal-directed search and exploratory search using mobile eye-tracking. In: Proceedings of the International Conference on Information Systems (2014)
Maalej, W., Nayebi, M., Johann, T., Ruhe, G.: Toward data-driven requirements engineering. IEEE Softw. 33(1), 48–54 (2016)
Slaats, T.: Declarative and hybrid process discovery: recent advances and open challenges. J. Data Semant. 9, 1–18 (2020)
Pesic, M.: Constraint-based workflow management systems: shifting control to users. Ph.D. thesis, TU Eindhoven (2008)
Fahland, D., Lübke, D., Mendling, J., Reijers, H., Weber, B., Weidlich, M., Zugal, S.: Declarative versus imperative process modeling languages: the issue of understandability. In: Proceedings of EMMSAD, pp. 353–366. Springer, Berlin (2009)
Fahland, D., Mendling, J., Reijers, H.A., Weber, B., Weidlich, M., Zugal, S.: Declarative versus imperative process modeling languages: the issue of maintainability. In: Rinderle-Ma, S., Sadiq, S., Leymann, F. (eds.) Business Process Management Workshops, pp. 477–488. Springer, Berlin (2010)
Miller, G.A.: The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63(2), 81 (1956)
Debois, Søren, Hildebrandt, T., Marquard, M., Slaats, T.: Bridging the valley of death: a success story on danish funding schemes paving a path from technology readiness level 1 to 9. In: Proceedings of the Second International Workshop on Software Engineering Research and Industrial Practice, Piscataway, NJ, USA, pp. 54–57. IEEE Press (2015)
Slaats, T.: Flexible process notations for cross-organizational case management systems. Ph.D. thesis, IT University of Copenhagen (2015)
Debois, S., Hildebrandt, T.T., Marquard, M., Slaats, T.: The DCR graphs process portal. In: BPM (Demos), pp. 7–11 (2016)
Kurzhals, K., Fisher, B., Burch, M., Weiskopf, D.: Eye tracking evaluation of visual analytics. Inf. Vis. 15(4), 340–358 (2016)
Holmqvist, K., Nyström, M., Mulvey, F.: Eye tracker data quality: what it is and how to measure it. In: Proceedings of the Symposium on Eye Tracking Research and Applications, pp. 45–52. ACM, London (2012)
Gulden, J., Burattin, A., Andaloussi, A.A., Weber, B.: From analytical purposes to data visualizations: a decision process guided by a conceptual framework and eye tracking. Softw. Syst. Model. 19, 531–554 (2020)
Kurzhals, K., Fisher, B., Burch, M., Weiskopf, D.: Evaluating visual analytics with eye tracking. In: Proceedings of the Fifth Workshop on Beyond Time and Errors: Novel Evaluation Methods for Visualization, BELIV ’14, pp. 61–69. ACM, New York (2014)
Zhao, F., Schnotz, W., Wagner, I., Gaschler, R.: Eye tracking indicators of reading approaches in text-picture comprehension. Frontline Learn. Res. 2(5), 46–66 (2014)
Zugal, S.: Applying cognitive psychology for improving the creation, understanding and maintenance of business process models. Ph.D. Thesis, University of Innsbruck (2013)
Figl, K.: Comprehension of procedural visual business process models. Bus. Inf. Syst. Eng. 59(1), 41–67 (2017)
Garcia, F., Ruiz, F., Visaggio, C.A.: A proposal and empirical validation of metrics to evaluate the maintainability of software process models. In: 2006 IEEE Instrumentation and Measurement Technology Conference Proceedings, pp. 1093–1097 (2006)
Claes, J., Vanderfeesten, I., Gailly, F., Grefen, P., Poels, G.: The structured process modeling theory (SPMT) a cognitive view on why and how modelers benefit from structuring the process of process modeling. Inf. Syst. Front. 17(6), 1401–1425 (2015)
Zugal, S., Haisjackl, C., Pinggera, J., Weber, B.: Empirical evaluation of test driven modeling. Int. J. Inf. Syst. Model. Des. 4(2), 23–43 (2013)
Duchowski, A.T.: Eye Tracking Methodology: Theory and Practice. Springer, Berlin (2007)
Petrusel, R., Mendling, J.: Eye-tracking the factors of process model comprehension tasks. In: Salinesi, C., Norrie, M.C., Pastor, Ó. (eds.) Advanced Information Systems Engineering, pp. 224–239. Springer, Berlin (2013)
Object Management Group OMG. Business Process Modeling Notation V 2.0. (2006)
Bera, P., Soffer, P., Parsons, J.: Using eye tracking to expose cognitive processes in understanding conceptual models. MIS Q. Manag. Inf. Syst. 43, 1105–1126 (2019)
Keller, G., Nüttgens, M., Scheer, A.W.: Semantische Prozessmodellierung auf der Grundlage “ereignisgesteuerter Prozessketten (EPK)”. Veröffentlichungen des Instituts für Wirtschaftsinformatik. Inst. für Wirtschaftsinformatik, Saarbrücken (1992)
Wang, W., Indulska, M., Sadiq, S., Weber, B.: Effect of linked rules on business process model understanding. In: Business Process Management, pp. 200–215. Springer, Cham (2017)
Sapkota, B., Sinderen, M.V.: Exploiting rules and processes for increasing flexibility in service composition. In: 2010 14th IEEE International Enterprise Distributed Object Computing Conference Workshops, pp. 177–185 (2010)
Zimoch, M., Pryss, R., Schobel, J., Reichert, M.: Eye tracking experiments on process model comprehension: lessons learned. In: Reinhartz-Berger, I., Gulden, J., Nurcan, S., Guédria, W., Bera, P. (eds.) Enterprise, Business-Process and Information Systems Modeling, pp. 153–168. Springer, Cham (2017)
Haisjackl, C., Barba, I., Zugal, S., Soffer, P., Hadar, I., Reichert, M., Pinggera, J., Weber, B.: Understanding declare models: strategies, pitfalls, empirical results. Softw. Syst. Model. 15(2), 325–352 (2016)
Haisjackl, C., Soffer, P., Lim, S.Y., Weber, B.: How do humans inspect BPMN models: an exploratory study. Softw. Syst. Model. 17(2), 655–673 (2018)
Pinggera, J.: The process of process modeling. Ph.D. Thesis, Department of Computer Science (2014)
Pinggera, J., Soffer, P., Fahland, D., Weidlich, M., Zugal, S., Weber, B., Reijers, H.A., Mendling, J.: Styles in business process modeling: an exploration and a model. Softw. Syst. Model. 14(3), 1055–1080 (2015)
Caldiera, V.R.B.G., Rombach, H.D.: The goal question metric approach. In: Encyclopedia of Software Engineering, pp. 528–532 (1994)
Charmaz, K.: Constructing Grounded Theory. Introducing Qualitative Methods Series. SAGE Publications, London (2014)
Richardson, D.C., Dale, R.: Looking to understand: the coupling between speakers’ and listeners’ eye movements and its relationship to discourse comprehension. Cogn. Sci. 29(6), 1045–1060 (2005)
Olsen, A.: The Tobii i-vt Fixation Filter. Tobii Technology, Danderyd (2012)
IEEE Standard for eXtensible Event Stream (XES) for Achieving Interoperability in Event Logs and Event Streams. IEEE Std 1849-2016, pp. 1–50 (2016)
Process mining with the heuristics miner algorithm: Weijters, A.J.M.M., van der Aalst, W.M.P., Alves De Medeiros, A.K. TU/e Tech. Report 166, 1–34 (2006)
Coombes, K.R., Brock, G., Abrams, Z.B.: Polychrome: creating and assessing qualitative palettes with many colors. J. Stat. Softw. (2019)
Gulden, J.: Visually comparing process dynamics with rhythm-eye views. In: BPM Workshops. LNBIP, pp. 474–485. Springer, Berlin (2016)
Saldana, J.: The Coding Manual for Qualitative Researchers. SAGE Publications, London (2012)
Friese, S.: Qualitative Data Analysis with ATLAS ti. SAGE Publications Limited, London (2019)
Ehmke, C., Wilson, S.: Identifying web usability problems from eye-tracking data. In: Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... But Not as We Know It-Volume 1, pp. 119–128. British Computer Society (2007)
Dumas, M., La Rosa, M., Mendling, J., Reijers, H.A.: Fundamentals of Business Process Management. Springer, Berlin (2018)
Glaholt, M.G., Mei-Chun, W., Reingold, E.M.: Evidence for top-down control of eye movements during visual decision making. J. Vis. 10(5), 15–15 (2010)
Burattin, A., Kaiser, M., Neurauter, M., Weber, B.: Learning process modeling phases from modeling interactions and eye tracking data. Data Knowl. Eng. 121, 1–17 (2019)
Ottensooser, A., Fekete, A., Reijers, H.A., Mendling, J., Menictas, C.: Making sense of business process descriptions: an experimental comparison of graphical and textual notations. J. Syst. Softw. 85(3), 596–606 (2012). Novel approaches in the design and implementation of systems/software architecture
Curtis, B., Sheppard, S.B., Kruesi-Bailey, E., Bailey, J., Boehm-Davis, D.A.: Experimental evaluation of software documentation formats. J. Syst. Softw. 9(2), 167–207 (1989)
Roth, W.-M., Bowen, G.M.: When are graphs worth ten thousand words? An expert-expert study. Cogn. Instr. 21(4), 429–473 (2003)
Moher, T.G., Mak, K.H., Blumenthal, B.B., Levanthal, L.M.: Comparing the comprehensibility of textual and graphical programs (1993)
Haisjackl, C., Zugal, S.: Investigating differences between graphical and textual declarative process models. In: Iliadis, L., Papazoglou, M., Pohl, K. (eds.) Advanced Information Systems Engineering Workshops, pp. 194–206. Springer, Cham (2014)
Larkin, J.H., Simon, H.A.: Why a diagram is (sometimes) worth ten thousand words. Cogn. Sci. 11(1), 65–100 (1987)
Moody, D.: The “physics” of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Trans. Softw. Eng. 35(6), 756–779 (2009)
Siau, K.: Informational and computational equivalence in comparing information modeling methods. J. Database Manag. 15, 73–86 (2004)
Paivio, A.: Dual coding theory: retrospect and current status (1991)
Many, C.W.W.N., Sufficient, Y.W.W.: Estimating the number of subjects needed for a thinking aloud test. Int. J. Hum. Comput. Stud. 41, 385–397 (1994)
Costain, G.: Cognitive support during object-oriented software development: the case of UML diagrams. Ph.D. thesis, ResearchSpace@ Auckland (2008)
Kalyuga, S., Ayres, P., Chandler, P., Sweller, J.: The expertise reversal effect. Educ. Psychol. 38(1), 23–31 (2003)
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Jens Gulden and Rainer Schmidt.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Work supported by the Innovation Fund Denmark project EcoKnow (7050-00034A); the fourth author additionally by the Danish Council for Independent Research project Hybrid Business Process Management Technologies (DFF-6111-00337).
Rights and permissions
About this article
Cite this article
Abbad Andaloussi, A., Zerbato, F., Burattin, A. et al. Exploring how users engage with hybrid process artifacts based on declarative process models: a behavioral analysis based on eye-tracking and think-aloud. Softw Syst Model 20, 1437–1464 (2021). https://doi.org/10.1007/s10270-020-00811-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10270-020-00811-8