Measuring the Understandability of Business Process Models - Are We Asking the Right Questions?

  • Ralf Laue
  • Andreas Gadatsch
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 66)


In this paper, we show how experiments on the understandability of business process models can depend on the exact wording used in the experiments’ questionnaires. For this purpose, we partially replicated a published experiment. We asked a group of students a number of questions on relations between tasks in a business process model. Alternatively, we used a set of modified questions which were aimed to ask for exactly the same relations. The result was that there was a significant difference in the number of correct answers between the two systems to construct a question. We argue that a non-negligible part of the wrong answers given in the experiment did not result from problems to understand the model, but rather from problems to understand the question. It follows that it is dangerous to draw conclusions from such an experiment until enough effort has been taken to select appropriate questions.


Business Process Correct Answer Activity Period Wrong Answer Business Process Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lindland, O.I., Sindre, G., Sølvberg, A.: Understanding quality in conceptual modeling. IEEE Softw. 11, 42–49 (1994)CrossRefGoogle Scholar
  2. 2.
    Becker, J., Rosemann, M., Uthmann, C.v.: Guidelines of business process modeling. In: Business Process Management, Models, Techniques, and Empirical Studies, pp. 30–49. Springer, London (2000)CrossRefGoogle Scholar
  3. 3.
    Rittgen, P.: Quality and perceived usefulness of process models. In: SAC 2010: Proceedings of the 2010 ACM Symposium on Applied Computing, pp. 65–72. ACM, New York (2010)Google Scholar
  4. 4.
    Schuman, H., Presser, S.: Questions and Answers in Attitude Surveys. Academic Press, San Diego (1981)Google Scholar
  5. 5.
    Carmines, E., Zeller, R.: Reliability and Validity Assessment. Sage Univ. papers, Thousand Oaks (1979)CrossRefGoogle Scholar
  6. 6.
    Cardoso, J.: Process control-flow complexity metric: An empirical validation. In: IEEE International Conference on Services Computing, pp. 167–173 (2006)Google Scholar
  7. 7.
    Holschke, O., Rake, J., Levina, O.: Granularity as a cognitive factor in the effectiveness of business process model reuse. In: Dayal, U., Eder, J., Koehler, J., Reijers, H.A. (eds.) BPM 2009. LNCS, vol. 5701, pp. 245–260. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  8. 8.
    Aguilar, E.R., Sanchez, L., Carballeira, F.G., Ruiz, F., Piattini, M., Caivano, D., Visaggio, G.: Prediction models for BPMN usability and maintainability. In: 2009 IEEE Conference on Commerce and Enterprise Computing, pp. 383–390 (2009)Google Scholar
  9. 9.
    Sarshar, K., Loos, P.: Comparing the control-flow of EPC and petri net from the end-user perspective. In: van der Aalst, W.M.P., Benatallah, B., Casati, F., Curbera, F. (eds.) BPM 2005. LNCS, vol. 3649, pp. 434–439. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  10. 10.
    Canfora, G., García, F., Piattini, M., Ruiz, F., Visaggio, C.A.: A family of experiments to validate metrics for software process models. J. Syst. Softw. 77, 113–129 (2005)CrossRefGoogle Scholar
  11. 11.
    Lara Proano, M.D.: Visual layout for drawing understandable process models. Master’s thesis, Technische Universiteit Eindhoven (2008)Google Scholar
  12. 12.
    Reijers, H., Mendling, J.: Modularity in process models: Review and effects. In: Dumas, M., Reichert, M., Shan, M.-C. (eds.) BPM 2008. LNCS, vol. 5240, pp. 20–35. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Mendling, J., Reijers, H.A., Cardoso, J.: What makes process models understandable? In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 48–63. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Recker, J., Dreiling, A.: Does it matter which process modelling language we teach or use? In: 18th Australasian Conference on Information Systems (2007)Google Scholar
  15. 15.
    Mendling, J., Strembeck, M.: Influence factors of understanding business process models. In: 11th International Conference, Business Information Systems, BIS 2008, Innsbruck, Austria, pp. 142–153. Springer, Heidelberg (2008)Google Scholar
  16. 16.
    Vanderfeesten, I.T.P., Reijers, H.A., Mendling, J., van der Aalst, W.M.P., Cardoso, J.: On a quest for good process models: The cross-connectivity metric. In: Bellahsène, Z., Léonard, M. (eds.) CAiSE 2008. LNCS, vol. 5074, pp. 480–494. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  17. 17.
    Mayer, R.: Models for understanding. Rev. of Educational Research 59, 43 (1989)CrossRefGoogle Scholar
  18. 18.
    Melcher, J., Seese, D.: Process measurement: Insights from software measurement on measuring process complexity, quality and performance. Technical report, Universität Karlsruhe, TH (2008)Google Scholar
  19. 19.
    Melcher, J., Seese, D.: Towards validating prediction systems for process understandability: Measuring process understandability. In: 10th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, pp. 564–571. IEEE Computer Society, Los Alamitos (2008)Google Scholar
  20. 20.
    Melcher, J., Mendling, J., Reijers, H.A., Seese, D.: On measuring the understandability of process models. In: Revised Papers of the BPM 2009 International Workshops. LNBIP, vol. 43, pp. 465–476. Springer, Ulm (2010)Google Scholar
  21. 21.
    Dwyer, M.B., Avrunin, G.S., Corbett, J.C.: Patterns in property specifications for finite-state verification. In: Proc. of the 21st International Conference on Software Engineering, pp. 411–420. IEEE Computer Society Press, Los Alamitos (1999)Google Scholar
  22. 22.
    Sudman, S., Bradburn, N.M.: Asking Questions: A Practical Guide to Questionnaire Design. Jossey-Bass, San Francisco (1982)Google Scholar
  23. 23.
    Converse, J.M., Presser, S.: Survey Questions: Handcrafting the Standardized Questionnaire. Sage Publications, Thousand Oaks (1986)CrossRefGoogle Scholar
  24. 24.
    Aranda, J., Ernst, N., Horkoff, J., Easterbrook, S.M.: A framework for empirical evaluation of model comprehensibility. In: International Workshop on Modeling in Software Engineering, MiSE 2007 (2007)Google Scholar
  25. 25.
    Patig, S.: A practical guide to testing the understandability of notations. In: Fifth Asia-Pacific Conference on Conceptual Modelling (APCCM 2008). CRPIT, vol. 79, pp. 49–58. Australian Computer Society (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Ralf Laue
    • 1
  • Andreas Gadatsch
    • 2
  1. 1.Chair of Applied Telematics / e-Business Computer Science FacultyUniversity of LeipzigGermany
  2. 2.Bonn-Rhine-Sieg University of Applied SciencesGermany

Personalised recommendations