Quality Assessment of Business Process Models Based on Thresholds

  • Laura Sánchez-González
  • Félix García
  • Jan Mendling
  • Francisco Ruiz
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6426)


Process improvement is recognized as the main benefit of process modelling initiatives. Quality considerations are important when conducting a process modelling project. While the early stage of business process design might not be the most expensive ones, they tend to have the highest impact on the benefits and costs of the implemented business processes. In this context, quality assurance of the models has become a significant objective. In particular, understandability and modifiability are quality attributes of special interest in order to facilitate the evolution of business models in a highly dynamic environment. These attributes can only be assessed a posteriori, so it is of central importance for quality management to identify significant predictors for them. A variety of structural metrics have recently been proposed, which are tailored to approximate these usage characteristics. The aim of this paper is to verify how understandable and modifiable BPMN models relate to these metrics by means of correlation and regression analyses. Based on the results we determine threshold values to distinguish different levels of process model quality. As such threshold values are missing in prior research, we expect to see strong implications of our approach on the design of modelling guidelines.


Business process measurement correlation analysis regression analysis BPMN 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pfleeger, S.L.: Integrating Process and Measurement. In: Melton, A. (ed.) Software Measurement, pp. 53–74. International Thomson Computer Press (1996)Google Scholar
  2. 2.
    Rosemann, M.: Potential pitfalls of process modeling: part A. Business Process Management Journal 12(2), 249–254 (2006)CrossRefGoogle Scholar
  3. 3.
    Indulska, M., Green, P., Recker, J., Rosemann, M.: Business Process Modeling: Perceived Benefits. In: Laender, A.H.F. (ed.) ER 2009. LNCS, vol. 5829, pp. 458–471. Springer, Heidelberg (2009)Google Scholar
  4. 4.
    Moody, D.: Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions. Data and Knowledge Engineering, 55, 243–276 (2005)CrossRefGoogle Scholar
  5. 5.
    Mylopoulos, J.: Conceptual modeling and telos. In: Conceptual Modeling, Databases, and Case: an Integrated View of Information Systems Development, ch. 2, pp. 49–68 (1992)Google Scholar
  6. 6.
    Dandekar, A., Perry, D.E., Votta, L.G.: Studies in Process Simplification. In: Proceedings of the Fourth International Conference on the Software Process, pp. 27–35 (1996)Google Scholar
  7. 7.
    ISO/IEC, ISO Standard 9000-2000: Quality Management Systems: Fundamentals and Vocabulary (2000)Google Scholar
  8. 8.
    ISO/IEC, 9126-1, Software engineering - product quality - Part 1: Quality Model (2001)Google Scholar
  9. 9.
    Sánchez González, L., García, F., Ruiz, F., Piattini, M.: Measurement in Business Processes: a Systematic Review. Business Process Management Journal 16(1), 114–134 (2010)CrossRefGoogle Scholar
  10. 10.
    Zelkowitz, M., Wallace, D.: Esperimental models for validating technology. IEEE Computer, Computing practices (1998)Google Scholar
  11. 11.
    Mendling, J.: Metrics for Process Models: Empirical Foundations of Verification, Error Prediction, and Guidelines for Correctness. Springer Publishing Company, Incorporated, Heidelberg (2008)CrossRefGoogle Scholar
  12. 12.
    OMG. Business Process Modeling Notation (BPMN), Final Adopted Specification (2006),
  13. 13.
    Gilmore, D., Green, T.: Comprehension and Recall of miniature programs. International Jounal of Man-Machine Studies archive 21(1), 31–48 (1984)CrossRefGoogle Scholar
  14. 14.
    Rittgen, P.: Negotiating Models. In: Krogstie, J., Opdahl, A.L., Sindre, G. (eds.) CAiSE 2007 and WES 2007. LNCS, vol. 4495, pp. 561–573. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Rosemann, M.: Potential pitfalls of process modeling: part B. Business Process Management Journal 12(3), 377–384 (2006)CrossRefGoogle Scholar
  16. 16.
    Mendling, J., Reijers, H.A., Cardoso, J.: What makes process models understandable? In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 48–63. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Green, T.R.G., Petre, M.: Usability analysis of visual programming environments: a cognitive dimensions framework. J. Visual Languages and Computing 7, 131–174 (1996)CrossRefGoogle Scholar
  18. 18.
  19. 19.
    Foss, T., Stensrud, E., Kitchenham, B., Myrtveit, I.: A Simulation Study of the Model Evaluation Criterion MMRE. IEEE Transactions on Software Engineering 29, 985–995 (2003)CrossRefGoogle Scholar
  20. 20.
    Kitchenham, B., Charters, S.: Guidelines for performing systematic literature reviews in software engineering, T. report, Editor, Keele University and University of Durham (2007)Google Scholar
  21. 21.
    Cardoso, J.: Process control-flow complexity metric: An empirical validation. In: SCC 2006: Proceedings of the IEEE International Conference on Services Computing, pp. 167–173 (2006)Google Scholar
  22. 22.
    Rolón, E., García, F., Ruiz, F.: Evaluation Measures for Business Process Models. In: Symposium in Applied Computing SAC 2006 (2006)Google Scholar
  23. 23.
    Rolón, E., Cardoso, J., García, F., Ruiz, F., Piattini, M.: Analysis and Validation of Control-Flow Complexity Measures with BPMN Process Models. In: The 10th Workshop on Business Process Modeling, Development, and Support (2009)Google Scholar
  24. 24.
    Rolón, E., Ruiz, F., García, F., Piattini, M.: Applying Software Process Metrics in Business Process. Procesos y Métricas, Asociación Española de Métricas del Software 3(2) (2006)Google Scholar
  25. 25.
    Rolon, E., Sanchez, L., Garcia, F., Ruiz, F., Piattini, M., Caivano, D., Visaggio, G.: Prediction Models for BPMN Usability and Maintainability. In: BPMN 2009 - 1st International Workshop on BPMN, pp. 383–390 (2009)Google Scholar
  26. 26.
    Vanderfeesten, I., Reijers, H.A., van der Aalst, W.M.P.: Evaluating Workflow Process Designs using Cohesion and Coupling Metrics. In: Computer in Industry (2008)Google Scholar
  27. 27.
    Vanderfeesten, I., Reijers, H.A., Mendling, J., van der Aalst, W.M.P., Cardoso, J.: On a Quest for Good Process models: the Cross Conectivity Metric. In: Bellahsène, Z., Léonard, M. (eds.) CAiSE 2008. LNCS, vol. 5074, pp. 480–494. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  28. 28.
    Mendling, J.: Testing Density as a complexity Metric for EPCs, in Technical Report JM-2006-11-15 (2006)Google Scholar
  29. 29.
    Cardoso, J.: How to Measure the Control-Flow Complexity of Web Processes and Workflows. In: Workflow Handbook 2005 (2005)Google Scholar
  30. 30.
    Cardoso, J.: Business Process Quality Metrics: Log-based Complexity of Workflow Patterns. In: Meersman, R., Tari, Z. (eds.) OTM 2007, Part I. LNCS, vol. 4803, pp. 427–434. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  31. 31.
    Jung, J.Y.: Measuring entropy in business process models. In: International Conference on Innovative Computing, Information and Control, pp. 246–252 (2008)Google Scholar
  32. 32.
    Latva-Koivisto, A.M.: Finding a Complexity Measure for Business Process Models. Individual Research Projects in Applied Mathematics (2001)Google Scholar
  33. 33.
    Gruhn, V., Laue, R.: Complexity Metrics for business Process Models. In: International Conference on Business Information Systems (2006)Google Scholar
  34. 34.
    Gruhn, V., Laue, R.: Adopting the Cognitive Complexity Measure for Business Process Models. In: 5th IEEE International Conference on Cognitive Informatics, ICCI 2006, vol. 1, pp. 236–241 (2006)Google Scholar
  35. 35.
    Rozinat, A., van der Aalst, W.M.P.: Conformance checking of processes based on monitoring real behavior. Information Systems 33, 64–95 (2008)CrossRefGoogle Scholar
  36. 36.
    Laue, R., Mendling, J.: Structuredness and its Significance for Correctness of Process Models. In: Information Systems and E-Business Management (2009)Google Scholar
  37. 37.
    Meimandi Parizi, R., Ghani, A.A.A.: An Ensemble of Complexity Metrics for BPEL Web Processes. In: Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, pp. 753–758 (2008)Google Scholar
  38. 38.
    Bisgaard Lassen, K., Van der Aalst, W.: Complexity Metrics for Workflow Nets. In: Information and Software Technology, pp. 610–626 (2008)Google Scholar
  39. 39.
    Huan, Z., Kumar, A.: New quality metrics for evaluating process models. In: Business Process Intelligence Workshop (2008)Google Scholar
  40. 40.
    Henderson-Sellers, B.: Object-Oriented Metrics: Measures of Complexity. Prentice-Hall, Englewood Cliffs (1996)Google Scholar
  41. 41.
    Shatnawi, R., Li, W., Swain, J., Newman, T.: Finding Software Metrics Threshold values using ROC Curves. In: Sofware Maintenance and Evolution: Research and Practice (2009)Google Scholar
  42. 42.
    Churchill, G.A., Doerge, R.W.: Empirical Threshold Values for Quantitative Trait Mapping. Genetics Society of America 138, 963–971 (1995)Google Scholar
  43. 43.
    Bender, R.: Quantitative Risk Assessment in Epidemiological Studies Investigatin Threshold Effects. Biometrical Journal 41(3), 305–319 (1999)CrossRefzbMATHGoogle Scholar
  44. 44.
    Royston, P., Douglas, G.A., Sauerbrei, W.: Dichotomizing continuous predictors in multiple regression: a bad idea. Statistics in Medicine 25, 127–141 (2005)MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Laura Sánchez-González
    • 1
  • Félix García
    • 1
  • Jan Mendling
    • 2
  • Francisco Ruiz
    • 1
  1. 1.Grupo AlarcosUniversidad de Castilla La ManchaCiudad RealEspaña
  2. 2.Humboldt-Universität zu BerlinBerlinGermany

Personalised recommendations