Towards the Reconstruction and Evaluation of Conceptual Model Quality Discourses – Methodical Framework and Application in the Context of Model Understandability

  • Peter Fettke
  • Constantin Houy
  • Armella-Lucia Vella
  • Peter Loos
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 113)


Within the information systems (IS) discipline conceptual models have gained tremendous importance in the past years. Different approaches for systematic model quality evaluation have emerged. However, these approaches are based on different understandings, definitions as well as operationalizations of the term “model quality”. In this article we refrain from conceptualizing and operationalizing model quality a priori. In contrast, assuming that the determination of model quality and appropriate criteria are negotiated in a discourse between modelers and model users based on their different perspectives, we develop a methodical framework for the critical reconstruction and evaluation of conceptual model quality discourses in order to identify relevant model quality criteria and understandings. Our method is exemplarily applied for the reconstruction of the discourse on the quality criterion model understandability based on relevant laboratory experiments. This application shows that many research results on model understandability are hardly comparable due to their different basic assumptions and should preferably be interpreted based on a methodical reconstruction of underlying understandings.


conceptual models model quality discourse orientation discourse reconstruction model understandability 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Fettke, P.: How Conceptual Modeling Is Used. Communications of the AIS 25, 571–592 (2009)Google Scholar
  2. 2.
    Germonprez, M., Hovorka, D., Gal, U.: Secondary Design: A Case of Behavioral Design Science Research. Journal of the AIS 12, 662–683 (2011)Google Scholar
  3. 3.
    Moody, D.L.: Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions. Data & Knowledge Eng 55, 243–276 (2005)CrossRefGoogle Scholar
  4. 4.
    Hevner, A.R., March, S.T., Park, J., Ram, S.: Design Science in Information Systems Research. MIS Quarterly 28, 75–105 (2004)Google Scholar
  5. 5.
    Frank, U.: Towards a Pluralistic Conception of Research Methods in Information Systems Research. Institut für Informatik und Wirtschaftsinformatik (ICB) der Universität Duisburg-Essen, Essen (2006)Google Scholar
  6. 6.
    Garvin, D.A.: What Does Product Quality Really Mean. Sloan Management Review 26, 25 (1984)Google Scholar
  7. 7.
    Moody, D.L.: Metrics for Evaluating the Quality of Entity Relationship Models. In: Ling, T.-W., Ram, S., Li Lee, M. (eds.) ER 1998. LNCS, vol. 1507, pp. 211–225. Springer, Heidelberg (1998)Google Scholar
  8. 8.
    Krogstie, J., Sindre, G., Jørgensen, H.: Process Models as Knowledge for Action: A Revised Quality Framework. European Journal of Information Systems 15, 91–102 (2006)CrossRefGoogle Scholar
  9. 9.
    Frank, U.: Evaluation of Reference Models. In: Fettke, P., Loos, P. (eds.) Reference Modeling for Business Systems Analysis, pp. 118–140. Idea Group, Hershey (2007)Google Scholar
  10. 10.
    Becker, J., Rosemann, M., von Uthmann, C.: Guidelines of Business Process Modeling. In: van der Aalst, W.M.P., Desel, J., Oberweis, A. (eds.) Business Process Management. LNCS, vol. 1806, pp. 30–49. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  11. 11.
    Janiesch, C., Brelage, C., Holten, R.: Exploration of Conceptual Models - Application of the GoM Framework. In: Information Resources Management Association International Conference (IRMA), San Diego, CA, USA, pp. 254–257 (2005)Google Scholar
  12. 12.
    Ortner, E., Schienmann, B.: Normative Language Approach - A Framework for Understanding. In: Thalheim, B. (ed.) ER 1996. LNCS, vol. 1157, pp. 261–276. Springer, Heidelberg (1996)CrossRefGoogle Scholar
  13. 13.
    Moody, D.L.: Validation of a Method for Representing Large Entity Relationship Models: An Action Research Study. In: Wrycza, S. (ed.) Proceedings of the 10th European Conference on Information Systems (ECIS 2002), Gdansk, Poland, pp. 391–405 (2002)Google Scholar
  14. 14.
    Moody, D.L., Shanks, G.G.: Improving the Quality of Data Models: Empirical Validation of a Quality Management Framework. Information Systems 28, 619–650 (2003)CrossRefGoogle Scholar
  15. 15.
    Sedera, W., Rosemann, M., Gable, G.: Measuring Process Modelling Success. In: Wrycza, S. (ed.) Proceedings of the 10th European Conference on Information Systems (ECIS 2002), Gdansk, Poland, pp. 331–341 (2002)Google Scholar
  16. 16.
    Schütte, R.: Evaluation of information models (in German). In: Heinrich, L.J., Häntschel, I. (eds.) Evaluation und Evaluationsforschung in der Wirtschaftsinformatik - Handbuch für Praxis, Lehre und Forschung, Oldenbourg, München, Wien, pp. 307–382 (2000)Google Scholar
  17. 17.
    Blommaert, J.: Discourse: a Critical Introduction. Cambridge Univ. Press, Cambr. (2005)CrossRefGoogle Scholar
  18. 18.
    Potter, J., Wetherell, M.: Discourse and Social Psychology. Sage, London (2011)Google Scholar
  19. 19.
    Harris, Z., Mattick Jr., P.: Science Sublanguages and the Prospects for a Global Language of Science. Annals of the American Academy of Pol. and Social Science 495, 73–83 (1988)CrossRefGoogle Scholar
  20. 20.
    Halpin, T.: Fact-Oriented Modeling: Past, Present and Future. In: Krogstie, J., Opdahl, A.L., Brinkkemper, S. (eds.) Conceptual Modelling in Information Systems Engineering, pp. 19–38. Springer, New York (2007)CrossRefGoogle Scholar
  21. 21.
    Sindre, G., Opdahl, A.L.: Capturing Dependability Threats in Conceptual Modeling. In: Krogstie, J., Opdahl, A.L., Brinkkemper, S. (eds.) Conceptual Modelling in Information Systems Engineering, pp. 247–260. Springer, New York (2007)CrossRefGoogle Scholar
  22. 22.
    Auramäki, E., Hirschheim, R., Lyytinen, K.: Modelling Offices Through Discourse Analysis: A Comparison and Evaluation of SAMPO with OSSAD and ICN. The Computer Journal 35, 492–500 (1991)CrossRefGoogle Scholar
  23. 23.
    Auramäki, E., Hirschheim, R., Lyytinen, K.: Modelling Offices Through Discourse Analysis: The SAMPO Approach. The Computer Journal 35, 342–352 (1991)CrossRefGoogle Scholar
  24. 24.
    Cimiano, P., Reyle, U., Sanic, J.: Ontology-driven discourse analysis for information extraction. Data & Knowl. Eng. 55, 59–83 (2005)CrossRefGoogle Scholar
  25. 25.
    Ulrich, W.: Critically Systemic Discourse - A Discursive Approach to Reflective Practice in ISD (Part 2). The Journal of Information Technology Theory and Application (JITTA) 3, 85–106 (2001)Google Scholar
  26. 26.
    Ulrich, W.: Philosophical Staircase for Information Systems Definition, Design, and Development - A Discoursive Approach to Reflective Practice in ISD (Part 1). The Journal of Information Technology Theory and Application (JITTA) 3, 55–84 (2001)Google Scholar
  27. 27.
    Winograd, T.: A Language/Action Perspective on the Design of Cooperative Work. Human-Computer Interaction 3, 3–30 (1987-1988)Google Scholar
  28. 28.
    Hoppenbrouwers, S., Proper, H., van der Weide, T.: Formal Modelling as a Grounded Conversation. In: Goldkuhl, G., Lind, M., Haraldson, S. (eds.) Proceedings of the 10th International Working Conference on the Language Action Perspective on Communication Modelling (LAP 2005), Kiruna, Sweden, pp. 139–155 (2005)Google Scholar
  29. 29.
    Cooper, H., Hedges, L.V.: Research Synthesis As a Scientific Enterprise. In: Cooper, H., Hedges, L.V. (eds.) The Handbook of Research Synthesis, pp. 3–14. Sage, New York (1994)Google Scholar
  30. 30.
    Agarwal, R., De, P., Sinha, A.P.: Comprehending Object and Process Models: An Empirical Study. IEEE Trans. Softw. Eng. 25, 541–556 (1999)CrossRefGoogle Scholar
  31. 31.
    Bodart, F., Patel, A., Sim, M., Weber, R.: Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests. Information Systems Research 12, 384–405 (2001)CrossRefGoogle Scholar
  32. 32.
    Burton-Jones, A., Weber, R.: Understanding Relationships with Attributes in Entity-Relationship Diagrams. In: De, P., DeGross, J.I. (eds.) Proceedings of the International Conference on Information Systems (ICIS 1999), pp. 214–228. Charlotte, North Carolina (1999)Google Scholar
  33. 33.
    Burton-Jones, A., Meso, P.N.: Conceptualizing Systems for Understanding: An Empirical Test of Decomposition Principles in Object-Oriented Analysis. Information System Research 17, 38–60 (2006)CrossRefGoogle Scholar
  34. 34.
    Burton-Jones, A., Meso, P.N.: The Effects of Decomposition Quality and Multiple Forms of Information on Novices’ Understanding of a Domain from a Conceptual Model. Journal of the AIS 9, 748–802 (2008)Google Scholar
  35. 35.
    Juhn, S., Naumann, J.: The Effectiveness of Data Representation Characteristics on User Validation. In: Proceedings of the International Conference on Information Systems (ICIS 1985), pp. 212–226 (1985)Google Scholar
  36. 36.
    Palvia, T., Lio, C., To, P.: The Impact of Conceptual Data Models on End User Performance. J. Database Manage. 3, 4–14 (1992)Google Scholar
  37. 37.
    Kim, Y.-G., March, S.T.: Comparing Data Modelling Formalisms. Communications of the ACM 38, 103–115 (1995)CrossRefGoogle Scholar
  38. 38.
    Nordbotten, J.C., Crosby, M.E.: The Effect of Graphic Style on Data Model Interpretation. Information Systems Journal 9, 139–155 (1999)CrossRefGoogle Scholar
  39. 39.
    Moody, D.L.: Complexity Effects on End User Understanding of Data Models: An Experimental Comparison of Large Data Model Representation Methods. In: Wrycza, S. (ed.) Proceedings of the 10th European Conference on Information Systems (ECIS 2002), Gdansk, Poland, pp. 482–496 (2002)Google Scholar
  40. 40.
    Moody, D.L.: Cognitive Load Effects on End User Understanding of Conceptual Models: An Experimental Analysis. In: Benczúr, A.A., Demetrovics, J., Gottlob, G. (eds.) ADBIS 2004. LNCS, vol. 3255, pp. 129–143. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  41. 41.
    Serrano, M., Calero, C., Trujillo, J., Lujan, S., Piattini, M.: Empirical Validation of Metrics for Conceptual Models of Data Warehouses. In: Persson, A., Stirna, J. (eds.) CAiSE 2004. LNCS, vol. 3084, pp. 506–520. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  42. 42.
    Gemino, A., Wand, Y.: Complexity and clarity in conceptual modeling: Comparison of mandatory and optional properties. Data & Knowledge Eng. 55, 301–326 (2005)CrossRefGoogle Scholar
  43. 43.
    Sarshar, K., Loos, P.: Comparing the Control-Flow of EPC and Petri Net from the End-User Perspective. In: van der Aalst, W.M.P., Benatallah, B., Casati, F., Curbera, F. (eds.) BPM 2005. LNCS, vol. 3649, pp. 434–439. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  44. 44.
    Poels, G., Gailly, F., Maes, A., Paemeleire, R.: Object Class or Association Class? Testing the User Effect on Cardinality Interpretation. In: Akoka, J., Liddle, S.W., Song, I.-Y., Bertolotto, M., Comyn-Wattiau, I., van den Heuvel, W.-J., Kolp, M., Trujillo, J., Kop, C., Mayr, H.C. (eds.) ER Workshops 2005. LNCS, vol. 3770, pp. 33–42. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  45. 45.
    Khatri, V., Vessey, I., Ramesh, V., Clay, P., Park, S.-J.: Understanding conceptual schemas: Exploring the role of application and IS domain knowledge. Information Systems Research 17, 81–99 (2006)CrossRefGoogle Scholar
  46. 46.
    Mendling, J., Reijers, H.A., Cardoso, J.: What Makes Process Models Understandable? In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 48–63. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  47. 47.
    Recker, J., Dreiling, A.: Does it matter which process modelling language we teach or use? An experimental study on understanding process modelling languages without formal education. In: Toleman, M., Cater-Steel, A., Roberts, D. (eds.) Proceedings of the 18th Australasian Conference on Inf. Systems, Toowoomba, Australia, pp. 356–366 (2007)Google Scholar
  48. 48.
    Serrano, M., Trujillo, J., Calero, C., Piattini, M.: Metrics for data warehouse conceptual models understandability. Inf. Softw. Technol. 49, 851–870 (2007)CrossRefGoogle Scholar
  49. 49.
    De Lucia, A., Gravino, C., Oliveto, R., Tortora, G.: Data model comprehension an empirical comparison of ER and UML class diagrams. In: IEEE International Conference on Program Comprehension, Amsterdam, pp. 93–102 (2008)Google Scholar
  50. 50.
    Genero, M., Poels, G., Piattini, M.: Defining and validating metrics for assessing the understandability of entity-relationship diagrams. Data & Knowl. Eng. 64, 534–557 (2008)CrossRefGoogle Scholar
  51. 51.
    Mendling, J., Strembeck, M.: Influence Factors of Understanding Business Process Models. In: Abramowicz, W., Fensel, D. (eds.) Business Information Systems. LNBIP, vol. 7, pp. 142–153. Springer, Berlin (2008)CrossRefGoogle Scholar
  52. 52.
    Reijers, H.A., Mendling, J.: Modularity in Process Models: Review and Effects. In: Dumas, M., Reichert, M., Shan, M.-C. (eds.) BPM 2008. LNCS, vol. 5240, pp. 20–35. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  53. 53.
    Vanderfeesten, I., Reijers, H.A., Mendling, J., van der Aalst, W.M.P., Cardoso, J.: On a Quest for Good Process Models: The Cross-Connectivity Metric. In: Bellahsène, Z., Léonard, M. (eds.) CAiSE 2008. LNCS, vol. 5074, pp. 480–494. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  54. 54.
    Recker, J., Dreiling, A.: The Effects of Content Presentation Format and User Characteristics on Novice Developers Understanding of Process Models. Communications of the AIS 28, 65–84 (2011)Google Scholar
  55. 55.
    Reijers, H.A., Mendling, J.: A Study Into the Factors That Influence the Understandability of Business Process Models. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans 41, 449–462 (2011)CrossRefGoogle Scholar
  56. 56.
    Schalles, C., Creagh, J., Rebstock, M.: Usability of Modelling Languages for Model Inter-pretation: An Empirical Research Report. In: Bernstein, A., Schwabe, G. (eds.) Proceedings of the 10th International Conference on Wirtschaftsinformatik, Zurich, Switzerland, pp. 787–796 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Peter Fettke
    • 1
  • Constantin Houy
    • 1
  • Armella-Lucia Vella
    • 1
  • Peter Loos
    • 1
  1. 1.Institute for Information Systems (IWi), at the German Research Center for Artificial Intelligence (DFKI)Saarland UniversitySaarbrückenGermany

Personalised recommendations