Evaluating Quality of Conceptual Models Based on User Perceptions

  • Ann Maes
  • Geert Poels
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4215)


This paper presents the development of a user evaluations based quality model for conceptual modeling applying the model of DeLone and McLean [6] for evaluating information systems in general. Given the growing awareness about the importance of high-quality conceptual models, it is surprising that there is no practical evaluation framework that considers the quality of conceptual models from a user’s perspective. Human factors research in conceptual modeling is still scarce and the perception of quality by model users has been largely ignored. A first research goal is therefore to determine what the appropriate dimensions are for evaluating conceptual models from a user’s perspective. Secondly, we investigate the relationships between these dimensions. Furthermore, we present the results of two experiments with 187 and 124 business students respectively, designed to test the proposed model and the generated hypotheses. The results largely support the developed model and have implications for both theory and practice of quality evaluation of conceptual models.


Conceptual Model Partial Little Square User Satisfaction Conceptual Schema Business Student 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Basili, V., Shull, F., Lanubile, F.: Building Knowledge through Families of Ex-periments. IEEE Transactions on Software Engineering 25(4), 456–473 (1999)CrossRefGoogle Scholar
  2. 2.
    Bodart, F., Patel, A., Sim, M., Weber, R.: Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests. Information Systems Research 12(4), 384–405 (2001)Google Scholar
  3. 3.
    Bollen, K.: Structural Equations with Latent Variables. John Wiley & Sons, NY (1989)MATHGoogle Scholar
  4. 4.
    Burton-Jones, A., Weber, R.: Understanding Relationships with Attributes in Entity-Relationship Diagrams. In: Proc. of the 20th International Conference on Information Systems, pp. 214–228 (1999)Google Scholar
  5. 5.
    Davis, F.: Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13(3), 319–339 (1989)CrossRefGoogle Scholar
  6. 6.
    DeLone, W.H., McLean, E.R.: Information Systems Success: The Quest for the dependent variable. Information Systems Journal 3(1), 60–95 (1992)Google Scholar
  7. 7.
    DeLone, W.H., McLean, E.R.: The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems 19(4), 9–30 (2003)Google Scholar
  8. 8.
    Diamantopoulos, A., Winklhofer, H.M.: Index Construction with Formative Indicators: An Alternative to Scale Development. Journal of Marketing Research 38(2), 269–277 (2001)CrossRefGoogle Scholar
  9. 9.
    Dunn, C.L., Grabski, S.V.: Perceived semantic expressiveness of accounting systems and task accuracy effects. International Journal of Accounting Information Systems 1(2), 79–87 (2000)CrossRefGoogle Scholar
  10. 10.
    Dunn, C.L., Grabski, S.V.: An investigation of localization as an element of cognitive fit in accounting model representations. Decision Science 32(1), 55–94 (2001)CrossRefGoogle Scholar
  11. 11.
    Fornell, C., Larcker, D.F.: Evaluating Structural Equation Models with unobservable variables and measurement error. Journal of marketing research 18(1), 39–50 (1981)CrossRefGoogle Scholar
  12. 12.
    Gemino, A., Wand, Y.: Foundations for Empirical Comparisons of Conceptual Modeling Techniques. In: Batra, D., Parsons, J., Ramesh, E. (eds.) Proc. of the Second Annual Symposium on Research in Systems Analysis and Design, Miami, Florida (2003)Google Scholar
  13. 13.
    Gemino, A., Wand, Y.: Complexity and Clarity in Conceptual Modeling: Comparison of Mandatory and Optional Properties. Data and Knowledge Engineering 55(3), 301–328 (2005)CrossRefGoogle Scholar
  14. 14.
    Hair, J.F., Anderson, R.E., Tatham, R.L.: Multivariate Data Analysis, 2nd edn. (1987)Google Scholar
  15. 15.
    Hulland, John: Use of Partial Least Squares (PLS) in strategic management research: A review of four recent studies. Strategic Management Journal 20(2), 195–204 (1999)CrossRefGoogle Scholar
  16. 16.
    Krogstie, J., Lindland, O.I., Sindre, G.: Defining quality aspects for conceptual models. In: Falkenberg, E.D., Hesse, W., Olive, A. (eds.) Proc. of the 3rd IFIP8.1 Working Conference on Information Systems, Marburg, Germany, pp. 216–231 (1995)Google Scholar
  17. 17.
    Lindland, O.I., Sindre, G., Sølvberg, A.: Understanding Quality in Conceptual Modeling. IEEE Software 11(2), 42–49 (1994)CrossRefGoogle Scholar
  18. 18.
    Moody, D.L.: Dealing with Complexity: A practical Method for representing Large Entity Relationship Models, Doctoral Dissertation, University of Melbourne (2001)Google Scholar
  19. 19.
    Moody, D.L.: Theoretical and Practical Issues in Evaluating the Quality of Conceptual Models: Current state and Future directions. Data and Knowledge Engineering 55(3), 243–276 (2005)CrossRefGoogle Scholar
  20. 20.
    Moore, G.C., Benbasat, I.: Development of an Instrument to Measure the Perceptions of Adopting and Information Technology Innovation. Information Systems Research 2(3), 192–222 (1991)CrossRefGoogle Scholar
  21. 21.
    Nelson, R.R., Todd, P.A., Wixom, B.H.: Antecedents of Information and System Quality: An Empirical Examination Within the Context of Data Warehousing. Journal of Management Information Systems 21(4), 199–235 (2005)Google Scholar
  22. 22.
    Parsons, J., Cole, L.: What do the pictures mean? Guidelines for the experimental evaluation of representation fidelity in diagrammatical conceptual modeling techniques. Data & Knowledge Engineering 55(3), 327–342 (2005)CrossRefGoogle Scholar
  23. 23.
    Poels, G., Nelson, J., Genero, M., Piattini, M.: Quality in Conceptual Modeling. New Research Directions. In: Olivé, À., Yoshikawa, M., Yu, E.S.K. (eds.) ER 2003. LNCS, vol. 2784, pp. 243–250. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  24. 24.
    Poels, G., Maes, A., Gailly, F., Paemeleire, R.: Measuring the Perceived Semantic Quality of Information Models. In: Akoka, J., Liddle, S.W., Song, I.-Y., Bertolotto, M., Comyn-Wattiau, I., van den Heuvel, W.-J., Kolp, M., Trujillo, J., Kop, C., Mayr, H.C. (eds.) ER Workshops 2005. LNCS, vol. 3770, pp. 376–385. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  25. 25.
    Rai, A., Lang, S.S., Welker, R.B.: Assessing the validity of IS success models: An empirical test and theoretical analysis. Information Systems Research 13(1), 50–69 (2002)CrossRefGoogle Scholar
  26. 26.
    Seddon, P.: A Respecification and Extension of the DeLone and McLean Model of IS Success. Information Systems Research 8(3), 240–253 (1997)CrossRefGoogle Scholar
  27. 27.
    Seddon, P., Kiew, M.-Y.: A partial test and development of the DeLone and McLean model of IS success. In: DeGross, J.I., Huff, S.L., Munro, M.C. (eds.) Proceedings of the International Conference on Information Systems, Atlanta, pp. 99–110 (1994)Google Scholar
  28. 28.
    Seddon, P., Yip, S.-K.: An Empirical Evaluation of User Information Satisfaction (UIS) Measures for Use with General Ledger Accounting Software. Journal of Information Systems 6(1), 75–92 (1992)Google Scholar
  29. 29.
    Shanks, G., Tansley, E., Weber, R.: Using ontology to validate conceptual models. Communications of the ACM 46(10), 85–89 (2003)CrossRefGoogle Scholar
  30. 30.
    Shannon, C.E., Weaver, W.: The Mathematical theory of Communication. University of Illinois Press, Urbana (1949)MATHGoogle Scholar
  31. 31.
    Siau, K., Wand, Y., Benbasat, I.: The Relative Importance of Structural Constraints and Surface Semantics in Information Modeling. Information Systems 22(2/3), 155–170 (1997)CrossRefGoogle Scholar
  32. 32.
    Topi, H., Ramesh, V.: Human Factors Research on Data Modeling: A Review of Prior Research, An Extended Framework and Future Research Directions. Journal of Database Management 13(2), 3–19 (2002)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Ann Maes
    • 1
  • Geert Poels
    • 1
  1. 1.Faculty of Economics and Business AdministrationGhent UniversityGhentBelgium

Personalised recommendations