Multiperspective Evaluation of Reference Models – Towards a Framework

  • Peter Fettke
  • Peter Loos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2814)


Within the information systems field, reference models have been known for many years. Despite the relevance of reference model quality, little research has been done on their systematic evaluation. Based on an analysis of prior work on reference model quality, we propose a framework for the multiperspective evaluation of reference models. The framework comprises 15 perspectives. Each perspective is discussed with respect to its strengths and limitations. As well, we provide examples of the types of research that have already been undertaken on each perspective.


Reference Model Unify Modeling Language Information System Research Information System Development Capability Maturity Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Frank, U.: Conceptual Modelling as the Core of the Information Systems Discipline – Perspectives and Epistemological Challenges. In: Proceedings of the Fifth Americas Conference on Information Systems (AMCIS 1999), August 13-15, pp. 695–697 (1999)Google Scholar
  2. 2.
    Mylopoulos, J.: Information Modeling in the Time of the Revolution. Information Systems 23, 127–155 (1998)CrossRefGoogle Scholar
  3. 3.
    Wand, Y., Weber, R.: Research Commentary: Information Systems and Conceptual Modelling – A Research Agenda. Information Systems Research 13, 363–377 (2002)CrossRefGoogle Scholar
  4. 4.
    Scheer, A.-W., Hars, A.: Extending Data Modeling to Cover the Whole Enterprise. Communications of the ACM 35, 166–172 (1992)CrossRefGoogle Scholar
  5. 5.
    Mertins, K., Bernus, P.: Reference Models. In: Bernus, P., Mertins, K., Schmidt, G. (eds.) Handbook on Architectures of Information Systems, pp. 615–617. Springer, Heidelberg (1998)Google Scholar
  6. 6.
    Mišic, V.B., Zhao, J.L.: Evaluating the Quality of Reference Models. In: Laender, A.H.F., Liddle, S.W., Storey, V.C. (eds.) ER 2000. LNCS, vol. 1920, pp. 484–498. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Scheer, A.-W., Nüttgens, M.: ARIS Architecture and Reference Models for Business Process Management. In: Aalst, W., Desel, J., Oberweis, A. (eds.) Business Process Management – Models, Techniques, and Empirical Studies, pp. 376–389. Springer, Heidelberg (2000)Google Scholar
  8. 8.
    Scheer, A.-W.: Business Process Engineering – Reference Models for Industrial Companies, 2nd edn. Springer, Berlin (1994)Google Scholar
  9. 9.
    Schütte, R.: Architectures for Evaluating the Quality of Information Models – A Meta and Object Level Comparison. In: Akoka, J., Bouzeghoub, M., Comyn-Wattiau, I., Métais, E. (eds.) ER 1999. LNCS, vol. 1728, pp. 490–505. Springer, Heidelberg (1999)Google Scholar
  10. 10.
    Bernus, P., Mertins, K., Schmidt, G.: Handbook on Architectures of Information Systems. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  11. 11.
    Fettke, P., Loos, P.: Classification of reference models – a methodology and its application. Information Systems and e-Business Management 1, 35–53 (2003)CrossRefGoogle Scholar
  12. 12.
    Vessey, I., Ramesh, V., Glass, R.L.: A Unified Classification System for Research in the Computing Disciplines. Available:
  13. 13.
    Siau, K., Rossi, M.: Evaluating of Information Modeling Methods – A Review. In: Proceedings of the 31th Hawaii International Conference on Systems Science, HICSS 1998 (1998) Google Scholar
  14. 14.
    Moody, D.L., Shanks, G.G.: Improving the quality of data models: empirical validation of a quality management framework. Information Systems 28, 619–650 (2003)zbMATHCrossRefGoogle Scholar
  15. 15.
    Mišic, V. B., Zhao, J. L.: Reference Models for Electronic Commerce (2003), Available
  16. 16.
    Wisse, P.: Metapattern – Context and Time in Information Models. Addison-Wesley, Reading (2001)Google Scholar
  17. 17.
    Rising, L.: The Pattern Almanac 2000. Addison-Wesley, Reading (2000)Google Scholar
  18. 18.
    Schütte, R.: Grundsätze ordnungsmäßiger Referenzmodellierung – Konstruktion konfigurations- und anpassungsorientierter Modelle. Gabler, Wiesbaden (1998)Google Scholar
  19. 19.
    Schuette, R., Rotthowe, T.: The Guidelines of Modeling – An Approach to Enhance the Quality in Information Models. In: Ling, T.-W., Ram, S., Li Lee, M. (eds.) ER 1998. LNCS, vol. 1507, pp. 240–254. Springer, Heidelberg (1998)Google Scholar
  20. 20.
    Lindland, O.I., Sindre, G., Sølvberg, A.: Understanding Quality in Conceptual Modeling. IEEE Software, 42–49 (1994)Google Scholar
  21. 21.
    Krogstie, J.: Conceptual Modeling for Computerized Information Systems Support in Organizations. University of Trondheim (1995)Google Scholar
  22. 22.
    Daneva, M., Scheer, A.-W.: Benchmarking Business Process Models. Institut für Wirtschaftsinformatik, Universität des Saarlandes, Arbeitsbericht 136 Saarbrücken (1996)Google Scholar
  23. 23.
    Moody, D.L.: Metrics for Evaluating the Quality of Entity Relationship Models. In: Ling, T.-W., Ram, S., Li Lee, M. (eds.) ER 1998. LNCS, vol. 1507, pp. 211–225. Springer, Heidelberg (1998)Google Scholar
  24. 24.
    Mühlen zur, M.: Evaluation of Workflow Management Systems Using Meta Models. In: Proceedings of the 32th Hawaii International Conference on Systems Science, HICSS 1999 (1999)Google Scholar
  25. 25.
    Paulk, M.C., Curtis, B., Chrissis, M.B., Weber, C.V.: Capability Maturity Model for Software, Version 1.1. Software Engineering Institute – Carnegie Mellon University, CMU/SEI-93-TR-024 Pittsburgh, Pennsylvania (1993) Google Scholar
  26. 26.
    Klein, H.K., Lyytinen, K.: Towards a New Understanding of Data Modelling. In: Floyd, C., Züllighoven, H., Budde, R., Keil-Slawik, R. (eds.) Software Development and Reality Construction, pp. 203–219. Springer, Heidelberg (1992)Google Scholar
  27. 27.
    Hirschheim, R., Klein, H.K., Lyytinen, K.: Information Systems Development and Data Modeling – Conceptual and Philosophical Foundations. Press Syndicate for the University of Cambridge, Cambridge (1995)Google Scholar
  28. 28.
    Loos, P.: Produktionslogistik in der chemischen Industrie – Betriebstypologische Merkmale und Informationsstrukturen. Gabler, Wiesbaden (1997)Google Scholar
  29. 29.
    Malone, T.W., Crowston, K., Lee, J., Pentland, B., Dellarocas, C., Wyner, G., Quimby, J., Osborn, C.S., Bernstein, A., Herman, G., Klein, M., O’Donnell, E.: Tools for inventing organizations. Toward a handbook of organizational processes. Management Science 45, 425–443 (1999)Google Scholar
  30. 30.
    Bunge, M.: Ontology I: The Furniture of the World. D. Reidel, Dordrecht (1977)zbMATHGoogle Scholar
  31. 31.
    Wand, Y., Monarchi, D.E., Parsons, J., Woo, C.C.: Theoretical foundations for conceptual modelling in information systems development. Decision Support Systems 15, 285–304 (1995)CrossRefGoogle Scholar
  32. 32.
    Fettke, P., Loos, P.: Ontological evaluation of reference models using the Bunge-Wand-Weber-model. In: Americas Conference on Information Systems (2003) (accepted) Google Scholar
  33. 33.
    Opdahl, A.L., Henderson-Sellers, B.: Ontological Evaluation of the UML Using the Bunge-Wand-Weber Model. Software and Systems Modeling 1, 43–67 (2002)Google Scholar
  34. 34.
    Bodart, F., Patel, A., Sim, M., Weber, R.: Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests. Information Systems Research 12, 384–405 (2001)CrossRefGoogle Scholar
  35. 35.
    Milton, S., Kazmierczak, E., Thomas, L.: Ontological Foundations of Data Modeling in Information Systems. In: Proceedings of the Sixth Americas Conference on Information Systems (AMCIS 2000), August 10-13, pp. 1537–1543 (2000)Google Scholar
  36. 36.
    Siau, K.: Information Modeling and Method Engineering: A Psychological Perspective. Journal of Database Management 10, 44–50 (1999)Google Scholar
  37. 37.
    Kim, J., Hahn, J., Hahn, H.: How Do We Unterstand a System with (So) Many Diagrams? Cognitive Integration Processes in Diagrammatic Reasoning. Information Systems Research 11, 284–303 (2000)CrossRefGoogle Scholar
  38. 38.
    Maier, R.: Qualität von Datenmodellen. Gabler, Wiesbaden (1996)Google Scholar
  39. 39.
    Prechelt, L., Unger, B., Tichy, W.F., Brössler, P., Votta, L.G.: A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions. IEEE Transactions on Software Engineering 27, 1134–1144 (2001)CrossRefGoogle Scholar
  40. 40.
    Burton-Jones, A., Meso, P.: How Good are these UML Diagrams? An Empirical Test of the Wand and Weber Good Decomposition Model. In: Twenty-Third International Conference on Information Systems, pp. 101–114 (2002)Google Scholar
  41. 41.
    Kim, Y.-G., March, S.T.: Comparing data modeling formalisms. Communications of the ACM 38, 103–115 (1995)CrossRefGoogle Scholar
  42. 42.
    Shanks, G., Tansley, E., Nuredini, J., Tobin, D., Weber, R.: Representing Part-Whole Relationships in Conceptual Modeling: An Empirical Evaluation. In: Twenty-Third International Conference on Information Systems, pp. 89–100 (2002)Google Scholar
  43. 43.
    Weber, R.: Are Attributes Entities? A Study of Database Designer’s Memory Structures. Information Systems Research 7, 137–162 (1996)CrossRefGoogle Scholar
  44. 44.
    Schwegmann, A.: Objektorientierte Referenzmodellierung – Theoretische Grundlagen und praktische Anwendung. DUV, Wiesbaden (1999)Google Scholar
  45. 45.
    Buchwalter, J.: Elektronische Ausschreibungen in der Beschaffung – Referenzprozeßmodell und prototypische Realisierung. Eul, Lohmar, Köln (2002)Google Scholar
  46. 46.
    Becker, J., Kugeler, M., Rosemann, M.: Process Management. Springer, Heidelberg (2003)Google Scholar
  47. 47.
    Avison, D.E., Lau, F., Myers, M.D., Nielsen, P.A.: Action Research. Communications of the ACM 42, 94–97 (1999)CrossRefGoogle Scholar
  48. 48.
    Moody, D.L., Shanks, G.G.: Improving the Quality of Entity Relationship Models: An Action Research Programme. The Australian Computer Journal 30, 129–138 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Peter Fettke
    • 1
  • Peter Loos
    • 1
  1. 1.Lehrstuhl für Wirtschaftsinformatik und Betriebswirtschaftslehre, ISYM – Information Systems & ManagementJohannes Gutenberg-University MainzMainzGermany

Personalised recommendations