Instantiation Validity in IS Design Research

  • Roman Lukyanenko
  • Joerg Evermann
  • Jeffrey Parsons
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8463)


Studies that involve information technology artifacts play a prominent role in Information Systems (IS) research. We argue that special attention needs to be paid to ensuring the validity of such studies. This paper makes three contributions to IS research. First, it introduces the concept of instantiation validity as broadly applicable to IS design research, and distinct from existing notions of validity. Second, the paper identifies several sources of instantiation validity threats that can arise in IS design research. Third, it points to the need for guidelines to address these threats and demonstrate validity in design research.


IS research validity design science instantiation validity 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kamis, A., Koufaris, M., Stern, T.: Using an Attribute-Based Decision Support System for User-Customized Products Online: An Experimental Investigation. MIS Quarterly 32, 159–177 (2008)Google Scholar
  2. 2.
    Komiak, S.Y.X., Benbasat, I.: The Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents. MIS Quarterly 30, 941–960 (2006)Google Scholar
  3. 3.
    Gregor, S.: The Nature of Theory in Information Systems. MIS Quarterly 30, 611–642 (2006)Google Scholar
  4. 4.
    Weber, R.: Evaluating and Developing Theories in the Information Systems Discipline. Journal of the Association for Information Systems 13, 1–30 (2012)Google Scholar
  5. 5.
    Gregor, S., Jones, D.: The Anatomy of Design Theory. Journal of the Association for Information Systems 8, 312–335 (2007)Google Scholar
  6. 6.
    Gregor, S., Hevner, A.R.: Positioning and Presenting Design Science Research for Maximum Impact. MIS Quarterly 37, 337–355 (2013)Google Scholar
  7. 7.
    Straub, D., Boudreau, M., Gefen, D.: Validation Guidelines for IS Positivist Research. Communications of the Association for Information Systems 13, 380–427 (2004)Google Scholar
  8. 8.
    Venkatesh, V., Brown, S.A., Bala, H.: Bridging the Qualitative-Quantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems. MIS Quarterly 37, 21–54 (2013)Google Scholar
  9. 9.
    Cook, T.D., Campbell, D.T., Peraccio, L.: Quasi-experimentation. In: Dunnette, M., Hough, L. (eds.) Handbook of Industrial and Organizational Psychology, vol. 1, pp. 491–576. Consulting Psychologists Press, Palo Alto (1990)Google Scholar
  10. 10.
    Benbasat, I.: Laboratory experiments in information systems studies with a focus on individuals: A critical appraisal, vol. 2. Harvard Business School, Cambridge (1989)Google Scholar
  11. 11.
    March, S.T., Smith, G.F.: Design and Natural Science Research on Information Technology. Decision Support Systems 15, 251–266 (1995)CrossRefGoogle Scholar
  12. 12.
    Gregg, D.G., Kulkarni, U.R., Vinzé, A.S.: Understanding the Philosophical Underpinnings of Software Engineering Research in Information Systems. Information Systems Frontiers 3, 169–183 (2001)CrossRefGoogle Scholar
  13. 13.
    Hevner, A., March, S., Park, J., Ram, S.: Design Science in Information Systems Research. MIS Quarterly 28, 75–105 (2004)Google Scholar
  14. 14.
    Parsons, J., Cole, L.: What do the Pictures Mean? Guidelines for Experimental Evaluation of Representation Fidelity in Diagrammatical Conceptual Modeling Techniques. Data & Knowledge Engineering 55, 327–342 (2005)Google Scholar
  15. 15.
    Burton-Jones, A., Wand, Y., Weber, R.: Guidelines for Empirical Evaluations of Conceptual Modeling Grammars. Journal of the Association for Information Systems 10, 495–532 (2009)Google Scholar
  16. 16.
    Shanks, G., Weber, R.: A Hole in the Whole: A Response to Allen and March. MIS Quarterly 36, 965–980 (2012)Google Scholar
  17. 17.
    Allen, G.N., March, S.T.: A Research Note on Representing Part-Whole Relations in Conceptual Modeling. MIS Quarterly 36, 945–964 (2012)Google Scholar
  18. 18.
    Arazy, O., Kumar, N., Shapira, B.: A Theory-Driven Design Framework for Social Recommender Systems. Journal of the Association for Information Systems 11, 455–490 (2010)Google Scholar
  19. 19.
    Hovorka, D., Gregor, S.: Untangling Causality in Design Science Theorizing. In: 5th Biennial ANU Workshop on Information Systems Foundations (2010)Google Scholar
  20. 20.
    Brewer, M.B.: Research design and issues of validity. In: Reis, H., Judd, C. (eds.) Handbook of Research Methods in Social and Personality Psychology, pp. 3–16. Cambridge University Press, Cambridge (2000)Google Scholar
  21. 21.
    Straub, D.W.: Validating Instruments in MIS Research. MIS Quarterly 13, 147–169 (1989)CrossRefGoogle Scholar
  22. 22.
    Churchill, G.A.: A Paradigm for Developing Better Measures of Marketing Constructs. Journal of Marketing Research, 64–73 (1979)Google Scholar
  23. 23.
    Hinkin, T.R.: A Review of Scale Development Practices in the Study of Organizations. Journal of Management 21, 967–988 (1995)CrossRefGoogle Scholar
  24. 24.
    Moore, G.C., Benbasat, I.: Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation. Information Systems Research 2, 192–222 (1991)CrossRefGoogle Scholar
  25. 25.
    Snodgrass, J.G., Vanderwart, M.: A Standardized Set of 260 Pictures: Norms for Name Agreement, Image Agreement, Familiarity, and Visual Complexity. Journal of Experimental Psychology: Human Learning and Memory 6, 174–215 (1980)Google Scholar
  26. 26.
    Kuechler, W., Vaishnavi, V.: A Framework for Theory Development in Design Science Research: Multiple Perspectives. Journal of the Association for Information Systems 13, 395–423 (2012)Google Scholar
  27. 27.
    Lukyanenko, R., Parsons, J.: Reconciling Theories with Design Choices in Design Science Research. In: vom Brocke, J., Hekkala, R., Ram, S., Rossi, M. (eds.) DESRIST 2013. LNCS, vol. 7939, pp. 165–180. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  28. 28.
    Merton, R.: On Sociological Theories of the Middle Range. In: Merton, R. (ed.) Social Theory and Social Structure, pp. 39–53. The Free Press, New York (1949)Google Scholar
  29. 29.
    Shepherd, M.M., Briggs, R.O., Reinig, B.A., Yen, J., Nunamaker, J., Jay, F.: Invoking Social Comparison to Improve Electronic Brainstorming: Beyond Anonymity. Journal of Management Information Systems 12, 155–170 (1995)Google Scholar
  30. 30.
    Briggs, R.O., Nunamaker, J., Jay, F., Sprague, J., Ralph, H.: 1001 Unanswered Research Questions in GSS. Journal of Management Information Systems 14, 3–21 (1997)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Roman Lukyanenko
    • 1
  • Joerg Evermann
    • 1
  • Jeffrey Parsons
    • 1
  1. 1.Faculty of Business AdministrationMemorial University of NewfoundlandSt. John’sCanada

Personalised recommendations