Journal of Information Technology

, Volume 30, Issue 1, pp 30–43 | Cite as

Generalization in IS research: a critique of the conflicting positions of Lee & Baskerville and Tsang & Williams

Research Article

Abstract

This paper is a companion to the paper on generalization in IS research by Williams and Tsang published in this edition of the Journal of Information Technology. Its purpose is to discuss the implications of the robust exchange of views about the meaning of the term ‘generalization’ in four papers, two by Lee and Baskerville, and two by Tsang and Williams. The objectives of this paper are, first, to help the reader understand the issues by summarizing the arguments in the four papers, and second, to assess the implications of the debate for future IS research. Our conclusion is that when the papers are interpreted from the perspectives of the respective pairs of authors, most of what they say is sound. However, because their perspectives are so different, their differences of opinion are also very real. As a way of showing that neither pair of authors’ conception of generalization is the ‘last word’ on this topic, the paper also compares key concepts from both pairs of authors with those from Seddon and Scheepers. It is argued that although the Seddon and Scheepers’ framework is also not the ‘last word’, it may prove more useful than either of the two preceding frameworks.

Keywords

generalization IS research induction knowledge claim representativeness other settings 

References

  1. Archer, M., Bhaskar, R., Collier, A., Lawson, T. and Norrie, A. (1998). Critical Realism: Essential readings, Milton Park: Routledge.Google Scholar
  2. Ariño, A. and Ring, P.S. (2010). The Role of Fairness in Alliance Formation, Strategic Management Journal 31 (10): 1054–1087.CrossRefGoogle Scholar
  3. Berger, J.O. (2003). Could Fisher, Jeffreys and Neyman Have Agreed on Testing, Statistical Science 18 (1): 1–12.CrossRefGoogle Scholar
  4. Bhaskar, R. (1978). A Realist Theory of Science, Hassocks, Sussex: Harvester, UK.Google Scholar
  5. Bunge, M. (1996). Finding Philosophy in Social Science, New Haven, CT: Yale University Press.Google Scholar
  6. Campbell, D. (1975). ‘Degrees of Freedom’ and the Case Study, Comparative Political Stud 8 (2): 178–193.Google Scholar
  7. Campbell, D. and Stanley, J. (1963). Experimental and Quasi-experimental Designs for Research, Boston, MA: Houghton Mifflin.Google Scholar
  8. Collier, A. (1994). Critical Realism: An introduction to Roy Bhaskar’s philosophy, London: Verso Books.Google Scholar
  9. Compeau, D., Marcolin, B., Kelley, H. and Higgins, C. (2012). Research Commentary – Generalizability of information systems research using student subjects – A reflection on our practices and recommendations for future research, Information Systems Research 23 (4): 1093–1109.CrossRefGoogle Scholar
  10. Cronbach, L.J. (1982). Designing Evaluations of Educational and Social Programs, San Francisco, CA: Jossey-Bass.Google Scholar
  11. Dubin, R. (1978). Theory Building. (Rev. ed.) London: Free Press.Google Scholar
  12. Eisenhardt, K.M. (1989). Building Theories from Case Study Research, Academy of Management Review 14 (4): 532–550.Google Scholar
  13. Galileo, G. (1610). Galileo Galileo [WWW document] http://en.wikipedia.org/wiki/Galileo_Galilei (accessed November 2014).
  14. Geertz, C. (1956). Religious Belief and Economic Behavior in a Central Javanese Town: Some preliminary considerations, Economic Development and Cultural Change 4 (2): 134–158.CrossRefGoogle Scholar
  15. Geertz, C. (1960). The Religion of Java, Chicago, IL: The Free Press.Google Scholar
  16. Gillham, B. (2005). Research Interviewing: The range of techniques: A practical guide, New York, NY: McGraw-Hill International.Google Scholar
  17. Goodman, S.N. (2005). Introduction to Bayesian Methods 1: Measuring the strength of evidence, Clinical Trials 2 (4): 282–290.CrossRefGoogle Scholar
  18. Gregor, S. (2006). The Nature of Theory in Information Systems, MIS Quarterly 30 (3): 611–642.Google Scholar
  19. Hume, D. (1748). An Enquiry Concerning Human Understanding, in L.A. Selby-Bigge (1777) (ed.) Enquiries Concerning the Human Understanding, and Concerning the Principles of Morals, by David Hume, Oxford: Clarenden Press, [WWW document] http://www.gutenberg.org/ebooks/9662 (accessed November 2014).
  20. King, W.R. and He, J. (2005). External Validity in Survey Research, Communications of the AIS 16 (article 45): 880–894.Google Scholar
  21. Kolakowski, L. (1968). The Alienation of Reason: A history of positivist thought, Garden City, NY: Doubleday.Google Scholar
  22. Lacity, M.C. and Willcocks, L.P. (1998). An Empirical Investigation of Information Technology Sourcing Practices: Lessons from experience, MIS Quarterly 22 (3): 363–408.CrossRefGoogle Scholar
  23. Lee, A.S. (1989). Case Studies as Natural Experiments, Human Relations 42 (2): 117–137.CrossRefGoogle Scholar
  24. Lee, A.S. and Baskerville, R.L. (2003). Generalizing Generalizability in Information Systems Research, Information Systems Research 14 (3): 221–243.CrossRefGoogle Scholar
  25. Lee, A.S. and Baskerville, R.L. (2012). Conceptualizing Generalizability: New contributions and a reply, MIS Quarterly 36 (3): 749–761.Google Scholar
  26. Mingers, J. (2006). Realising Systems Thinking: Knowledge and action in management science, New York: Springer.Google Scholar
  27. NASA (2014). Our Solar System: Moons, [WWW document] http://solarsystem.nasa.gov/planets/profile.cfm?Display=Moons (accessed November 2014).
  28. Peirce, C.S. (1903). A Syllabus of Certain Topics of Logic, in the Peirce Edition Project, The Essential Peirce. Vol. 2. Indiana University Press, (see also [WWW document] http://www.helsinki.fi/science/commens/terms/abduction.html (accessed November 2014)).Google Scholar
  29. Pettigrew, A.M. (1985a). Contextualist Research: A natural way to think, in E.E. Lawler, III, A.M. Mohrman, Jr., S.A. Mohrman, G.E. Ledford, Jr. and T.G. Cummings (eds.) Doing Research that Is Useful for Theory and Practice, San Francisco, CA: Jossey-Bass, Publishers, pp. 223–274.Google Scholar
  30. Pettigrew, A.M. (1985b). Contextualist Research and the Study of Organizational Change Processes, in E. Mumford, R. Hirschheim, G. Fitzgerald and T. Wood-Harper (eds.) Research Methods in Information Systems, Amsterdam: Elsevier Science Publishers, B.V. (North-Holland), pp. 53–78.Google Scholar
  31. Pettigrew, A.M. (1990). Longitudinal Field Research on Change: Theory and practice, Organization Science 1 (3): 267–292.CrossRefGoogle Scholar
  32. Pinsonneault, A. and Kraemer, K.L. (1993). Survey Research in Management Information Systems: An assessment, Journal of Management Information Systems 10 (2): 75–105.CrossRefGoogle Scholar
  33. Popper, K. (2002a). The Logic of Scientific Discovery, London: Routledge.Google Scholar
  34. Popper, K. (2002b). Unended Quest: An intellectual autobiography, London: Routledge.Google Scholar
  35. Seddon, P.B. and Scheepers, R. (2006). Other-settings generalization in IS Research, in Proceedings, International Conference on Information Systems (Milwaukee), 1141–1158.Google Scholar
  36. Seddon, P.B. and Scheepers, R. (2012). Drawing General Conclusions from Samples: Towards the improved treatment of generalization of knowledge claims in IS research, European Journal of Information Systems 21: 6–21.CrossRefGoogle Scholar
  37. Shadish, W.R., Cook, T.D. and Campbell, D.T. (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference, Boston, MA: Houghton Mifflin Company.Google Scholar
  38. Sivo, S.A., Saunders, C., Chang, Q. and Jiang, J.J. (2006). How Low Should You Go? Low Response Rates and the Validity of Inference in IS Questionnaire Research, Journal of the Association for Information Systems 7 (6): 351–414.Google Scholar
  39. Tsang, E.W.K. (2014). Case Studies and Generalization in Information Systems Research: A critical realist perspective, Journal of Strategic Information Systems 23 (2): 174–186.CrossRefGoogle Scholar
  40. Tsang, E.W.K. and Williams, J.N. (2012). Generalization and Induction: Misconceptions, clarifications, and a classification of induction, MIS Quarterly 36 (3): 729–748.Google Scholar
  41. Williams, J.N. and Tsang, E.W.K. (2015). Classifying Generalization: Paradigm war or abuse of terminology? Journal of Information Technology 30 (1): 19–29.CrossRefGoogle Scholar
  42. Yin, R.K. (1994). Case Study Research, Design and Methods, 2nd edn California: Sage Publications.Google Scholar
  43. Yin, R.K. (2003). Case Study Research, Design and Methods, 3rd edn California: Sage Publications.Google Scholar

Copyright information

© Association for Information Technology Trust 2015

Authors and Affiliations

  1. 1.The University of MelbourneAustralia
  2. 2.Deakin UniversityAustralia

Personalised recommendations