Skip to main content
SpringerLink
Log in

Re-examining the causal structure of information technology impact research

  • Original Article
  • Published:
European Journal of Information Systems

Abstract

About 20 years ago, Markus and Robey noted that most research on IT impacts had been guided by deterministic perspectives and had neglected to use an emergent perspective, which could account for contradictory findings. They further observed that most research in this area had been carried out using variance theories at the expense of process theories. Finally, they suggested that more emphasis on multilevel theory building would likely improve empirical reliability. In this paper, we reiterate the observations and suggestions made by Markus and Robey on the causal structure of IT impact theories and carry out an analysis of empirical research published in four major IS journals, Management Information Systems Quarterly (MISQ), Information Systems Research (ISR), the European Journal of Information Systems (EJIS), and Information and Organization (I&O), to assess compliance with those recommendations. Our final sample consisted of 161 theory-driven articles, accounting for approximately 21% of all the empirical articles published in these journals. Our results first reveal that 91% of the studies in MISQ, ISR, and EJIS focused on deterministic theories, while 63% of those in I&O adopted an emergent perspective. Furthermore, 91% of the articles in MISQ, ISR, and EJIS adopted a variance model; this compares with 71% from I&O that applied a process model. Lastly, mixed levels of analysis were found in 14% of all the surveyed articles. Implications of these findings for future research are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

Notes

  1. The Senior Scholar Forum consists of senior information systems academics, who have served as editors-in-chief of MIS Quarterly, and ISR, plus former ICIS program chairs and presidents of AIS.

  2. The authors would like to thank an anonymous reviewer for this insight.

References

  • Ancona DG, Goodman PS, Lawrence BS and Tushman ML (2001) Time: a new research lens. Academy of Management Review 26 (4), 645–663.

    Google Scholar 

  • Ang S, Cummings LL, Straub SW and Earley PC (1993) The effects of information technology and the perceived mood of the feedback giver on feedback seeking. Information Systems Research 4 (3), 240–261.

    Article  Google Scholar 

  • Ang S, Slaughter S and Ng KY (2002) Human capital and institutional determinants of information technology compensation: modeling multilevel and cross-level interactions. Management Science 48 (11), 1427–1445.

    Article  Google Scholar 

  • Attewell P (1994) Information technology and the productivity paradox. In Organizational Linkages: Understanding the Productivity Paradox (HARRIS DH, Ed), pp 13–53, National Academy Press, Washington, DC.

    Google Scholar 

  • Avital M (2000) Dealing with time in social inquiry: a tension between method and lived experience. Organization Science 11 (6), 665–673.

    Article  Google Scholar 

  • Bannister F and Remenyi D (2003) The societal value of ICT: first steps towards an evaluation framework. Electronic Journal of Information Systems Evaluation 6 (2), 197–206.

    Google Scholar 

  • Barley S (1986) Technology as an occasion for structuring: evidence from observation of CT scanners and the social order of radiology departments. Administrative Science Quarterly 35 (1), 61–103.

    Article  Google Scholar 

  • Barnes SJ (2005) Assessing the value of IS journals. Communications of the ACM 48 (1), 110–112.

    Article  Google Scholar 

  • Barrett M and Scott S (2004) Electronic trading and the process of globalization in traditional future exchanges: a temporal perspective. European Journal of Information Systems 13 (1), 65–79.

    Article  Google Scholar 

  • Barua A, Kriebel CH and Mukhopadhyay T (1995) Information technologies and business value: an analytic and empirical investigation. Information Systems Research 6 (1), 3–23.

    Article  Google Scholar 

  • Baskerville R (2004) An editor's values. European Journal of Information Systems 13 (1), 1–2.

    Article  Google Scholar 

  • Boudreau M-C, Gefen D and Straub DW (2001) Validation in information systems research: a state-of-the-art assessment. MIS Quarterly 25 (1), 1–16.

    Article  Google Scholar 

  • Brynjolfsson E (1993) The productivity paradox of information technology. Communications of the ACM 36 (12), 67–77.

    Article  Google Scholar 

  • Brynjolfsson E, Malone TW, Gurbaxani V and Kambil A (1994) Does information technology lead to smaller firms? Management Science 40 (12), 1628–1644.

    Article  Google Scholar 

  • Burrell G and Morgan G (1979) Sociological Paradigms and Organizational Analysis. Heinemann, London.

    Google Scholar 

  • Chan Y (2000) IT value: the great divide between qualitative and quantitative and individual and organizational measures. Journal of Management Information Systems 16 (4), 225–261.

    Article  Google Scholar 

  • Chen W and Hirschheim R (2004) A paradigmatic and methodological examination of information systems research from 1991 to 2001. Information Systems Journal 14 (3), 197–235.

    Article  Google Scholar 

  • Clegg CW, Axtell C, Damadoran L, Farbey B, Hull R, Lloyd-Jones R, Nicholls J, Sell R and Tomlinson C (1997) Information technology: a study of performance and the role of human and organizational factors. Ergonomics 40 (9), 851–871.

    Article  Google Scholar 

  • Coopersmith J (1996) Texas politics and the fax revolution. Information Systems Research 7 (1), 37–51.

    Article  Google Scholar 

  • Cousins KC and Robey D (2005) Human agency in a wireless world: patterns of technology use in nomadic computing environments. Information and Organization 15 (2), 151–180.

    Article  Google Scholar 

  • Culnan MJ and Swanson EB (1986) Research in management information systems, 1980–1984: points of work and reference. MIS Quarterly 10 (3), 289–302.

    Article  Google Scholar 

  • Dennis AR and Kinney ST (1998) Testing media richness in the new media: the effects of cues, feedback, and task equivocality. Information Systems Research 9 (3), 256–274.

    Article  Google Scholar 

  • Dennis AR, Valacich JS, Fuller MA and Schneider C (2006) Research standards for promotion and tenure in information systems. MIS Quarterly 30 (1), 1–12.

    Google Scholar 

  • Desouza KC, El Sawy OA, Galliers RD, Loebbecke C and Watson RT (2005) Information systems research that really matters: beyond the IS rigor versus relevance debate. Proceedings of the ICIS 957–959, Las Vegas, NV, USA, 2005.

    Google Scholar 

  • Dickson GW, Partridge LG and Robinson LH (1993) Exploring modes of facilitative support for GDSS technology. MIS Quarterly 17 (2), 173–194.

    Article  Google Scholar 

  • Doherty NF and King M (2005) From technical to socio-technical change: tackling the human and organizational aspects of systems development projects. European Journal of Information Systems 14 (1), 1–5.

    Article  Google Scholar 

  • Dos Santos BL, Peffer K and Mauer DC (1993) The impact of information technology investment announcements on the market value of the firm. Information Systems Research 4 (1), 1–23.

    Article  Google Scholar 

  • Dubé L and Paré G (2003) Rigor in IS positivist case research: current practices, trends, and recommendations. MIS Quarterly 27 (4), 597–635.

    Google Scholar 

  • Eisenhardt KM (1989) Building theories from case study research. Academy of Management Review 14 (4), 532–550.

    Google Scholar 

  • Giddens A (1984) The Constitution of Society. University of California Press, Berkeley and Los Angeles.

    Google Scholar 

  • Glesne C and Peshkin A (1992) Becoming Qualitative Researchers: An Introduction. Longman, New York.

    Google Scholar 

  • Gregor S (2006) The nature of theory in information systems. MIS Quarterly 30 (3), 611–642.

    Google Scholar 

  • Hayes N (2001) Boundless and bounded interactions in the knowledge work process: the role of groupware technologies. Information and Organization 11 (2), 79–101.

    Article  Google Scholar 

  • Hess CM and Kemerer CF (1994) Computerized loan origination system: an industry case study of the electronic markets hypothesis. MIS Quarterly 18 (3), 251–276.

    Article  Google Scholar 

  • Im KS, Dow DE and Grover V (2001) A reexamination of IT investment and the market value of the firm – an event study methodology. Information Systems Research 12 (1), 103–117.

    Article  Google Scholar 

  • King WR and He J (2005) External validity in IS survey research. Communications of the Association for Information Systems 16, 880–894.

    Google Scholar 

  • Klein KJ and Kozlowski SWJ (2000) Multilevel Theory, Research and Methods in Organizations: Foundations, Extensions and New Directions. Jossey-Bass Inc., San Francisco, CA.

    Google Scholar 

  • Klein KJ, Tosi J and Cannella Jr. AA (1999) Multilevel theory building: benefits, barriers, and new developments. Academy of Management Review 24 (2), 243–248.

    Article  Google Scholar 

  • Kohli R and Devaraj S (2003) Measuring information technology payoff: a meta-analysis of structural variables in firm-level empirical research. Information Systems Research 14 (2), 127–145.

    Article  Google Scholar 

  • Kuhn TS (1962) The Structure of Scientific Revolutions. University of Chicago Press, Chicago, IL.

    Google Scholar 

  • Landis JR and Kock GG (1977) The measurement of observer agreement for categorical data. Biometrics 33 (1), 159–174.

    Article  Google Scholar 

  • Langley A (1999) Strategies for theorizing from process data. Academy of Management Review 24 (4), 691–710.

    Google Scholar 

  • Lapointe L and Rivard S (2005) A multilevel model of resistance to IT implementation. MIS Quarterly 29 (3), 461–491.

    Google Scholar 

  • Leavitt HJ and Whisler TL (1958) Management in the 1980's. Harvard Business Review 36 (6), 41–48.

    Google Scholar 

  • Lee AS (2000) Researchable directions for ERP and other new information technologies. MIS Quarterly 24 (1), iii–viii.

    Google Scholar 

  • Lee H (1999) Time and information technology: monochronicity, polychronicity and temporal symmetry. European Journal of Information Systems 8 (1), 16–26.

    Article  Google Scholar 

  • Lee H and Liebenau J (2000) Temporal effects of information systems on business processes: focusing on the dimensions of temporality. Accounting, Management and Information Technologies 10 (3), 157–185.

    Article  Google Scholar 

  • Liebenau J and Smithson S (1993) Editorial. European Journal of Information Systems 2 (1), 1.

    Article  Google Scholar 

  • Liebenau J and Smithson S (1995) Too many conferences? European Journal of Information Systems 4 (1), 1–2.

    Article  Google Scholar 

  • Lowry PB, Romans D and Curtis A (2004) Global journal prestige and supporting disciplines: a scientometric study of information systems journals. Journal of the Association for Information Systems 5 (2), 29–77.

    Google Scholar 

  • Macredie RD and Sandom C (1999) IT-enabled change: evaluating an improvisational perspective. European Journal of Information Systems 8 (4), 247–259.

    Article  Google Scholar 

  • Majchrzak A, Rice RE, Malhotra A, King N and Ba S (2000) Technology adaptation: the case of computer-supported interorganizational virtual team. MIS Quarterly 24 (4), 569–600.

    Article  Google Scholar 

  • Markus L and Robey D (1988) Information technology and organizational change: causal structure in theory and research. Management Science 34 (5), 583–598.

    Article  Google Scholar 

  • Miles MB and Huberman AM (1994) Qualitative Data Analysis, An Expanded Sourcebook. Sage, Beverly Hills, CA.

    Google Scholar 

  • Nerur S, Sikora R, Mangalaraj G and Balijepally VG (2005) Assessing the relative influence of journals in a citation network. Communications of the ACM 48 (11), 71–74.

    Article  Google Scholar 

  • O’Keefe B and Paul R (2000) Editorial. European Journal of Information Systems 9 (1), 1–2.

    Article  Google Scholar 

  • Oliver C (1991) Strategic responses to institutional processes. Academy of Management Review 16 (1), 145–179.

    Article  Google Scholar 

  • Orlikowski WJ (1993) CASE tools as organizational change – investigating incremental and radical changes in systems development. MIS Quarterly 17 (3), 309–340.

    Article  Google Scholar 

  • Orlikowski WJ (1996) Improvising organizational transformation over time: a situated change perspective. Information Systems Research 7 (1), 63–92.

    Article  Google Scholar 

  • Paré G and Trudel MC (2007) Knowledge barriers to PACS adoption and implementation in hospitals. International Journal of Medical Informatics 76 (1), 22–33.

    Article  Google Scholar 

  • Paul RJ (2005) Editor's view: an opportunity for editors of IS journals to relate their experiences and offer advice. The editorial view of Ray J Paul. First in a series. European Journal of Information Systems 14 (3), 207–212.

    Article  Google Scholar 

  • Pentland BT (1995) Information systems and organizational learning: the social epistemology of organizational knowledge systems. Accounting, Management & Information Technology 5 (1), 1–21.

    Article  Google Scholar 

  • Pinsonneault A and Kraemer KL (1997) Middle management downsizing: an empirical investigation of the impact of information technology. Management Science 43 (5), 659–679.

    Article  Google Scholar 

  • Pozzebon M (2004) The influence of a structurationist view on strategic management research. Journal of Management Studies 41 (2), 247–272.

    Article  Google Scholar 

  • Richardson H and Robinson B (2007) The mysterious case of the missing paradigm: a review of critical information systems research 1991–2001. Information Systems Journal 17 (3), 251–270.

    Article  Google Scholar 

  • Rivard S, Aubert BA, Patry M, Paré G and Smith H (2004) Information Technology and Organizational Transformation. Solving the Management Puzzle. Butterworth-Heinemann, Elsevier Science, Oxford, UK.

    Google Scholar 

  • Robey D (1996) Diversity in information systems research: threat, promise, and responsibility. Information Systems Research 7 (4), 400–408.

    Article  Google Scholar 

  • Robey D and Boudreau MC (1999) Accounting for the contradictory organizational consequences of information technology: theoretical directions and methodological implications. Information Systems Research 10 (2), 167–185.

    Article  Google Scholar 

  • Sabherwal R and Chan YE (2001) Alignment between business and IS strategies: a study of prospectors, analyzers and defenders. Information Systems Research 12 (1), 11–33.

    Article  Google Scholar 

  • Sabherwal R and Robey D (1995) Reconciling variance and process strategies for studying information system development. Information Systems Research 6 (4), 303–327.

    Article  Google Scholar 

  • Sambamurthy V and Poole MS (1992) The effects of variations in capabilities of GDSS designs on management of cognitive conflict in groups. Information Systems Research 3 (3), 224–251.

    Article  Google Scholar 

  • Schultze I and Orlikowski WJ (2004) A practice perspective on technology-mediated network relations: the use of internet-based self-serve technologies. Information Systems Research 15 (1), 87–106.

    Article  Google Scholar 

  • Scott WR (1995) Institutions and Organizations. Sage, Thousand Oaks, CA.

    Google Scholar 

  • Sengupta K and Te’eni D (1993) Cognitive feedback in GDSS: improving control and convergence. MIS Quarterly 17 (1), 87–114.

    Article  Google Scholar 

  • Short JC, Piccolo G, Powell A and Ives B (2005) Investigating multilevel relationships in information systems research: an application to virtual teams research using hierarchical linear modeling. Journal of Information Technology Theory and Application 7 (3), 1–26.

    Article  Google Scholar 

  • Subramani M (2004) How do suppliers benefit from information technology use in supply chain relationships? MIS Quarterly 28 (1), 45–73.

    Google Scholar 

  • Subramani M and Walden E (2001) The impact of E-commerce announcements on the market value of firms. Information Systems Research 12 (2), 135–154.

    Article  Google Scholar 

  • Suh KS and Lee YE (2005) The effects of virtual reality on consumer learning: an empirical investigation. MIS Quarterly 29 (4), 673–697.

    Google Scholar 

  • Swanson EB and Dans E (2000) System life expectancy and the maintenance effort: exploring their equilibration. MIS Quarterly 24 (2), 277–298.

    Article  Google Scholar 

  • Tolsby J (1998) Effects of organizational culture on a large scale IT introduction effort: a case study of the Norwegian army's EDBLF project. European Journal of Information Systems 7 (2), 108–114.

    Article  Google Scholar 

  • Trauth EM (2001) The choice of qualitative methods in IS research. In Qualitative Research in IS: Issues and Trends (TRAUTH EM, Ed), pp 1–19, Idea Group Publishing, Hershey, PA.

    Chapter  Google Scholar 

  • Vaast E and Walsham G (2005) Representations and actions: the transformation of work practices with IT use. Information and Organization 15 (1), 65–89.

    Article  Google Scholar 

  • Vessey I, Ramesh V and Glass RL (2002) Research in information systems: an empirical study of diversity in the discipline and its journal. Journal of Management Information Systems 19 (2), 129–174.

    Google Scholar 

  • Weber R (2003a) The problem of the problem. MIS Quarterly 27 (1), iii–ix.

    Google Scholar 

  • Weber R (2003b) Theoretically speaking. MIS Quarterly 27 (3), iii–xii.

    Google Scholar 

  • Weber R (2004) The rhetoric of positivism versus interpretivism: a personal view. MIS Quarterly 28 (1), iii–xii.

    Google Scholar 

  • Wells JD, Fuerst WL and Palmer JW (2005) Designing consumer interfaces for experiential tasks: an empirical investigation. European Journal of Information Systems 14 (3), 273–287.

    Article  Google Scholar 

  • Zmud RW (1995) Editor's comments. MIS Quarterly 19 (4), iii–viii.

    Google Scholar 

Download references

Acknowledgements

We gratefully acknowledge Henri Barki, Kathleen Greenaway, M. Lynne Markus, Alain Pinsonneault, Suzanne Rivard and Daniel Robey who provided helpful comments and suggestions on earlier versions of the manuscript. We also thank Joanne Griffith for editorial assistance.

Author information

Authors and Affiliations

  1. Canada Research Chair in Information Technology in Health Care, HEC Montréal, Montréal, Québec, Canada

    Guy Paré

  2. Department of Information Technologies, HEC Montréal, Montréal, Québec, Canada

    Simon Bourdeau, Josianne Marsan & Shadi Shuraida

  3. Management and Technology Department, Université du Québec à Montréal, Montréal, Québec, Canada

    Hamid Nach

Authors
  1. Guy Paré
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simon Bourdeau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Josianne Marsan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hamid Nach
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shadi Shuraida
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guy Paré.

Appendices

Appendix A

Coding scheme

Causal agency

1=Technological imperative

2=Organizational imperative

3=Emergent perspective

Logical structure

1=Variance model

2=Process model

3=Not applicable (descriptive studies)

Main level of analysis

1=Individual

2=Task

3=Group/team

4=Department/division/unit

5=Organization

6=Network of organizations

7=Society

8=Mix of levels

Primary methodology

1=Questionnaire survey

2=Case study

3=Qualitative survey (interviews)

4=Experiment

5=Action research

6=Secondary data

7=Ethnography/grounded theory

8=Field study

Overall nature of the study

1=Descriptive

2=Theory building

3=Theory testing

Appendix B

Examples of coding decision rules

See Table B1.

Table 6 Table a1
Full size table

Rights and permissions

Reprints and permissions

About this article

Cite this article

Paré, G., Bourdeau, S., Marsan, J. et al. Re-examining the causal structure of information technology impact research. Eur J Inf Syst 17, 403–416 (2008). https://doi.org/10.1057/ejis.2008.34

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/ejis.2008.34

Keywords

Search

Navigation