Skip to main content
SpringerLink
Log in

Engaging scientometrics in information systems

  • State of the Art
  • Published:
Journal of Information Technology

Abstract

Although scientometrics is seeing increasing use in Information Systems (IS) research, in particular for evaluating research efforts and measuring scholarly influence; historically, scientometric IS studies are focused primarily on ranking authors, journals, or institutions. Notwithstanding the usefulness of ranking studies for evaluating the productivity of the IS field’s formal communication channels and its scholars, the IS field has yet to exploit the full potential that scientometrics offers, especially towards its progress as a discipline. This study makes a contribution by raising the discourse surrounding the value of scientometric research in IS, and proposes a framework that uncovers the multi-dimensional bases for citation behaviour and its epistemological implications on the creation, transfer, and growth of IS knowledge. Having identified 112 empirical research evaluation studies in IS, we select 44 substantive scientometric IS studies for in-depth content analysis. The findings from this review allow us to map an engaging future in scientometric research, especially towards enhancing the IS field’s conceptual and theoretical development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1

Similar content being viewed by others

Notes

  1. At the time of data collection, IS journals not indexed by the WoS included DataBase, Communications of the AIS, Journal of the AIS, Journal of Management Systems, and Informing Science. The Journal of the AIS and DataBase have since been indexed.

  2. Lotka’s Law measures the frequency of publication for authors in any discipline. It states that the number of authors making n contributions is about 1/n 2 of those making one contribution.

  3. Yule-Simon’s distribution is a kind of power distribution that models the distribution and numbers of citations over time.

  4. For example, the word ‘citation’ frequently occurs in all articles; however, it points to different dimensions depending on which other words it is associated with. Most of the KWIC words are searched using a modifier-referent or subject-object structure. The phrase or 2-gram ‘citation evidence’ and its variants (e.g., 3-gram ‘evidence from citations’) may occur more frequently in articles focused on the evidentiary dimension, whereas the phrase ‘verify citations’ and its variants, (e.g., ‘verified citations’) can be expected to occur more in the tool dimension using citations as a verification tool.

  5. Researchers wishing to replicate the study can take advantage of the exact NVivo syntax or the format offered in their chosen software research tool by using various conjugations of terms, wild-card forms, or stemmed versions of these terms (e.g., ‘eval’ for evaluate and evaluation).

  6. The issues of native theories (Moody et al., 2010) and cumulative tradition (Taylor et al., 2010) continue to be studied outside of scientometrics too (Straub, 2012).

  7. For example, arguably, Takeda et al. (2010) could be primarily categorized in the science management tool dimension based on its main goal (it is weakly categorized as such) of evaluating scholars. However, because most of the paper is dedicated to social network content and related implications, as well as the evaluation of the influence of IS scholars, the coding process categorized it primarily into the social network and sociological dimensions.

  8. In 1996, a physics professor, Alan Sokal submitted a paper to Social Text, an academic journal of postmodern cultural studies. He wanted to test the publication’s intellectual rigour, specifically, to see if the journal would publish a paper containing non-sensical citations if it sounded good and agreed with the journal editors’ leanings. The paper was published (Sokal, 1996). The instance raised many questions concerning the scholarly merit of humanistic commentaries on the natural sciences and ethics in academics (Sokal, 2000). It also points at journal editors’ ability to be sensitive about persuasive rhetorical tools in academic works and the decline of standards of rigour in the academic community.

References

  • Alvesson, M. and Sandberg, J. (2013). Constructing Research Questions: Doing interesting research, Thousand Oaks, CA: SAGE Publications.

    Book  Google Scholar 

  • Anderson, M.H. (2006). How Can We Know What We Think until We See What We Said?: A citation and citation context analysis of Karl Weick’s The Social Psychology of Organizing, Organization Studies 27(11): 1675–1692.

    Article  Google Scholar 

  • Ball, P. (2005). Index Aims for Fair Ranking of Scientists, Nature 436(7053): 900.

    Article  Google Scholar 

  • Banting, F.G. and Best, C.H. (1922). The Internal Secretion of the Pancreas, Journal of Laboratory and Clinical Medicine 7(5): 251–266.

    Google Scholar 

  • Banville, C. and Landry, M. (1989). Can the Field of MIS be Disciplined? Communications of the ACM 32(1): 48–60.

    Article  Google Scholar 

  • Bazeley, P. (2013). Qualitative Data Analysis: Practical strategies, London: SAGE.

    Google Scholar 

  • Benbasat, I. and Zmud, R.W. (1999). Empirical Research in Information Systems: The Practice of Relevance, MIS Quarterly 23(1): 3–16.

    Article  Google Scholar 

  • Benbasat, I. and Zmud, R. (2003). The Identity Crisis Within the IS Discipline: Defining and Communicating the discipline’s core Properties, MIS Quarterly 27(2): 183–194.

    Google Scholar 

  • Blumer, H. (1969). Symbolic Interactionism: Perspective and Method, Berkeley: University of California Press.

    Google Scholar 

  • Bontis, N. and Serenko, A. (2009). A Follow-up Ranking of Academic Journals, Journal of Knowledge Management 13(1): 16–26.

    Article  Google Scholar 

  • Bornmann, L. and Daniel, H.-D. (2008). What Do Citation Counts Measure? A review of studies on citing behavior, Journal of Documentation 64(1): 45–80.

    Article  Google Scholar 

  • Brooks, T.A. (1986). Evidence of Complex Citer Motivations, Journal of the American Society for Information Science 37(1): 34–36.

    Article  Google Scholar 

  • Brooks, T.A. (1995). People, Words, and Perceptions: A phenomenological investigation of textuality, Journal of the American Society for Information Science 46(2): 103–115.

    Article  Google Scholar 

  • Butler, T. (1998). Towards a Hermeneutic Method for Interpretive Research in Information Systems, Journal of Information Technology 13(4): 285–300.

    Article  Google Scholar 

  • Callon, M., Courtial, J.-P. and Laville, F. (1991). Co-Word Analysis as a Tool for Describing the Network of Interactions Between Basic and Technological Research: The case of polymer chemsitry, Scientometrics 22(1): 155–205.

    Article  Google Scholar 

  • Callon, M., Law, J. and Rip, A. (1986). Mapping the Dynamics of Science and Technology: Sociology of science in the real world, London: Palgrave Macmillan.

    Book  Google Scholar 

  • Chen, C. and Hicks, D. (2004). Tracing Knowledge Diffusion, Scientometrics 59(2): 199–211.

    Article  Google Scholar 

  • Cheon, M.J., Choong, C.L. and Grover, V. (1992). Research in MIS – Points of Work and Reference: A replication and extension of the Culnan and Swanson study, Data Base 23(2): 21–29.

    Article  Google Scholar 

  • Chua, C., Cao, L., Cousins, K. and Straub, D.W. (2002). Measuring Researcher-Production in Information Systems, Journal of the Association for Information Systems 3(6): 145–215.

    Google Scholar 

  • Chubin, D.E. and Moitra, S.D. (1975). Content Analysis of References: Adjunct or alternative to citation counting, Social Studies of Science 5(4): 423–441.

    Article  Google Scholar 

  • Clark, K.E. (1957). America's Psychologists: A survey of a growing profession, Washington DC: American Psychological Association.

    Book  Google Scholar 

  • Cocosila, M., Serenko, A. and Turel, O. (2009). A Scientometric Study of Information Systems Conferences: Exploring ICIS, PACIS and ASAC, Americas Conference on Information Systems (AMCIS), (6–9 August, San Francisco, CA) Paper 434.

  • Cocosila, M., Serenko, A. and Turel, O. (2011). Exploring the Management Information Systems Discipline: A scientometric study of ICIS, PACIS and ASAC, Scientometrics 87(1): 1–16.

    Article  Google Scholar 

  • Cohen, A.M. and Hersh, W.R. (2005). A Survey of Current Work in Biomedical Text Mining, Briefings in Bioinformatics 6(1): 57–71.

    Article  Google Scholar 

  • Cole, J.R. and Cole, S. (1973). Social Stratification in Science, Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Cole, M. and Avison, D. (2007). The Potential of Hermeneutics in Information Systems Research, European Journal of Information Systems 16(6): 820–833.

    Article  Google Scholar 

  • Cole, S. and Cole, J.R. (1967). Scientific Output And Recognition: A study in the operation of the reward system in science, American Sociological Review 32(3): 377–390.

    Article  Google Scholar 

  • Conway, M. (2006). The Subjective Precision of Computers: A methodological comparison with human coding in content analysis, Journalism & Mass Communication Quarterly 83(1): 183–200.

    Article  Google Scholar 

  • Cooper, R.B., Blair, D. and Pao, M. (1993). Communicating MIS Research: A citation study of journal influence, Information Processing and Management 29(1): 113–127.

    Article  Google Scholar 

  • Costanza, R., Stern, D., Fisher, B., He, L. and Ma, C. (2004). Influential Publications in Ecological Economics: A citation analysis, Ecological Economics 50(3–4): 261–292.

    Article  Google Scholar 

  • Coulter, N., Monarch, I. and Konda, S. (1998). Software Engineering as Seen Through its Research Literature: A study in co-word analysis, Journal of the American Society for Information Science 49(13): 1206–1223.

    Article  Google Scholar 

  • Cozzens, S.E. (1981). Taking the Measure of Science: A review of citation theories, Newsletter of the International Society for the Sociology of Knowledge 7(1–2): 16–21.

    Google Scholar 

  • Cozzens, S.E. (1985). Comparing the Sciences: Citation context analysis of papers from neuropharmacology and the sociology of science, Social Studies of Science 15(1): 127–153.

    Article  Google Scholar 

  • Cozzens, S.E. (1989). What do Citations Count? The rhetoric-first model, Scientometrics 15(5–6): 437–447.

    Article  Google Scholar 

  • Cronin, B. (1984). The Citation Process: The role and significance of citations in scientific communication, London: Taylor Graham.

    Google Scholar 

  • Cronin, B. (1998). Metatheorizing Citation, Scientometrics 43(1): 45–55.

    Article  Google Scholar 

  • Cronin, B. (2000). Semiotics and Evaluative Bibliometrics, Journal of Documentation 56(4): 440–453.

    Article  Google Scholar 

  • Cuellar, M., Takeda, H. and Truex, III D.P. (2008). The Hirsch family of bibliometric indices as an improved measure of IS academic journal impact, Americas Conference on Information Systems (AMCIS 2008), (14–17 August, Toronto, CA) Article 295.

  • Culnan, M.J. (1986). The Intellectual Development Of Management Information Systems, 1972–1982: A co-citation analysis, Management Science 32(2): 156–172.

    Article  Google Scholar 

  • Culnan, M.J. (1987). Mapping the Intellectual Structure of MIS, 1980–85: A co-citation analysis, MIS Quarterly 11(3): 340–353.

    Article  Google Scholar 

  • Culnan, M.J. and Swanson, E.B. (1986). Research in Management Information Systems, 1980–1984: Points of work and reference, MIS Quarterly 10(3): 289–302.

    Article  Google Scholar 

  • DiMaggio, P.J. and Powell, W.W. (1983). The Iron Cage Revisited: Institutional isomorphism and collective rationality in organizational fields, American Sociological Review 48(2): 147–160.

    Article  Google Scholar 

  • Duncan, W.J., Ford, E.W., Rousculp, M.D. and Ginter, P.M. (2002). Community of Scholars: An exploratory study of management laureates, Scientometrics 55(3): 395–409.

    Article  Google Scholar 

  • Duriau, V.J., Reger, R.K. and Pfarrer, M.D. (2007). A Content Analysis of the Content Analysis Literature in Organization Studies: Research themes, data sources, and methodological refinements, Organizational Research Methods 10(1): 5–34.

    Article  Google Scholar 

  • Eom, S.B. (1996). Mapping the Intellectual Structure of Research in Decision Support Systems through Author Cocitation Analysis, Decision Support Systems 16(4): 315–338.

    Article  Google Scholar 

  • Eom, S.B. and Farris, R.S. (1996). The Contributions of Organizational Science to the Development of Decision Support Systems research Subspecialties, Journal of the American Society for Information Science 47(12): 941–952.

    Article  Google Scholar 

  • Eom, S.B., Lee, S.M. and Kim, J. (1993). The Intellectual Structure of Decision Support Systems (1971–1989), Decision Support Systems 10(1): 19–36.

    Article  Google Scholar 

  • Fairclough, N. (2003). Analysing Discourse: Textual Analysis for Social Research, London: Routledge.

    Google Scholar 

  • Fanelli, D. (2009). How Many Scientists Fabricate and Falsify Research? A systematic review and meta-analysis of survey data, PLoS ONE 4(5): e5738.

    Article  Google Scholar 

  • Gadamer, H.-G. (1976). Philosophical Hermeneutics, Berkeley: University of California Press.

    Google Scholar 

  • Gallivan, M. (2010). Analyzing Frequent Collaborators in the IS Field: What can we learn about IS coauthorship in general? Americas Conference on Information Systems (AMCIS 2010), (12–15 August, Lima, Peru) Paper 581.

  • Garfield, E. (1955). Citation Indexes for Science: A new dimension in documentation through association of ideas, Science 122(3159): 108–111.

    Article  Google Scholar 

  • Garfield, E. (1963). Science Citation Index: Introduction, in science citation index, Philadelphia, PA: Institute for Scientific Information.

    Google Scholar 

  • Garfield, E. (1970). Citation Indexing, Historio-Bibliography, and the Sociology of Science, in Proceedings of the Third International Congress of Medical Librarianship, (Amsterdam, The Netherlands) Excerpta Medica, 187–204.

  • Garfield, E. (1972). Citation Analysis as a Tool in Journal Evaluation, Science 178(4060): 471–479.

    Article  Google Scholar 

  • Garfield, E. (1979a). The Application of Citation Indexing to the Patent Literature, in Citation Indexing – Its theory and application in science, technology, and humanities, New York: John Wiley & Sons, pp. 37–40.

    Google Scholar 

  • Garfield, E. (1979b). The Citation Index as a Search Tool, in Citation Indexing – Its theory and application in science, technology, and humanities, New York: John Wiley & Sons, pp. 41–61.

    Google Scholar 

  • Garfield, E. (1979c). A Science-Management Tool, in Citation Indexing – Its theory and application in science, technology, and humanities, New York: John Wiley & Sons, pp. 62–80.

    Google Scholar 

  • Garfield, E. (2004). Historiographic Mapping of Knowledge Domains Literature, Journal of Information Science 30(2): 119–145.

    Article  Google Scholar 

  • Garfield, E. and Malin, M.V. (1968). Can Nobel Prize Winners be Predicted? University of Pennsylvania [WWW document] http://garfield.library.upenn.edu/papers/nobelpredicted.pdf (accessed 30 December 2006).

  • Garfield, E., Sher, I.H. and Torpie, R.J. (1964). The Use of Citation Data in Writing the History of Science, Philadelphia, PA: Institute for Scientific Information (ISI).

    Google Scholar 

  • Garfield, E. and Welljams-Dorof, A. (1990). The Impact of Fraudulent Research on the Scientific Literature: The Stephen E. Breuning case, Journal of the American Medical Association 263(10): 1424–1426.

    Article  Google Scholar 

  • Gephart, R. (1997). Hazardous Measures: An interpretive textual analysis of quantitative sensemaking during crises, Journal of Organizational Behavior 18(S1): 583–622.

    Article  Google Scholar 

  • Ghose, A. (2009). Internet Exchanges for used Goods: An empirical analysis of trade patterns and adverse selection, MIS Quarterly 33(2): 263–291.

    Google Scholar 

  • Gibbons, M. and Georghiou, L. (1987). Evaluation of Research: A selection of current practices, Paris: Organization for Economic Cooperation and Development (OECD).

    Google Scholar 

  • Gilbert, G.N. (1976). The Transformation of Research Findings into Scientific Knowledge, Social Studies of Science 6(3/4): 281–306.

    Article  Google Scholar 

  • Gilbert, G.N. (1977). Referencing as Persuasion, Social Studies of Science 7(1): 113–122.

    Article  Google Scholar 

  • Gilbert, G.N. (1978). Measuring the Growth of Science, Scientometrics 1(1): 9–34.

    Article  Google Scholar 

  • Goes, P.B. (2015). Editor’s Comment. Inflection Point: Looking back or looking forward? MIS Quarterly 39(3): iii–vii.

    Google Scholar 

  • Gray, P. (2009). Journal Self-Citation I: Overview of the journal self-citation papers – The wisdom of the IS crowd, Communications of the Association for Information Systems 25(1): 1–10.

    Google Scholar 

  • Griffith, B.C. and Mullins, N.C. (1972). Coherent Social Groups in Scientific Change, Science 177(4053): 959–964.

    Article  Google Scholar 

  • Griffith, B.C., Small, H.G., Stonehill, J.A. and Dey, S. (1974). The Structure of Scientific Literatures II: Toward a macro- and microstructure for science, Science Studies 4(4): 339–365.

    Article  Google Scholar 

  • Groot, T. and García-Valderrama, T. (2006). Research Quality and Efficiency: An analysis of assessments and management issues in Dutch economics and business research programs, Research Policy 35(9): 1362–1376.

    Article  Google Scholar 

  • Grover, V., Ayyagari, R., Gokhale, R. and Lim, J. (2006a). About Reference Disciplines and Reference Differences: A critique of Wade et al., Journal of the Association for Information Systems 7(5): 336–350.

    Google Scholar 

  • Grover, V., Ayyagari, R., Gokhale, R., Lim, J. and Coffey, J. (2006b). A Citation Analysis of the Evolution and State of Information Systems Within a Constellation of Reference Disciplines, Journal of the Association for Information Systems 7(5): 270–325.

    Google Scholar 

  • Hagstrom, W.O. (1971). Inputs, Outputs, and the Prestige of University Science Departments, Sociology of Education 44(4): 375–397.

    Article  Google Scholar 

  • Hamilton, S. and Ives, B. (1982). Knowledge Utilization among MIS Researchers, MIS Quarterly 20(3): 61–77.

    Article  Google Scholar 

  • Hansen, S., Lyytinen, K. and Markus, M.L. (2006). The Legacy of ‘Power and Politics’ in Disciplinary Discourse: A citation analysis, 27th International Conference for Information Systems (ICIS), (10–13 December, Milwaukee, WI).

  • Hassan, N.R. (2014). Paradigm Lost … Paradigm Gained: A hermeneutical rejoinder to Banville and Landry’s ‘can the field of MIS be disciplined?’, European Journal of Information Systems 23(6): 600–615.

    Article  Google Scholar 

  • Hirsch, J.E. (2005). An Index to Quantify an Individual’s Scientific Research Output, Proceedings of the National Academy of Sciences of the United States of America 102(46): 16569–16572.

    Article  Google Scholar 

  • Hjørland, B. (2000). Relevance research: The missing perspective(s): ‘Non-relevance’ and ‘epistemological relevance’, Journal of the American Society for Information Science and Technology 51(2): 209–211.

    Article  Google Scholar 

  • Holsti, O.R. (1969). Content Analysis for the Social Sciences and Humanities, Reading, MA: Addison-Wesley Publishing Company.

    Google Scholar 

  • Husserl, E. (1911/1965). Philosophy as Rigorous Science, in Q. Lauer (ed.) Phenomenology and the Crisis of Philosophy, New York: Harper and Row.

    Google Scholar 

  • Igami, M. (2008). Exploration of the Evolution of Nanotechnology via Mapping of Patent Applications, Scientometrics 77(2): 289–308.

    Article  Google Scholar 

  • Jaffe, A.B. and Trajtenberg, M. (2002). Patents, Citations, and Innovations a Window on the Knowledge Economy, Cambridge, MA: MIT Press.

    Google Scholar 

  • Jaffe, A.B., Trajtenberg, M. and Henderson, R. (1993). Geographic Localization of Knowledge Spillovers as Evidenced by Patent Citations, Quarterly Journal of Economics 108(3): 577–598.

    Article  Google Scholar 

  • Kabanoff, B. (1997). Introduction: Computers can Read as well as Count: Computer-aided text analysis in organizational research, Journal of Organizational Behavior 18(S1): 507–511.

    Article  Google Scholar 

  • Kaplan, N. (1965). The Norms of Citation Behavior: Prolegomena to the footnote, American Documentation 16(3): 179–184.

    Article  Google Scholar 

  • Katerattanakul, P., Han, B. and Hong, S. (2003). Objective Quality Ranking of Computing Journals, Communications of the ACM 46(10): 111–114.

    Article  Google Scholar 

  • Katerattanakul, P., Han, B. and Rea, A. (2006). Is Information Systems a Reference Discipline? Communications of the ACM 49(5): 114–118.

    Article  Google Scholar 

  • Keen, P.G.W. (1980). MIS Research: Reference disciplines and a cumulative tradition, International Conference on Information Systems (ICIS), (Philadelphia, PA), ACM Press, 9–18.

  • Klein, H.K. and Hirschheim, R. (2008). The Structure of the IS Discipline Reconsidered: Implications and reflections from a community of practice perspective, Information and Organization 18(4): 280–302.

    Article  Google Scholar 

  • Klein, H.K. and Myers, M.D. (1999). A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems, MIS Quarterly 23(1): 67–94.

    Article  Google Scholar 

  • Korpela, K.M. (2010). How Long does it take for the Scientific Literature to Purge itself of Fraudulent Material?: The Breuning case revisited, Current Medical Research & Opinion 26(4): 843–847.

    Article  Google Scholar 

  • Kostoff, R.N., Stump, J.A., Johnson, D., Murday, J.S., Lau, C.G.Y. and Tolles, W.M. (2006). The Structure and Infrastructure of the Global Nanotechnology Literature, Journal of Nanoparticle Research 8(3–4): 301–321.

    Article  Google Scholar 

  • Kostoff, R.N., Toothman, D.R., Eberhart, H.J. and Humenik, J.A. (2001). Text Mining using Database Tomography and Bibliometrics: A review, Technological Forecasting and Social Change 68(3): 223–253.

    Article  Google Scholar 

  • Krippendorff, K. (2004). Content Analysis: An introduction to its methodology, Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  • Latour, B. (1987). Science in Action: How to follow scientists and engineers through society, Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Latour, B. and Woolgar, S. (1979). Laboratory Life: The social construction of scientific facts, Beverly Hills: Sage Publications.

    Google Scholar 

  • Law, J. and French, D. (1974). Normative and Interpretive Sociologies of Science, Sociological Review 22(4): 581–595.

    Article  Google Scholar 

  • Leydesdorff, L. (1987). Towards a Theory of Citation? Scientometrics 12(5–6): 305–309.

    Article  Google Scholar 

  • Leydesdorff, L. (1998). Theories of Citations? Scientometrics 43(1): 5–25.

    Article  Google Scholar 

  • Leydesdorff, L. and Amsterdamska, O. (1990). Dimensions of Citation Analysis, Science, Technology and Human Values 15(3): 305–335.

    Article  Google Scholar 

  • Lim, S., Saldanha, T., Malladi, S. and Melville, N.P. (2009). Theories used in information systems research: Identifying theory networks in leading IS journals, International Conference on Information Systems, (15–18 December, Phoenix, AZ), 1–10.

  • Locke, K. and Golden-Biddle, K. (1997). Constructing Opportunities for Contribution: Structuring intertextual coherence and ‘problematizing’ in organizational studies, Academy of Management Journal 40(5): 1023–1062.

    Article  Google Scholar 

  • Loebbecke, C. and Leidner, D. (2012). The Contribution of Top IS Publications to Subsequent Research: A citation analysis, Communications of the Association for Information Systems 30(1): 423–438.

    Google Scholar 

  • Long, R., Crawford, A., White, M. and Davis, K. (2009). Determinants of Faculty Research Productivity in Information Systems: An empirical analysis of the impact of academic origin and academic affiliation, Scientometrics 78(2): 231–260.

    Article  Google Scholar 

  • Lowry, P.B., Romans, D. and Curtis, A. (2004). Global Journal Prestige and Supporting Disciplines: A scientometric study of information systems journals, Journal of the Association for Information Systems 5(2): 29–77.

    Google Scholar 

  • Luukkonen, T. (1997). Why has Latour’s Theory of Citations been Ignored by the Bibliometric Community? Discussion of sociological interpretations of citation analysis, Scientometrics 38(1): 27–37.

    Article  Google Scholar 

  • MacRoberts, M.H. and MacRoberts, B.R. (1986). Quantitative Measures of Communication in Science: A study of the formal level, Social Studies of Science 16(1): 151–172.

    Article  Google Scholar 

  • Marion, L.S., Garfield, E., Hargens, L.L., Lievrouw, L.A., White, H. and Concepción, C.S. (2003). Social Network Analysis and Citation Network Analysis: Complementary approaches to the study of scientific communication, Proceedings of the American Society for Information Science and Technology 40(1): 486–487.

    Article  Google Scholar 

  • Markus, M.L. (1983). Power, Politics, and MIS Implementation, Communications of the ACM 26(6): 430–444.

    Article  Google Scholar 

  • Marshakova-Shaikevich, I. (2001). Scientometric Perspectives of the Analysis of Chemical Terminology, Scientometrics 52(2): 323–336.

    Article  Google Scholar 

  • Matthes, J. and Kohring, M. (2008). The Content Analysis of Media Frames: Toward improving reliability and validity, Journal of Communication 58(2): 258–279.

    Article  Google Scholar 

  • McCain, K.W. (1983). The Author Cocitation Structure of Macroeconomics, Scientometrics 5(5): 277–289.

    Article  Google Scholar 

  • McKelvey, B. (1982). Organizational Systematics – Taxonomy, evolution, classification, Berkeley, CA: University of California Press.

    Google Scholar 

  • Merton, R.K. (1957). Priorities in Scientific Discovery, American Sociological Review 22(6): 635–659.

    Article  Google Scholar 

  • Merton, R.K. (1965). On the Shoulders of Giants: A shandean postscript, New York: Free Press.

    Google Scholar 

  • Merton, R.K. (1968a). The Matthew Effect in Science, Science 159(3810): 56–63.

    Article  Google Scholar 

  • Merton, R.K. (1968b). Social Theory and Social Structure, New York: Free Press.

    Google Scholar 

  • Merton, R.K. (1969). Behavior Patterns of Scientists, American Scientist 57(1): 1–23.

    Google Scholar 

  • Merton, R.K. (1973a). Singletons and Multiples in Scientific Discovery: A chapter in the sociology of science, in The Sociology of Science: Theoretical and empirical investigations, Chicago: University of Chicago Press, pp. 343–370.

    Google Scholar 

  • Merton, R.K. (1973b). The Sociology of Science: Theoretical and empirical investigations, Chicago, IL: The University of Chicago Press.

    Google Scholar 

  • Mingers, J. and Harzing, A.-W. (2007). Ranking Journals in Business and Management: A statistical analysis of the harzing data set, European Journal of Information Systems 16(4): 303–316.

    Article  Google Scholar 

  • Mingers, J. and Leydesdorff, L. (2015). A Review of Theory and Practice in Scientometrics, in arXiv, New York: Cornell University.

    Google Scholar 

  • Mizruchi, M.S. and Fein, L.C. (1999). The social Construction of Organizational Knowledge: A study of the uses of coercive, mimetic, and normative isomorphism, Administrative Science Quarterly 44(4): 653–683.

    Article  Google Scholar 

  • Mohr, L.B. (1982). Explaining Organizational Behavior, San Francisco, CA: Jossey-Bass Publishers.

    Google Scholar 

  • Moody, D., Iacob, M.-E. and Amrit, C. (2010). In Search of Paradigms: Identifying the theoretical foundations of the IS field, European Conference on Information Systems, (6–9 June, Pretoria, South Africa).

  • Moody, D.L. (2000). Building Links between IS Research and Professional Practice: Improving the relevance and impact of IS research, International Conference on Information Systems (ICIS), (Brisbane, Australia) ACM Press, 351–360.

  • Moravcsik, M.J. (1984). Life in a Multidimensional World, Scientometrics 6(2): 75–85.

    Article  Google Scholar 

  • Moravcsik, M.J. (1985). Applied Scientometrics: An assessment methodology for developing countries, Scientometrics 7(3–6): 165–176.

    Article  Google Scholar 

  • Moravcsik, M.J. and Murugesan, P. (1975). Some Results on the Function and Quality of Citations, Social Studies of Science 5(1): 86–92.

    Article  Google Scholar 

  • Morris, R. (1994). Computerized Content Analysis in Management Research: A demonstration of advantages & limitations, Journal of Management 20(4): 903–931.

    Article  Google Scholar 

  • Mulkay, M.J. (1974). Methodology in the Sociology of Science: Some reflections on the study of radio astronomy, Social Science Information 13(2): 107–119.

    Article  Google Scholar 

  • Mullins, N.C., Hargens, L.L., Hecht, P.K. and Kick, E.L. (1977). The Group Structure of Cocitation Clusters: A comparative study, American Sociological Review 42(4): 552–562.

    Article  Google Scholar 

  • Narin, F. (1976). Evaluative Bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity, Cherry Hill, NJ: Computer Horizons.

    Google Scholar 

  • Narin, F., Hamilton, K.S. and Olivastro, D. (1997). The Increasing Linkage Between U.S. Technology and Public Science, Research Policy 26(3): 317–330.

    Article  Google Scholar 

  • Nath, R. and Jackson, W.M. (1991). Productivity of Management Information Systems Researchers: Does Lotka’s law apply? Information Processing and Management 27(2–3): 203–209.

    Article  Google Scholar 

  • Nerur, S.P., Mahapatra, R., Balijepally, V. and Mangalaraj, G. (2006). Is Information Systems a Reference Discipline? 39th Annual Hawaii International Conference on System Sciences (HICSS 39), (Lihue, Hawaii).

  • Nerur, S.P., Sikora, R., Mangalaraj, G. and Balijepally, V. (2005). Assessing the Relative Influence of Journals in a Citation Network, Communications of the ACM 48(11): 71–74.

    Article  Google Scholar 

  • Neuendorf, K.A. (2002). The Content Analysis Guidebook, Thousand Oaks, CA: Sage Publications.

    Google Scholar 

  • Newton, I. (1676/1959). The Correspondence of Isaac Newton, in H.W. Turnbull, J.F. Scott and A.R. Hall (eds.) Vol. 1. Cambridge, UK: Royal Society at the University Press.

  • Nicolaisen, J. (2007). Citation Analysis, Annual Review of Science and Technology 41(1): 609–641.

    Article  Google Scholar 

  • Noruzi, A. (2005). Google Scholar: The new generation of citation indexes, Libri 55(4): 170–180.

    Article  Google Scholar 

  • Oh, W., Choi, J.N. and Kim, K. (2006). Coauthorship Dynamics and Knowledge Capital: The patterns of cross-disciplinary collaboration in information systems research, Journal of Management Information Systems 22(3): 265–292.

    Article  Google Scholar 

  • Orlikowski, W.J. and Baroudi, J.J. (1991). Studying Information Technology in Organizations: Research approaches and assumptions, Information Systems Research 2(1): 1–28.

    Article  Google Scholar 

  • Orlikowski, W.J. and Iacono, C.S. (2001). Research Commentary: Desperately seeking the ‘IT’ in IT research – A call to theorizing the IT artifact, Information Systems Research 12(2): 121–134.

    Article  Google Scholar 

  • Oromaner, M. (1985). The Ortega Hypothesis and Influential Articles in American Sociology, Scientometrics 7(1–2): 3–10.

    Article  Google Scholar 

  • Ortega y Gasset, J. (1957). The Revolt of the Masses, New York: W. W. Norton & Company.

    Google Scholar 

  • Palevitz, B.A. (1997). The Ethics of Citation: A matter of science’s family values, The Scientist 11(12): 8.

    Google Scholar 

  • Park, T.K. (1993). The Nature of Relevance in Information Retrieval – An empirical study, Library Quarterly 63(3): 318–351.

    Article  Google Scholar 

  • Pavlou, P.A. and Dimoka, A. (2006). The Nature and Role of Feedback Text Comments in Online Marketplaces: Implications for trust building, price premiums, and seller differentiation, Information Systems Research 17(4): 392–414.

    Article  Google Scholar 

  • Pecorari, D. (2008). Academic Writing and Plagiarism: A linguistic analysis, New York: Continuum International Publishing Group.

    Google Scholar 

  • Peffers, K. and Hui, W. (2003). Collaboration and Author Order: Changing patterns in IS research, Communications of the Association for Information Systems 11(1): 166–190.

    Google Scholar 

  • Peffers, K. and Ya, T. (2003). Identifying and Evaluating the Universe of Outlets for Information Systems Research: Ranking the journals, The Journal of Information Technology Theory and Application 5(1): 63–84.

    Google Scholar 

  • Polites, G.L. and Watson, R.T. (2009). Using Social Network Analysis to Analyze Relationships Among IS Journals, Journal of the Association for Information Systems 10(8): 595–636.

    Google Scholar 

  • Price, D.J.d.S. (1963). Little Science, Big Science, New York: Columbia University Press.

    Google Scholar 

  • Price, D.J.d.S. (1965). Networks of Scientific Papers, Science 149(3683): 510–515.

    Article  Google Scholar 

  • Price, D.J.d.S. (1970). Citation Measures of Hard Science, Soft Science, Technology, and Non Science, in C.E. Nelson and D. Pollack (eds.) Communication among Scientists and Engineers, Lexington, MA: D. C. Heath and Company, pp. 3–22.

    Google Scholar 

  • Price, D.J.d.S. (1978). Editorial Statements, Scientometrics 1(1): 3–18.

    Article  Google Scholar 

  • Price, D.J.d.S. and Beaver, D. (1966). Collaboration in an Invisible College, American Psychologist 21(11): 1011–1018.

    Article  Google Scholar 

  • Rainer, R.K., Jr. and Miller, M.D. (2005). Examining Differences Across Journal Rankings, Communications of the ACM 48(2): 91–94.

    Article  Google Scholar 

  • Ramiller, N.C., Swanson, B. and Wang, P. (2008). Research Directions in Information Systems: Toward an institutional ecology, Journal of the Association for Information Systems 9(1): 1–22.

    Article  Google Scholar 

  • Roche, I., Besagni, D., François, C., Hörlesberger, M. and Schiebel, E. (2010). Identification and Characterisation of Technological Topics in the Field of Molecular Biology, Scientometrics 82(3): 663–676.

    Article  Google Scholar 

  • Salton, G. and McGill, M.J. (1983). Introduction to Modern Information Retrieval, New York: McGraw-Hill.

    Google Scholar 

  • Schubert, A. (2014). Sentences to Remember from the First 100 Volumes of the Journal Scientometrics, Scientometrics 100(1): 1–13.

    Article  Google Scholar 

  • Schutz, A. (1973). Concept and Theory Formulation in the Social Sciences, in M. Natanson (ed.) Collected Papers Vol. 1, The Hague: Martinus Nijhoff, pp. 48–66.

    Google Scholar 

  • SCImago (2007). SJR – Scimago journal & country rank [WWW document] http://www.scimagojr.com (accessed 9 December 2012).

  • Serenko, A., Bontis, N., Booker, L.D., Sadeddin, K. and Hardie, T. (2010). A Scientometric Analysis of Knowledge Management and Intellectual Capital Academic Literature (1994–2008), Journal of Knowledge Management 14(1): 3–23.

    Article  Google Scholar 

  • Serenko, A., Cox, R.A.K., Bontis, N. and Booker, L.D. (2011). The Superstar Phenomenon in the Knowledge Management and Intellectual Capital Academic Discipline, Journal of Informetrics 5(3): 333–345.

    Google Scholar 

  • Short, J.C., Broberg, J.C., Cogliser, C.C. and Brigham, K.H. (2010). Construct Validation Using Computer-Aided Text Analysis (CATA): An illustration using entrepreneurial orientation, Organizational Research Methods 13(2): 320–347.

    Article  Google Scholar 

  • Small, H. (1973). Co-Citation in the Scientific Literature: A new measure of the relationship between two documents, Journal of the American Society for Information Science 24(4): 265–269.

    Article  Google Scholar 

  • Small, H. (1978). Cited Documents as Concept Symbols, Social Studies of Science 8(3): 327–340.

    Article  Google Scholar 

  • Small, H. (2003). Paradigms, Citations, and Maps of Science: A personal history, Journal of the American Society for Information Science 54(5): 394–399.

    Article  Google Scholar 

  • Small, H. (2004). On the Shoulders of Robert Merton: Towards a normative theory of citation, Scientometrics 60(1): 71–79.

    Article  Google Scholar 

  • Small, H. and Griffith, B.C. (1974). The Structure of Scientific Literatures I: Identifying and graphing specialties, Science Studies 4(1): 17–40.

    Article  Google Scholar 

  • Sokal, A.D. (1996). Transgressing the Boundaries – Toward a transformative hermeneutics of quantum gravity, Social Text 46/47(Summer): 217–252.

    Article  Google Scholar 

  • Sokal, A.D. (2000). The Sokal Hoax: The sham that shook the academy, Lincoln: University of Nebraska Press.

    Google Scholar 

  • Stamper, R. (1991). The Semiotic Framework for Information Systems Research, in H.-E. Nissen, H.K. Klein and R. Hirschheim (eds.) Information Systems Research: Contemporary approaches and emergent traditions, North-Holland: Elsevier Science Publishers B. V., pp. 515–528.

    Google Scholar 

  • Straub, D. (2006). The Value of Scientometric Studies: An introduction to a debate on IS as a reference discipline, Journal of the Association for Information Systems 7(5): 241–246.

    Google Scholar 

  • Straub, D. (2012). Editorial: Does MIS have native theories, MIS Quarterly 36(2): iii–xii.

    Google Scholar 

  • Straub, D.W. (2015). The Critical Role of Historiography in Writing IS History, Communications of the Association for Information Systems 36(1): 593–598 Article 29.

    Google Scholar 

  • Stryker, J.E., Way, R.J., Hornik, R.C. and Yanovitzky, I. (2006). The Validation of Database Search Terms for Content Analysis: The case of cancer news coverage, Journalism & Mass Communication Quarterly 83(2): 413–430.

    Article  Google Scholar 

  • Swanson, D.R. (1990). Somatomedin C and Arginine: Implicit connections between mutually isolated literatures, Perspectives in Biology and Medicine 33(2): 157–186.

    Article  Google Scholar 

  • Swanson, D.R. and Smalheiser, N.R. (1997). An Interactive System for Finding Complementary Literatures: A stimulus to scientific discovery, Artificial Intelligence 91(2): 183–203.

    Article  Google Scholar 

  • Takeda, H., Cuellar, M., Truex, III D.P. and Vidgen, R. (2011). Networks of Innovation in IS research: An exploration of the relationship between co-authorship networks and H-family indices, in Proceedings of the European Conference on Information Systems (ECIS 2011), 9–11 June, Helsinki, Finland, pp. 1–14.

  • Takeda, H., Truex, D.P. and Cuellar, M.J. (2010). Evaluating Scholarly Influence through Social Network Analysis: The next step in evaluating scholarly influence, Americas Conference on Information Systems (AMCIS 2010), (12–15 August, Lima, Peru).

  • Taylor, H., Dillon, S. and Van Wingen, M. (2010). Focus and Diversity in Information Systems Research: Meeting the dual demands of a healthy applied discipline, MIS Quarterly 34(4): 647–667.

    Google Scholar 

  • Thomson Reuters (2008). Successful 2008 predictions, Thomson Reuters [WWW document] http://scientific.thomsonreuters.com/nobel/success-08/ (accessed 23 October 2008).

  • Thomson Reuters (2013). Science watch: Predicting Nobel Prizes, Thomson Reuters [WWW document] http://sciencewatch.com/nobel (27 March 2013).

  • Thorne, F.C. (1977). The Citation Index, Another Case of Spurious Validity Journal of Clinical Psychology 33(4): 1157–1161.

    Google Scholar 

  • Todd, P.A., McKeen, J.D. and Gallupe, R.B. (1995). The Evolution of IS Job Skills: A content analysis of IS job advertisements from 1970 to 1990, MIS Quarterly 19(1): 1–27.

    Article  Google Scholar 

  • Truex, D., Cuellar, M., Takeda, H. and Vidgen, R. (2011). The Scholarly Influence of Heinz Klein: Ideational and social measures of his impact on IS research and IS scholars, European Journal of Information Systems 20(4): 422–439.

    Article  Google Scholar 

  • Truex, D.P., Cuellar, M. and Takeda, H. (2009). Assessing Scholarly Influence: Using the Hirsch indices to reframe the discourse, Journal of the Association for Information Systems 10(7): 560–594.

    Google Scholar 

  • Truex, D.P., Cuellar, M.J. and Takeda, H. (2008). Assessing Scholarly Influence: Proposing new metrics, in Proceedings of the International Conference on Information Systems (ICIS 2008), (Paris, France), 1–19.

  • Vessey, I., Ramesh, V. and Glass, R.L. (2002). Research in Information Systems: An empirical study of diversity in the discipline and its journals, Journal of Management Information Systems 19(2): 129–174.

    Google Scholar 

  • Vidgen, R., Henneberg, S. and Naude, P. (2007). What Sort of Community is the European Conference on Information Systems? A social network analysis 1993–2005, European Journal of Information Systems 16(1): 5–19.

    Article  Google Scholar 

  • Wade, M., Biehl, M. and Kim, H. (2006a). If the Tree of IS Knowledge Falls in a Forest, will Anyone Hear? A commentary on Grover et al., Journal of the Association for Information Systems 7(5): 326–335.

    Google Scholar 

  • Wade, M., Biehl, M. and Kim, H. (2006b). Information Systems is not a Reference Discipline (and What We Can do About it), Journal of the Association for Information Systems 7(5): 247–269.

    Google Scholar 

  • Walsham, G. (1993). Interpreting Information Systems in Organizations, New York: John Wiley & Sons.

    Google Scholar 

  • Walsham, G. (1995a). The Emergence of Interpretivism in IS Research, Information Systems Research 6(4): 376–394.

    Article  Google Scholar 

  • Walsham, G. (1995b). Interpretive Case Studies in IS Research: Nature and method, European Journal of Information Systems 4(2): 74–81.

    Article  Google Scholar 

  • Walsham, G. (2006). Doing Interpretive Research, European Journal of Information Systems 15(3): 320–330.

    Article  Google Scholar 

  • Walstrom, K.A. and Leonard, L.N.K. (2000). Citation Classics from the Information Systems Literature, Information & Management 38(2): 59–72.

    Article  Google Scholar 

  • Wasko, M.M. and Faraj, S. (2005). Why Should I Share? Examining Social Capital and Knowledge Contribution in Electronic Networks of Practice, MIS Quarterly 29(1): 35–57.

    Google Scholar 

  • Wasserman, S., Faust, K., Iacobucci, D. and Granovetter, M. (1994). Social Network Analysis: Methods and applications, London: Cambridge University Press.

    Book  Google Scholar 

  • Weber, R.P. (1990). Basic Content Analysis: Quantitative applications in the social sciences, Thousand Oaks: Sage.

    Book  Google Scholar 

  • Webster, J. and Watson, R.T. (2002). Analyzing the Past to Prepare for the Future: Writing a literature review, MIS Quarterly 26(2): xiii–xxiii.

    Google Scholar 

  • Weick, K.E. (2001). Making Sense of the Organization, Malden, MA: Blackwell Publishers.

    Google Scholar 

  • Weick, K.E., Sutcliffe, K.M. and Obstfeld, D. (2005). Organizing and the Process of Sensemaking, Organization Science 16(4): 409–421.

    Article  Google Scholar 

  • Weinstock, M. (1971). Citation Indexes, in Encyclopedia of Library and Information Science. vol. 5. New York: Marcel Dekker, pp. 16–40.

    Google Scholar 

  • White, H.C. (1992). Identity and Control: A structural theory of social action, Princeton, NJ: Princeton University Press.

    Google Scholar 

  • White, H.D., Wellman, B. and Nazer, N. (2004). Does Citation Reflect Social Structure? Longitudinal Evidence From the ‘Globenet’ Interdisciplinary Research Group, Journal of the American Society for Information Science and Technology 55(2): 111–126.

    Article  Google Scholar 

  • Whitley, E.A. and Galliers, R.D. (2007). An Alternative Perspective on Citation Classics: Evidence from the first 10 years of the European conference on information systems, Information & Management 44(5): 441–455.

    Article  Google Scholar 

  • Wouters, P. (1998). The Signs of Science, Scientometrics 41(1–2): 225–241.

    Article  Google Scholar 

  • Wouters, P. (1999). The Citation Culture. PhD diss., University of Amsterdam.

  • Xu, J. and Chau, M. (2006). The Social Identity of IS: Analyzing the collaboration network of the ICIS conferences (1980–2005), in Proceedings of the International Conference on Information Systems (ICIS 2006), (Milwaukee, WI), 569–590.

  • Yoon, B. and Park, Y. (2004). A Text-Mining-Based Patent Network: Analytical tool for high-technology trend, Journal of High Technology Management Research 15(1): 37–50.

    Article  Google Scholar 

  • Zhang, P. (2015). The IS History Initiative: Looking forward by looking back, Communications of the Association for Information Systems 36(1): 477–514.

    Google Scholar 

  • Zucker, L.G., Darby, M.R. and Brewer, M.B. (1998). Intellectual Human Capital and the Birth of U.S. Biotechnology Enterprises, The American Economic Review 88(1): 290–306.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Finance and MIS, University of Minnesota Duluth, Duluth, USA

    Nik R Hassan

  2. Department of Business, Media and Technology Management, University of Cologne, Koeln, Germany

    Claudia Loebbecke

Authors
  1. Nik R Hassan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Claudia Loebbecke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nik R Hassan.

Additional information

Supplementary information accompanies this article on the Journal of Information Technology website (www.palgrave-journals.com/jit)

Electronic supplementary material

Appendix

Appendix

Table A1

Table A1 Categorization of 44 substantive scientometric IS studies
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hassan, N., Loebbecke, C. Engaging scientometrics in information systems. J Inf Technol 32, 85–109 (2017). https://doi.org/10.1057/jit.2015.29

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/jit.2015.29

Keywords

Search

Navigation