Skip to main content
SpringerLink
Log in

Ranking journals in business and management: a statistical analysis of the Harzing data set

  • Opinion Piece
  • Published:
European Journal of Information Systems

Abstract

Creating rankings of academic journals is an important but contentious issue. It is of especial interest in the U.K. at this time (2007) as we are only one year away from getting the results of the next Research Assessment Exercise (RAE) the importance of which, for U.K. universities, can hardly be overstated. The purpose of this paper is to present a journal ranking for business and management based on a statistical analysis of the Harzing data set which contains 13 rankings. The primary aim of the analysis is two-fold – to investigate relationships between the different rankings, including that between peer rankings and citation behaviour; and to develop a ranking based on four groups that could be useful for the RAE. Looking at the different rankings, the main conclusions are that there is in general a high degree of conformity between them as shown by a principal components analysis. Cluster analysis is used to create four groups of journals relevant to the RAE. The higher groups are found to correspond well with previous studies of top management journals and also gave, unlike them, equal coverage to all the management disciplines. The RAE Business and Management panel have a huge and unenviable task in trying to judge the quality of over 10,000 publications and they will inevitably have to resort to some standard mechanistic procedures to do so. This work will hopefully contribute by producing a ranking based on a statistical analysis of a variety of measures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3

References

  • ABS (2007) Academic journal quality guide. Association of Business Schools, www.the-ABS.org.uk.

  • Ackoff R (1974) The social responsibility of operational research. Operational Research Quarterly 25, 361–371.

    Article  Google Scholar 

  • Agresti A (1988) A model for agreement between ratings on an ordinal scale. Biometrics 44, 539–548.

    Article  Google Scholar 

  • Aldenderfer M and Blashfield R (1984) Cluster Analysis. Sage, London.

    Book  Google Scholar 

  • Anderberg M (1973) Cluster Analysis for Applications. Academic Press, New York.

    Google Scholar 

  • Baden-Fuller C, Ravazzolo F and Schweizer T (2000) Making and measuring reputations – the research ranking of European business schools. Long Range Planning 33 (5), 621–650.

    Article  Google Scholar 

  • DFES (2006) Reform of Higher Education Research Assessment and Funding. Department for Education and Skills, London.

  • Doyle J and Arthurs A (1995) Judging the quality of research in business schools. Omega, International Journal of Management Science 23 (3), 257–270.

    Article  Google Scholar 

  • Doyle J, Arthurs A, Green R, McAulay L, Pitt M and Bottomly P (1996a) The judge, the model of the judge, and the model of the judged as judge: analysis of the UK 1992 Research Assessment Exercise data for Business and Management Studies. Omega 24 (1), 13–28.

    Article  Google Scholar 

  • Doyle J, Arthurs A, McAulay L and Osbourne P (1996b) Citation as effortful voting: a reply to Jones, Brinn and Pendlebury. Omega 24 (5), 603–606.

    Article  Google Scholar 

  • Dubois FL and Reeb D (2000) Ranking the international business journals. Journal of International Business Studies 31 (4), 689–704.

    Article  Google Scholar 

  • Easton G and Easton D (2003) Marketing journals and the research assessment exercise. Journal of Marketing Management 19, 5–24.

    Article  Google Scholar 

  • Everitt B (1980) Cluster Analysis. Heinemann, London.

    Google Scholar 

  • Forgionne G and Kohli R (2001) A multiple criteria assessment of decision technology system journal quality. Information and Management 38, 421–435.

    Article  Google Scholar 

  • Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178, 471–479.

    Article  Google Scholar 

  • Geary J, Marriott L and Rowlinson M (2004) Journal rankings in business and management and the 2001 research assessment exercise in the UK. British Journal of Management 15, 95–141.

    Article  Google Scholar 

  • Gibbons J (1993) Nonparametric Measures of Association. Sage, London.

    Book  Google Scholar 

  • Glänzel W and Moed HK (2002) Journal impact measures in bibliometric research. Scientometrics 53 (2), 171–193.

    Article  Google Scholar 

  • Gordon A (1981) Classification. Chapman and Hall, London.

    Google Scholar 

  • Gower J (1971) A general coefficient of similarity and some of its properties. Biometrics 27, 857–871.

    Article  Google Scholar 

  • Hair J, Anderson R, Tatham R and Black W (1998) Multivariate Data Analysis. Prentice-Hall, New Jersey.

    Google Scholar 

  • Harvey C and Morris H (2005) Classification of Academic Journals in the Field of Business and Management Studies. University of West of England, Bristol.

    Google Scholar 

  • Harzing A-W (2005) Journal Quality List. Anne-Wil Harzing, http://www.harzing.com/.

  • Horowitz I (2003) Preference-neutral attribute weights in the journal-ranking problem. Journal of Operational Research Society 54, 452–457.

    Article  Google Scholar 

  • Jennings C (1998) Citation data: the wrong impact? Nature Neuroscience 1 (8), 641–642.

    Article  Google Scholar 

  • Jones M, Brinn T and Pendlebury M (1996a) Journal evaluation methodologies: a balanced response. Omega 24 (5), 607–612.

    Article  Google Scholar 

  • Jones M, Brinn T and Pendlebury M (1996b) Judging the quality of research in business schools: a comment from accounting. Omega, International Journal of Management Science 24 (5), 597–602.

    Article  Google Scholar 

  • Katerattanakul P and Han B (2003) Are European IS journals under-rated? An answer based on citation analysis. European Journal of Information Systems 12 (1), 60–71.

    Article  Google Scholar 

  • Kim J (1975) Multivariate analysis of ordinal data. American Journal of Sociology 81, 261–298.

    Article  Google Scholar 

  • Labovitz S (1967) Some observations on measurement and statistics. Social Forces 46 (2), 151–160.

    Article  Google Scholar 

  • Labovitz S (1970) The assignment of numbers to rank order categories. American Sociological Review 35, 515–525.

    Article  Google Scholar 

  • Liebertrau A (1983) Measures of Association. Sage, London.

    Book  Google Scholar 

  • Lowry PB, Romans D and Curtis A (2004) Global journal prestige and supporting disciplines: a scientometric study of information systems journals. Journal of the Association for Information Systems 5 (2), 29–75.

    Google Scholar 

  • Mingers J (2007) Exploring the dynamics of journal citations: modelling with S-curves. Journal of Operational Research Society (forthcoming).

  • Mingers J and Burrell Q (2006) Modelling citation behavior in Management Science journals. Information Processing and Management 42 (6), 1451–1464.

    Article  Google Scholar 

  • Peffers K and Tang Y (2003) Identifying and evaluating the universe of outlets for information systems research: ranking the journals. The Journal of Information Technology Theory and Application 5 (1), 63–84.

    Google Scholar 

  • Rainer R and Miller M (2005) Examining differences across journal rankings. Communications of the ACM 48 (2), 91–94.

    Article  Google Scholar 

  • Schuster C and von Eye A (2001) Models for ordinal agreement data. Biometrical Journal 43 (7), 795–808.

    Article  Google Scholar 

  • Seglen P (1997) Why the impact factor of journals should not be used for evaluating research. British Medical Journal 314 (7079), 498–502.

    Article  Google Scholar 

  • Sneath P and Sokal R (1973) Numerical Taxonomy. Freeman, San Franciso.

    Google Scholar 

  • Tahai A and Meyer M (1999) A revealed preference study of management journals direct influences. Strategic Management Journal 20, 279–296.

    Article  Google Scholar 

  • Tse A (2001) Using mathematical programming to solve large ranking problems. Journal of Operational Research Society 52, 1144–1150.

    Article  Google Scholar 

  • Zumbo B and Zimmerman D (1993) Is the selection of statistical methods governed by level of measurement? Canadian Psychology 34, 390–399.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kent Business School, University of Kent, Kent, U.K.

    John Mingers

  2. Department of Management, University of Melbourne, Victoria, Australia

    Anne-Wil Harzing

Authors
  1. John Mingers
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne-Wil Harzing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John Mingers.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mingers, J., Harzing, AW. Ranking journals in business and management: a statistical analysis of the Harzing data set. Eur J Inf Syst 16, 303–316 (2007). https://doi.org/10.1057/palgrave.ejis.3000696

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1057/palgrave.ejis.3000696

Keywords

Search

Navigation