Advertisement

Scientometrics

, Volume 100, Issue 2, pp 339–361 | Cite as

OR/MS journals evaluation based on a refined PageRank method: an updated and more comprehensive review

  • Brenda Cheang
  • Samuel Kai Wah Chu
  • Chongshou LiEmail author
  • Andrew Lim
Article

Abstract

The purpose of this research is to furnish the OR/MS research community with an updated assessment of the discipline’s journals set with refinements that also highlight the various characteristics of OR/MS journals. More specifically, we apply a refined PageRank method initially proposed by Xu et al. (2011) to evaluate the top 31 OR/MS journals for 2010, and report our findings. We also report the shifts in the rankings that span 5 years, from 2006 to 2010. We observe that Manufacturing and Service Operations Management, indexed by the SCI only in 2008, is a specialized journal that is consistently highly regarded within the discipline. The rankings also suggest that Management Science is more established as a generalized journal as it has more external impact. In general, our ranking results correlate with expert opinions, and we also observe, report and discuss some interesting patterns that have emerged over the past 5 years from 2006 to 2010.

Keywords

Journals evaluation Citations analysis Journal influence Journal impact Impact factor Influence index OR/MS PageRank 

References

  1. Agrawal, V. K., Agrawal, V., & Rungtusanatham, M. (2011). Theoretical and interpretation challenges to using the author affiliation index method to rank journals. Production and Operations Management, 20(2), 280–300.CrossRefGoogle Scholar
  2. Barman, S., Hanna, M. D., & LaForge, R. L. (2001). Perceived relevance and quality of POM journals: a decade later. Journal of Operations Management, 19(3), 367–385.CrossRefGoogle Scholar
  3. Barman, S., Tersine, R. J., & Buckley, M. R. (1991). An empirical assessment of the perceived relevance and quality of POM-related journals by academicians. Journal of Operations Management, 10(2), 194–212.CrossRefGoogle Scholar
  4. Bressan, M., & Peserico, E. (2010). Choose the damping, choose the ranking? Journal of Discrete Algorithms, 8(2), 199–213.CrossRefzbMATHMathSciNetGoogle Scholar
  5. Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual Web search engine. Computer Networks and ISDN Systems, 30(1), 107–117.CrossRefGoogle Scholar
  6. Butler, D. (2008). Free journal-ranking tool enters citation market. Nature, 451(7174), 6.CrossRefGoogle Scholar
  7. Cheang, B., Chu, S. K. W., Li, C., & Lim, A, (2014). A multidimensional approach to evaluating management journals: Refining PageRank via the differentiation of citation types and identifying the roles that management journals play. Journal of the Association for Information Science and Technology, Accepted, forthcoming.Google Scholar
  8. Chen, C. R., & Huang, Y. (2007). Author affiliation index, finance journal ranking, and the pattern of authorship. Journal of Corporate Finance, 13(5), 1008–1026.CrossRefGoogle Scholar
  9. DuBois, F. L., & Reeb, D. (2000). Ranking the international business journals. Journal of International Business Studies, 31(4), 689–704.CrossRefGoogle Scholar
  10. Feng, Y., & Chen, Y. F. (2011). Technical Note—A computational approach for optimal joint inventory-pricing control in an infinite-horizon periodic-review system. Operations Research, 59(5), 1297–1303.CrossRefzbMATHMathSciNetGoogle Scholar
  11. Glanzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171–193.CrossRefGoogle Scholar
  12. Goh, C. H., Holsapple, C. W., Johnson, L. E., & Tanner, J. (1996). An empirical assessment of influences on POM research. Omega, 24(3), 337–345.CrossRefGoogle Scholar
  13. Goh, C. H., Holsapple, C. W., Johnson, L. E., & Tanner, J. R. (1997). Evaluating and classifying POM journals. Journal of Operations Management, 15(2), 123–138.CrossRefGoogle Scholar
  14. Gonzalez-Pereira, B., Guerrero-Bote, V. P., & Moya-Anegón, F. (2010). A new approach to the metric of journals’ scientific prestige: The SJR indicator. Journal of Informetrics, 4(3), 379–391.CrossRefGoogle Scholar
  15. Gorman, M. F., & Kanet, J. J. (2005). Evaluating operations management-related journals via the author affiliation index. Manufacturing & Service Operations Management, 7(1), 3–19.CrossRefGoogle Scholar
  16. Gorman, M. F., & Kanet, J. J. (2007). Note—“OM Forum: Evaluating Operations Management-Related Journals via the Author Affiliation Index”—Do professors at top US business schools do what they say? Manufacturing & Service Operations Management, 9(1), 51–53.CrossRefGoogle Scholar
  17. Gorraiz, J., Gumpenberger, C., Schloegl, C., & Wieland, M. (2012, September). On the temporal stability of Garfield’s impact factor and its suitability to identify hot papers. In: 17th International Conference on Science and Technology Indicators (STI) (pp. 5–8).Google Scholar
  18. Guerrero-Bote, V. P., & Moya-Anegón, F. (2012). A further step forward in measuring journals’ scientific prestige: The SJR2 indicator. Journal of Informetrics, 6(4), 674–688.CrossRefGoogle Scholar
  19. Harless, D., & Reilly, R. (1998). Revision of the journal list for doctoral designation, Unpublished report. Richmond, VA: Virginia Commonwealth University.Google Scholar
  20. Harzing, A. (2007). Publish or Perish. Available from her website at http://www.harzing.com/pop.htm. Accessed 20 May 2013.
  21. Leydesdorff, L. (2008). Caveats for the use of citation indicators in research and journal evaluations. Journal of the American Society for Information Science and Technology, 59(2), 278–287.CrossRefGoogle Scholar
  22. Lim, A., Ma, H., Wen, Q., Xu, Z., & Cheang, B. (2009). Distinguishing citation quality for journal impact assessment. Communications of the ACM, 52(8), 111–116.CrossRefGoogle Scholar
  23. Lim, A., Ma, H., Wen, Q., Xu, Z., Cheang, B., Tan, B., & Zhu, W. (2007). Journal-Ranking. com: an online interactive journal ranking system. In Proceeding of the National Conference on Artificial Intelligence (Vol. 22, No. 2, p. 1723). Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999.Google Scholar
  24. Moed, H., Colledge, L., Reedijk, J., Moya-Anegon, F., Guerrero-Bote, V., Plume, A., et al. (2012). Citation-based metrics are appropriate tools in journal assessment provided that they are accurate and used in an informed way. Scientometrics, 92, 367–376.CrossRefGoogle Scholar
  25. Olson, J. E. (2005). Top-25-business-school professors rate journals in operations management and related fields. Interfaces, 35(4), 323–338.CrossRefMathSciNetGoogle Scholar
  26. Page, L., Brin, S., Motwani, R., & Winograd, T. (1999). The PageRank citation ranking: bringing order to the web. Technical Report. Stanford InfoLab.Google Scholar
  27. Palacios-Huerta, I., & Volij, O. (2004). The measurement of intellectual influence. Econometrica, 72(3), 963–977.CrossRefzbMATHGoogle Scholar
  28. Pendlebury, D. A. (2009). The use and misuse of journal metrics and other citation indicators. Archivum immunologiae et therapiae experimentalis, 57(1), 1–11.CrossRefGoogle Scholar
  29. Pinski, G., & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing and Management, 12(5), 297–312.CrossRefGoogle Scholar
  30. Robey, D., & Markus, M. L. (1998). Beyond rigor and relevance: producing consumable research about information systems. Information Resources Management Journal, 11(1), 7–16.CrossRefGoogle Scholar
  31. Rousseau, R. (2002). Journal evaluation: technical and practical issues. Library Trends, 50(3), 418–439.Google Scholar
  32. Saladin, B. (1985). Operations management research: Where should we publish. Operations Management Review, 3(4), 3–9.Google Scholar
  33. Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ British Medical Journal, 314(7079), 498.CrossRefGoogle Scholar
  34. Smith, R. (2006). Commentary: The power of the unrelenting impact factor—Is it a force for good or harm? International Journal of Epidemiology, 35(5), 1129–1130.CrossRefGoogle Scholar
  35. Vokurka, R. J. (1996). The relative importance of journals used in operations management research a citation analysis. Journal of Operations Management, 14(4), 345–355.CrossRefGoogle Scholar
  36. Weingart, P. (2005). Impact of bibliometrics upon the science system: Inadvertent consequences? Scientometrics, 62(1), 117–131.CrossRefGoogle Scholar
  37. Wen, Q. (2008). Journal Impact Assessment: Methodology and Experiments. PhD Thesis, Chapter 2. The Hong Kong University of Science and Technology, Hong Kong).Google Scholar
  38. West, J., Althouse, B., & Rosvall, M. (2008). Eigenfactor™ Score and Article Influence™ Score: Detailed Methods, Available at [http://www.eigenfactor.org/methods.pdf]. Accessed 25 Jan 2014.
  39. Xu, Z., Cheang, B., Lim, A., & Wen, Q. (2011). Evaluating OR/MS journals via PageRank. Interfaces, 41(4), 375–388.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  • Brenda Cheang
    • 1
    • 2
  • Samuel Kai Wah Chu
    • 2
  • Chongshou Li
    • 3
    Email author
  • Andrew Lim
    • 1
  1. 1.International Center of Management Science and Engineering, School of Management and EngineeringNanjing UniversityNanjingChina
  2. 2.Division of Information and Technology Studies, Faculty of EducationUniversity of Hong KongHong Kong, S.A.R.China
  3. 3.Department of Management Sciences, College of BusinessCity University of Hong KongHong Kong, S.A.R.China

Personalised recommendations