Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison

Abstract

This article aims to provide a systematic and comprehensive comparison of the coverage of the three major bibliometric databases: Google Scholar, Scopus and the Web of Science. Based on a sample of 146 senior academics in five broad disciplinary areas, we therefore provide both a longitudinal and a cross-disciplinary comparison of the three databases. Our longitudinal comparison of eight data points between 2013 and 2015 shows a consistent and reasonably stable quarterly growth for both publications and citations across the three databases. This suggests that all three databases provide sufficient stability of coverage to be used for more detailed cross-disciplinary comparisons. Our cross-disciplinary comparison of the three databases includes four key research metrics (publications, citations, h-index, and hI, annual, an annualised individual h-index) and five major disciplines (Humanities, Social Sciences, Engineering, Sciences and Life Sciences). We show that both the data source and the specific metrics used change the conclusions that can be drawn from cross-disciplinary comparisons.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Notes

  1. 1.

    Two professors in Law and Physics unfortunately had to be removed from the final sample, as their publication patterns were very uncharacteristic of their field. The Law professor (now a Professor at Stanford with a dual appointment in Medicine and Law, and an Honorary Professor in the Medical Faculty at Melbourne) specialised in Health Law and had published mainly in medical journals. His publication pattern (large number of publications and many co-authors) and citation pattern (very high level of citations) was very atypical for the field of Law, which is characteristic by very low citation levels. He was also a Federation Fellow, i.e. a very high performing academic. Including him in the Humanities sample would have completely changed our results; in the Web of Science he had four times as many citations on his own as all of the 19 other Humanities academics combined. The Physics professor was a Particle Physicist. Many of her publications had more than a thousand authors. As a result her publication and citation metrics far exceeded that of any of the other academics in our sample, thus distorting any comparisons. Moreover, as the number of co-authors resulted in very large datafiles, we experienced problems in exporting the data from Scopus for this author as well as difficulties in importing the WoS datafiles into Publish or Perish, thus making data collection impossible in some months. Unfortunately, by the time we realised these problems, it was too late to select a replacement.

  2. 2.

    This obviously begs the question why we didn’t use the WoS Cited Reference search or the Scopus secondary documents in our comparison. For a comprehensive review of the many reasons why this is not a feasible option, please see Harzing (2013, pp. 1064–1065).

References

  1. Adler, N., & Harzing, A. W. (2009). When knowledge wins: Transcending the sense and nonsense of academic rankings. The Academy of Management Learning & Education, 8(1), 72–95.

    Article  Google Scholar 

  2. Adriaanse, L. S., & Rensleigh, C. (2013). Web of Science, Scopus and Google Scholar: A content comprehensiveness comparison. The Electronic Library, 31(6), 727–744.

    Article  Google Scholar 

  3. Amara, N., & Landry, R. (2012). Counting citations in the field of business and management: Why use Google Scholar rather than the Web of Science. Scientometrics, 93(3), 553–581.

    Article  Google Scholar 

  4. Bakkalbasi, N., Bauer, K., Glover, J., & Wang, L. (2006). Three options for citation tracking: Google Scholar, Scopus and Web of Science. Biomedical digital libraries, 3(1), 7.

    Article  Google Scholar 

  5. Bar-Ilan, J. (2010). Citations to the “Introduction to informetrics” indexed by WOS. Scopus and Google Scholar. Scientometrics, 82(3), 495–506.

    Google Scholar 

  6. Bergman, E. M. L. (2012). Finding citations to social work literature: The relative benefits of using Web of Science, Scopus, or Google Scholar. The Journal of Academic Librarianship, 38(6), 370–379.

    MathSciNet  Article  Google Scholar 

  7. Chrysomallis, M. (2014, December 8). Scopus continues to add pre-1996 citations [Web log post]. http://blog.scopus.com/posts/scopus-continues-to-add-pre-1996-citations

  8. De Groote, S. L., & Raszewski, R. (2012). Coverage of Google Scholar, Scopus, and Web of Science: A case study of the h-index in nursing. Nursing Outlook, 60(6), 391–400.

    Article  Google Scholar 

  9. De Winter, J. C., Zadpoor, A. A., & Dodou, D. (2014). The expansion of Google Scholar versus Web of Science: A longitudinal study. Scientometrics, 98(2), 1547–1565.

    Article  Google Scholar 

  10. Delgado-López-Cózar, E., & Repiso-Caballero, R. (2013). El impacto de las revistas de comunicación: Comparando Google Scholar Metrics, Web of Science y Scopus. Comunicar: Revista Científica de Comunicación y Educación, 21(41), 45–52.

    Article  Google Scholar 

  11. Etxebarria, G., & Gomez-Uranga, M. (2010). Use of Scopus and Google Scholar to measure social sciences production in four major Spanish universities. Scientometrics, 82(2), 333–349.

    Article  Google Scholar 

  12. Franceschet, M. (2010). A comparison of bibliometric indicators for computer science scholars and journals on Web of Science and Google Scholar. Scientometrics, 83(1), 243–258.

    Article  Google Scholar 

  13. Harzing, A. W. (2007) Publish or Perish. http://www.harzing.com/pop.htm

  14. Harzing, A. W. (2013). A preliminary test of Google Scholar as a source for citation data: A longitudinal study of Nobel Prize winners. Scientometrics, 93(3), 1057–1075.

    Article  Google Scholar 

  15. Harzing, A. W. (2014). A longitudinal study of Google Scholar coverage between 2012 and 2013. Scientometrics, 98(1), 565–575.

    Article  Google Scholar 

  16. Harzing, A. W., Alakangas, S., & Adams, D. (2014). hIa: An individual annual h-index to accommodate disciplinary and career length differences. Scientometrics, 99(3), 811–821.

    Article  Google Scholar 

  17. Harzing, A. W., & Mijnhardt, W. (2015). Proof over promise: Towards a more inclusive ranking of Dutch academics in Economics & Business. Scientometrics, 102(1), 727–749.

    Article  Google Scholar 

  18. Jacso, P. (2008). Testing the calculation of a realistic h-index in Google Scholar, Scopus, and Web of Science for FW Lancaster. Library Trends, 56(4), 784–815.

    Article  Google Scholar 

  19. Jacsó, P. (2010). Metadata mega mess in Google Scholar. Online Information Review, 34(1), 175–191.

    Article  Google Scholar 

  20. Kulkarni, A. V., Aziz, B., Shams, I., & Busse, J. W. (2009). Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. JAMA, 302(10), 1092–1096.

    Article  Google Scholar 

  21. Levine-Clark, M., & Gil, E. L. (2008). A comparative citation analysis of Web of Science, Scopus, and Google Scholar. Journal of Business & Finance Librarianship, 14(1), 32–46.

    Article  Google Scholar 

  22. London School of Economics and Political Science. (2011). Maximizing the impacts of your research: A handbook for social scientists. http://www2.lse.ac.uk/government/research/resgroups/LSEPublicPolicy/Docs/LSE_Impact_Handbook_April_2011.pdf

  23. Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105–2125.

    Article  Google Scholar 

  24. Mikki, S. (2010). Comparing Google Scholar and ISI Web of Science for earth sciences. Scientometrics, 82(2), 321–331.

    Article  Google Scholar 

  25. Minasny, B., Hartemink, A. E., McBratney, A., & Jang, H. J. (2013). Citations and the h index of soil researchers and journals in the Web of Science, Scopus, and Google Scholar. PeerJ, 1, e183.

    Article  Google Scholar 

  26. Mingers, J., & Lipitakis, E. (2010). Counting the citations: A comparison of Web of Science and Google Scholar in the field of business and management. Scientometrics, 85(2), 613–625.

    Article  Google Scholar 

  27. Mingers, J., & Willmott, H. (2013). Taylorizing business school research: On the ‘one best way’performative effects of journal ranking lists. Human Relations, 66(8), 1051–1073.

    Article  Google Scholar 

  28. Roales-Nieto, J. G., & O’Neill, B. (2012). A comparative study of journals quality based on web of science, scopus and google scholar: A case study with IJP&PT. International Journal of Psychology and Psychological Therapy, 12(3), 453–480.

    Google Scholar 

  29. Torres-Salinas, D., Lopez-Cózar, E. D., & Jiménez-Contreras, E. (2009). Ranking of departments and researchers within a university using two different databases: Web of Science versus Scopus. Scientometrics, 80(3), 761–774.

    Article  Google Scholar 

  30. Vaughan, L., & Shaw, D. (2008). A new look at evidence of scholarly citations in citation indexes and from web sources. Scientometrics, 74(2), 317–330.

    Article  Google Scholar 

  31. Vieira, E., & Gomes, J. (2009). A comparison of Scopus and Web of Science for a typical university. Scientometrics, 81(2), 587–600.

    Article  Google Scholar 

  32. Wildgaard, L. (2015). A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar. Scientometrics, 104(3), 1–34.

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Anne-Wil Harzing.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Harzing, AW., Alakangas, S. Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison. Scientometrics 106, 787–804 (2016). https://doi.org/10.1007/s11192-015-1798-9

Download citation

Keywords

  • Google Scholar
  • Scopus
  • Web of Science
  • H-index
  • hIa
  • Citation analysis
  • Research metrics