Abstract
Google Scholar is an appealing data source for the measurement of scientific production in the social sciences and humanities (SSH) fields. Its appeal derives from its extensive coverage of the literature. This contrasts with issues of data quality, which are still quite controversial. This chapter aims to evaluate the reliability and validity of bibliometric indicators taken from Scholar as well as their coverage of the scientific production in the social sciences. The analysis will be based on a comparison of Scholar with other bibliometric data sources (Web of Science and Scopus) plus an institutional dataset. The reliability of Scholar indicators will be investigated through correlational analysis, while their validity will be assessed using different external criteria (the results of national evaluation procedures based on a peer review approach). The analysis will be developed for the population of Italian university professors in a subset of SSH: political philosophy, history, political science and sociology. The final discussion of the results will take into account the various purposes that bibliometric exercises try to achieve.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
- 1.
- 2.
We would like to thank Andrea Bonaccorsi and Marco Malgarini, who allowed us to use SUA-RD 2011–2013 data.
- 3.
Given that the VQR scores are highly confidential, these analyses were conducted directly by ANVUR. In this regard, we thank Marco Malgarini, Tindaro Cicero and Marco De Santis Puzzonia from ANVUR for their precious collaboration. We also need to report that given some difficulties in combining the CAVIB dataset with that of the VQR 2004–2010, the analysis was conducted for 1394 over 1697 cases (82% of the total sample).
- 4.
If instead of the Spearman correlation coefficients we had calculated the Pearson’s coefficient, the results would have been even lower.
References
ANVUR. (30 June2013). Valutazione della Qualità̀ della Ricerca 2004-2010 (VQR 2004–2010). Rapporto finale di area. Gruppo di Esperti della Valutazione dell’Area 14 (GEV14), Rome.
Baccini, A. (2010). Valutare la ricerca scientifica. Uso e abuso degli indicatori bibliometrici. Bologna: Il Mulino.
Clemens, E. S., Powell, W. W., McIlwaine, K., & Okamoto, D. (1995). Careers in print: Books, journals, and scholarly reputations. American Journal of Sociology, 101(2), 433–494.
Cole, S., & Cole, J. R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American Sociological Review, 32(3), 377–390.
Corbetta, P. (2003). Social research. Theory, methods and techniques. Thousand Oaks: Sage.
Corbetta P., Gasperoni G. and Pisati M. (2001). Statistica per la ricerca sociale, il Mulino, Bologna.
Franceschet, M. (2011). PageRank: Standing on the shoulders of giants. Communications of the ACM, 54(6), 92–101.
Hamilton, L. C. (2012). Statistics with Stata: Updated for version 12. Belmont: Brooks/Cole, Cengage Learning.
Hicks, D. (2004). The four literatures of social science. In H. F. Moed, W. Glaenzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 473–496). Dordrecht: Kluwer Academic Publishers.
Hirsch, J.E. (15 Nov 2005). An index to quantify an individual’s scientific research output, in PNAS. Proceedings of the National Academy of Sciences of the United States of America, 102(46).
Martin, B. R., & Irvine, J. (1983). Assessing basic research. Some partial indicators of scientific progress in radio astronomy. In Research policy, vol. 12, (pp. 61–90); partial Italian translation in Viale, R. and Cerroni, A. (2003) (eds.) Valutare la scienza. Soveria Mannelli: Rubettino.
Menard, S. (2002). Applied logistic regression analysis (2nd ed.). Thousand Oaks: Sage.
Merton, R.K. (1973). The sociology of science. Chicago: The University of Chicago Press; Italian translation: Scienza, religione e politica, Bologna, Il Mulino, 2011.
Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.
Nederhof, A. J. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81–100.
Research Evaluation and Policy Project. (2005). Quantitative indicators for research assessment – A literature review, Discussion Paper 05/1, 30 March.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Biolcati-Rinaldi, F., Molteni, F., Salini, S. (2018). Assessing the Reliability and Validity of Google Scholar Indicators. The Case of Social Sciences in Italy. In: Bonaccorsi, A. (eds) The Evaluation of Research in Social Sciences and Humanities. Springer, Cham. https://doi.org/10.1007/978-3-319-68554-0_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-68554-0_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-68553-3
Online ISBN: 978-3-319-68554-0
eBook Packages: Social SciencesSocial Sciences (R0)