, Volume 114, Issue 3, pp 1251–1273 | Cite as

A novel method for depicting academic disciplines through Google Scholar Citations: The case of Bibliometrics

  • Alberto Martín-Martín
  • Enrique Orduna-Malea
  • Emilio Delgado López-Cózar


This article describes a procedure to generate a snapshot of the structure of a specific scientific community and their outputs based on the information available in Google Scholar Citations (GSC). We call this method multifaceted analysis of disciplines through academic profiles (MADAP). The international community of researchers working in Bibliometrics, Scientometrics, Informetrics, Webometrics, and Altmetrics was selected as a case study. The records of the top 1000 most cited documents by these authors according to GSC were manually processed to fill any missing information and deduplicate fields like the journal titles and book publishers. The results suggest that it is feasible to use GSC and the MADAP method to produce an accurate depiction of the community of researchers working in Bibliometrics (both specialists and occasional researchers) and their publication habits (main publication venues such as journals and book publishers). Additionally, the wide document coverage of Google Scholar (specially books and book chapters) enables more comprehensive analyses of the documents published in a specific discipline than were previously possible with other citation indexes, finally shedding light on what until now had been a blind spot in most citation analyses.


Academic profiles Google Scholar Citations Bibliometrics Scientometrics Informetrics Webometrics Altmetrics Academic search engines Scientific disciplines MADAP method 



Funding was provided by Ministerio de Educación, Cultura y Deporte (FPU2013/05863), Universitat Politècnica de València (PAID-10-14).


  1. Aguillo, I. F. (2012). Is Google Scholar useful for bibliometrics? A webometric analysis. Scientometrics, 91(2), 343–351.CrossRefGoogle Scholar
  2. Almind, T. C., & Ingwersen, P. (1997). Informetric analyses on the world wide web: methodological approaches to ‘webometrics’. Journal of Documentation, 53(4), 404–426.CrossRefGoogle Scholar
  3. Bar-Ilan, J. (2008). Informetrics at the beginning of the 21st century—A review. Journal of Informetrics, 2(1), 1–52.CrossRefGoogle Scholar
  4. Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars’ visibility on the social Web. In É. Archambault, Y. Gingras, & V. Larivière (Eds.). Proceedings of the international conference on science and technology indicators (STI 2012) (pp. 99–109).Google Scholar
  5. Becher, T., & Trowler, P. (2001). Academic tribes and territories: Intellectual enquiry and the culture of disciplines. London: McGraw-Hill Education.Google Scholar
  6. Bensman, S. J. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41(1), 93–155.CrossRefGoogle Scholar
  7. Björneborn, L., & Ingwersen, P. (2004). Toward a basic framework for webometrics. Journal of the American Society for Information Science and Technology, 55(14), 1216–1227.CrossRefGoogle Scholar
  8. Bonitz, M. (1982). Scientometrie, bibliometrie, informetrie. Zentralblatt für Bibliothekswesen, 96(2), 19–24.Google Scholar
  9. Borgman, C. L., & Furner, J. (2002). Scholarly communication and bibliometrics. Annual Review of Information Science and Technology, 36, 3–72.Google Scholar
  10. Bornmann, L., Thor, A., Marx, W., & Schier, H. (2016). The application of bibliometrics to research evaluation in the humanities and social sciences: An exploratory study using normalized Google Scholar data for the publications of a research institute. Journal of the Association for Information Science and Technology, 67(11), 2778–2789.CrossRefGoogle Scholar
  11. Braun, T. (1994). Little scientometrics, big scientometrics… and beyond? Scientometrics, 30(2–3), 373–537.CrossRefGoogle Scholar
  12. Broadus, R. N. (1987a). Early approaches to bibliometrics. Journal of the American Society for Information Science, 38(2), 127–129.CrossRefGoogle Scholar
  13. Broadus, R. N. (1987b). Toward a definition of ‘bibliometrics’. Scientometrics, 12(5–6), 373–379.CrossRefGoogle Scholar
  14. Brookes, B. C. (1988). Comments on the scope of bibliometrics. In L. Egghe & R. Rousseau (Eds.), Informetrics 87/88. Select proceedings of the first international conference on bibliometrics and theoretical aspects of information retrieval (pp. 29–41). Amsterdam: Elsevier.Google Scholar
  15. Brookes, B. C. (1990). Biblio-, Sciento-, Infor-metrics??? What are we talking about? In L. Egghe & R. Rousseau (Eds.), Informetrics 89/90. Selection of Papers Submitted for the Second International Conference on Bibliometrics, Scientometrics and Informetrics (pp. 31–43). Amsterdam: Elsevier.Google Scholar
  16. Cronin, B. (2001). Bibliometrics and beyond: Some thoughts on web-based citation analysis. Journal of Information Science, 27(1), 1–7.CrossRefGoogle Scholar
  17. De Bellis, N. (2009). Bibliometrics and citation analysis: from the science citation index to cybermetrics. Maryland: Scarecrow Press.Google Scholar
  18. Delgado López-Cózar, E., Robinson-García, N., & Torres-Salinas, D. (2014). The Google Scholar experiment: How to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65(3), 446–454.CrossRefGoogle Scholar
  19. Egghe, L., & Rousseau, R. (1990). Introduction to informetrics: Quantitative methods in library, documentation and information science. Amsterdam: Elsevier.Google Scholar
  20. Garfield, E. (1970). Citation indexing for studying science. Nature, 227(5259), 669–671.CrossRefGoogle Scholar
  21. Glänzel, W., & Schoepflin, U. (1994). Little scientometrics, big scientometrics … and beyond? Scientometrics, 30(2–3), 375–384.CrossRefGoogle Scholar
  22. Godin, B. (2006). On the origins of bibliometrics. Scientometrics, 68(1), 109–133.CrossRefGoogle Scholar
  23. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163.CrossRefGoogle Scholar
  24. Hertzel, D. H. (1987). History of the development of ideas in bibliometrics. In A. Kent (Ed.), Encyclopedia of library and information sciences (pp. 144–219). New York: Marcel Dekker.Google Scholar
  25. Hood, W., & Wilson, C. (2001). The literature of bibliometrics, scientometrics, and informetrics. Scientometrics, 52(2), 291–314.CrossRefGoogle Scholar
  26. Jacsó, P. (2005). Google Scholar: The pros and the cons. Online Information Review, 29(2), 208–214.CrossRefGoogle Scholar
  27. Jacsó, P. (2008). Google scholar revisited. Online information review, 32(1), 102–114.CrossRefGoogle Scholar
  28. Jacsó, P. (2012). Google Scholar Author Citation Tracker: Is it too little, too late? Online Information Review, 36(1), 126–141.CrossRefGoogle Scholar
  29. Kramer, B., & Bosman, J. (2015). 101 innovations in scholarly communicationThe changing research workflow. Available at
  30. Larivière, V. (2012). The decade of metrics? Examining the evolution of metrics within and outside LIS. Bulletin of the American Society for Information Science and Technology, 38(6), 12–17.CrossRefGoogle Scholar
  31. Larivière, V., Sugimoto, C., & Cronin, B. (2012). A bibliometric chronicling of library and information science’s first hundred years. Journal of the American Society for Information Science and Technology, 63(5), 997–1016.CrossRefGoogle Scholar
  32. Lawani, S. M. (1981). Bibliometrics: Its theoretical foundations, methods and applications. Libri, 31(1), 294–315.CrossRefGoogle Scholar
  33. Lin, J., & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quarterly, 25(2), 20–26.CrossRefGoogle Scholar
  34. Martín-Martín, A., Orduna-Malea, E., Ayllón, Juan M., & Delgado López-Cózar, E. (2016). A two-sided academic landscape: Snapshot of highly-cited documents in Google Scholar (1950–2013). Revista Española de Documentación Científica, 39(4), e149.CrossRefGoogle Scholar
  35. Martin-Martin, A., Orduna-Malea, E., Harzing, A. W., & Delgado López-Cózar, E. (2017). Can we use Google Scholar to identify highly-cited documents? Journal of Informetrics, 11(1), 152–163.CrossRefGoogle Scholar
  36. McCain, K. W. (2010). The view from Garfield’s shoulders: Tri-citation mapping of Eugene Garfield’s citation image over three successive decades. Annals of Library and Information Studies, 57, 261–270.Google Scholar
  37. Meho, L. I., & Yang, K. (2007). Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar. Journal of the Association for Information Science and Technology, 58(13), 2105–2125.Google Scholar
  38. Mikki, S., Zygmuntowska, M., Gjesdal, Ø. L., & Al Ruwehy, H. A. (2015). Digital presence of Norwegian Scholars on academic network sites—Where and who are they?’. PLoS ONE, 10(11), e0142709.CrossRefGoogle Scholar
  39. Milojević, S., & Leydesdorff, L. (2013). Information metrics (iMetrics): A research specialty with a socio-cognitive identity? Scientometrics, 95(1), 141–157.CrossRefGoogle Scholar
  40. Mingers, J., & Meyer, M. (2017). Normalizing Google Scholar data for use in research evaluation. Scientometrics, 112(2), 1111–1121.CrossRefGoogle Scholar
  41. Mingers, J., O’Hanley, J. R., & Okunola, M. (2017). Using Google Scholar institutional level data to evaluate the quality of university research. Scientometrics, 113(3), 1627–1643.CrossRefGoogle Scholar
  42. Narin, F., & Moll, J. K. (1977). Bibliometrics. Annual Review of Information Science and Technology, 12, 35–58.Google Scholar
  43. Nicolaisen, J., & Frandsen, T. F. (2015). Bibliometric evolution: Is the journal of the association for information science and technology transforming into a specialty Journal? Journal of the Association for Information Science and Technology, 66(5), 1082–1085.CrossRefGoogle Scholar
  44. Orduna-Malea, E., Ayllón, J. M., Martín-Martín, A., & Delgado López-Cózar, E. (2015). Methods for estimating the size of Google Scholar. Scientometrics, 104(3), 931–949.CrossRefGoogle Scholar
  45. Orduna-Malea, E., Martín-Martín, A., Ayllón, Juan M., & Delgado López-Cózar, E. (2016a). La revolución Google Scholar: Destapando la caja de Pandora académica. Granada: UNE.Google Scholar
  46. Orduna-Malea, E., Martín-Martín, A., & Delgado-López-Cózar, E. (2016b). The next bibliometrics: ALMetrics (Author Level Metrics) and the multiple faces of author impact. El Profesional de la Información, 25(3), 485–496.CrossRefGoogle Scholar
  47. Ortega, Jose L. (2015a). How is an academic social site populated? A demographic study of Google Scholar Citations population. Scientometrics, 104(1), 1–18.CrossRefGoogle Scholar
  48. Ortega, Jose L. (2015b). Relationship between altmetric and bibliometric indicators across academic social sites: The case of CSIC’s members. Journal of Informetrics, 9(1), 39–49.MathSciNetCrossRefGoogle Scholar
  49. Ortega, Jose L., & Aguillo, Isidro F. (2012). Science is all in the eye of the beholder: Keyword maps in Google Scholar Citations. Journal of the American Society for Information Science and Technology, 63(12), 2370–2377.CrossRefGoogle Scholar
  50. Ortega, Jose L., & Aguillo, Isidro F. (2013). Institutional and country collaboration in an online service of scientific profiles: Google Scholar Citations. Journal of Informetrics, 7(2), 394–403.CrossRefGoogle Scholar
  51. Ortega, Jose L., & Aguillo, Isidro F. (2014). Microsoft academic search and google scholar citations: Comparative analysis of author profiles. Journal of the Association for Information Science and Technology, 65(6), 1149–1156.CrossRefGoogle Scholar
  52. Otlet, P. (1934). Traité de documentation: Le livre sur le livre, théorie et pratique. Brussels: Editiones Mundaneum.Google Scholar
  53. Peritz, B. (1984). On the careers of terminologies; The case of bibliometrics. Libri, 34(1), 233–242.CrossRefGoogle Scholar
  54. Portal, S. G. (1994). Principios teóricos y metodológicos de los estudios métricos de la información. Investigación Bibliotecológica, 8, 23–32.Google Scholar
  55. Priem, J., & Hemminger, B. H. (2010). Scientometrics 2.0: New metrics of scholarly impact on the social Web. First Monday, 15(7).Google Scholar
  56. Prins, A. A. M., Costas, R., Van Leeuwen, T. N., & Wouters, Paul F. (2016). Using Google Scholar in research evaluation of humanities and social science programs: A comparison with Web of Science data. Research Evaluation, 25(3), 264–270.CrossRefGoogle Scholar
  57. Ranganathan, S. R. (1969). Documentation Research and Training Centre: Annual Seminar (7) (1969): Subject analysis for document finding systems: Quantification and librametric studies: Management of translation service (pp. 285–301). Bangalore: Documentation Research and Training Centre.Google Scholar
  58. Sengupta, I. N. (1992). Bibliometrics, informetrics, scientometrics and librametrics: an overview. Libri, 42(2), 75–98.CrossRefGoogle Scholar
  59. Shapiro, Fred R. (1992). Origins of Bibliometrics, Citation Indexing, and Citation Analysis: The Neglected Legal Literature. Journal of the American Society for Information Science, 43(5), 337–339.CrossRefGoogle Scholar
  60. Small, H. (2017). A tribute to Eugene Garfield: Information innovator and idealist. Journal of Informetrics, 1(3), 599–612.CrossRefGoogle Scholar
  61. Stuart, D. (2014). Web metrics for library and information professionals. London: Facet publishing.Google Scholar
  62. Tague-Sutcliffe, J. (1992). An introduction to informetrics. Information Processing and Management, 28(1), 1–3.CrossRefGoogle Scholar
  63. Thelwall, M. (2008). Bibliometrics to webometrics. Journal of Information Science, 34(4), 605–621.CrossRefGoogle Scholar
  64. Thelwall, M. (2009). Introduction to webometrics: Quantitative web research for the social sciences. San Diego: Morgan & Claypool.Google Scholar
  65. Thelwall, M., Vaughan, L., & Björneborn, L. (2005). Webometrics. Annual Review of Information Science and Technology, 39(1), 81–135.CrossRefGoogle Scholar
  66. Van Raan, A. (1997). Scientometrics: State-of-the-art. Scientometrics, 38(1), 205–218.CrossRefGoogle Scholar
  67. White, H. D., & McCain, K. W. (1989). Bibliometrics. Annual Review of Information Science and Technology, 24, 119–186.Google Scholar
  68. White, H. D., & McCain, K. W. (1998). Visualizing a discipline: An author co-citation analysis of information science, 1972–1995. Journal of the American Society for Information Science, 49(4), 327–355.Google Scholar
  69. Whitley, R. (1984). The intellectual and social organization of the sciences. Oxford: Oxford University Press.Google Scholar
  70. Wilson, C. S. (1999). Informetrics. Annual Review of Information Science and Technology, 34, 107–247.Google Scholar
  71. Wouters, P. (2017). Eugene Garfield (1925–2017). Nature, 543(7642), 492.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  • Alberto Martín-Martín
    • 1
  • Enrique Orduna-Malea
    • 2
  • Emilio Delgado López-Cózar
    • 1
  1. 1.Facultad de Comunicación y DocumentaciónUniversidad de GranadaGranadaSpain
  2. 2.Universitat Politècnica de ValènciaValenciaSpain

Personalised recommendations