, Volume 113, Issue 3, pp 1371–1384 | Cite as

Filling the citation gap: measuring the multidimensional impact of the academic book at institutional level with PlumX

  • Daniel Torres-Salinas
  • Nicolás Robinson-GarciaEmail author
  • Juan Gorraiz


More than 5 years after their emergence, altmetrics are still seen as a promise to complement traditional citation-based indicators. However, no study has focused on their potential usefulness to capture the impact of scholarly books. While recent literature shows that citation indicators cannot fully capture the impact of books, other studies have suggested alternative indicators such as usage, publishers’ prestige or library holdings. In this paper, we calculate 18 indicators which range from altmetrics to library holdings, views, downloads or citations to the production of monographs of a Spanish university using the bibliometric suite PlumX from EBSCO. The objective of the study is to adopt a multidimensional perspective on the analysis of books and understand the level of complementarity between these different indicators. Also, we compare the overview offered by this range of indicators when applied to monographs with the traditional bibliometric perspective focused on journal articles and citation impact. We observe a low presence of altmetric indicators for monographs, even lower than for journal articles and a predominance of library holdings, confirming this indicator as the most promising one towards the analysis of the impact of books.


Books Monographs Altmetrics Usage metrics Plum analytics Citation analysis 



The authors thank Stephan Buettgen (EBSCO) for granted trial access to PlumX. Nicolas Robinson-Garcia is currently supported by a Juan de la Cierva-Formacion grant from the Spanish Ministry of Economy and Competitiveness.


  1. Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.CrossRefGoogle Scholar
  2. Cabezas-Clavijo, A., Robinson-García, N., Torres-Salinas, D., Jiménez-Contreras, E., Mikulka, T., Gumpenberger, C., Wemisch, A., & Gorraiz, J. (2013). Most borrowed is most cited? Library loan statistics as a proxy for monograph selection in citation indexes. In Proceedings of the 14th international conference on scientometrics and informetrics, pp. 1237–1252.Google Scholar
  3. Chi, P.-S. (2016). Differing disciplinary citation concentration patterns of book and journal literatura? Journal of Informetrics, 10(3), 814–829.CrossRefGoogle Scholar
  4. Costas, R., Zahedi, Z., & Wouters, P. (2015). Do ‘altmetrics’ correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.CrossRefGoogle Scholar
  5. Eve, M. P. (2014). Monographs. In Open access and the humanities (pp. 86–111). Cambridge: Cambridge University Press.Google Scholar
  6. Giménez-Toledo, E., Mañana-Rodríguez, J., Engels, T. C. E., Ingwersen, P., Pölönen, J., Sivertsen, G., et al. (2016). Taking scholarly books into account: current developments in five European countries. Scientometrics, 107(2), 685–699.CrossRefGoogle Scholar
  7. Giménez-Toledo, E., Tejada-Artigas, C., & Mañana-Rodríguez, J. (2013). Evaluation of scientific books’ publishers in social sciences and humanities: Results of a survey. Research Evaluation, 22(1), 64–77.CrossRefGoogle Scholar
  8. Glänzel, W., & Gorraiz, J. (2015). Usage metrics versus altmetrics: Confusing terminology? Scientometrics, 102(3), 2161–2164.CrossRefGoogle Scholar
  9. Gorraiz, J., Gumpenberger, C., & Glade, T. (2016). On the bibliometric coordinates of four different research fields in geography. Scientometrics, 107(2), 873–897.CrossRefGoogle Scholar
  10. Gorraiz, J., Gumpenberger, C., & Purnell, P. J. (2014a). The power of book reviews: A simple and transparent enhancement approach for Book Citation Indexes. Scientometrics, 98(2), 841–852.CrossRefGoogle Scholar
  11. Gorraiz, J., Gumpenberger, C., & Schloegl, C. (2014b). Usage versus citation behaviours in four subject areas. Scientometrics, 101(2), 1077–1095.CrossRefGoogle Scholar
  12. Gorraiz, J., Purnell, P. J., & Glänzel, W. (2013). Opportunities for and limitations of the Book Citation Index. Journal of the American Society for Information Science and Technology, 64(7), 1388–1398.CrossRefGoogle Scholar
  13. Haustein, S. (2014). Readership metrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 327–344). Cambridge: MIT Press.Google Scholar
  14. Haustein, S. (2016). Grand challenges in altmetrics: heterogeneity, data quality and dependencies. Scientometrics, 108(1), 413–423.CrossRefGoogle Scholar
  15. Huang, M., & Chang, Y. (2008). Characteristics of research output in social sciences and humanities: From a research evaluation perspective. Journal of the American Society for Information Science and Technology, 59(11), 1819–1828.CrossRefGoogle Scholar
  16. Kousha, K., & Thelwall, M. (2009). Google book search: Citation analysis for social science and the humanities. Journal of the American Society for Information Science and Technology, 60(8), 1537–1549.CrossRefGoogle Scholar
  17. Kousha, K., & Thelwall, M. (2015a). Alternative metrics for book impact assessment: Can choice reviews be a useful source? In Proceedings of the 15th international conference on scientometrics and informetrics, pp. 59–70.Google Scholar
  18. Kousha, K., & Thelwall, M. (2015b). Can reviews help to assess the wider impacts of books? Journal of the Association for Information Science and Technology, 67(3), 566–581.CrossRefGoogle Scholar
  19. Kousha, K., & Thelwall, M. (2015c). Web indicators for research evaluation. Part 3: books and non-standard outputs. El profesional de la información, 24(6), 724–736.CrossRefGoogle Scholar
  20. Kousha, K., & Thelwall, M. (2016). Can Amazon. com reviews help to assess the wider impacts of books? Journal of the Association for Information Science and Technology, 67(3), 566–581.CrossRefGoogle Scholar
  21. Kousha, K., Thelwall, M., & Abdoli, M. (2016). Goodreads reviews to assess the wider impacts of books. Journal of the Association for Information Science and Technology, 68(8), 2004–2016.CrossRefGoogle Scholar
  22. Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62(11), 2147–2164.CrossRefGoogle Scholar
  23. Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, 1–64.CrossRefGoogle Scholar
  24. Leydesdorff, L., & Felt, U. (2012). Edited volumes, monographs and book chapters in the Book Citation Index (BKCI) and science citation index (SCI, SoSCI, A&HCI). Journal of Scientometric Research, 1(1), 28–34.CrossRefGoogle Scholar
  25. Linmans, A. J. M. (2010). Why with bibliometrics the humanities does not need to be the weakest link. Indicators for research evaluation based on citations, library bindings and productivity measures. Scientometrics, 83(2), 337–354.CrossRefGoogle Scholar
  26. Nederhof, A. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81–100.CrossRefMathSciNetGoogle Scholar
  27. Peters, I., Kraker, P., Lex, E., Gumpenberger, C., & Gorraiz, J. (2016). Research data explored: An extended analysis of citations and altmetrics. Scientometrics, 107(2), 723–744.CrossRefGoogle Scholar
  28. Priem, J. (2010). I like the term #articlelevelmetrics, but it fails to imply *diversity* of measures. Lately, I’m liking #altmetrics. Retrieved from
  29. Priem, J. (2014). Altmetrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 263–288). Cambridge, MA: MIT Press.Google Scholar
  30. Rafols, I., Ciarli, T., van Zwanenberg, P., & Stirling, A. (2012). Towards indicators for ‘opening up’ science and technology policy. In Proceedings of 17 th International Conference on Science and Technology Indicators (Vol. 2, pp. 663–674).Google Scholar
  31. REF (2014). Results and submissions in Research Excellence Framework.
  32. Robinson-Garcia, N., Torres-Salinas, D., Zahedi, Z., & Costas, R. (2014). New data, new possibilities: Exploring the insides of El Profesional de la Información, 23(4), 359–366.CrossRefGoogle Scholar
  33. Schlögl, C., & Gorraiz, J. (2006). Document delivery as a source for bibliometric analyses: the case of Subito. Journal of Information Science, 32(3), 223–237.CrossRefGoogle Scholar
  34. Small, H. (2013). The value of the humanities. Oxford: Oxford University Press.CrossRefGoogle Scholar
  35. Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–2062. doi: 10.1002/asi.23833.CrossRefGoogle Scholar
  36. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.CrossRefGoogle Scholar
  37. Torres-Salinas, D., & Moed, H. F. (2009). Library catalog analysis as a tool in studies of social sciences and humanities: An exploratory study on published book titles in economics. Journal of Informetrics, 3(1), 9–26.CrossRefGoogle Scholar
  38. Torres-Salinas, D., Robinson-García, N., Campanario, J. M., & López-Cózar, Delgado. (2014a). Coverage, specialization and impact of scientific publishers in the ‘Book Citation Index’. Online Information Review, 38(1), 24–42.CrossRefGoogle Scholar
  39. Torres-Salinas, D., Robinson-García, N., Fernandez-Valdivia, J., & García, J. A. (2014b). Analyzing the citation characteristics of books: Edited books, book series and publisher types in the Book Citation Index. Scientometrics, 98(3), 2113–2127.CrossRefGoogle Scholar
  40. Torres-Salinas, D., Robinson-García, N., Jiménez-Contreras, E., & Delgado López-Cózar, E. (2012). Towards a ‘Book Publishers Citation Reports’. First approach using the ‘Book Citation Index’. Revista Española de Documentación Científica, 35(4), 615–620.CrossRefGoogle Scholar
  41. Torres-Salinas, D., Rodriguez-Sánchez, R., Robinson-García, N., Fdez-Valdivia, J., & García, J. A. (2013). Mapping citation patterns of book chapters using the Book Citation Index. Journal of Informetrics, 7(2), 412–424.CrossRefGoogle Scholar
  42. Watkinson, A., Nicholas, D., Thornley, C., Herman, E., Jamali, H. R., Volentine, R., et al. (2016). Changes in the digital scholarly environment and issues of trust: An exploratory, qualitative analysis. Information Processing and Management, 52(3), 446–458.CrossRefGoogle Scholar
  43. White, H. D., Boell, S. K., Yu, H., Davis, M., Wilson, C. S., & Cole, F. T. H. (2009). Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences. Journal of the American Society for Information Science and Technology, 60(6), 1083–1096.CrossRefGoogle Scholar
  44. Williams, P., Stevenson, L., Nicholas, D., Watkinson, A., & Rowlands, I. (2009). The role and future of the monograph in arts and humanities research. Aslib Proceedings, 61(1), 67–82.CrossRefGoogle Scholar
  45. Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., et al. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management.. doi: 10.13140/RG.2.1.4929.1363.
  46. Zuccala, A., & Cornacchia, R. (2016). Data matching, integration, and interoperability for a metric assessment of monographs. Scientometrics, 108(1), 465–484.CrossRefGoogle Scholar
  47. Zuccala, A., & Van Leeuwen, T. (2011). Book reviews in humanities research evaluations. Journal of the American Society for Information Science and Technology, 62(10), 1979–1991.CrossRefGoogle Scholar
  48. Zuccala, A. A., Verleysen, F. T., Cornacchia, R., & Engels, T. C. (2015). Altmetrics for the humanities: Comparing Goodreads reader ratings with citations to history books. Aslib Journal of Information Management, 67(3), 320–336.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2017

Authors and Affiliations

  1. 1.Universidad de Granada (EC3metrics & Medialab UGR)GranadaSpain
  2. 2.Universidad de NavarraPamplonaSpain
  3. 3.INGENIO (CSIC-UPV)Universitat Politècnica de ValènciaValenciaSpain
  4. 4.Vienna University LibraryUniversity of ViennaViennaAustria

Personalised recommendations