Skip to main content
SpringerLink
Log in

Filling the citation gap: measuring the multidimensional impact of the academic book at institutional level with PlumX

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

More than 5 years after their emergence, altmetrics are still seen as a promise to complement traditional citation-based indicators. However, no study has focused on their potential usefulness to capture the impact of scholarly books. While recent literature shows that citation indicators cannot fully capture the impact of books, other studies have suggested alternative indicators such as usage, publishers’ prestige or library holdings. In this paper, we calculate 18 indicators which range from altmetrics to library holdings, views, downloads or citations to the production of monographs of a Spanish university using the bibliometric suite PlumX from EBSCO. The objective of the study is to adopt a multidimensional perspective on the analysis of books and understand the level of complementarity between these different indicators. Also, we compare the overview offered by this range of indicators when applied to monographs with the traditional bibliometric perspective focused on journal articles and citation impact. We observe a low presence of altmetric indicators for monographs, even lower than for journal articles and a predominance of library holdings, confirming this indicator as the most promising one towards the analysis of the impact of books.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://plumanalytics.com/about/leadership/.

References

  • Bornmann, L. (2014). Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. Journal of Informetrics, 8(4), 895–903.

    Article  Google Scholar 

  • Cabezas-Clavijo, A., Robinson-García, N., Torres-Salinas, D., Jiménez-Contreras, E., Mikulka, T., Gumpenberger, C., Wemisch, A., & Gorraiz, J. (2013). Most borrowed is most cited? Library loan statistics as a proxy for monograph selection in citation indexes. In Proceedings of the 14th international conference on scientometrics and informetrics, pp. 1237–1252.

  • Chi, P.-S. (2016). Differing disciplinary citation concentration patterns of book and journal literatura? Journal of Informetrics, 10(3), 814–829.

    Article  Google Scholar 

  • Costas, R., Zahedi, Z., & Wouters, P. (2015). Do ‘altmetrics’ correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Science and Technology, 66(10), 2003–2019.

    Article  Google Scholar 

  • Eve, M. P. (2014). Monographs. In Open access and the humanities (pp. 86–111). Cambridge: Cambridge University Press.

  • Giménez-Toledo, E., Mañana-Rodríguez, J., Engels, T. C. E., Ingwersen, P., Pölönen, J., Sivertsen, G., et al. (2016). Taking scholarly books into account: current developments in five European countries. Scientometrics, 107(2), 685–699.

    Article  Google Scholar 

  • Giménez-Toledo, E., Tejada-Artigas, C., & Mañana-Rodríguez, J. (2013). Evaluation of scientific books’ publishers in social sciences and humanities: Results of a survey. Research Evaluation, 22(1), 64–77.

    Article  Google Scholar 

  • Glänzel, W., & Gorraiz, J. (2015). Usage metrics versus altmetrics: Confusing terminology? Scientometrics, 102(3), 2161–2164.

    Article  Google Scholar 

  • Gorraiz, J., Gumpenberger, C., & Glade, T. (2016). On the bibliometric coordinates of four different research fields in geography. Scientometrics, 107(2), 873–897.

    Article  Google Scholar 

  • Gorraiz, J., Gumpenberger, C., & Purnell, P. J. (2014a). The power of book reviews: A simple and transparent enhancement approach for Book Citation Indexes. Scientometrics, 98(2), 841–852.

    Article  Google Scholar 

  • Gorraiz, J., Gumpenberger, C., & Schloegl, C. (2014b). Usage versus citation behaviours in four subject areas. Scientometrics, 101(2), 1077–1095.

    Article  Google Scholar 

  • Gorraiz, J., Purnell, P. J., & Glänzel, W. (2013). Opportunities for and limitations of the Book Citation Index. Journal of the American Society for Information Science and Technology, 64(7), 1388–1398.

    Article  Google Scholar 

  • Haustein, S. (2014). Readership metrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 327–344). Cambridge: MIT Press.

    Google Scholar 

  • Haustein, S. (2016). Grand challenges in altmetrics: heterogeneity, data quality and dependencies. Scientometrics, 108(1), 413–423.

    Article  Google Scholar 

  • Huang, M., & Chang, Y. (2008). Characteristics of research output in social sciences and humanities: From a research evaluation perspective. Journal of the American Society for Information Science and Technology, 59(11), 1819–1828.

    Article  Google Scholar 

  • Kousha, K., & Thelwall, M. (2009). Google book search: Citation analysis for social science and the humanities. Journal of the American Society for Information Science and Technology, 60(8), 1537–1549.

    Article  Google Scholar 

  • Kousha, K., & Thelwall, M. (2015a). Alternative metrics for book impact assessment: Can choice reviews be a useful source? In Proceedings of the 15th international conference on scientometrics and informetrics, pp. 59–70.

  • Kousha, K., & Thelwall, M. (2015b). Can Amazon.com reviews help to assess the wider impacts of books? Journal of the Association for Information Science and Technology, 67(3), 566–581.

    Article  Google Scholar 

  • Kousha, K., & Thelwall, M. (2015c). Web indicators for research evaluation. Part 3: books and non-standard outputs. El profesional de la información, 24(6), 724–736.

    Article  Google Scholar 

  • Kousha, K., & Thelwall, M. (2016). Can Amazon. com reviews help to assess the wider impacts of books? Journal of the Association for Information Science and Technology, 67(3), 566–581.

    Article  Google Scholar 

  • Kousha, K., Thelwall, M., & Abdoli, M. (2016). Goodreads reviews to assess the wider impacts of books. Journal of the Association for Information Science and Technology, 68(8), 2004–2016.

    Article  Google Scholar 

  • Kousha, K., Thelwall, M., & Rezaie, S. (2011). Assessing the citation impact of books: The role of Google Books, Google Scholar, and Scopus. Journal of the American Society for Information Science and Technology, 62(11), 2147–2164.

    Article  Google Scholar 

  • Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, 1–64.

    Article  Google Scholar 

  • Leydesdorff, L., & Felt, U. (2012). Edited volumes, monographs and book chapters in the Book Citation Index (BKCI) and science citation index (SCI, SoSCI, A&HCI). Journal of Scientometric Research, 1(1), 28–34.

    Article  Google Scholar 

  • Linmans, A. J. M. (2010). Why with bibliometrics the humanities does not need to be the weakest link. Indicators for research evaluation based on citations, library bindings and productivity measures. Scientometrics, 83(2), 337–354.

    Article  Google Scholar 

  • Nederhof, A. (2006). Bibliometric monitoring of research performance in the social sciences and the humanities: A review. Scientometrics, 66(1), 81–100.

    Article  MathSciNet  Google Scholar 

  • Peters, I., Kraker, P., Lex, E., Gumpenberger, C., & Gorraiz, J. (2016). Research data explored: An extended analysis of citations and altmetrics. Scientometrics, 107(2), 723–744.

    Article  Google Scholar 

  • Priem, J. (2010). I like the term #articlelevelmetrics, but it fails to imply *diversity* of measures. Lately, I’m liking #altmetrics. Retrieved from https://twitter.com/jasonpriem/status/25844968813.

  • Priem, J. (2014). Altmetrics. In B. Cronin & C. R. Sugimoto (Eds.), Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact (pp. 263–288). Cambridge, MA: MIT Press.

    Google Scholar 

  • Rafols, I., Ciarli, T., van Zwanenberg, P., & Stirling, A. (2012). Towards indicators for ‘opening up’ science and technology policy. In Proceedings of 17 th International Conference on Science and Technology Indicators (Vol. 2, pp. 663–674).

  • REF (2014). Results and submissions in Research Excellence Framework. http://results.ref.ac.uk/.

  • Robinson-Garcia, N., Torres-Salinas, D., Zahedi, Z., & Costas, R. (2014). New data, new possibilities: Exploring the insides of Altmetric.com. El Profesional de la Información, 23(4), 359–366.

    Article  Google Scholar 

  • Schlögl, C., & Gorraiz, J. (2006). Document delivery as a source for bibliometric analyses: the case of Subito. Journal of Information Science, 32(3), 223–237.

    Article  Google Scholar 

  • Small, H. (2013). The value of the humanities. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017). Scholarly use of social media and altmetrics: A review of the literature. Journal of the Association for Information Science and Technology, 68(9), 2037–2062. doi:10.1002/asi.23833.

    Article  Google Scholar 

  • Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.

    Article  Google Scholar 

  • Torres-Salinas, D., & Moed, H. F. (2009). Library catalog analysis as a tool in studies of social sciences and humanities: An exploratory study on published book titles in economics. Journal of Informetrics, 3(1), 9–26.

    Article  Google Scholar 

  • Torres-Salinas, D., Robinson-García, N., Campanario, J. M., & López-Cózar, Delgado. (2014a). Coverage, specialization and impact of scientific publishers in the ‘Book Citation Index’. Online Information Review, 38(1), 24–42.

    Article  Google Scholar 

  • Torres-Salinas, D., Robinson-García, N., Fernandez-Valdivia, J., & García, J. A. (2014b). Analyzing the citation characteristics of books: Edited books, book series and publisher types in the Book Citation Index. Scientometrics, 98(3), 2113–2127.

    Article  Google Scholar 

  • Torres-Salinas, D., Robinson-García, N., Jiménez-Contreras, E., & Delgado López-Cózar, E. (2012). Towards a ‘Book Publishers Citation Reports’. First approach using the ‘Book Citation Index’. Revista Española de Documentación Científica, 35(4), 615–620.

    Article  Google Scholar 

  • Torres-Salinas, D., Rodriguez-Sánchez, R., Robinson-García, N., Fdez-Valdivia, J., & García, J. A. (2013). Mapping citation patterns of book chapters using the Book Citation Index. Journal of Informetrics, 7(2), 412–424.

    Article  Google Scholar 

  • Watkinson, A., Nicholas, D., Thornley, C., Herman, E., Jamali, H. R., Volentine, R., et al. (2016). Changes in the digital scholarly environment and issues of trust: An exploratory, qualitative analysis. Information Processing and Management, 52(3), 446–458.

    Article  Google Scholar 

  • White, H. D., Boell, S. K., Yu, H., Davis, M., Wilson, C. S., & Cole, F. T. H. (2009). Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences. Journal of the American Society for Information Science and Technology, 60(6), 1083–1096.

    Article  Google Scholar 

  • Williams, P., Stevenson, L., Nicholas, D., Watkinson, A., & Rowlands, I. (2009). The role and future of the monograph in arts and humanities research. Aslib Proceedings, 61(1), 67–82.

    Article  Google Scholar 

  • Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., et al. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management.. doi:10.13140/RG.2.1.4929.1363.

  • Zuccala, A., & Cornacchia, R. (2016). Data matching, integration, and interoperability for a metric assessment of monographs. Scientometrics, 108(1), 465–484.

    Article  Google Scholar 

  • Zuccala, A., & Van Leeuwen, T. (2011). Book reviews in humanities research evaluations. Journal of the American Society for Information Science and Technology, 62(10), 1979–1991.

    Article  Google Scholar 

  • Zuccala, A. A., Verleysen, F. T., Cornacchia, R., & Engels, T. C. (2015). Altmetrics for the humanities: Comparing Goodreads reader ratings with citations to history books. Aslib Journal of Information Management, 67(3), 320–336.

    Article  Google Scholar 

Download references

Acknowledgements

The authors thank Stephan Buettgen (EBSCO) for granted trial access to PlumX. Nicolas Robinson-Garcia is currently supported by a Juan de la Cierva-Formacion grant from the Spanish Ministry of Economy and Competitiveness.

Author information

Authors and Affiliations

  1. Universidad de Granada (EC3metrics & Medialab UGR), Granada, Spain

    Daniel Torres-Salinas

  2. Universidad de Navarra, Pamplona, Spain

    Daniel Torres-Salinas

  3. INGENIO (CSIC-UPV), Universitat Politècnica de València, Valencia, Spain

    Nicolás Robinson-Garcia

  4. Vienna University Library, University of Vienna, Vienna, Austria

    Juan Gorraiz

Authors
  1. Daniel Torres-Salinas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicolás Robinson-Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Juan Gorraiz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicolás Robinson-Garcia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Torres-Salinas, D., Robinson-Garcia, N. & Gorraiz, J. Filling the citation gap: measuring the multidimensional impact of the academic book at institutional level with PlumX. Scientometrics 113, 1371–1384 (2017). https://doi.org/10.1007/s11192-017-2539-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2539-z

Keywords

Search

Navigation