Assessing the impact of scholarly books is a difficult research evaluation problem. Library Catalog Analysis facilitates the quantitative study, at different levels, of the impact and diffusion of academic books based on data about their availability in libraries. The WorldCat global catalog collates data on library holdings, offering a range of tools including the novel WorldCat Identities. This is based on author profiles and provides indicators relating to the availability of their books in library catalogs. Here, we investigate this new tool to identify its strengths and weaknesses based on a sample of Bibliometrics and Scientometrics authors. We review the problems that this entails and compare Library Catalog Analysis indicators with Google Scholar and Web of Science citations. The results show that WorldCat Identities can be a useful tool for book impact assessment but the value of its data is undermined by the provision of massive collections of ebooks to academic libraries.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
All data are available at https://github.com/Wences91/library_catalog_wi/.
Information drawn from the Directory of OCLC Members: https://www.oclc.org/en/contacts/libraries.html. Note that some OCLC sources put the number of member libraries at 17,983: https://www.oclc.org/en/about.html.
Scholar Mirrors: [2020-02-11]: http://www.scholar-mirrors.infoec3.es/.
Scholar Mirrors: Methodology [11/02/2020]: http://www.scholar-mirrors.infoec3.es/layout.php?id=methodology.
WorldCat Identities API: https://pypi.org/project/worldcatidentities/.
Archambault, É., Vignola-Gagné, É., Côté, G., Larivière, V., & Gingrasb, Y. (2006). Benchmarking scientific output in the social sciences and humanities: The limits of existing databases. Scientometrics, 68(3), 329–342. https://doi.org/10.1007/s11192-006-0115-z.
Biagetti, M. T., Iacono, A., & Trombone, A. (2018). Testing library catalog analysis as a bibliometric indicator for research evaluation in Social Sciences and Humanities. In Challenges and opportunities for knowledge organization in the digital age: Proceedings of the fifteenth international ISKO conference 9-11 July 2018 Porto, Portugal (pp. 892–899). Baden-Baden: Ergon-Verlag. https://doi.org/10.5771/9783956504211-892.
Halevi, G., Nicolas, B., & Bar-Ilan, J. (2016). The Complexity Of Measuring The Impact Of Books. Publishing Research Quarterly, 32(3), 187–200. https://doi.org/10.1007/s12109-016-9464-5.
Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44(2), 193–215. https://doi.org/10.1007/BF02457380.
Huang, M., & Chang, Y. (2008). Characteristics of research output in social sciences and humanities: From a research evaluation perspective. Journal of the American Society for Information Science and Technology, 59(11), 1819–1828. https://doi.org/10.1002/asi.20885.
Jacsó, P., & Lancaster, F. W. (1999). Build your own database. American Library Association.
Kousha, K., & Thelwall, M. (2015). Web indicators for research evaluation: part 3: books and non standard outputs. El Profesional de La Información, 24(6), 724–736. https://doi.org/10.3145/epi.2015.nov.04.
Kousha, K., & Thelwall, M. (2016). Can Amazon.com reviews help to assess the wider impacts of books? Journal of the Association for Information Science and Technology, 67(3), 566–581. https://doi.org/10.1002/asi.23404.
Lewis, R. M., & Kennedy, M. R. (2019). The big picture: A holistic view of E-book acquisitions. Library Resources & Technical Services, 63(2), 160. https://doi.org/10.5860/lrts.63n2.160.
Linmans, A. J. M. (2008). Een exploratieve studie van de onderzoeksprestaties van de Faculteit Letteren aan de Universiteit Leiden (in Dutch). Internal CWTS report.
Linmans, A. J. M. (2010). Why with bibliometrics the Humanities does not need to be the weakest link—Indicators for research evaluation based on citations, library holdings, and productivity measures. Scientometrics, 83(2), 337–354. https://doi.org/10.1007/s11192-009-0088-9.
Martín-Martín, A., Orduna-Malea, E., & Delgado López-Cózar, E. (2016). The role of ego in academic profile services: Comparing Google Scholar, ResearchGate, Mendeley, and ResearcherID (SSRN Scholarly Paper ID 2745892). Social Science Research Network. https://doi.org/10.2139/ssrn.2745892.
Moed, H. F. (2005). Citation analysis in research evaluation. Berlin: Springer. https://doi.org/10.1007/1-4020-3714-7.
Neville, T. M., & Henry, D. B. (2014). Evaluating scholarly book publishers—A case study in the field of journalism. The Journal of Academic Librarianship, 40(3), 379–387. https://doi.org/10.1016/j.acalib.2014.05.005.
Nilges, C. (2006). The online computer library center’s open WorldCat program. Library Trends, 54(3), 430–447. https://doi.org/10.1353/lib.2006.0027.
Price, D. J. D. S. (1962). Science since babylon. New Haven: Yale University Press.
Priem, J., Taraborelli, D., Groth, P., & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved March 20, 2020 from http://altmetrics.org/manifesto/.
Thelwall, M., & Kousha, K. (2017). ResearchGate versus Google Scholar: Which finds more early citations? Scientometrics, 112(2), 1125–1131. https://doi.org/10.1007/s11192-017-2400-4.
Torres-Salinas, D., & Arroyo-Machado, W. (2020). Library catalog analysis and library holdings counts: Origins, methodological issues and application to the field of informetrics. In C. Daraio & W. Glänzel (Eds.), Evaluative informetrics: The art of metrics-based research assessment: Festschrift in honour of Henk F. Moed (pp. 287–308). Berlin: Springer. https://doi.org/10.1007/978-3-030-47665-6_13.
Torres-Salinas, D., Gumpenberger, C., & Gorraiz, J. (2017a). PlumX As a potential tool to assess the macroscopic multidimensional impact of books. Frontiers in Research Metrics and Analytics, 2, 5. https://doi.org/10.3389/frma.2017.00005.
Torres-Salinas, D., & Moed, H. F. (2008). Library catalog analysis is a useful tool in studies of social sciences and humanities. In A new challenge for the combination of quantitative and qualitative approaches. 10th International conference on science and technology indicators, Viena.
Torres-Salinas, D., & Moed, H. F. (2009). Library Catalog Analysis as a tool in studies of social sciences and humanities: An exploratory study of published book titles in Economics. Journal of Informetrics, 3(1), 9–26. https://doi.org/10.1016/j.joi.2008.10.002.
Torres-Salinas, D., Robinson-Garcia, N., & Gorraiz, J. (2017b). Filling the citation gap: Measuring the multidimensional impact of the academic book at institutional level with PlumX. Scientometrics, 113(3), 1371–1384. https://doi.org/10.1007/s11192-017-2539-z.
Wakeling, S., Clough, P., Connaway, L. S., Sen, B., & Tomás, D. (2017). Users and uses of a global union catalog: A mixed-methods study of WorldCat.org. Journal of the Association for Information Science and Technology, 68(9), 2166–2181. https://doi.org/10.1002/asi.23708.
White, H. D., Boell, S. K., Yu, H., Davis, M., Wilson, C. S., & Cole, F. T. H. (2009). Libcitations: A measure for comparative assessment of book publications in the humanities and social sciences. Journal of the American Society for Information Science and Technology, 60(6), 1083–1096. https://doi.org/10.1002/asi.21045.
White, H. D., & Zuccala, A. (2018). Libcitations, worldcat, cultural impact, and fame. Journal of the Association for Information Science and Technology, 69(12), 1502–1512. https://doi.org/10.1002/asi.24064.
Zhang, H., Zhou, Q., & Zhang, C. (2018). Multi-discipline correlation analysis between citations and detailed features of library holdings. Proceedings of the Association for Information Science and Technology, 55(1), 946–947. https://doi.org/10.1002/pra2.2018.14505501188.
Zuccala, A., Breum, M., Bruun, K., & Wunsch, B. T. (2018). Metric assessments of books as families of works. Journal of the Association for Information Science and Technology, 69(1), 146–157. https://doi.org/10.1002/asi.23921.
Zuccala, A., & Guns, R. (2013). Comparing book citations in humanities journals to library holdings: Scholarly use versus ‘perceived cultural benefit’. In 14th International society of scientometrics and informetrics conference, ISSI 2013, Vienna, pp. 353–360.
Zuccala, A., & White, H. D. (2015). Correlating libcitations and citations in the humanities with WorldCat and scopus data. In A. A. Salah, Y. Tonta, A. A. Akdag Salah, C. Sugimoto, & U. Al (Eds.), Proceedings of the 15th international society of scientometrics and informetrics conference, (ISSI), Istanbul, Turkey, 29th June to 4th July, 2015 (pp. 305–316). Denmark: Bogazici Universitesi.
Zuccala, A., & Zuccala, A. (2018). Language, culture and traversing the scholarly evaluation landscape. In A. Bonaccorsi (Ed.), The evaluation of research in social sciences and humanities. Cham: Springer. https://doi.org/10.1007/978-3-319-68554-0_17.
This work has been possible thanks to financial support from “InfluScience—Scientists with social influence: a model to measure knowledge transfer in the digital society” (PID2019-109127RB-I00/SRA/10.13039/501100011033), a project funded by scientific research team grants from the Ministry of Science and Innovation of Spain. Daniel Torres-Salinas has received funding from the University of Granada’s “Plan Propio de Investigación y Transferencia” under the “Reincorporación de Jóvenes Doctores” grant. Wenceslao Arroyo-Machado has received funding from the Spanish Ministry of Universities under the FPU Grant (FPU18/05835).
This work has been possible thanks to financial support from “InfluScience - Scientists with social influence: a model to measure knowledge transfer in the digital society” (PID2019-109127RB-I00/SRA/10.13039/501100011033), a project funded by scientific research team grants from the Ministry of Science and Innovation of Spain.
Conflict of interest
The authors declare that they have no conflict of interest.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Torres-Salinas, D., Arroyo-Machado, W. & Thelwall, M. Exploring WorldCat identities as an altmetric information source: a library catalog analysis experiment in the field of Scientometrics. Scientometrics 126, 1725–1743 (2021). https://doi.org/10.1007/s11192-020-03814-w
- Library catalog analysis
- Library holdings analysis
- WorldCat Identities