Skip to main content

Methodological issues in measuring citations in Wikipedia: a case study in Library and Information Science

Scientometrics volume 113, pages 455–464 (2017)Cite this article

Abstract

Wikipedia citations have been suggested as a metric that partially captures the impact of research, providing an indication of the transfer of scholarly output to a wider audience beyond the academic community. In this article, we explore the coverage of Library and Information Science literature published between 2001 and 2010 in Wikipedia, paying special attention to the methodological issues involved in counting Wikipedia citations. The results reveal severe limitations in the use of Wikipedia citations for research evaluation. Lack of standardization and incompleteness of Wikipedia references make it difficult to retrieve them. The number of Wikipedia citations is very low, with less than 3% of articles in the sample having been cited. A significant number of references are cited in biographical entries about the authors of the articles, resulting in a phenomenon of accumulated advantage, which is similar to the Matthew effect. Nearly one-third of the Wikipedia citations link to an open access source, although this result is probably an underestimate of open access availability, given the incompleteness of Wikipedia citations.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Notes

  1. https://en.wikipedia.org/wiki/Wikipedia:Size_of_Wikipedia.

  2. https://en.wikipedia.org/wiki/Wikipedia:Five_pillars.

  3. https://www.altmetric.com/blog/new-source-alert-wikipedia/.

  4. https://help.altmetric.com/support/solutions/articles/6000060980-how-does-altmetric-track-mentions-on-wikipedia.

  5. https://en.wikipedia.org/wiki/Template:Cite_journal.

References

  • Evans, P., & Krauthammer, M. (2011). Exploring the use of social media to measure journal article impact. AMIA Annual Symposium Proceedings, 374–381. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3243242/. Accessed 2 May 2017.

  • Franceschini, F., Maisano, D., & Mastrogiacomo, L. (2015). Errors in DOI indexing by bibliometric databases. Scientometrics, 102(3), 2181–2186. doi:10.1007/s11192-014-1503-4.

    Article  Google Scholar 

  • Franceschini, F., Maisano, D., & Mastrogiacomo, L. (2016). The museum of errors/horrors in Scopus. Journal of Informetrics, 10(1), 174–182. doi:10.1016/j.joi.2015.11.006.

    Article  Google Scholar 

  • Halfaker, A., & Taraborelli, D. (2015). Scholarly article citations in Wikipedia. Figshare. doi:10.6084/m9.figshare.1299540.v8.

  • Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014). Coverage and adoption of altmetrics sources in the bibliometric community. Scientometrics, 101(2), 1145–1163. doi:10.1007/s11192-013-1221-3.

    Article  Google Scholar 

  • Huvila, I. (2010). Where does the information come from? Information source use patterns in Wikipedia. Information Research, 15(3). http://www.informationr.net/ir/15-3/paper433.html. Accessed 2 May 2017.

  • Kousha, K., & Thelwall, M. (2017). Are Wikipedia citations important evidence of the impact of scholarly articles and books? Journal of the Association for Information Science and Technology, 68(3), 762–779. doi:10.1002/asi.23694.

    Article  Google Scholar 

  • Lin, J., & Fenner, M. (2014). An analysis of Wikipedia references across PLOS publications. Figshare. doi:10.6084/m9.figshare.1048991.v3.

  • Luyt, B., & Tan, D. (2010). Improving Wikipedia’s credibility: References and citations in a sample of history articles. Journal of the Association for Information Science and Technology, 61(4), 715–722. doi:10.1002/asi.21304.

    Google Scholar 

  • Mesgari, M., Okoli, C., Mehdi, M., Nielsen, F. A., & Lanamäki, A. (2015). The sum of all human knowledge: A systematic review of scholarly research on the content of Wikipedia. Journal of the Association for Information Science and Technology, 66(2), 219–245. doi:10.1002/asi.23172.

    Article  Google Scholar 

  • Nielsen, F. (2007). Scientific citations in Wikipedia. First Monday, 12(8). http://firstmonday.org/article/view/1997/1872. Accessed 2 May 2017.

  • Stankus, T., & Spiegel, S. E. (2010). Wikipedia, Scholarpedia, and references to journals in the brain and behavioral sciences: A comparison of cited sources and recommended readings in matching free online encyclopedia entries. Science & Technology Libraries, 29(3), 258–265. doi:10.1080/0194262X.2010.497711.

    Article  Google Scholar 

  • Tarango, J., Ascencio-Baca, G., Romo-González, J. R., & Gutiérrez-Balderrama, J. P. (2017). Tendencias de información en la categoría de artículos destacados de Wikipedia: una perspectiva sobre la obsolescencia de los contenidos virtuales y de libre acceso. Anales de Documentación. doi:10.6018/analesdoc.20.1.259721.

    Google Scholar 

  • Teplitskiy, M., Lu, G., & Duede, E. (2017). Amplifying the impact of open access: Wikipedia and the diffusion of science. Journal of the Association for Information Science and Technology, in press.. doi:10.1002/asi.23687.

    Google Scholar 

  • Thelwall, M. (2016). Does astronomy research become too dated for the public? Wikipedia citations to astronomy and astrophysics journal articles 1996–2014. El Profesional de la Información, 25(6), 893–900. doi:10.3145/epi.2016.nov.06.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Facultat de Biblioteconomia i Documentació, Universitat de Barcelona, Melcior de Palau 140, 08014, Barcelona, Spain

    Aida Pooladian & Ángel Borrego

Authors
  1. Aida Pooladian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ángel Borrego
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ángel Borrego.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Pooladian, A., Borrego, Á. Methodological issues in measuring citations in Wikipedia: a case study in Library and Information Science. Scientometrics 113, 455–464 (2017). https://doi.org/10.1007/s11192-017-2474-z

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-017-2474-z

Keywords

  • Altmetrics
  • Citation analysis
  • Library and Information Science
  • Wikipedia