Skip to main content
SpringerLink
Log in

Comparison of downloads, citations and readership data for two information systems journals

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

In our article we compare downloads from ScienceDirect, citations from Scopus and readership data from the social reference management system Mendeley for articles from two information systems journals (“Journal of Strategic Information Systems” and “Information and Management”) published between 2002 and 2011. Our study shows a medium to high correlation between downloads and citations (Spearman r = 0.77/0.76) and between downloads and readership data (Spearman r = 0.73/0.66). The correlation between readership data and citations, however, was only medium-sized (Spearman r = 0.51/0.59). These results suggest that there is at least “some” difference between the two usage measures and the citation impact of the analysed information systems articles. As expected, downloads and citations have different obsolescence characteristics. While the highest number of downloads are usually made in the publication year and immediately afterwards, it takes several years until the citation maximum is reached. Furthermore, there was a re-increase in the downloads in later years which might be an indication that citations also have an effect on downloads to some degree.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Bar-Ilan, J. (2012). JASIST@mendeley. ACM Web Science Conference 2012 Workshop. Retrieved January 23, 2013 from http://altmetrics.org/altmetrics12/bar-ilan/.

  • Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: scholars’ visibility on the social web. In Proceedings of the 17th International Conference on Science and Technology Indicators (pp. 98–109). Montréal: Science-Metrix and OST.

  • Bollen, J., Luce, R., Vemulapalli S. S., & Xu, W. (2003). Usage analysis for the identification of research trends in digital libraries. D-Lib Magazine, 9(5). http://public.lanl.gov/herbertv/papers/ipm05jb-final.pdf. Accessed November 26, 2008.

  • Bollen, J., & Van de Sompel, H. (2008). Usage impact factor: The effects of sample characteristics on usage-based impact metrics. Journal of the American Society for Information Science and Technology, 59(1), 136–149.

    Article  Google Scholar 

  • Bollen, J., Van de Sompel, H., Smith, J. A., & Luce, R. (2005). Toward alternative metrics of journal impact: A comparison of download and citation data. Information Processing and Management, 41, 1419–1440. http://public.lanl.gov/herbertv/papers/ipm05jb-final.pdf. Accessed November 26, 2008.

  • Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology, 57(8), 1060–1072.

    Article  Google Scholar 

  • Chu, H., & Krichel, T. (2007). Downloads vs. citations in economics: Relationships, contributing factors and beyond. In Proceedings of the 11th International Conference on Scientometrics and Informetrics, Madrid (pp. 207–215).

  • Duy, J., & Vaughan, L. (2006). Can electronic journal usage data replace citation data as a measure of journal use? An empirical examination. The Journal of Academic Librarianship, 32(5), 512–517.

    Article  Google Scholar 

  • Gorraiz, J., Gumpenberger, C., Jack, K., Kraker, P., & Schlögl, C. (2013). What do citations, downloads and readership data of an information systems journal have in common and where do they differ? In S. Hinze & A. Lottmann (Eds.), Translational twists and turns: Science as a socio-economic endeavor, Proceedings of the 18th International Conference of Science and Technology Indicators (STI 2013) (pp. 140–145). Berlin: ENID and iFQ.

  • Gorraiz, J., Gumpenberger, C., & Schlögl, C. (2013). Difference and similarities in usage versus citation behaviours observed for five subject areas. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 519–535.

  • Hammerton, J., Granitzer, M., Harvey, D., Hristakeva, M., & Jack, K. (2012). On generating large-scale ground truth datasets for the deduplication of bibliographic records. In International Conference on Web Intelligence, Mining and Semantics 2012 (p. 18). ACM. doi:10.1145/2254129.2254153.

  • Haustein, S. (2012). Using social bookmarks and tags as alternative indicators of journal content description. First Monday, 17- 5 November 2012, http://firstmonday.org/ojs/index.php/fm/article/view/4110 Accessed 2014-02-25.

  • Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2013). Coverage and adoption of altmetrics sources in the bibliometric community. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 2, pp. 1–12.

  • Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of Informetrics, 5(3), 446–457.

    Google Scholar 

  • Henning, V., & Reichelt, J. (2012). Mendeley—A Last.fm for research? In IEEE Fourth International Conference on eScience (pp. 327–328). IEEE. doi:10.1109/eScience.

  • Huntington, P., Nicholas, D., Jamali, H. R., & Tenopir, C. (2006). Article decay in the digital environment: an analysis of usage of OhioLINK by date of publication, employing deep log methods. Journal of the American Society for Information Science and Technology, 57(13), 1840–1851.

  • Jack, K. (2014). E-Mail information from Kris Jack, chief data scientist at Mendeley, from April 28, 2014.

  • Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, 3–64.

    Article  Google Scholar 

  • Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Henneken, E., et al. (2005a). The effect of use and access on citations. Information Processing and Management, 41, 1395–1402.

    Article  Google Scholar 

  • Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Murray, S. S., et al. (2005b). The bibliometric properties of article readership information. Journal of the American Society for Information Science and Technology, 56(2), 111–128.

    Article  Google Scholar 

  • Li, X., & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In Proceedings of 17th International Conference on Science and Technology Indicators (STI 2012), (pp. 541–551). Montréal: Science-Metrix and OST.

  • Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471.

    Article  Google Scholar 

  • Moed, H. F. (2005). Statistical relationships between downloads and citations at the level of individual documents within a single journal. Journal of the American Society for Information Science and Technology, 56(10), 1088–1097.

    Article  Google Scholar 

  • Mohammadi, E., & Thelwall, M. (2013). Assessing the Mendeley readership of social sciences and humanities research. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 1, pp. 200–214.

  • Nicholoas, D., Huntington, P., Dobrowsolski, T., Rowlands, I., Jamali, H. R., & Polydoratou, P. (2005). Revisiting ‘obsolescence’ and journal article ‘decay’ through usage data: An analysis of digital journal use by year of publication. Information Processing and Management, 41, 1441–1461.

    Article  Google Scholar 

  • O’Leary, D. E. (2008). The relationship between citations and number of downloads in decision support systems. Decision Support Systems, 45, 972–980.

    Article  Google Scholar 

  • Schloegl, C., & Gorraiz, J. (2010). Comparison of citation and usage indicators: The case of oncology journals. Scientometrics, 82(3), 567–580.

    Article  Google Scholar 

  • Schloegl, C., & Gorraiz, J. (2011). Global usage versus global citation metrics: The case of pharmacology journals. Journal of the American Society for Information Science and Technology, 62(1), 161–170.

    Article  Google Scholar 

  • Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2013): Download vs. citation vs. readership data: The case of an information systems journal. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference (pp. 626–634). Wien: AIT Austrian Institute of Technology.

  • Takei, C., Yoshikane F., & Itsumura, H. (2013). Use of electronic journals in university libraries: an analysis of obsolescence regarding citations and access. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 2, pp. 1772–1783.

  • Thelwall, M., Haustein, S., Lariviere, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.

    Article  Google Scholar 

  • Tsay, M. (1998a). Library journal use and citation half-life in medical science. Journal of the American Society for Information Science, 49(14), 1283–1292.

    Article  Google Scholar 

  • Tsay, M. (1998b). The relationship between journal use in a medical library and citation use. Bulletin of the Medical Library Association, 86(1), 31–39.

    Google Scholar 

  • Zahedi, Z., Costas, R., & Wouters, P. (2013). How well developed are Altmetrics? Cross disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications (RIP). In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 876–884.

Download references

Acknowledgments

This report is based in part on the analysis of anonymous ScienceDirect usage data and/or Scopus citation data provided by Elsevier within the framework of the Elsevier Bibliometric Research Program (EBRP). Readership data were provided by Mendeley. The authors would like to thank both Elsevier and Mendeley for their great support. The Know-Center, which is the affiliation of one co-author, is funded within the Austrian COMET programme—Competence Centers for Excellent Technologies—under the auspices of the Austrian Federal Ministry for Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth, and the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.

Author information

Authors and Affiliations

  1. Institute of Information Science and Information Systems, University of Graz, Universitätsstr. 15/F3, 8010, Graz, Austria

    Christian Schlögl

  2. Vienna University Library, Bibliometrics Department, University of Vienna, Boltzmanngasse 5, 1090, Vienna, Austria

    Juan Gorraiz & Christian Gumpenberger

  3. Mendeley, 144a Clerkenwell Road, London, EC1R5DF, UK

    Kris Jack

  4. Know-Center, Inffeldgasse 13, 8010, Graz, Austria

    Peter Kraker

Authors
  1. Christian Schlögl
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Gorraiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Gumpenberger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kris Jack
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peter Kraker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Schlögl.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Schlögl, C., Gorraiz, J., Gumpenberger, C. et al. Comparison of downloads, citations and readership data for two information systems journals. Scientometrics 101, 1113–1128 (2014). https://doi.org/10.1007/s11192-014-1365-9

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-014-1365-9

Keywords

Search

Navigation