Abstract
In our article we compare downloads from ScienceDirect, citations from Scopus and readership data from the social reference management system Mendeley for articles from two information systems journals (“Journal of Strategic Information Systems” and “Information and Management”) published between 2002 and 2011. Our study shows a medium to high correlation between downloads and citations (Spearman r = 0.77/0.76) and between downloads and readership data (Spearman r = 0.73/0.66). The correlation between readership data and citations, however, was only medium-sized (Spearman r = 0.51/0.59). These results suggest that there is at least “some” difference between the two usage measures and the citation impact of the analysed information systems articles. As expected, downloads and citations have different obsolescence characteristics. While the highest number of downloads are usually made in the publication year and immediately afterwards, it takes several years until the citation maximum is reached. Furthermore, there was a re-increase in the downloads in later years which might be an indication that citations also have an effect on downloads to some degree.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11192-014-1365-9/MediaObjects/11192_2014_1365_Fig1_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11192-014-1365-9/MediaObjects/11192_2014_1365_Fig2_HTML.gif)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs11192-014-1365-9/MediaObjects/11192_2014_1365_Fig3_HTML.gif)
Similar content being viewed by others
References
Bar-Ilan, J. (2012). JASIST@mendeley. ACM Web Science Conference 2012 Workshop. Retrieved January 23, 2013 from http://altmetrics.org/altmetrics12/bar-ilan/.
Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: scholars’ visibility on the social web. In Proceedings of the 17th International Conference on Science and Technology Indicators (pp. 98–109). Montréal: Science-Metrix and OST.
Bollen, J., Luce, R., Vemulapalli S. S., & Xu, W. (2003). Usage analysis for the identification of research trends in digital libraries. D-Lib Magazine, 9(5). http://public.lanl.gov/herbertv/papers/ipm05jb-final.pdf. Accessed November 26, 2008.
Bollen, J., & Van de Sompel, H. (2008). Usage impact factor: The effects of sample characteristics on usage-based impact metrics. Journal of the American Society for Information Science and Technology, 59(1), 136–149.
Bollen, J., Van de Sompel, H., Smith, J. A., & Luce, R. (2005). Toward alternative metrics of journal impact: A comparison of download and citation data. Information Processing and Management, 41, 1419–1440. http://public.lanl.gov/herbertv/papers/ipm05jb-final.pdf. Accessed November 26, 2008.
Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology, 57(8), 1060–1072.
Chu, H., & Krichel, T. (2007). Downloads vs. citations in economics: Relationships, contributing factors and beyond. In Proceedings of the 11th International Conference on Scientometrics and Informetrics, Madrid (pp. 207–215).
Duy, J., & Vaughan, L. (2006). Can electronic journal usage data replace citation data as a measure of journal use? An empirical examination. The Journal of Academic Librarianship, 32(5), 512–517.
Gorraiz, J., Gumpenberger, C., Jack, K., Kraker, P., & Schlögl, C. (2013). What do citations, downloads and readership data of an information systems journal have in common and where do they differ? In S. Hinze & A. Lottmann (Eds.), Translational twists and turns: Science as a socio-economic endeavor, Proceedings of the 18th International Conference of Science and Technology Indicators (STI 2013) (pp. 140–145). Berlin: ENID and iFQ.
Gorraiz, J., Gumpenberger, C., & Schlögl, C. (2013). Difference and similarities in usage versus citation behaviours observed for five subject areas. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 519–535.
Hammerton, J., Granitzer, M., Harvey, D., Hristakeva, M., & Jack, K. (2012). On generating large-scale ground truth datasets for the deduplication of bibliographic records. In International Conference on Web Intelligence, Mining and Semantics 2012 (p. 18). ACM. doi:10.1145/2254129.2254153.
Haustein, S. (2012). Using social bookmarks and tags as alternative indicators of journal content description. First Monday, 17- 5 November 2012, http://firstmonday.org/ojs/index.php/fm/article/view/4110 Accessed 2014-02-25.
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2013). Coverage and adoption of altmetrics sources in the bibliometric community. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 2, pp. 1–12.
Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of Informetrics, 5(3), 446–457.
Henning, V., & Reichelt, J. (2012). Mendeley—A Last.fm for research? In IEEE Fourth International Conference on eScience (pp. 327–328). IEEE. doi:10.1109/eScience.
Huntington, P., Nicholas, D., Jamali, H. R., & Tenopir, C. (2006). Article decay in the digital environment: an analysis of usage of OhioLINK by date of publication, employing deep log methods. Journal of the American Society for Information Science and Technology, 57(13), 1840–1851.
Jack, K. (2014). E-Mail information from Kris Jack, chief data scientist at Mendeley, from April 28, 2014.
Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, 3–64.
Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Henneken, E., et al. (2005a). The effect of use and access on citations. Information Processing and Management, 41, 1395–1402.
Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Murray, S. S., et al. (2005b). The bibliometric properties of article readership information. Journal of the American Society for Information Science and Technology, 56(2), 111–128.
Li, X., & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In Proceedings of 17th International Conference on Science and Technology Indicators (STI 2012), (pp. 541–551). Montréal: Science-Metrix and OST.
Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471.
Moed, H. F. (2005). Statistical relationships between downloads and citations at the level of individual documents within a single journal. Journal of the American Society for Information Science and Technology, 56(10), 1088–1097.
Mohammadi, E., & Thelwall, M. (2013). Assessing the Mendeley readership of social sciences and humanities research. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 1, pp. 200–214.
Nicholoas, D., Huntington, P., Dobrowsolski, T., Rowlands, I., Jamali, H. R., & Polydoratou, P. (2005). Revisiting ‘obsolescence’ and journal article ‘decay’ through usage data: An analysis of digital journal use by year of publication. Information Processing and Management, 41, 1441–1461.
O’Leary, D. E. (2008). The relationship between citations and number of downloads in decision support systems. Decision Support Systems, 45, 972–980.
Schloegl, C., & Gorraiz, J. (2010). Comparison of citation and usage indicators: The case of oncology journals. Scientometrics, 82(3), 567–580.
Schloegl, C., & Gorraiz, J. (2011). Global usage versus global citation metrics: The case of pharmacology journals. Journal of the American Society for Information Science and Technology, 62(1), 161–170.
Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2013): Download vs. citation vs. readership data: The case of an information systems journal. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference (pp. 626–634). Wien: AIT Austrian Institute of Technology.
Takei, C., Yoshikane F., & Itsumura, H. (2013). Use of electronic journals in university libraries: an analysis of obsolescence regarding citations and access. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 2, pp. 1772–1783.
Thelwall, M., Haustein, S., Lariviere, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.
Tsay, M. (1998a). Library journal use and citation half-life in medical science. Journal of the American Society for Information Science, 49(14), 1283–1292.
Tsay, M. (1998b). The relationship between journal use in a medical library and citation use. Bulletin of the Medical Library Association, 86(1), 31–39.
Zahedi, Z., Costas, R., & Wouters, P. (2013). How well developed are Altmetrics? Cross disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications (RIP). In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 876–884.
Acknowledgments
This report is based in part on the analysis of anonymous ScienceDirect usage data and/or Scopus citation data provided by Elsevier within the framework of the Elsevier Bibliometric Research Program (EBRP). Readership data were provided by Mendeley. The authors would like to thank both Elsevier and Mendeley for their great support. The Know-Center, which is the affiliation of one co-author, is funded within the Austrian COMET programme—Competence Centers for Excellent Technologies—under the auspices of the Austrian Federal Ministry for Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth, and the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Schlögl, C., Gorraiz, J., Gumpenberger, C. et al. Comparison of downloads, citations and readership data for two information systems journals. Scientometrics 101, 1113–1128 (2014). https://doi.org/10.1007/s11192-014-1365-9
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11192-014-1365-9