Advertisement

Scientometrics

, Volume 101, Issue 2, pp 1113–1128 | Cite as

Comparison of downloads, citations and readership data for two information systems journals

  • Christian SchlöglEmail author
  • Juan Gorraiz
  • Christian Gumpenberger
  • Kris Jack
  • Peter Kraker
Article

Abstract

In our article we compare downloads from ScienceDirect, citations from Scopus and readership data from the social reference management system Mendeley for articles from two information systems journals (“Journal of Strategic Information Systems” and “Information and Management”) published between 2002 and 2011. Our study shows a medium to high correlation between downloads and citations (Spearman r = 0.77/0.76) and between downloads and readership data (Spearman r = 0.73/0.66). The correlation between readership data and citations, however, was only medium-sized (Spearman r = 0.51/0.59). These results suggest that there is at least “some” difference between the two usage measures and the citation impact of the analysed information systems articles. As expected, downloads and citations have different obsolescence characteristics. While the highest number of downloads are usually made in the publication year and immediately afterwards, it takes several years until the citation maximum is reached. Furthermore, there was a re-increase in the downloads in later years which might be an indication that citations also have an effect on downloads to some degree.

Keywords

Downloads Citations Readership Mendeley Information systems journals 

Notes

Acknowledgments

This report is based in part on the analysis of anonymous ScienceDirect usage data and/or Scopus citation data provided by Elsevier within the framework of the Elsevier Bibliometric Research Program (EBRP). Readership data were provided by Mendeley. The authors would like to thank both Elsevier and Mendeley for their great support. The Know-Center, which is the affiliation of one co-author, is funded within the Austrian COMET programme—Competence Centers for Excellent Technologies—under the auspices of the Austrian Federal Ministry for Transport, Innovation and Technology, the Austrian Federal Ministry of Economy, Family and Youth, and the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.

References

  1. Bar-Ilan, J. (2012). JASIST@mendeley. ACM Web Science Conference 2012 Workshop. Retrieved January 23, 2013 from http://altmetrics.org/altmetrics12/bar-ilan/.
  2. Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: scholars’ visibility on the social web. In Proceedings of the 17th International Conference on Science and Technology Indicators (pp. 98–109). Montréal: Science-Metrix and OST.Google Scholar
  3. Bollen, J., Luce, R., Vemulapalli S. S., & Xu, W. (2003). Usage analysis for the identification of research trends in digital libraries. D-Lib Magazine, 9(5). http://public.lanl.gov/herbertv/papers/ipm05jb-final.pdf. Accessed November 26, 2008.
  4. Bollen, J., & Van de Sompel, H. (2008). Usage impact factor: The effects of sample characteristics on usage-based impact metrics. Journal of the American Society for Information Science and Technology, 59(1), 136–149.CrossRefGoogle Scholar
  5. Bollen, J., Van de Sompel, H., Smith, J. A., & Luce, R. (2005). Toward alternative metrics of journal impact: A comparison of download and citation data. Information Processing and Management, 41, 1419–1440. http://public.lanl.gov/herbertv/papers/ipm05jb-final.pdf. Accessed November 26, 2008.
  6. Brody, T., Harnad, S., & Carr, L. (2006). Earlier web usage statistics as predictors of later citation impact. Journal of the American Society for Information Science and Technology, 57(8), 1060–1072.CrossRefGoogle Scholar
  7. Chu, H., & Krichel, T. (2007). Downloads vs. citations in economics: Relationships, contributing factors and beyond. In Proceedings of the 11th International Conference on Scientometrics and Informetrics, Madrid (pp. 207–215).Google Scholar
  8. Duy, J., & Vaughan, L. (2006). Can electronic journal usage data replace citation data as a measure of journal use? An empirical examination. The Journal of Academic Librarianship, 32(5), 512–517.CrossRefGoogle Scholar
  9. Gorraiz, J., Gumpenberger, C., Jack, K., Kraker, P., & Schlögl, C. (2013). What do citations, downloads and readership data of an information systems journal have in common and where do they differ? In S. Hinze & A. Lottmann (Eds.), Translational twists and turns: Science as a socio-economic endeavor, Proceedings of the 18th International Conference of Science and Technology Indicators (STI 2013) (pp. 140–145). Berlin: ENID and iFQ.Google Scholar
  10. Gorraiz, J., Gumpenberger, C., & Schlögl, C. (2013). Difference and similarities in usage versus citation behaviours observed for five subject areas. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 519–535.Google Scholar
  11. Hammerton, J., Granitzer, M., Harvey, D., Hristakeva, M., & Jack, K. (2012). On generating large-scale ground truth datasets for the deduplication of bibliographic records. In International Conference on Web Intelligence, Mining and Semantics 2012 (p. 18). ACM. doi: 10.1145/2254129.2254153.
  12. Haustein, S. (2012). Using social bookmarks and tags as alternative indicators of journal content description. First Monday, 17- 5 November 2012, http://firstmonday.org/ojs/index.php/fm/article/view/4110 Accessed 2014-02-25.
  13. Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2013). Coverage and adoption of altmetrics sources in the bibliometric community. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 2, pp. 1–12.Google Scholar
  14. Haustein, S., & Siebenlist, T. (2011). Applying social bookmarking data to evaluate journal usage. Journal of Informetrics, 5(3), 446–457.Google Scholar
  15. Henning, V., & Reichelt, J. (2012). Mendeley—A Last.fm for research? In IEEE Fourth International Conference on eScience (pp. 327–328). IEEE. doi: 10.1109/eScience.
  16. Huntington, P., Nicholas, D., Jamali, H. R., & Tenopir, C. (2006). Article decay in the digital environment: an analysis of usage of OhioLINK by date of publication, employing deep log methods. Journal of the American Society for Information Science and Technology, 57(13), 1840–1851.Google Scholar
  17. Jack, K. (2014). E-Mail information from Kris Jack, chief data scientist at Mendeley, from April 28, 2014.Google Scholar
  18. Kurtz, M. J., & Bollen, J. (2010). Usage bibliometrics. Annual Review of Information Science and Technology, 44, 3–64.CrossRefGoogle Scholar
  19. Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Henneken, E., et al. (2005a). The effect of use and access on citations. Information Processing and Management, 41, 1395–1402.CrossRefGoogle Scholar
  20. Kurtz, M. J., Eichhorn, G., Accomazzi, A., Grant, C., Demleitner, M., Murray, S. S., et al. (2005b). The bibliometric properties of article readership information. Journal of the American Society for Information Science and Technology, 56(2), 111–128.CrossRefGoogle Scholar
  21. Li, X., & Thelwall, M. (2012). F1000, Mendeley and traditional bibliometric indicators. In Proceedings of 17th International Conference on Science and Technology Indicators (STI 2012), (pp. 541–551). Montréal: Science-Metrix and OST.Google Scholar
  22. Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461–471.CrossRefGoogle Scholar
  23. Moed, H. F. (2005). Statistical relationships between downloads and citations at the level of individual documents within a single journal. Journal of the American Society for Information Science and Technology, 56(10), 1088–1097.CrossRefGoogle Scholar
  24. Mohammadi, E., & Thelwall, M. (2013). Assessing the Mendeley readership of social sciences and humanities research. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 1, pp. 200–214.Google Scholar
  25. Nicholoas, D., Huntington, P., Dobrowsolski, T., Rowlands, I., Jamali, H. R., & Polydoratou, P. (2005). Revisiting ‘obsolescence’ and journal article ‘decay’ through usage data: An analysis of digital journal use by year of publication. Information Processing and Management, 41, 1441–1461.CrossRefGoogle Scholar
  26. O’Leary, D. E. (2008). The relationship between citations and number of downloads in decision support systems. Decision Support Systems, 45, 972–980.CrossRefGoogle Scholar
  27. Schloegl, C., & Gorraiz, J. (2010). Comparison of citation and usage indicators: The case of oncology journals. Scientometrics, 82(3), 567–580.CrossRefGoogle Scholar
  28. Schloegl, C., & Gorraiz, J. (2011). Global usage versus global citation metrics: The case of pharmacology journals. Journal of the American Society for Information Science and Technology, 62(1), 161–170.CrossRefGoogle Scholar
  29. Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2013): Download vs. citation vs. readership data: The case of an information systems journal. In Proceedings of the 14th International Society of Scientometrics and Informetrics Conference (pp. 626–634). Wien: AIT Austrian Institute of Technology.Google Scholar
  30. Takei, C., Yoshikane F., & Itsumura, H. (2013). Use of electronic journals in university libraries: an analysis of obsolescence regarding citations and access. In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics, Vol. 2, pp. 1772–1783.Google Scholar
  31. Thelwall, M., Haustein, S., Lariviere, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841.CrossRefGoogle Scholar
  32. Tsay, M. (1998a). Library journal use and citation half-life in medical science. Journal of the American Society for Information Science, 49(14), 1283–1292.CrossRefGoogle Scholar
  33. Tsay, M. (1998b). The relationship between journal use in a medical library and citation use. Bulletin of the Medical Library Association, 86(1), 31–39.Google Scholar
  34. Zahedi, Z., Costas, R., & Wouters, P. (2013). How well developed are Altmetrics? Cross disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications (RIP). In J. Gorraiz, E. Schiebel, C. Gumpenberger, M. Hörlesberger, & H. Moed (Eds.), Proceedings of the 14th International Society of Scientometrics and Informetrics Conference, Vol. 1, pp. 876–884.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  • Christian Schlögl
    • 1
    Email author
  • Juan Gorraiz
    • 2
  • Christian Gumpenberger
    • 2
  • Kris Jack
    • 3
  • Peter Kraker
    • 4
  1. 1.Institute of Information Science and Information SystemsUniversity of GrazGrazAustria
  2. 2.Vienna University Library, Bibliometrics DepartmentUniversity of ViennaViennaAustria
  3. 3.MendeleyLondonUK
  4. 4.Know-CenterGrazAustria

Personalised recommendations