Skip to main content

Usage, content and citation in open access publication: any interaction effects?

Abstract

This study aims to validate an empirical model, at document level, that explains the interaction among content, usage, and citation within open access publications. The PLoS site was the information source for this study. Using an R API (Application Programming Interface) for PLoS ONE, 776,465 records were downloaded on August 24, 2018. Those records (from 2006 to 2018) were organized according to the PLoS’ thematic areas. The empirical framework was validated using mediation analysis. For computing the parameters’ significance, bootstrapping with 500 replications for the general model and each thematic area was used. When usage was included as the mediating variable within the model, the total effects of cognitive and social variables got better predictive capability, as expressed by the explained variance of citation (R2 = 0.282) and usage (R2 = 0.333). The same trend was observed for the indirect effects after carrying out the mediation analysis by categories. Promotion campaigns of scientific publication should reinforce the widespread adoption of easy-to-use social media because, besides the velocity and variety of diffusion channels, the extended use guarantees that journal’s papers will reach increasing audiences. This is one of the first studies that analyze the interaction effects of variables at the article-level within open access publications.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

References

  1. Akella, A., Alhoori, H., Kondamudi, P. R., Freeman, C., & Zhou, H. (2021). Early indicators of scientific impact: Predicting citations with altmetrics. Journal of Informetrics, 15(2), 101128. https://doi.org/10.1016/j.joi.2020.101128

    Article  Google Scholar 

  2. Baron, R. M., & Kenny, D. A. (1986). The moderator–mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51(6), 1173–1182. https://doi.org/10.1037//0022-3514.51.6.1173

    Article  Google Scholar 

  3. Bjekić, J., Lazarević, L. B., Živanović, M., & Knežević, G. (2014). Psychometric evaluation of the Serbian dictionary for automatic text analysis—LIWCser. Psihologija, 47(1), 5–32. https://doi.org/10.2298/PSI1401005B

    Article  Google Scholar 

  4. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3(4–5), 993–1022.

    MATH  Google Scholar 

  5. Bornmann, L., & Haunschild, R. (2018). Do altmetrics correlate with the quality of papers? A large-scale empirical study based on F1000Prime data. PLoS ONE, 13(5), e0197133. https://doi.org/10.1371/journal.pone.0197133

    Article  Google Scholar 

  6. Bornmann, L., Wolf, M., & Daniel, H.-D. (2012). Closed versus open reviewing of journal manuscripts: How far do comments differ in language use? Scientometrics, 91(3), 843–856. https://doi.org/10.1007/s11192-011-0569-5

    Article  Google Scholar 

  7. Chen, W. M. Y., Bukhari, M., Cockshull, F., & Galloway, J. (2020). The relationship between citations, downloads and alternative metrics in rheumatology publications: A bibliometric study. Rheumatology, 59(2), 277–280. https://doi.org/10.1093/rheumatology/kez163

    Article  Google Scholar 

  8. Cho, J. (2021). Altmetrics analysis of highly cited academic papers in the field of Library and Information Science. Scientometrics, 126(9), 7623–7635. https://doi.org/10.1007/s11192-021-04084-w

    Article  Google Scholar 

  9. Dehdaridad, T. (2020). Could early tweets counts predict later citation counts? A gender study in Life Sciences and Biomedicine (2014–2016). PLoS ONE, 15(11), e0241723. https://doi.org/10.1371/journal.pone.0241723

    Article  Google Scholar 

  10. Ebrahimy, S., Mehrad, J., Setareh, F., & Hosseinchari, M. (2016). Path analysis of the relationship between visibility and citation: The mediating roles of save, discussion, and recommendation metrics. Scientometrics, 109(3), 1497–1510. https://doi.org/10.1007/s11192-016-2130-z

    Article  Google Scholar 

  11. Ekinci, E., & Omurca, I. (2020). Concept-LDA: Incorporating Babelfy into LDA for aspect extraction. Journal of Information Science, 46(3), 406–418. https://doi.org/10.1177/0165551519845854

    Article  Google Scholar 

  12. Gontijo, M. C. A., & de Araújo, R. F. (2021). Impacto acadêmico e atenção on-line de pesquisas sobre inteligência artificial na área da saúde: Análise de dados bibliométricos e altmétricos. Encontros Bibli: Revista Eletrônica De Biblioteconomia e Ciência Da Informação, 26, 01–21. https://doi.org/10.5007/1518-2924.2021.e76249

    Article  Google Scholar 

  13. Graybeal, A., Seagal, J. D., & Pennebaker, J. W. (2002). The role of story-making in disclosure writing: The psychometrics of narrative. Psychology and Health, 17(5), 571–581. https://doi.org/10.1080/08870440290025786

    Article  Google Scholar 

  14. Guerrero-Bote, V. P., & Moya-Anegón, F. (2014). Relationship between downloads and citations at journal and paper levels, and the influence of language. Scientometrics, 101(2), 1043–1065. https://doi.org/10.1007/s11192-014-1243-5

    Article  Google Scholar 

  15. Hassan, S.-U., Aljohani, N. R., Idrees, N., Sarwar, R., Nawaz, R., Martinez-Camara, R., Ventura, S., & Herrera, F. (2020). Predicting literature’s early impact with sentiment analysis in Twitter. Knowledge-Based Systems, 192(15), e105383. https://doi.org/10.1016/j.knosys.2019.105383

    Article  Google Scholar 

  16. Haustein, S., & Larivière, V. (2014). A multidimensional analysis of Aslib proceedings—Using everything but the impact factor. Aslib Journal of Information Management, 66(4), 358–380. https://doi.org/10.1108/AJIM-11-2013-0127

    Article  Google Scholar 

  17. Haustein, S., Costas, R., & Larivière, V. (2015). Characterizing social media metrics of scholarly papers: The effect of document properties and collaboration patterns. PLoS ONE, 10(3), e0120495. https://doi.org/10.1371/journal.pone.0120495

    Article  Google Scholar 

  18. Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis. The Guilford Press.

  19. Hu, Z., Fang, S., & Liang, T. (2014). Empirical study of constructing a knowledge organization system of patent documents using topic modeling. Scientometrics, 100(3), 787–799. https://doi.org/10.1007/s11192-014-1328-1

    Article  Google Scholar 

  20. Iacobucci, D. (2008). Mediation analysis. Sage.

  21. JASP Team. (2020). JASP, version 0.14.1 (computer software). https://jasp-stats.org/download.

  22. Jose, P. E. (2013). Doing statistical mediation and moderation. The Guilford Press.

  23. Li, X., Wu, C., & Mai, F. (2019). The effect of online reviews on product sales: A joint sentiment-topic analysis. Information Management, 56(2), 172–184. https://doi.org/10.1016/j.im.2018.04.007

    Article  Google Scholar 

  24. Maggio, L. A., Leroux, T. C., Meyer, H. S., & Artino, A. R., Jr. (2018). #MedEd: Exploring the relationship between altmetrics and traditional measures of dissemination in health professions education. Perspectives on Medical Education, 7(4), 239–247. https://doi.org/10.1007/s40037-018-0438-5

    Article  Google Scholar 

  25. Maricato, J. M., & Vilan-Filho, J. L. (2018). The potential for altmetrics to measure other types of impact in scientific production: academic and social impact dynamics in social media and networks. Information Research, 23(1), paper 780. Retrieved from http://InformationR.net/ir/23-1/paper780.html.

  26. Mehl, M. R., & Pennebaker, J. W. (2003). The sounds of social life: A psychometric analysis of students’ daily social environments and conversations. Journal of Personality and Social Psychology, 84(4), 857–870. https://doi.org/10.1037/0022-3514.84.4.857

    Article  Google Scholar 

  27. Moed, H. F., & Halevi, G. (2016). On full text download and citation distributions in scientific-scholarly journals. Journal of the Association for Information Science and Technology, 67(2), 412–431. https://doi.org/10.1002/asi.23405

    Article  Google Scholar 

  28. Ortega, J. L. (2015). Relationship between altmetric and bibliometric indicators across academic social sites: The case of CSIC’s members. Journal of Informetrics, 9(1), 39–49. https://doi.org/10.1016/j.joi.2014.11.004

    Article  Google Scholar 

  29. Pennebaker, J. W., Francis M. E., & Booth, R. J. (2001). Linguistic Inquiry and Word Count LIWC 2001 Manual. Retrieved from https://www.researchgate.net/publication/246699633_Linguistic_inquiry_and_word_count_LIWC.

  30. Priem, J., Taraborelli, D., Groth, P. & Neylon, C. (2010). Altmetrics: A manifesto. Retrieved from http://altmetrics.org/manifesto.

  31. Roemer, R. C., & Borchardt, R. (2015). Issues, controversies, and opportunities for altmetrics. Library Technology Reports, 51(5), 20–30.

    Google Scholar 

  32. Sathianathen, N. J., Lane, R., Condon, B., Murphy, D. G., Lawrentschuk, N., Weight, C. J., & Lamb, A. D. (2020). Early online attention can predict citation counts for urological publications: The #UroSoMe_Score. European Urology Focus, 6(3), 458–462. https://doi.org/10.1016/j.euf.2019.10.015

    Article  Google Scholar 

  33. Schlögl, C., & Gorraiz, J. (2010). Comparison of citation and usage indicators: The case of oncology journals. Scientometrics, 82(3), 567–580. https://doi.org/10.1007/s11192-010-0172-1

    Article  Google Scholar 

  34. Schlögl, C., Gorraiz, J., Gumpenberger, C., Jack, K., & Kraker, P. (2014). Comparison of downloads, citations and readership data for two information systems journals. Scientometrics, 101(2), 1113–1128. https://doi.org/10.1007/s11192-014-1365-9

    Article  Google Scholar 

  35. Smith, Z. L., Chiang, A. L., Bowman, D., & Wallace, M. B. (2019). Longitudinal relationship between social media activity and article citations in the journal Gastrointestinal Endoscopy. Gastrointestinal Endoscopy, 90(1), 77–83. https://doi.org/10.1016/j.gie.2019.03.028

    Article  Google Scholar 

  36. Smith-Keiling, B. L., & Hyun, H. I. F. (2019). Applying a computer-assisted tool for semantic analysis of writing: Uses for STEM and ELL. Journal of Microbiology & Biology Education, 20(1), 1–6. https://doi.org/10.1128/jmbe.v20i1.1709

    Article  Google Scholar 

  37. Strauβ, N., Alonso-Muñoz, L., & Zúñiga, H. G. (2020). Bursting the filter bubble: The mediating effect of discussion frequency on network heterogeneity. Online Information Review, 44(6), 1161–1181. https://doi.org/10.1108/OIR-11-2019-0345

    Article  Google Scholar 

  38. Thelwall, M. (2017). Are Mendeley reader counts useful impact indicators in all fields? Scientometrics, 113(1), 1721–1731. https://doi.org/10.1007/s11192-017-2557-x

    Article  Google Scholar 

  39. Thelwall, M., & Nevill, T. (2018). Could scientists use Altmetric.com scores to predict longer term citation counts? Journal of Informetrics, 12(1), 237–248. https://doi.org/10.1016/j.joi.2018.01.008

    Article  Google Scholar 

  40. Thelwall, M., & Wilson, P. (2014). Regression for citation data: An evaluation of different methods. Journal of Informetrics, 8(4), 963–971. https://doi.org/10.1016/j.joi.2014.09.011

    Article  Google Scholar 

  41. Thelwall, M., Haustein, S., Larivière, V., & Sugimoto, C. R. (2013). Do altmetrics work? Twitter and ten other social web services. PLoS ONE, 8(5), e64841. https://doi.org/10.1371/journal.pone.0064841

    Article  Google Scholar 

  42. Vílchez-Román, C., Huamán-Delgado, F., & Alhuay-Quispe, J. (2020). Social dimension activates the usage and academic impact of Open Access publications in Andean countries: A structural modeling-based approach. Information Development. https://doi.org/10.1177/0266666920901849

    Article  Google Scholar 

  43. Wang, Z., Chen, Y., & Glänzel, W. (2020a). Preprints as accelerator of scholarly communication: An empirical analysis in Mathematics. Journal of Informetrics, 14(4), 101097. https://doi.org/10.1016/j.joi.2020.101097

    Article  Google Scholar 

  44. Wang, Z., Glänzel, W., & Chen, Y. (2020b). The impact of preprints in Library and Information Science: An analysis of citations, usage and social attention indicators. Scientometrics, 125(2), 1403–1423. https://doi.org/10.1007/s11192-020-03612-4

    Article  Google Scholar 

  45. Wei, M., & Noroozi Chakoli, A. (2020). Evaluating the relationship between the academic and social impact of open access books based on citation behaviors and social media attention. Scientometrics, 125(3), 2401–2420. https://doi.org/10.1007/s11192-020-03678-0

    Article  Google Scholar 

  46. Wilson, T. D. (1999). Models of information behaviour research. Journal of Documentation, 55(3), 249–270. https://doi.org/10.1108/EUM0000000007145

    Article  Google Scholar 

  47. Williams, A. E. (2017). Altmetrics: An overview and evaluation. Online Information Review, 41(3), 311–317. https://doi.org/10.1108/OIR-10-2016-0294

    Article  Google Scholar 

  48. Wu, Q., Dbouk, W., Hasan, I., Kobeissi, N., & Zheng, L. (2021). Does gender affect innovation? Evidence from female chief technology officers. Research Policy, 50(9), 104327. https://doi.org/10.1016/j.respol.2021.104327

    Article  Google Scholar 

  49. Zhang, Y., Wu, Y. J., Goh, M., & Liu, X. (2019). Supply chain management scholar’s research impact: Moderated mediation analysis. Library Hi Tech, 39(1), 118–135. https://doi.org/10.1108/LHT-07-2017-0149

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Carlos Vílchez-Román.

Appendices

Appendix A

See Tables 5 and 6.

Table 5 Parameter estimates of mediation analysis for the general model (filtered dataset, n = 758,419)
Table 6 Parameter estimates of mediation effects for each category (filtered dataset, n = 758,419)

Appendix B

Links to the Comma-Separated-Values (CSV) files used in the study.


Filtered dataset (n = 758,419 records)

https://doi.org/10.6084/m9.figshare.16652707.


Analyzed dataset (n = 235,384 records)

https://doi.org/10.6084/m9.figshare.16652758.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Vílchez-Román, C., Vara-Horna, A. Usage, content and citation in open access publication: any interaction effects?. Scientometrics 126, 9457–9476 (2021). https://doi.org/10.1007/s11192-021-04178-5

Download citation

Keywords

  • Open access publications
  • Mediation analysis
  • Social media
  • Altmetrics

Mathematics Subject Classification

  • 62

JEL Classification

  • D80