Skip to main content
SpringerLink
Log in

What a difference a colon makes: how superficial factors influence subsequent citation

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

Getting cited is important for scholars and for the institutions in which they work. Whether because of the influence on scientific progress or because of the reputation of scholars and their institutions, understanding why some articles are cited more often than others can help scholars write more highly cited articles. This article builds upon earlier literature which identifies seemingly superficial factors that influence the citation rate of articles. Three Journal Citation Report subject categories are analyzed to identify these effects. From a set of 2,016 articles in Sociology, 6,957 articles in General & Internal Medicine, and 23,676 articles in Applied Physics, metadata from the Web of Knowledge was downloaded in addition to PDFs of the full articles. In this article number of words in title, number of pages, number of references, sentences in the abstract, sentences in the paper, number of authors and readability were identified as factors for analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ball, R., Mittermaier, B., & Tunger, D. (2009). Creation of journal-based publication profiles of scientific institutions—A methodology for the interdisciplinary comparison of scientific research based on the J-factor. Scientometrics, 81(2), 381–392. doi:10.1007/s11192-009-2120-5.

    Article  Google Scholar 

  • Booth, W. C., Colomb, G. G., & Williams, J. M. (2003). The craft of research (2nd ed., Chicago guides to writing, editing, and publishing). Chicago: University of Chicago press.

    Google Scholar 

  • Botton, A. D. (2001). The consolations of philosophy. Baltimore, MD: Penguin Books Ltd.

    Google Scholar 

  • Collins, H. M. (1990). Artificial experts: Social knowledge and intelligent machines (inside technology). Cambridge, MA: MIT Press.

    Google Scholar 

  • Crossley, S., Greenfield, J., & McNamara, D. (2008). Assessing text readability using cognitively based indices. Tesol Quarterly, 42(3), 475–493. doi:10.1002/j.1545-7249.2008.tb00142.x.

    Google Scholar 

  • Flesch, R. (1948). A new readability yardstick. Journal of Applied Psychology, 32(3), 221–233. doi:10.1037/h0057532.

    Article  Google Scholar 

  • Franceschet, M., & Costantini, A. (2010). The effect of scholar collaboration on impact and quality of academic papers. Journal of Informetrics, 4(4), 540–553. doi:http://dx.doi.org/10.1016/j.joi.2010.06.003.

  • Frenken, K., Hölzl, W., & Vor, F. D. (2005). The citation impact of research collaborations: the case of European biotechnology and applied microbiology (1988–2002). Journal of Engineering and Technology Management, 22, 9–30. doi:10.1111/j.1435-5957.2010.00309.x.

    Article  Google Scholar 

  • Friedman, D. B., Hoffman-Goetz, L., & Arocha, J. F. (2004). Readability of cancer information on the internet. Journal of Cancer Education, 19(2), 117–122. doi:10.1207/s15430154jce1902_13.

    Article  Google Scholar 

  • Fry, E. (1968). A readability formula that saves time. Journal of Reading, 11(7), 513–516, 575–578. doi:10.2307/40013635.

    Google Scholar 

  • Gilbert, G. N. (1977). Referencing as persuasion. Social Studies of Science, 7(1), 113–122. doi:10.2307/284636.

    Article  Google Scholar 

  • Glänzel, W., & Thijs, B. (2004). Does co-authorship inflate the share of self-citations? Scientometrics, 61(3), 395–404. doi:10.1023/B:SCIE.0000045117.13348.b1.

    Article  Google Scholar 

  • Hartley, J., Trueman, M., & Meadows, A. (1988). Readability and prestige in scientific journals. Journal of Information Science, 14(2), 69–75. doi:10.1177/016555158901500209.

    Article  Google Scholar 

  • Haslam, N., Ban, L., Kaufmann, L., Loughnan, S., Peters, K., Whelan, J., et al. (2008). What makes an article influential? Predicting impact in social and personality psychology. Scientometrics, 76(1), 169–185. doi:10.1007/s11192-007-1892-8.

    Article  Google Scholar 

  • Hayden, J. D. (2008). Readability of the British Journal of Surgery. British Journal of Surgery, 95, 119–124. doi:10.1002/bjs.5994.

    Article  Google Scholar 

  • Hudson, J. (2007). Be known by the company you keep: Citations—quality or chance? Scientometrics, 71(2), 231–238. doi:10.1007/s11192-007-1671-6.

    Article  Google Scholar 

  • Jacques, T. S., & Sebire, N. J. (2010). The impact of article titles on citation hits: An analysis of general and specialist medical journals. JRSM Short Reports, 1(1). doi:10.1258/shorts.2009.100020.

  • Jamali, H., & Nikzad, M. (2011). Article title type and its relation with the number of downloads and citations. Scientometrics, 88(2), 653–661. doi:10.1007/s11192-011-0412-z.

    Article  Google Scholar 

  • Kincaid, J. P., Fishburne, R. P, Jr., Rogers, R. L., & Chissom, B. S. (1975). Derivation of new readability formulas (automated readability index, fog count and flesch reading ease formula) for navy enlisted personnel. Springfield, VA: National Technical Information Service.

    Google Scholar 

  • Larivière, V., Archambault, É., & Gingras, Y. (2008). Long-term variations in the aging of scientific literature: From exponential growth to steady-state science (1900–2004). Journal of the American Society for Information Science and Technology, 59(2), 288–296. doi:10.1002/asi.20744.

    Article  Google Scholar 

  • Latour, B. (1987). Science in action: How to follow scientists and engineers through society. Cambridge, MA: Harvard University Press.

    Google Scholar 

  • Latour, B., & Woolgar, S. (1986). Laboratory life: The construction of scientific facts. Princeton, NJ: Princeton University Press.

    Google Scholar 

  • Levitt, J. M., & Thelwall, M. (2009). Citation levels and collaboration within library and information science. Journal of the American Society for Information Science and Technology, 60(3), 434–442. doi:10.1002/asi.21000.

    Article  Google Scholar 

  • Lin, S.-Y., Su, C.-C., Lai, Y.-D., Yang, L.-C., & Hsieh, S.-K. (2009). Assessing text readability using hierarchical lexical relations retrieved from WordNet. Computational Linguistics and Chinese Language Processing, 14(1), 45–84.

    Google Scholar 

  • Martin, B., & Groth, E. (1991). Scientific knowledge in controversy: The social dynamics of the fluoridation debate (SUNY series in science, technology, and society). Albany, NY: State University of New York Press.

    Google Scholar 

  • Merton, R. (1968). The Matthew effect in science: The reward and communication systems of science are considered. Science, 159(3810), 56–63. doi:10.1126/science.159.3810.56.

    Article  Google Scholar 

  • Microsoft. (2003). Readability scores. http://office.microsoft.com/en-us/word/HP051863181033.aspx. Accessed 23 March 2013.

  • Neuman, W. L. (1991). Social research methods: Qualitative and quantitative approaches. Boston: Allyn and Bacon.

    Google Scholar 

  • Price, D. J. D. S. (1963). Little science, big science (George B. Pegram lectures, Vol. 1962). New York: Columbia University Press.

  • Ramos, M. A., Melo, J. G., & Albuquerque, U. P. (2012). Citation behavior in popular scientific papers: What is behind obscure citations? The case of ethnobotany. Scientometrics, 92, 711–719. doi:10.1007/s11192-012-0662-4.

    Article  Google Scholar 

  • Roberts, J. C., Fletcher, R. H., & Fletcher, S. W. (1994). Effects of peer review and editing on the readability of articles published in annals of internal medicine. Journal of the American Medical Association, 272, 119–121. doi:10.1001/jama.1994.03520020045012.

    Article  Google Scholar 

  • Smart, J. C., & Bayer, A. E. (1986). Author collaboration and impact: a note on citation rates of single and multiple authored articles. Scientometrics, 10(5–6), 297–305. doi:10.1007/BF02016776.

    Article  Google Scholar 

  • Stremersch, S., Verniers, I., & Verhoef, P. (2007). The quest for citations: Drivers of article impact. Journal of Marketing, 71(3), 171–193. doi:10.1509/jmkg.71.3.171.

    Article  Google Scholar 

  • Vieira, E. S., & Gomes, J. A. N. F. (2010). Citations to scientific articles: Its distribution and dependence on the article features. Journal of Informetrics, 4, 1–13. doi:10.1016/j.joi.2009.06.002.

    Article  Google Scholar 

  • Villere, M. F., & Stearns, G. K. (1976). The readability of organizational behavior textbooks. The Academy of Management Journal, 19(1), 132–137. doi:10.2307/255455.

    Article  Google Scholar 

  • Wager, E., & Middleton, P. (2002). Effects of technical editing in biomedical journals: A systematic review. JAMA, 287(21), 2821–2824. doi:10.1001/jama.287.21.2821.

    Article  Google Scholar 

  • Wang, M., Yu, G., An, S., & Yu, D. (2012). Discovery of factors influencing citation impact based on a soft fuzzy rough set model. Scientometrics, 93, 635–644. doi:10.1007/s11192-012-0766-x.

    Google Scholar 

  • Webster, G. D., Jonason, P. K., & Schember, T. O. (2009). Hot topics and popular papers in evolutionary psychology: Analyses of title words and citation counts in evolution and human behavior, 1979–2008. Evolutionary Psychology, 7(3), 348–362.

    Google Scholar 

  • Weeks, W. B., & Wallace, A. E. (2002). Readability of British and American medical prose at the start of the 21st century. British Medical Journal, 325, 1451–1452. doi:10.1136/bmj.325. 7378.1451.

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Loet Leydesdorff for his helpful comments. Furthermore we believe that additional comments from the two anonymous reviewers have increased the quality of this article, for which we are grateful.

Author information

Authors and Affiliations

  1. Department of Family Medicine, Faculty of Health, Medicine and Life Sciences, Maastricht University, PO Box 616, 6200 MD, Maastricht, The Netherlands

    Maarten van Wesel

  2. Department of Technology and Society Studies, Faculty of Arts and Social Sciences, Maastricht University, PO Box 616, 6200 MD, Maastricht, The Netherlands

    Sally Wyatt

  3. Department of Education and Research Services, University Library, Maastricht University, PO Box 616, 6200 MD, Maastricht, The Netherlands

    Jeroen ten Haaf

Authors
  1. Maarten van Wesel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sally Wyatt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeroen ten Haaf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Maarten van Wesel.

Appendices

Appendix: limitations to the automatic processing of article text

The automatic processing of article text offers great advantages in terms of speed thus increasing the sheer number of articles that can be processed, but there are disadvantages in terms of accuracy. Some of the causes will be briefly discussed below. Even though the problems discussed below have an impact on the readability score of individual articles, our assessment is that, as all articles in a journal/year set seem to suffer the same problems, it reduces the inter-article differences per subset. Therefore we do not expect the problems to reduce the reliability of our analysis.

Continuous print

Many journals offer articles as a single unit in a PDF, but some journals print articles continuously, i.e. not always starting an article on a new page. Thus the PDF of an article may also contain pages of another article, most likely its reference list or first page. These pages are also included in the analysis of the target article.

OCR mistakes

As mentioned in the main text regarding titles of articles in the Web of Science, articles themselves also suffer from OCR problems. Unfortunately not all PDFs are created from their original source. One of the most common problems this respect is mistaking the letter “m” with the combination “rn”, and possibly also vice verse.

Footer/header

Depending on the way an article is created and presented, sentences across pages are read continuously (as a human reader would do) or are read as continuing in the footer and header, thus including items such as the page number, article or journal name.

Affiliations, addresses, and references

All text in the articles is extracted, icluding the affiliations and addresses. As these do not follow normal language conventions they will have an impact on readability.

Rights and permissions

Reprints and permissions

About this article

Cite this article

van Wesel, M., Wyatt, S. & ten Haaf, J. What a difference a colon makes: how superficial factors influence subsequent citation. Scientometrics 98, 1601–1615 (2014). https://doi.org/10.1007/s11192-013-1154-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11192-013-1154-x

Keywords

MSC

JEL Classification

Search

Navigation