Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles


Context. The use of citation frequency and impact factor as measures of research quality and journal prestige is being criticized. Citation frequency is augmented by self-citation and for most journals the majority of citations originate from a minority of papers. We hypothesized that citation frequency is also associated with the geographical origin of the research publication.

Objective. We determined whether citations originate more frequently from institutes that are located in the same country as the authors of the cited publication than would be expected by chance.

Design. We screened citations referring to 1200 cardiovascular publications in the 7 years following their publication. For the 1200 citation recipient publications we documented the country where the research originated (9 countries/regions) and the total number of received citations. For a selection of 8864 citation donor papers we registered the country/region where the citing paper originated.

Results. Self-citation was common in cardiovascular journals (n = 1534, 17.8%). After exclusion of self-citation, however, the number of citations that originated from the same country as the author of the citation recipient was found to be on average 31.6% higher than would be expected by chance (p<0.01 for all countries/regions). In absolute numbers, nation oriented citation bias was most pronounced in the USA, the country with the largest research output (p<0.001).

Conclusion. Citation frequency was significantly augmented by nation oriented citation bias. This nation oriented citation behaviour seems to mainly influence the cumulative citation number for papers originating from the countries with a larger research output.

This is a preview of subscription content, log in to check access.


  1. 1.

    E. Garfield, Citation indexing for studying science. Nature, 227 (1970) 669–671.

  2. 2.

    E. Garfield, Journal impact factor: a brief review. Can. Med. Assoc. J., 161 (1999) 979–800.

  3. 3.

    H. B. Hansen, J. H. Henriksen, How well does journal “impact” work in the assessment of papers on clinical physiology and nuclear medicine? Clin. Physiol., 17(4) (1997) 409–418.

  4. 4.

    V. Hachinski, The Impact of Impact Factors. Stroke, 32 (2001) 2729.

  5. 5.

    P. J. Van Diest, Impactitis: new cures for an old disease. J. Clin. Pathol., 54 (2001) 817–819.

  6. 6.

    A. Fassoulaki, K. Papilas, A. Paraskeva, K. Patris, Impact factor bias and proposed adjustments for its determination. Acta Anaesthesiol., 46 (2002) 902–905.

  7. 7.

    P. O. Seglen, Why the impact factor of journals should not be used for evaluating research. BMJ, 314 (1997) 497.

  8. 8.

    H. F. Moed, Th. N. Van Leeuwen, J. Reedijk, Towards appropriate indicators of journal impact. Scientometrics, 46(3) (1999) 575–589.

  9. 9.

    Th. N. Van Leeuwen, H. F. Moed, Characteristics of journal impact factors: the effects of uncitedness and citation distribution on the understanding of journal impact factors. Scientometrics, 63(2) (2005) 357–371.

  10. 10.

    M. Callaham, R. L. Wears, E. Weber, Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA, 287 (2002) 2847–2850.

  11. 11.

    E. Garfield, 100 citation classics from the Journal of the American Medical Association. JAMA, 257 (1987) 52–59.

  12. 12.

    G. Lundberg, The omnipotent science citation index impact factor. Med. J. Aust., 178 (2003) 253–254.

  13. 13.

    R. West, A. McIIlwaine, What do citation counts for in the field of addiction? An empirical evaluation of citation counts and their link with peer ratings of quality. Addiction, 97 (2002) 501–504.

  14. 14.

    R. Leimu, J. Koricheva, What determines the citation frequency of ecological papers? Trends Ecol. Evol., 20 (2005) 28–32.

  15. 15.

    R. Leimu, J. Koricheva, Does scientific collaboration increase the impact of ecological articles? Bioscience, 55 (2005) 438–443.

  16. 16.

    P. Borry, P. Schotsmans, K. Dierickx, How international is bioethics? A quantitative retrospective study. BMC Medical Ethics, 7 (2006) 1.

  17. 17.

    E. Frank, Authors’ criteria for selecting journals. JAMA, 272 (1994) 163–164.

Download references

Author information

Correspondence to Gerard Pasterkamp.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Pasterkamp, G., Rotmans, J.I., de Kleijn, D.V.P. et al. Citation frequency: A biased measure of research impact significantly influenced by the geographical origin of research articles. Scientometrics 70, 153–165 (2007).

Download citation


  • Impact Factor
  • Citation Behavior
  • Journal Impact Factor
  • European Heart Journal
  • Citation Frequency