Skip to main content
SpringerLink
Log in

Is citation analysis a legitimate evaluation tool?

  • Published:
Scientometrics Aims and scope Submit manuscript

Abstract

A comprehensive discussion on the use of citation analysis to rate scientific performance and the controversy surrounding it. The general adverse criticism that citation counts include an excessive number of negative citations (citations to incorrect results worthy of attack), self-citations (citations to the works of the citing authors), and citations to methodological papers is analyzed. Included are a discussion of measurement problems such as counting citations for multiauthored papers, distinguishing between more than one person with the same last name (homographs), and what it is that citation analysis actually measures. It is concluded that as the scientific enterprise becomes larger and more complex, and its role in society more critical, it will become more difficult, expensive and necessary to evaluate and identify the largest contributors. When properly used, citation analysis can introduce a useful measure of objectivity into the evaluation process at relatively low financial cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. A. A. JOHNSON, R. B. DAVIS. The Research Productivity of Academic Materials Scientists,Journal of Metals 27 (1975) No. 6, 28–29.

    Google Scholar 

  2. D. SHAPLEY. NSF: A ‘Populist’ Pattern in Metallurgy, Materials Research?Science, 189 (1975) No. 4203, 622–624.

    Google Scholar 

  3. C. A. WERT. The Citation Index Revisited,Journal of Metals 27 (1975) No. 12, 20–22.

    Google Scholar 

  4. T. GUSTAFSON, The Controversy Over Peer Review,Science, 190 (1975) No. 4219, 1060–1066.

    Google Scholar 

  5. D. SHAPLEY, Materials Research: Scientists Show Scant Taste for Breaking Ranks,Science, 191 (1976) No. 4222, 53.

    Google Scholar 

  6. R. ROY, Comments on Citation Study of Materials Science Departments,Journal of Metals, 28 (1976) No. 6, 29–30.

    Google Scholar 

  7. G. C. CROSBIE, R. W. HECKEL, Citation Criteria for Ranking Academic Departments,Journal of Metals, 28 (1976) No. 9, 27–28.

    Google Scholar 

  8. N. ARBITER, Letter to the Editor,Journal of Metals, 28 (1976) No. 12, 33.

    Google Scholar 

  9. J. C. AGARWAL, et al., Letter to the Editor,Journal of Metals, 28 (1976) No. 12, 33.

    Google Scholar 

  10. C. J. ALTSTETTER, Letter to the Editor,Journal of Metals, 28 (1976) No. 12, 33–35.

    Google Scholar 

  11. N. WADE, Citation Analysis: A New Tool for Science Administrators,Science, 188 (1975) No. 4187, 429–432.

    Google Scholar 

  12. Citation Analysis. Letters in response to Wade by M. Klerer, H. J. M. Hanley, J. Arditti, R. E. Machol,Science, 188 (1975) No. 4193, 1064. [N. WADE, Citation Analysis: A New Tool for Science Administrators,Science, 188 (1975) No. 4187, 429–432.]

    Google Scholar 

  13. G. P. KOSHY, The Citeability of a Scientific Paper.Proc. of Northeast Regional Conference of American Institute for Decision Sciences, Philadelphia, Pa., April/May 1976, 224–227.

  14. E. GARFIELD, Is the Ratio Between Number of Citations and Publications Cited a True Constant?Essays of an Information Scientist, Vol. 2, Philadelphia: ISI Press, 1977, 419–421.

    Google Scholar 

  15. E. GARFIELD, Will ISI'sArts & Humanities Citation Index Revolutionize Scholarship?Current Contents, No. 32 (August 8, 1977) 5–9.

    Google Scholar 

  16. I. H. SHER, E. GARFIELD, New Tools for Improving and Evaluating the Effectiveness of Research, inResearch Program Effectiveness, M. C. YOUITS, D. M. GILFORD, R. H. WILCOX, E. STAVELY, H. D. LEMER, (Eds). New York; Gordon and Breach, 1966, p. 135–146.

    Google Scholar 

  17. J. P. MARTINO, Citation Indexing for Research and Development Management,IEEE Transactions on Engineering Management, EM-18 (1971) No. 4, 146–151.

    Google Scholar 

  18. A. E. BAYER, J. FOLGER, Some Correlates of a Citation Measure of Productivity in Science,Sociology of Education, 39 (1966) 381–390.

    Google Scholar 

  19. J. A. VIRGO, A Statistical Procedure for Evaluating the Importance of Scientific Papers,Library Quarterly 47 (1977) No. 4, 415–430.

    Google Scholar 

  20. E. GARFIELD, Citation Indexing for Studying Science,Nature, 227 (1970) No. 5259, 669–671.

    Article  Google Scholar 

  21. C. L. BERNIER, W. N. GILL, R. G. HUNT, Measures of Excellence of Engineering and Science Departments: A Chemical Engineering Example,Chemical Engineering Education, 9 (1975) 94–97.

    Google Scholar 

  22. G. M. CARTER, Peer Review, Citations, and Biomedical Research Policy: NIH Grants to Medical School Faculty,Rand Report, R-1583-HEW, Santa Monica, California; Rand Corporation, 1974.

    Google Scholar 

  23. A. J. MEADOWS,Communication in Science, London Butterworths, 1974, p. 45.

    Google Scholar 

  24. H. G. SMALL,Characteristics of Frequently Cited Papers in Chemistry, Final Report on NSF Contract #C795, 1974. See also E. GARFIELD, A list of 100 Most Cited (Chemical) Articles,Current Contents, No. 10 (March 9, 1977) 5–12.

  25. O. H. LOWRY, N. J. ROSEBROUGH, A. L. FARR, R. J. RANDAL, Protein measurement with the Folin phenol reagent,Journal of Biological Chemistry, 193 (1951) 265–275.

    PubMed  Google Scholar 

  26. O. H. LOWRY, Personal communication to D. de S. PRICE, November 11, 1969.

  27. H. G. SMALL, B. C. GRIFFITH, The Structure of Scientific Literatures, I: Identifying and Graphing Specialties,Science Studies, 4 (1974) 17–40.

    Google Scholar 

  28. B. C. GRIFFITH, H. G. SMALL, J. A. STONEHILL, S. DEY, The Structure of Scientific Literatures, II: Towards a Macro- and Microstructure for Science,Science Studies 4 (1974) 339–365.

    Google Scholar 

  29. R. K. MERTON,On the Shoulders of Giants-A Shandean Postcript, New York, Harcourt Brace & World, 1965, p. 218–219.

    Google Scholar 

  30. J. LEDERBERG, A View of Genetics,Science, 131 (1960) 269–276.

    Google Scholar 

  31. E. GARFIELD, I. H. SHER, Citation Indexes in Sociological and Historical Research,American Documentation, 14 (1963) 289–291.

    Google Scholar 

  32. F. NARIN,Evaluative Bibliometrics: The USE of Publication and Citation Analysis in the Evaluation of Scientific Activity, Cherry Hill, N.J.; Computer Horizons, Inc., 1976, p. 500, NTIS-PB252339/AS.

    Google Scholar 

  33. J. R. COLE, S. COLE, The Ortega Hypothesis,Science, 178 (1975) 368–375.

    Google Scholar 

  34. E. GARFIELD, Citation Analysis and the Anti-Vivisection Controversy,Current Contents, No. 17 (April 25, 1977) 5–10.

    Google Scholar 

  35. E. GARFIELD, Citation Analysis and the Anti-Vivisection Controversy. Part II. An Assessment of Lester R. Aronson's Citation Record,Current Contents, No. 48 (November 28, 1977) 5–14.

    Google Scholar 

  36. E. GARFIELD, The 250 Most-Cited Primary Authors, 1961–1975. Part II. The Correlation Between Citedness, Nobel Prizes, and Academy Memberships,Current Contents, No. 50 (December 12, 1977) 5–15.

    Google Scholar 

  37. E. GARFIELD, The 250 Most-Cited Primary Authors, 1961–1975. Part III. Each Author's Most Cited Publication,Current Contents, No. 51 (December 19, 1977) 5–20.

    Google Scholar 

  38. E. GARFIELD, The 300 Most-Cited Authors, 1961–1976, Including Co-Authors at Last. Part I. How the Names Were Selected,Current Contents, No. 28 (July 10, 1978) 5–18.

    Google Scholar 

  39. J. R. COLE, S. COLE,Social Stratification in Science, Chicago; University of Chicago Press, 1973, p. 32–33.

    Google Scholar 

  40. D. LINDSEY, G. W. BROWN, Problems of Measurement in the Sociology of Science: Taking Account of Collaboration, (unpublished, 1977).

  41. R. ROY, Approximating Total Citation Counts From First-Author Counts and Total Papers, Working Paper, April 1977.

  42. D. de S. PRICE,Little Science, Big Science, New York, Columbia University Press, 1963.

    Google Scholar 

  43. D. de S. PRICE, D. B. BEAVER, Collaboration in an Invisible College,American Psychologist, 21 (1966) 1011–1018.

    Google Scholar 

  44. N. L. GELLER, J. S. CANI, R. E. DAVIES, Lifetime-Citation Rates as a Basis for Comparisons Within a Scientific Field,Proc. of the American Statistical Association. Social Statistics Section, Washington, D. C., 1975, p. 429–433.

  45. N. L. GELLER, J. S. de CANI, R. E. DAVIES, Lifetime-Citation Rates to Compare Scientists' Work,Social Science Research, (in press).

  46. E. GARFIELD, Caution Urged in the Use of Citation Analysis,Trends in Biochemical Sciences, 2 (1977) No. 4, N84.

    Article  Google Scholar 

  47. D. LINDSEY, G. W. BROWN, The Measurement of Quality in Social Studies of Science: Critical Reflections on the Use of Citation Counts, (unpublished, 1977).

Download references

Author information

Authors and Affiliations

  1. Institute for Scientific Information® (ISI®), 325 Chestnut Street, 19106, Philadelphia, PA, (USA)

    E. Garfield

Authors
  1. E. Garfield
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Modification of a chapter in E. Garfield:Citation Indexing: Its Theory and Application in Science, Technology and the Humanities, New York, Wiley, 1979.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Garfield, E. Is citation analysis a legitimate evaluation tool?. Scientometrics 1, 359–375 (1979). https://doi.org/10.1007/BF02019306

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF02019306

Keywords

Search

Navigation