A comprehensive discussion on the use of citation analysis to rate scientific performance and the controversy surrounding it. The general adverse criticism that citation counts include an excessive number of negative citations (citations to incorrect results worthy of attack), self-citations (citations to the works of the citing authors), and citations to methodological papers is analyzed. Included are a discussion of measurement problems such as counting citations for multiauthored papers, distinguishing between more than one person with the same last name (homographs), and what it is that citation analysis actually measures. It is concluded that as the scientific enterprise becomes larger and more complex, and its role in society more critical, it will become more difficult, expensive and necessary to evaluate and identify the largest contributors. When properly used, citation analysis can introduce a useful measure of objectivity into the evaluation process at relatively low financial cost.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
A. A. JOHNSON, R. B. DAVIS. The Research Productivity of Academic Materials Scientists,Journal of Metals 27 (1975) No. 6, 28–29.
D. SHAPLEY. NSF: A ‘Populist’ Pattern in Metallurgy, Materials Research?Science, 189 (1975) No. 4203, 622–624.
C. A. WERT. The Citation Index Revisited,Journal of Metals 27 (1975) No. 12, 20–22.
T. GUSTAFSON, The Controversy Over Peer Review,Science, 190 (1975) No. 4219, 1060–1066.
D. SHAPLEY, Materials Research: Scientists Show Scant Taste for Breaking Ranks,Science, 191 (1976) No. 4222, 53.
R. ROY, Comments on Citation Study of Materials Science Departments,Journal of Metals, 28 (1976) No. 6, 29–30.
G. C. CROSBIE, R. W. HECKEL, Citation Criteria for Ranking Academic Departments,Journal of Metals, 28 (1976) No. 9, 27–28.
N. ARBITER, Letter to the Editor,Journal of Metals, 28 (1976) No. 12, 33.
J. C. AGARWAL, et al., Letter to the Editor,Journal of Metals, 28 (1976) No. 12, 33.
C. J. ALTSTETTER, Letter to the Editor,Journal of Metals, 28 (1976) No. 12, 33–35.
N. WADE, Citation Analysis: A New Tool for Science Administrators,Science, 188 (1975) No. 4187, 429–432.
Citation Analysis. Letters in response to Wade by M. Klerer, H. J. M. Hanley, J. Arditti, R. E. Machol,Science, 188 (1975) No. 4193, 1064. [N. WADE, Citation Analysis: A New Tool for Science Administrators,Science, 188 (1975) No. 4187, 429–432.]
G. P. KOSHY, The Citeability of a Scientific Paper.Proc. of Northeast Regional Conference of American Institute for Decision Sciences, Philadelphia, Pa., April/May 1976, 224–227.
E. GARFIELD, Is the Ratio Between Number of Citations and Publications Cited a True Constant?Essays of an Information Scientist, Vol. 2, Philadelphia: ISI Press, 1977, 419–421.
E. GARFIELD, Will ISI'sArts & Humanities Citation Index Revolutionize Scholarship?Current Contents, No. 32 (August 8, 1977) 5–9.
I. H. SHER, E. GARFIELD, New Tools for Improving and Evaluating the Effectiveness of Research, inResearch Program Effectiveness, M. C. YOUITS, D. M. GILFORD, R. H. WILCOX, E. STAVELY, H. D. LEMER, (Eds). New York; Gordon and Breach, 1966, p. 135–146.
J. P. MARTINO, Citation Indexing for Research and Development Management,IEEE Transactions on Engineering Management, EM-18 (1971) No. 4, 146–151.
A. E. BAYER, J. FOLGER, Some Correlates of a Citation Measure of Productivity in Science,Sociology of Education, 39 (1966) 381–390.
J. A. VIRGO, A Statistical Procedure for Evaluating the Importance of Scientific Papers,Library Quarterly 47 (1977) No. 4, 415–430.
E. GARFIELD, Citation Indexing for Studying Science,Nature, 227 (1970) No. 5259, 669–671.
C. L. BERNIER, W. N. GILL, R. G. HUNT, Measures of Excellence of Engineering and Science Departments: A Chemical Engineering Example,Chemical Engineering Education, 9 (1975) 94–97.
G. M. CARTER, Peer Review, Citations, and Biomedical Research Policy: NIH Grants to Medical School Faculty,Rand Report, R-1583-HEW, Santa Monica, California; Rand Corporation, 1974.
A. J. MEADOWS,Communication in Science, London Butterworths, 1974, p. 45.
H. G. SMALL,Characteristics of Frequently Cited Papers in Chemistry, Final Report on NSF Contract #C795, 1974. See also E. GARFIELD, A list of 100 Most Cited (Chemical) Articles,Current Contents, No. 10 (March 9, 1977) 5–12.
O. H. LOWRY, N. J. ROSEBROUGH, A. L. FARR, R. J. RANDAL, Protein measurement with the Folin phenol reagent,Journal of Biological Chemistry, 193 (1951) 265–275.
O. H. LOWRY, Personal communication to D. de S. PRICE, November 11, 1969.
H. G. SMALL, B. C. GRIFFITH, The Structure of Scientific Literatures, I: Identifying and Graphing Specialties,Science Studies, 4 (1974) 17–40.
B. C. GRIFFITH, H. G. SMALL, J. A. STONEHILL, S. DEY, The Structure of Scientific Literatures, II: Towards a Macro- and Microstructure for Science,Science Studies 4 (1974) 339–365.
R. K. MERTON,On the Shoulders of Giants-A Shandean Postcript, New York, Harcourt Brace & World, 1965, p. 218–219.
J. LEDERBERG, A View of Genetics,Science, 131 (1960) 269–276.
E. GARFIELD, I. H. SHER, Citation Indexes in Sociological and Historical Research,American Documentation, 14 (1963) 289–291.
F. NARIN,Evaluative Bibliometrics: The USE of Publication and Citation Analysis in the Evaluation of Scientific Activity, Cherry Hill, N.J.; Computer Horizons, Inc., 1976, p. 500, NTIS-PB252339/AS.
J. R. COLE, S. COLE, The Ortega Hypothesis,Science, 178 (1975) 368–375.
E. GARFIELD, Citation Analysis and the Anti-Vivisection Controversy,Current Contents, No. 17 (April 25, 1977) 5–10.
E. GARFIELD, Citation Analysis and the Anti-Vivisection Controversy. Part II. An Assessment of Lester R. Aronson's Citation Record,Current Contents, No. 48 (November 28, 1977) 5–14.
E. GARFIELD, The 250 Most-Cited Primary Authors, 1961–1975. Part II. The Correlation Between Citedness, Nobel Prizes, and Academy Memberships,Current Contents, No. 50 (December 12, 1977) 5–15.
E. GARFIELD, The 250 Most-Cited Primary Authors, 1961–1975. Part III. Each Author's Most Cited Publication,Current Contents, No. 51 (December 19, 1977) 5–20.
E. GARFIELD, The 300 Most-Cited Authors, 1961–1976, Including Co-Authors at Last. Part I. How the Names Were Selected,Current Contents, No. 28 (July 10, 1978) 5–18.
J. R. COLE, S. COLE,Social Stratification in Science, Chicago; University of Chicago Press, 1973, p. 32–33.
D. LINDSEY, G. W. BROWN, Problems of Measurement in the Sociology of Science: Taking Account of Collaboration, (unpublished, 1977).
R. ROY, Approximating Total Citation Counts From First-Author Counts and Total Papers, Working Paper, April 1977.
D. de S. PRICE,Little Science, Big Science, New York, Columbia University Press, 1963.
D. de S. PRICE, D. B. BEAVER, Collaboration in an Invisible College,American Psychologist, 21 (1966) 1011–1018.
N. L. GELLER, J. S. CANI, R. E. DAVIES, Lifetime-Citation Rates as a Basis for Comparisons Within a Scientific Field,Proc. of the American Statistical Association. Social Statistics Section, Washington, D. C., 1975, p. 429–433.
N. L. GELLER, J. S. de CANI, R. E. DAVIES, Lifetime-Citation Rates to Compare Scientists' Work,Social Science Research, (in press).
E. GARFIELD, Caution Urged in the Use of Citation Analysis,Trends in Biochemical Sciences, 2 (1977) No. 4, N84.
D. LINDSEY, G. W. BROWN, The Measurement of Quality in Social Studies of Science: Critical Reflections on the Use of Citation Counts, (unpublished, 1977).
Modification of a chapter in E. Garfield:Citation Indexing: Its Theory and Application in Science, Technology and the Humanities, New York, Wiley, 1979.