Citation ranking versus expert judgment in evaluating communication scholars: Effects of research specialty size and individual prominence
- 183 Downloads
Numerous attempts have been made to validate the use of citation as an evaluation method by comparing it with peer review. Unlike past studies using journals, research articles or universities as the subject matter, the present study extends the comparison to the ranking of individual scholars. Results show that citation ranking and expert judgment of communication scholars are highly correlated. The citation method and the expert judgment method are found to work better in smaller research areas and yield more valid evaluation results for more prominent scholars.
Unable to display preview. Download preview PDF.
Notes and references
- 1.R. C. Anderson, F. Narin, P. McAllister, Publication ratings versus peer ratings of universities,Journal of the American Society for Information Science, 29 (1978) 91–103.Google Scholar
- 3.M. E. D. Koenig, Bibliometric indicators versus expert opinion in assessing research performance,Journal of the American Society for Information Science, 34 (1983) 136–145.Google Scholar
- 4.S. M. Lawani, A. E. Bayer, Validity of citation criteria for assessing the influence of scientific publications: New evidence with peer assessment,Journal of the American Society for Information Science, 34 (1983) 59–66.Google Scholar
- 6.P. R. McAllister, R. C. Anderson, F. Narin, Comparison of peer and citation assessment of the influence of scientific journals,Journal of the American Society for Information Science, 31 (1980) 147–152.Google Scholar
- 7.M. D. Gordon, Citation ranking versus subjective evaluation in the determination of journal hierarchies in the social sciences,Journal of the American Society for Information Science, 33 (1982) 55–57.Google Scholar
- 8.A. Singleton, Journal ranking and selection: A review in physics,Journal of Documentation, 32 (1976) 258–289.Google Scholar
- 10.The citation data collected in 1985–87 and the questionnaire data collected in 1991 were both for the author's dissertation research. Collecting both types of data simultaneously should be more ideal, but the time lapse between the two sets of data should not pose a major threat to validity because overall citation patterns as well as peer judgment usually remain quite stable over a short period of time. For more details on the data collection methods, seeC. Y. K. So,Mapping the Intellectual Landscape of Communication Studies: An Evaluation of Its Disciplinary Status, Unpublished Ph.D. dissertation, Annenberg School for Communication, University of Pennsylvania, 1995 (UMI Number 9615129).Google Scholar
- 12.E. Garfield,Citation Indexing: Its Theory and Application in Science, Technology, and Humanities, ISI Press, Philadelphia, 1979.Google Scholar