Abstract
Although co-citation techniques are very powerful structuring tools, the use of science policy indicators based on co-citation has often been criticized, especially on ISI research fronts. A major issue is the small fraction of literature retrieved, i.e. the “recall rate” problem. Our investigations indicate that at the level of micro/meso studies high recall rates can be achieved by (a) the use of appropriate clustering techniques limiting singletons and (b) the enrichment of cocited cores by medium-cited items. This combination of appropriate clustering and extension of recall proves to be efficient, provided that careful trade-offs are sought between the extension and relevance of recall. It leads to a reassessment of the performance of the co-citation approach for structuring scientific fields and providing related indicators not limited to the ‘leading edge’. It also opens new opportunities for comparison/combination with other relational methods such as co-word analysis.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H. G. Small, Co-citation in the scientific literature,Journal of the American Society for Information Science, 24 (1973) 265–269.
I. V. Marshakova, Document coupling system based on references taken fromScience Citation Index (in Russian),Nauchno-Teknicheskaya Informatsiya, Ser. 2 (1973) No. 6,3.
H. D. White, K. W. McCain, Bibliometrics,Annual Review of Information Science and Technology, 24 (1989) 119–186.
H. D. White, B. C. Griffith, Author Co-citation: A Literature Measure of Intellectual Structure,Journal of the American Society for Information Science, 32, 3 (1981) 163–172.
K. W. McCain, The Author Cocitation Structure of Macro-Economics,Scientometrics 5 (1983) 277–289.
H. Penan, Analyse Des Citations, Principes, Applications à la Théorie Microéconomique, inDesvals, Dou,La Veille Technologique, Dunod 1992, 277–330.
K. W. McCain, Mapping Economics through the Journal Literature: An Experiment in Journal Cocitation Analysis,Journal of the American Society for Information Science, 42, 4 (1991) 290–296.
H. G. Small, B. C. Griffith, The Structure of Scientific Literature. I: Identifying and Graphing Specialties,Science Studies, 4, (1974) 17–40.B. C. Griffith, H. G. Small, J. A. Stonehill, S. Dey, The Structure of Scientific Literature. II: Toward a Macro and Microstructure for Science,Science Studies, 4, (1974) 339–365.
J. King, A Review of Bibliometric and Other Science Indicators and their Role in Research Evaluation,Journal of Information Science, 13 (1987) 261–276.
D. Hicks, Limitations of Co-citation Analysis/Bibliometric Modelling: A Reply to FranklinSocial Studies of Science, 17 (1987) 295–316.
R. R. Braam, H. F. Moed, A. F. J. Van Raan, Mapping of Science by combined Co-citation and Word Analysis, I Structural Aspects,Journal of the American Society of Information Science, 42 (1991) 233–251.
L. Leydesdorff, The Generation of Aggregated Journal-journal Citation Maps on the Basis of the CD-ROM Version of theScience Citation Index, Scientometrics, 31, 1 (1994) 59–84.
H. F. Moed, M. Vriens, Possible Inaccuracies Occurring in Citation Analysis,Journal of Information Science, 15 (1989) 95–107.
D. Sullivan, D. H. White, E. J. Barboni Co-Citation Analyses of Science-Evaluation,Social Studies of Science, 7, 2 (1977) 223–240.
D. Hicks, Limitations and More Limitations of Co-citation Analysis as a Tool for Science Policy,Social Studies of Science, 18 (1988) 375–384.
H. G. Small, E. Sweeney, Clustering theScience Citation Index using Co-citations. I A comparison of methods,Scientometrics, 7, 3–6 (1985) 391–409H. G. Small, E. Sweeney, E. Greenlee, Clustering theScience Citation Index using Co-citations. II Mapping Science,Scientometrics, 8, 5–6 (1985) 321–340.
P. Healey, H. Rothman, P. K. Hoch, An Experiment in Science Mapping for Research Planning,Research Policy, 15 (1986) 233–251.
W. Glänzel, H. J. Czerwon, A New Methodological Approach to Bibliographic Coupling and its Application to Research-Front and Other Core Documents,Proceedings of the Fifth Biennial Conference of the International Society for Scientometrics and Informetrics, River Forest, 7–10 June 1995, Learned Information Inc., Medford, New Jersey, 167–176.
A. Rip, Mapping of Science: Possibilities and LimitationsHandbook of Quantitative Studies of Science and Technology,A. F. J. van Raan (Ed.), Elsevier Science Publishers, Amsterdam, 1988, 253–273.
M. Zitt, E. Bassecoulard, Development of a method for detection and trend analysis of research fronts built by lexical or co-citation analysis.Scientometrics, 30, 1 (1994) 333–351.
M. Zitt, E. Bassecoulard, Recall Rates of Co-citation Techniques: field Bibliometric Constraints and Strategies of Improvement in Micro Studies,Proceedings of the Fifth Biennial Conference of the International Society for Scientometrics and Informetrics, River Forest, 7–10 June 1995, Learned Information Inc., Medford, New Jersey, 167–176.
B. L. Milman, Yu.A. Gavrilova, Citations in Chemical Engineering.,Scientometrics, 27, 1 (1993) 53–74.
M. H. McRoberts, B. R. McRoberts, Problems of Citation Analysis: A Critical Review.,Journal of the American Society for Information Science, 40, 5 (1989) 342–349.
J. J. Franklin, Testing and Using Quantitative Methods in Science Policy Contexts: A Response to Hicks,Social Studies of Science, 18 (1988) 365–375.
J. A. Hartigan,Clustering Algorithms, John Wiley, New York, 1975.
Sas Institute Inc., Introduction to Clustering Procedures, inSAS/STAT TM User's Guide, Release 6,03 Edition. Cary, NC: SAS Institute Inc., 1988, 47–83.
L. Leydesdorff, Various Methods for the Mapping of Science,Scientometrics, 11, 5–6 (1987) 295–324.
R. Todorov, J. Vlachy, National Patterns in Physics Output by Subfields,Czechoslowak Journal of Physics, B36, (1986), 163–166.
G. W. Milligan, An Examination of the Effects of Six Types of Error Perturbation on Fifteen Clustering Algorithms,Psychometrika, 45, 3, (1980) 325–342.
M. M. Kessler, Comparison of Results of Bibliographic Coupling and Analytic Subject Indexing,American Documentation, 16 (1965) 223–233.
W. A. Turner, G. Chartron, F. Laville, F. B. Michelet, Packaging Information for Peer Review: new Co-word Analysis Techniques,Handbook of Quantitative Studies of Science and Technology,A. F. J. Van Raan (Ed.), Elsevier Science Publishers, Amsterdam, 1988, 291–323.
H. G. Small, Macro-level Changes in the Structure of Co-citation Clusters: 1983–1989,Scientometrics, 26, 1 (1993) 5–20.
P. Stern, Bibliometric Maps as Valid Representations of Scientific Practices: Controversies and Closure in the Social Sciences,Proceedings of the Fifth Biennial Conference of the International Society for Scientometrics and Informetrics, River Forest, 7–10 June 1995, Learned Information Inc., Medford, New Jersey, 167–176.
R. H. Coward, R. R. Fresne, Taxonomies of Science and Bibliometric Indicators: Developing a Methodology for Classifying Research Fronts in a Co-citation Database,Proceedings of the Joint EC-Leiden Conference on Science and Technology Indicators Leiden, 23–25 Oct. 1991, DSWO Press, Leiden University.
J. P. Benzecri, et al.,L'Analyse des Données, 1 La Taxinomie Paris, Dunod, 1973, 180–181.
C. Hennequet,Exploration et Validation de Méthodes de Classification en vue d'Analyses Bibliometriques, DESS Report, Univ. Paul Sabatier, Toulouse/INRA LERECO, 1993.
P. Willett, Recent trends in hierarchic document clustering: a critical review.,Information Processing and Management, 24, 5 (1988) 577–597.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Zitt, M., Bassecoulard, E. Reassessment of co-citation methods for science indicators: Effect of methods improving recall rates. Scientometrics 37, 223–244 (1996). https://doi.org/10.1007/BF02093622
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF02093622