Scientometrics

, Volume 21, Issue 1, pp 49–68 | Cite as

The utility of scientific papers

  • J. -L. Sikorav
Article

Abstract

This article investigates the function of scientific papers in the production of scientific knowledge. For this production, the citations made of these papers in the scientific literature can be considered as economic utilities. The work of the scientist is described as the production of citations by means of citations. The number of citations received by a given paper can be used to measure the paper's formal utility. The formal utility of scientific papers is studied empirically. It is concluded that the references contained in a scientific paper are a major determinant of its future utility.

Keywords

Scientific Literature Scientific Knowledge Major Determinant Scientific Paper Future Utility 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes and references

  1. 1.
    A classical introduction to the sociology of sciences is provided byR.K. Merton,The Sociology of Science. Theoretical and Empirical Investigations, University of Chicago Press, Chicago, 1973. The various modes of communication between scientists are described by D.,Crane Invisible Colleges: Diffusion of Knowledge in Scientific Communities, University of Chicago Press, Chicago, 1972.Google Scholar
  2. 2.
    D.M. Lamberton, The Economics of information and organisation,Annual Review of Information Science and Technology, 19 (1984) 3;B. Cronin, M. Gudin, Information and productivity: a review of research,International Journal of Information Management, 6 (1986) 85.Google Scholar
  3. 3.
    Reviewed byB. Cronin,The Citation Process, Taylor Graham, London, 1984.Google Scholar
  4. 4.
    E. Garfield Citation Indexing: Its Theory and Application in Science, Technology and Humanities, Wiley, New York, 1979. These indexes are published by the Institute for Scientific Information and can be consulted through printed copies, compact-discs or on-line information systems.Google Scholar
  5. 5.
    D. Bernoulli, Specimen theoriae novae de mensura sortis, 1738, translated inEconometrica, 22 (1953) 23; the sentences in which the concept of utility is introduced read: “the determination of the value of an item must not be based on its price, but rather on the utility it yields. The price of an item is dependent only on the thing itself and is equal for everyone; the utility, however, is dependent on the particular circumstances of the person making the estimate”. The evolution of the concept of utility in economics can be found inP.A. Samuelson,The Foundations of Economics, Harvard University Press, Cambridge, 1983, p. 90–96. That a citation reflects the utility of a scientific paper (in the common sense of the word) has long been known.D.J. de Solla Price, in his classical bookLittle Science, Big Science (first published in 1963) considered “the use of a paper in terms of references made to it in other papers” (reprinted inD. de Solla Price, iitLittle Science, Big Science and beyond Columbia University Press, New-York, 1986, p. 69). See also E. Garfield's statement: “People talk about citation counts being a measure of the “importance”, “significance”, or “impact” of scientific work, but those who are knowledgeable about the subject use these words in a very pragmatic sense: what they really are talking about is utility”.E. Garfield, op. cit. p. 246.Google Scholar
  6. 6.
    M. Polanyi has made a similar analogy between the basic research system and the free market economy:M. Polanyi, The republic of science: its political and economic theory,Minerva, 1 (1962) 54.Google Scholar
  7. 7.
    P.A. Samuelson,op. cit.;K.J. Arrow,Social Choice and Individual Values, John Wiley and Sons, New York, 1963. Economists distinguish ordinal utility (the utility expressed, by an ordering relation) from cardinal utility (the utility measured by a number).Google Scholar
  8. 8.
    K.J. Arrowop. cit.,.Google Scholar
  9. 9.
    This practice is defined as an “obliteration by incorporation”; seeR.K. Merton,op. cit..Google Scholar
  10. 10.
    P.A. Scales, Citation analyses as indicators of the use of serials,Journal of documentation, 32 (1976) 17Google Scholar
  11. 11.
    W.V. Quine,Word and Object, MIT Press, Cambridge, 1960Google Scholar
  12. 12.
    H.G. Small, Cited documents as concept symbols,Social Studies of Science, 8 (1978) 327.Google Scholar
  13. 13.
    E. Garfield,op. cit.. (Garfield speaks of the “semantic stability of citations”); see alsoM.L. Pao, D.B. Worthen, Retrieval effectiveness by semantic and citation searching,Journal of the American Society for Information Science, 40 (1989) 226.Google Scholar
  14. 14.
    Cf. the statement “meaning is use”L. Wittgenstein,Philosophical Investigations (MacMillan, New York, 1953). (In our case the use of citations by the author(s) leads to the meaning of the paper). It is of interest to mention the classical distinction introduced byG. Frege between sense and denotation.Frege uses the term “reference” to describe expressions which have no sense, but simply denote the individuals or the entities called by them. Proper names provide a possible example of such pure referring expressions (seeC. Semenga, M. Zettin, Evidence from aphasia for the role of proper names as pure referring expressionsNature, 342 (1989) 678). That citations carry some sence implies that they do not belong to the category of “references” described byFrege.Google Scholar
  15. 15.
    On the notions of truth, statement, corroboration and refutation seeK. Popper,The Logic of Scientific Discovery, Harper Torchbooks, New York, 1968.Google Scholar
  16. 16.
    That the truth value of a paper is obtained by the removal of the citations can be compared with the notion of disquotation introduced byTarski; cfW.V. Quine,Philosophy of Logic, Prentice Hall, Englewood Cliffs, 1970. According to Frege's distinction we can observe that a citation carries not only sense but also denotation.Google Scholar
  17. 17.
    For a recent review of the different methods of research evaluation seeT. Lukkonen-Gronow, Scientific research evaluation: a review of methods and various contexts of their application,R & D Management, 17 (1987) 207.Google Scholar
  18. 18.
    D. de Solla Price, Networks of scientific papers,Science, 149 (1965) 510, (reprinted inD. de Solla Price, op. cit. p. 103–118).Google Scholar
  19. 19.
    B.R. Martin, J. Irvine, Assessing basic research: some partial indicators of scientific progress in radio astronomy,Research Policy, 12 (1983) 61;M. H. Mac Roberts, B.R. Mac Roberts, Problems of citation analysis: a critical review,Journal of the American Society for Information Science, 40 (1989) 342;S. Cole, Citations and the evaluation of individual scientists,Trends in Biochemistry, 14 (1989) 9.Google Scholar
  20. 20.
    A.G. Holtmann, A.E. Bayer, Determinants of professional income among recent recipients of natural sciences doctorates,Journal of Business, 43 (1970) 410:D.S. Hamermesh, G. E. Johnson, B.A. Weisbrod, Scholarship, citations and salaries: economic rewards in economics,Southern Economic Journal, 49 (1982) 472,A.M. Diamomd, Jr., The money value of citations to singleauthored and multiple-authored articles,Scientometrics, 8 (1985) 315,A.M. Diamond, Jr., What is a citation worth?,Journal of Human Resources, 21 (1986) 210.Google Scholar
  21. 21.
    C. Freeman,The Economics of Industrial Innovation, M.I.T. Press, Cambridge, 1986, p. 27–47.Google Scholar
  22. 22.
    O.H. Lowry, N.J. Rosebrough, A.L. Farr, R.J. Randall, Protein measurement with the Folin phenol reagent,Journal of Biological Chemistry, 193 (1951) 265. For a recent citation analysis of this paper, seeE. Garfield, The most cited papers of all time, SCI 1945–1988,Current Contents, 33 (1990) No. 7, 3.PubMedGoogle Scholar
  23. 23.
    This is a rough estimate for a publication in biomedical research: seeF. Narin, R.P. Rozek, Bibliometric analysis of US pharmaceutical industry research performance,Research Policy, 17 (1988) 139.Google Scholar
  24. 24.
    M.J. Moravcsik, P. Murugesan, Some results on the function and quality of citations,Social Studies of Science, 5 (1975) 86; also discussed byB. Cronin, op. cit.The Citation Process, Taylor Graham, London, 1984.Google Scholar
  25. 25.
    P. Sraffa,Production of Commodities by Means of Commodities. Prelude to a Critique of Economic Theory, Cambridge University Press, Cambridge, 1960.Google Scholar
  26. 26.
    Discussed byJ. Margolis, Citation indexing and the evaluation of scientific papers,Science, 155 (1967) 1213. In this paper, a genetic metaphor is applied to describe the production of scientific papers: references cited in an article are the article's “ancestors”; subsequent papers citing this article are its “descendants”. Selection operates backward because the descendants choose their ancestors. What we consider here as the utility of a scientific paper is defined as its indirect influence (or impact). Theory of graphs is used byMargolis to compute the indirect impact of a scientific article in subsequent generations of scientific papers. This impact is maximal in the first generation of papers citing the considered article: this empirical finding justifies the decision taken here to limit the study of u2 to the first generation of citing papers.Google Scholar
  27. 28.
    A.M. Maxam, W. Gilbert, A new method for sequencing DNA,Proceedings of the National Academy of Sciences of the USA, 74 (1977) 560.Google Scholar
  28. 29.
    A.M. Maxam, W. Gilbert, Sequencing of end-labelled DNA with base-specific chemical cleavate,Methods in Enzymology, 65 (1980) 499.Google Scholar
  29. 30.
    Diachronous studies investigate time changes in citation counts to or from journal aggregates, individual journal or individual journal articles. SeeE.R. Stinson, W.F. Lancaster, Synchronous versus diachronous methods in the measurement of obsolescence by citations studies,Journal of Information Science, 13 (1987) 65;K.W.Mc Cain, K. Turner, Citation context analysis and aging patterns,Scientometrics, 17 (1989) 127.Google Scholar
  30. 31.
    P. Kotler,Marketing Management, Prentice Hall, Englewood Cliffs, 1988, p. 347–354.Google Scholar
  31. 32.
    The selection of a single year for the acquisition of citation counts makes it easier to obtain the data. The citations made to an article peak on average in the third year following its publication (cf.Margolis op. cit.); hence the choice of this third year for our investigation. Alternatively the experiments performed in this article could be made with the total number of citations received in their lifetimes by the citing papers: seeN.L. Geller, J.S. de Cani, R.E. Davies, Lifetime-citation rates to compare scientists' work,Social Science Research, 7 (1978) 345.Google Scholar
  32. 33.
    For a sample of n articles having received ci citations (1≤i≤n), the geometric mean (GM) number is defined as GM=[II(1+ci)]1/n-1. A geometric mean rather than an arithmetic mean was chosen in order to decrease the standard deviation of the means obtained: the data given in Fig. 1b reflect the actual geometric mean the of whole population with a 10% accuracy. Geometric mean performs better than the arithmetic mean because of the approximate lognormal character of the distribution of citation counts. SeeF.A. Cowell,Measuring Inequality, Philip Allan, Oxford, 1977, for an introduction to the mathematics of inequality. The lognormal character of citation count distribution is discussed inThe Evaluation of Scientific Research, D. Evered, S. Harnett (Eds), John Wiley & Sons, Chichester, 1989, p. 107 and ff.Google Scholar
  33. 34.
    M.B. Line, A. Sandison, Obsolescence and change in the use of literature with time,Journal of Documentation, 30 (1974) 283;B.C. Griffith, P. Servi, A. Anker, M.C, Drott, The aging of scientific literature,Journal of Documentation, 35 (1979) 179,V.M. Motylev, The main problems of studying literature aging,Scientometrics, 15 (1989) 97.Google Scholar
  34. 37.
    J. Margolis,op. cit. A particular interest of this experiment is that the Citation Indexes were not available before 1964, ruling out the possibility that these indexes have influenced the scientists.Google Scholar
  35. 38.
    This study is fully synchronous because both the cited papers (published in 1978, cited in 1981) and the citing papers (published in 1981, cited in 1984) are three years old. The study is not however time-independent: different results would be obtained with papers of earlier or more recent publication. As in the case of the diachronous study, the experiments performed in this article could be made with the total number of citations received in their lifetimes by the cited and/or the citing papers (cf. ref. 28)Google Scholar
  36. 39.
    R.K. Mertonop. cit.. p. 303.Google Scholar
  37. 41.
    On the notions of simplicity and falsifiability seeK. Popper,op. cit.. Unity as a criterion for scientific choice is advocated byA.M. Wienberg, Values in science: unity as a criterion of scientific choices,Minerva, 22 (1984) 1.Google Scholar
  38. 43.
    The notions of redundancy and of multiple discoveries are discussed byR.K. Merton,op. cit..Google Scholar
  39. 44.
    Citation analysis is an inappropriate tool to study this social utility. If, for example, the utility of science in technology is to be examined, the analysis of the citations made to scientific papers in patents provides an invaluable approach:M.P. Carpenter, M. Cooper, F. Narin, Linkage between basic research literature and patents,Research Management, 23 (1980) 30. For a different opinion on this question seeJ.R., Cole, S. Cole, The Ortega hypothesis,Science, 178, (1972) 368 (the data discussed in this paper can easily be interpreted in terms of convex utility).Google Scholar

Copyright information

© Akadéemiai Kiadó 1991

Authors and Affiliations

  • J. -L. Sikorav
    • 1
  1. 1.Department of GeneticsHarvard Medical SchoolBoston(USA)

Personalised recommendations