Advertisement

Scientometrics

, Volume 7, Issue 3–6, pp 281–308 | Cite as

Evaluating big science: CERN's past performance and future prospects

  • J. Irvine
  • B. R. Martin
Article

Abstract

After explaining the reasons why science policy-makers face a growing need for more rigorous forms of research evaluation, we outline an approach combining bibliometric and peer-evaluation data that has been developed at the Science Policy Research Unit in the course of a programme of studies of Big Science specialties. The paper describes the results obtained when this ‘method of converging partial indicators’ is applied to compared the past research performance of the accelerators at CERN — the joint European Laboratory for Particle Physics — with that of the world's other main accelerators. The paper concludes by demonstrating how, on the basis of an analysis of the factors that have structured research performance in the past, it is possible to arrive at a systematic set of conclusions about the future prospects for a major new research facility such as an accelerator.

Keywords

Research Performance Past Research Particle Physics Policy Research Future Prospect 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Notes and references

  1. 1.
    D. de SOLLA PRICE, Philosophical Mechanism and Mechanical Philosophy: Some Notes Towards a Philosophy of Scientific Instruments,Annali del'Institutio e Museo di Storia della Scienza di Firenze, V (1980) 75–85. D. de SOLLA PRICE, Selaing Wax and String: A Philosophy of the Experimenter's Craft and its Role in the Genesis of High Technology, Sarton Lecture at the AAAS, May 1983. D. de SOLLA PRICE, The Science/Technology Relationship; The Craft of Experimental Science, and Policy for the Improvement of High Technology Innovation,Research Policy 13 (1984) 3–20.Google Scholar
  2. 2.
    See D. O. EDGE, M. J. MULKAY,Astronomy Transformed, Wiley Interscience, New York, 1976.Google Scholar
  3. 3.
    See A. L. ROBINSON, CERN Vector Boson Hunt Successful,Science, 221 (26 August 1983) 840–42.Google Scholar
  4. 4.
    For evidence on the exponential growth of science, see e.g., N. RESCHER,Scientific Progress, Blackwell, Oxford, 1978.Google Scholar
  5. 5.
    cf. J. IRVINE, B. R. MARTIN, C. H. G. OLDHAM,Research Evaluation in British Science: A SPRU Review, a report prepared for the French Ministry of Research and Industry by the Science Policy Research Unit, University of Sussex, Brighton BN1 9RF, U. K.Google Scholar
  6. 6.
    The programme of research began in 1978 with a two-year study focusing on five British Big Science laboratories working in the areas of radio astronomy (see B. R. MARTIN and J. IRVINE, Assessing Basic Research: Some Partial Indicators of Scientific Progress in Radio Astronomy,Research Policy, 12 (1983) 61–90) optical astronomy (see J. IRVINE and B. R. MARTIN, Assessing Basic Research: The Case of the Isaac Newton Telescope,Social Studies of Science, 13 (1983) 49–86), and electron high-energy physics (see B. R. MARTIN and J. IRVINE, Internal Criteria for Scientific Choice: An Evaluation of Research in High-Energy Physics Using Electron Accelerators,Minerva, XIX (1981) 408–32). This was followed in 1981 by an evaluation of applied research (mechanical engineering and electronics) for a Norwegian Royal Commission (see M. SCHWARZ, J. IRVINE and B. R. MARTIN, with K. PAVITT and R. ROTHWELL, Government Support for Industrial Research: Lessons from a Study of Norway,R&D Management, 12 (1982) 155–67). The CERN project describeb here was carried out over a 15-month period between 1981 and 1982, this paper constituting a summary of the three articles listed in references 19, 20 and 27, below. Since then, the authors have undertaken evaluations of: (1) the Norwegian Institute for Energy Technology; (2). the steel-research programme of the European Coal and Steel Community (ECSC); the mechanisms used by policy-makers in Britain for evaluating past research performance (see reference 5, above); and (4) Britain's international standing in the fields of ocean currents and protein crystallography. We should like to record the fact that the late Professor de SOLLA PRICE played a central role in helping establish our work as a legitimate area of research, and we shall always be indebted to him for his crucial early interest and enthusiasm.Google Scholar
  7. 7.
    See also J. IRVINE, B. R. MARTIN, What Direction for Basic Scientific Research? in: M. GIBBONS, P. GUMMET, B. M. UDGAONKAR (Eds),Science and Technology Policy in the 1980s and Beyond, Longman, Harlow, 1984, 67–98.Google Scholar
  8. 8.
  9. 9.
    For an analysis of the very limited extent to which output indicators are currently used by science-policy makers in Britain, see J. IRVINE, B. R. MARTIN, C. H. G. OLDHAM, op. cit., note 5. J. IRVINE, B. R. MARTIN, C. H. G. OLDHAM,Research Evaluation in British Science: A SPRU Review, a report prepared for the French Miistry of Research and Industry by the Science Policy Research Unit, University of Sussex, Brighton BN1 9RF, U. K.Google Scholar
  10. 10.
    For example, the situation in Britain is now such that, within the Science and Engineering Research Council (the organization responsible for funding research in the natural sciences), one subsection of physics — nuclear physics — has its own Board, as does astronomy and space research, while all other areas of science — physics, chemistry, biology, mathematics and computing — have only one Board betwen them. The result is that Big Science consumes almost double the resources of all the remaining natural sciences together, whilst constituting only a small proportion of total scientific activity, at least in terms of numbers of researchers (see J. IRVINE, B. R. MARTIN,, for further discussion).Google Scholar
  11. 13.
    See, for example, H. COLLINS, Scientific Knowledge and Science Policy: Some Foreseeable Implications, paper presented at Council for Science and Society Conference on Growing Points in Science Studies, Imperial College, 25 June 1983.Google Scholar
  12. 14.
    The methodology is described in detail in B. R. MARTIN and J. IRVINE (1983).Google Scholar
  13. 15.
    J. IRVINE, B. R. MARTIN, The Economic Effects of Big Science: The Case of Radio Astronomy,Proceedings of the International Colloquium on Economic Effects of Space and Other Advanced Technologies, Strasbourg 28–30 April 1980, European Space Agency, Paris (Ref. ESA SP-151, September 1980).Google Scholar
  14. 16.
    See B. R. MARTIN, J. IRVINE (1983), for a fuller discussion of what is meant by the term ‘partial indicator’.Google Scholar
  15. 17.
    The distinction between the ‘quality’, ‘importance’, and ‘impact’ of a publication is discussed further in 69–70.Google Scholar
  16. 19.
    See B. R. MARTIN, J. IRVINE, CERN: Past Performance and Future Prospects — I — CERN's Position in World High-Energy Physics,Research Policy, 13 (1984) 183–210.Google Scholar
  17. 20.
    See J. IRVINE, B. R. MARTIN, CERN: Past Performance and Future Prospects — II — The Scientific Performance of the CERN Accelerators,Research Policy, 13 (1984) 247–84.Google Scholar
  18. 21.
    CERN Annual Report 1972, CERN, Geneva, 1972, pp. 11 and 26.Google Scholar
  19. 22.
    Of the 182 physicists, 169 were able to carry out this ranking. Further details of the interviews are given in J. IRVINE, B. R. MARTIN, 247–84.CrossRefGoogle Scholar
  20. 23.
    The degree of consistency is analyzed in detail in B. R. MARTIN, J. IRVINE,.CrossRefGoogle Scholar
  21. 24.
    See J. IRVINE and B. R. MARTIN, Basic Research in the East and West: A Comparison of the Scientific Performance of High-Energy Physics Accelerators,Social Studies of Science, (forthcoming).Google Scholar
  22. 25.
    See note 6.,Research Policy 12 (1983) 61–90.Google Scholar
  23. 27.
    B. R. MARTIN and J. IRVINE, CERN: Past Performance and Future Prospects — III — CERN and the Future of World High-Energy Physics,Research Policy, 13 (1984) 311–42.CrossRefGoogle Scholar

Copyright information

© Akadémiai Kiadó 1985

Authors and Affiliations

  • J. Irvine
    • 1
  • B. R. Martin
    • 1
  1. 1.Science Policy Research UnitUniversity of SussexBrighton(UK)

Personalised recommendations