Scientometrics

, Volume 100, Issue 2, pp 483–496

How expensive is Big Science? Consequences of using simple publication counts in performance assessment of large scientific facilities

Article

Abstract

Although the nuclear era and the Cold War superpower competition have long since passed, governments are still investing in Big Science, although these large facilities are nowadays mostly geared towards areas of use closer to utility. Investments in Big Science are also motivated not only by promises of scientific breakthroughs but also by expectations (and demands) of measurable impact, and with an emerging global market of competing user-oriented Big Science facilities, quantitative measures of productivity and quality have become mainstream. Among these are rather simple and one-sided publication counts. This article uses publication counts and figures of expenditure for three cases that are disparate but all represent the state-of-the-art of Big Science of their times, discussing at depth the problems of using simple publication counts as a measure of performance in science. Showing, quite trivially, that Big Science is very expensive, the article also shows the absurd consequences of consistently using simple publication counts to display productivity and quality of Big Science, and concludes that such measures should be deemed irrelevant for analyses on the level of organizations in science and replaced by qualitative assessment of the content of the science produced.

Keywords

Big Science Publication counts Quality assessment Performance assessment Productivity 

References

  1. Agrell, W. (2012). Framing prospects and risk in the public promotion of ESS-Scandinavia. Science and Public Policy, 39, 429–438.CrossRefGoogle Scholar
  2. Bodnarczuk, M., & Hoddeson, L. (2008). Megascience in particle physics: The birth of an experiment string at fermilab. Historical Studies in the Natural Sciences, 38(4), 508–534.CrossRefGoogle Scholar
  3. Eisler, M. (2013). The Ennobling Unity of Science and Technology: Materials Sciences and Engineering, the Department of Energy, and the Nanotechnology Enigma. Minerva, 51, 225–251.CrossRefGoogle Scholar
  4. Elzinga, A. (2012). Features of the current science policy regime: Viewed in historical perspective. Science and Public Policy, 39, 416–428.CrossRefGoogle Scholar
  5. Giudice, G. F. (2012). Big Science and the Large Hadron Collider. Physics in Perspective, 14, 95–112.CrossRefGoogle Scholar
  6. Guston, D. H. (1999). Stabilizing the boundary between US politics and science: The role of the office of technology transfer as a boundary organization. Social Studies of Science, 29(1), 87–111.CrossRefGoogle Scholar
  7. Hallonsten, O. (2013a). Introducing facilitymetrics: A first review and analysis of commonly used measures of scientific leadership among synchrotron radiation facilities worldwide. Scientometrics, 96(2), 497–513.CrossRefGoogle Scholar
  8. Hallonsten, O. (2013b). Myths and realities of the ESS project: A systematic scrutiny of readily accepted truths. In T. Kaiserfeld & T. O’Dell (Eds.), Legitimizing ESS: Big Science as a collaboration across boundaries. Lund: Nordic Academic Press.Google Scholar
  9. Hallonsten, O. (2014). The politics of European collaboration in big science. In M. Mayer, M. Carpes, & R. Knoblich (Eds.), International relations and the global politics of science and technology. Dordrecht: Springer.Google Scholar
  10. Hallonsten, O. (submitted). The parasites: Synchrotron radiation at SLAC, 1972–1992.Google Scholar
  11. Hallonsten, O., & Heinze, T. (2012). Institutional persistence through gradual adaptation: Analysis of national laboratories in the USA and Germany. Science and Public Policy, 39, 450–463.CrossRefGoogle Scholar
  12. Hallonsten, O., & Heinze, T. (2013). From particle physics to photon science: Multidimensional and multilevel renewal at DESY and SLAC. Science and Public Policy, 40, 591–603.CrossRefGoogle Scholar
  13. Hallonsten, O. & Heinze, T. (submitted). Formation and expansion of a new organizational field in experimental science: Synchrotron Radiation Labs in Europe and the United States, 1974–2012.Google Scholar
  14. Hazelkorn, E. (2011). Rankings and the reshaping of higher education. The battle for world-class excellence. Hampshire: Palgrave Macmillan.CrossRefGoogle Scholar
  15. Irvine, J., & Martin, B. (1984). CERN: Past performance and future prospects. I. CERN’s position in world high-energy physics. Research Policy, 13(4), 183–210.CrossRefGoogle Scholar
  16. Jacob, M., & Hallonsten, O. (2012). The persistence of big science and megascience in research and innovation policy. Science and Public Policy, 39, 411–415.CrossRefGoogle Scholar
  17. Johnson, A. (2004). The end of pure science: science policy from Bayh–Dole to the NNI. In D. Baird, A. Nordmann, & J. Schummer (Eds.), Discovering the nanoscale. Amsterdam: IOS Press.Google Scholar
  18. Kevles, D. J. (1997). Big Science and big politics in the United States: Reflections on the death of the SSC and the life of the Human Genome Project. Historical Studies in the Physical and Biological Sciences, 27(2), 269–297.CrossRefGoogle Scholar
  19. Krige, J. (2003). The politics of european scientific collaboration. In J. Krige & D. Pestre (Eds.), Companion to science in the twentieth century. London: Routledge.Google Scholar
  20. Krige, J. (2006). American hegemony and the postwar reconstruction of science in Europe. Cambridge: MIT Press.Google Scholar
  21. Lohrmann, E., & Söding, P. (2009). Von schnellen Teilchen und hellem Licht: 50 Jahre Deutsches Elektronen-Synchrotron DESY. New York: Wiley.Google Scholar
  22. McCray, W. P. (2000). Large telescopes and the moral economy of recent astronomy. Social Studies of Science, 30(5), 685–711.CrossRefGoogle Scholar
  23. Pestre, D., & Krige, J. (1992). Some thoughts on the early history of CERN. In P. Galison & B. Hevly (Eds.), Big Science—The growth of large-scale research. Stanford: Stanford University Press.Google Scholar
  24. Westfall, C. (2010). Surviving to tell the tale: Argonne’s intense pulsed neutron source from an ecosystem perspective. Historical Studies in the Natural Sciences, 40(3), 350–398.CrossRefGoogle Scholar
  25. Westfall, C. (2012). Institutional persistence and the material transformation of the US National Labs: The curious story of the advent of the advanced photon source. Science and Public Policy, 39, 439–449.CrossRefGoogle Scholar
  26. Wildavsky, B. (2010). The Great Brain race: How global universities are reshaping the world. Princeton: Princeton University Press.Google Scholar
  27. Woods, H. R. (2006). New life for a linac. Symmetry, 3(7), 10–15.Google Scholar

Copyright information

© Akadémiai Kiadó, Budapest, Hungary 2014

Authors and Affiliations

  1. 1.Department for Philosophy, Linguistics and Theory of ScienceUniversity of GothenburgGothenburgSweden

Personalised recommendations