Skip to main content
SpringerLink
Log in
Book cover

Incentives and Performance pp 223–239Cite as

Myths, Challenges, Risks and Opportunities in Evaluating and Supporting Scientific Research

  • Chapter
  • First Online:

Abstract

We summarize and discuss critically the various procedures used for decisions in evaluating and funding scientific research in view of the goal of obtaining the best research with a given funding volume. Merits, difficulties, and problems in some procedures are outlined. We identify a number of myths appearing frequently in the process of research evaluation. We indicate some similarities in deciding on funding research projects, making appointments to academic and leading research positions and selecting prize winners for high prizes for scientists. Challenges, risks and opportunities in research and its funding are identified.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Albert S, Meier BH, Quack M, Seyfang G, Trabesinger A (2006) On the possibility of stable nuclear spin symmetry isomers in H2O. Chimia 60:476

    Google Scholar 

  • Ausländer R (2002) Und nenne dich Glück. Gedichte. Fischer, Frankfurt

    Google Scholar 

  • Buchstein H (2011) Der Zufall in der Forschungsförderungspolitik. Swifts “Akademie von Lagado”. Forschung und Lehre 8:596–597 (cites Les Allen after Neil Duxbury, Random Justice, Oxford 1999, p 89)

    Google Scholar 

  • Diederich F (2013) Are we refereeing ourselves to death? The peer-review system at its limit. Angew Chem Int Ed 52:13828–13829

    Article  Google Scholar 

  • Einstein A (1931) Zum kosmologischen Problem der allgemeinen Relativitätstheorie. Sitzber Preuss Akad Wiss Phys-Math Kl:235–237

    Google Scholar 

  • Einstein A, Preuss SB (1931) Akad Wiss:235 (This is the very common false version of the citation)

    Google Scholar 

  • Ernst RR (1992) Nuclear Magnetic Resonance Fourier Transform Spectroscopy (Nobel Lecture). Angew Chem Int Ed Engl 31:805–823; Angew Chem (1992) 104:817–836

    Google Scholar 

  • Ernst RR (2010a) The follies of citation indices and academic ranking lists, a brief commentary to ‘bibliometrics as weapons of mass citation’. Chimia 64:90–90

    Article  Google Scholar 

  • Ernst RR (2010b) The follies of citation indices and academic ranking lists, a brief commentary to ‘bibliometrics as weapons of mass citation’. Bunsen-Magazin 5:199–200

    Google Scholar 

  • Ernst RR, Bodenhausen G, Wokaun A (1987) Principles of nuclear magnetic resonance in one and two dimensions. Clarendon, Oxford

    Google Scholar 

  • Gerhards J (2013) Der Deutsche Sonderweg in der Messung von Forschungsleistungen. In: Wissenschaftspolitik im Dialog, Schriftenreihe der Berlin-Brandenburgischen Akademie der Wissenschaften, Berlin

    Google Scholar 

  • Heilbron JL (1989) The detection of the antiproton In: De Maria M, Grilli M, Sebastiani F (eds) Proceedings of the international conference on the restructuring of physical sciences in Europe and the United States 1945–1960 (Rome, 1988). World Scientific, Singapore, pp 161–209

    Google Scholar 

  • Hirsch JE (2005) An index to quantify an individual’s scientific research output. Proc Natl Acad Sci USA 102:16569–16572

    Article  Google Scholar 

  • Kneißl D, and Schwarz H (2011) Editorial: Grundlagenforschung braucht exzellente Wissenschaftler – und Freiräume. Angew Chem 123:12578–12579

    Google Scholar 

  • Kuhn TS (1962) The structure of scientific revolutions. University of Chicago Press, Chicago

    Google Scholar 

  • Manca Tanner C, Quack M, Schmidiger D (2013) Nuclear spin symmetry conservation and relaxation in water (1H2 16O) studied by Cavity Ring-Down (CRD) spectroscopy of supersonic jets. J Phys Chem A 117:10105–10118

    Article  Google Scholar 

  • Merkt F, Quack M (2011) Molecular quantum mechanics and molecular spectra, molecular symmetry, and interaction of matter with radiation. In: Quack M, Merkt F (eds) Handbook of high-resolution spectroscopy. Wiley, Chichester; New York, pp 1–55 (see also preface to this Handbook)

    Google Scholar 

  • Molinié A, Bodenhausen G (2010a) Bibliometrics as weapons of mass citation. Chimia 64:78–89

    Article  Google Scholar 

  • Molinié A, Bodenhausen G (2010b) Bibliometrics as weapons of mass citation. Bunsen-Magazin 5:188–198

    Google Scholar 

  • Molinié A, Bodenhausen G (2011) The kinship or k-index as an antidote against the toxic effects of h-indices. Chimia 65(6):433–436

    Article  Google Scholar 

  • Moore WJ (1989) Schrödinger, life and thought. Cambridge University Press, Cambridge

    Google Scholar 

  • Osterloh M (2013) Das Paradox der Leistungsmessung und die Nachhaltigkeit der Forschung. Nova Acta Leopoldina NF 117:103–113

    Google Scholar 

  • Osterloh M, and Frey BS (2012) Input control and random choice improving the selection process for journal articles (Manuscript preprint); Ranking Games, to appear in Evaluation Review (2014)

    Google Scholar 

  • Parkinson CN (1957) Parkinson’s law and other studies in administration. The Riverside Press, Cambridge, MA

    Google Scholar 

  • Perutz MF (1982) Ging’s ohne Forschung besser? - Der Einfluss der Naturwissenschaften auf die Gesellschaft. Wissenschaftliche Verlagsgesellschaft mbH, Stuttgart

    Google Scholar 

  • Petsko GA (2008) Having an impact (factor). Genome Biol 9, 107

    Article  Google Scholar 

  • Pfaltz A, Van Gunsteren WF, Quack M, Thiel W, and Wiersma DA (2009) An investigation with respect to the possible fabrication of research data reported in the thesis ETH No 13629 and in the papers Journal of Chemical Physics (2000) 112:2575 and (2000) 113:561, July 2009, ETH. Available: http://www.ethlife.ethz.ch/archive_articles/120123_Expertenbericht_tl/120123_Expertenbericht

  • Popper K (1989) Popper on Schrödinger. Nature 342:337–337

    Article  Google Scholar 

  • Quack M (2004) Time and time reversal symmetry in quantum chemical kinetics. In: Brändas EJ, Kryachko ES (eds) Fundamental world of quantum chemistry a tribute to the memory of Per-Olov Löwdin. Kluwer, Dordrecht, pp 423–474

    Chapter  Google Scholar 

  • Quack M (2011a) Fundamental symmetries and symmetry violations from high resolution spectroscopy. In: Quack M, Merkt F (eds) Handbook of high resolution spectroscopy. Wiley, Chichester; New York, pp 659–722

    Chapter  Google Scholar 

  • Quack M (2011b) Naturwissenschaften! Warum überhaupt? Warum nicht? In: Gegenworte, Heft 26, “Zweckfreie Forschung”. Berlin-Brandenburgische Akademie der Wissenschaften, pp 28–35

    Google Scholar 

  • Quack M (2011c) Naturwissenschaften! Warum überhaupt? Warum nicht? Bulletin der Vereinigung der Schweizerischen Hochschuldozierenden VSH/AEU 37:7–14

    Google Scholar 

  • Quack M (2011d) Rede des Ersten Vorsitzenden anlässlich der 110. Hauptversammlung der Deutschen Bunsen-Gesellschaft für Physikalische Chemie in Berlin, 2. Juni 2011 (printed version of the speech). Bunsen-Magazin 13:138–143

    Google Scholar 

  • Quack M (2012a) Rede des Ersten Vorsitzenden anlässlich der 111. Hauptversammlung der Deutschen Bunsen-Gesellschaft für Physikalische Chemie in Leipzig, 17. Mai 2012 (im Druck publizierte Fassung). Bunsen-Magazin 14:181–189

    Google Scholar 

  • Quack M (2012b) Risiken und Chancen in der Forschungsförderung (Auszug aus der Rede des 1. Vorsitzenden der Deutschen Bunsen-Gesellschaft für Physikalische Chemie, Leipzig Mai 2012). Deutsche Zahnärztliche Zeitung 67:726–730

    Google Scholar 

  • Quack M (2013) Error and discovery: why repeating can be new. Angew Chem Int Ed 52:9362–9370; Angew Chem (2013) 125:9530–9538

    Google Scholar 

  • Quack M (2014) The concept of law and models in chemistry. Eur Rev 22:S50–S86

    Article  Google Scholar 

  • Quack M, Merkt F (eds) (2011) Handbook of high resolution spectroscopy. Wiley, Chichester; New York

    Google Scholar 

  • Schrödinger E (1926a) Der stetige Übergang von der Mikro- zur Makromechanik. Naturwissenschaften 14:664–666

    Article  Google Scholar 

  • Schrödinger E (1926b) Quantisierung als Eigenwertproblem I. Ann d Phys 79:361–376

    Article  Google Scholar 

  • Schrödinger E (1926c) Quantisierung als Eigenwertproblem II. Ann d Phys 79:489–527

    Article  Google Scholar 

  • Schrödinger E (1926d) Quantisierung als Eigenwertproblem III. Ann d Phys 80:437–490

    Article  Google Scholar 

  • Schrödinger E (1926e) Quantisierung als Eigenwertproblem IV. Ann d Phys 81:109–139

    Article  Google Scholar 

  • Straumann N (2008) Materie, Antimaterie, und Dunkle Energie. In: Walde P, and Kraus F (eds) An den Grenzen des Wissens. Vdf Publishers, Zürich, pp 103–126 [Here it is shown that a paper of Einstein A (1931) (see citation) has been transformed by citations (obviously from people who never read or even saw the paper) until it reached its ‘canonical’ form with a second coauthor Preuss SB (Einstein A and Preuss SB 1931)].

    Google Scholar 

  • Suhm MA (2010) Fahren mit angezogener Handbremse. Bunsen-Magazin 12:200

    Google Scholar 

  • Szilárd L (1961) The Mark Gable Foundation. In: The voice of the dolphins, and other stories. Simon & Schuster, New York

    Google Scholar 

  • Tikhonov VI, Volkov AA (2002) Separation of water into its ortho and para isomers. Science 296:2363–2363

    Article  Google Scholar 

  • Trommsdorff V (2002). Gesteinslaboratorium Alpen (Abschiedsvorlesung ETH Zürich). Bildarchiv der ETH-Bibliothek [Prod.]

    Google Scholar 

  • Weinberg S (1967) A model of leptons. Phys Rev Lett 19:1264–1266

    Article  Google Scholar 

  • Zare RN (2012a) Editorial: assessing academic researchers. Angew Chem Int Ed 51:7338–7339

    Article  Google Scholar 

  • Zare RN (2012b) Research impact. Curr Sci 102:9

    Google Scholar 

Download references

Acknowledgment

Our own scientific research receives support from the ETH Zurich, Schweizerischer Nationalfonds and by an Advanced Grant from the European Research Council. I am grateful to the hospitality of the Schmid family in their house in Arosa, where the present manuscript was written during a holiday and I am greatly indebted to Ruth Schüpbach for transferring my handwritten notes into a clean manuscript. The paper is dedicated to Richard Ernst on the occasion of his 80th anniversary.

Author information

Authors and Affiliations

  1. Physical Chemistry, ETH Zürich, CH-8093, Zürich, Switzerland

    Martin Quack

Authors
  1. Martin Quack
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin Quack .

Editor information

Editors and Affiliations

  1. TUM School of Management, Technische Universität München, Munich, Germany

    Isabell M. Welpe

  2. TUM School of Management, Technische Universität München, Munich, Germany

    Jutta Wollersheim

  3. TUM School of Management, Technische Universität München, Munich, Germany

    Stefanie Ringelhan

  4. Zeppelin University, Friedrichshafen, Germany, and University of Zurich, Zurich, Switzerland

    Margit Osterloh

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Quack, M. (2015). Myths, Challenges, Risks and Opportunities in Evaluating and Supporting Scientific Research. In: Welpe, I., Wollersheim, J., Ringelhan, S., Osterloh, M. (eds) Incentives and Performance. Springer, Cham. https://doi.org/10.1007/978-3-319-09785-5_14

Download citation

Publish with us

Policies and ethics

Search

Navigation