This is a preview of subscription content, log in via an institution to check access.
Notes
This section draws heavily on my paper of 1997, with minor updating only.
Given the role of peer review in warranting scientific knowledge claims, it is surprising there is not more science studies literature on it. The notable recent exception is Lamont (2010). In recent years, medical professionals have begun to address peer review reliability, paying substantial attention to the potentially biasing role of corporate sponsors, financial conflicts of interest and media attention, and to the problem of ghost-authorship. See for example, Flanagin et al. (1998). Richard Smith, a former editor of British Medical Journal, notes (2006) with irony that the basis of scientific vetting is a system that has not been well studied scientifically. For an earlier but still pertinent attempt to evaluate peer review scientifically, see Cole et al. (1981).
On the problem of claims that do not hold up under further scrutiny, see: Jonah Lehrer, The truth wears off. The New Yorker, 13 December 2010. On gender bias in peer review, see Wenneras and Wold (1997), Bornmann et al. (2007) or Abrevaya and Hamermesh (2010). While many studies do show some effect of gender bias, others suggest a stronger effect caused by prestige bias—grants and papers by authors who have published extensively before, or are affiliated with prestigious institutions, are more likely to receive positive reviews. See: Peters and Ceci (1983); Fisher et al. (1994). Other studies, however, showed no benefit to blinding or unmasking in the peer review process (e.g. Van et al. 1998), perhaps because blinding is less effective than one would imagine because reviewers are able to identify authors through their knowledge of the field. On the other hand, the role of bias may itself be exaggerated because there is bias against ‘no effect’ results. Cole et al. (1981) attribute much of the variability in peer review outcomes to chance.
The evidence that peer review may be deliberately undermined—and not just by Merchants of Doubt but in diverse ways—raises the question of whether this mechanism has out-lived its efficacy. See Flanagin et al. (1998), Rennie (1986, 1999) and Healy (2000, 2004). Some have suggested the time has come to replace traditional peer review with open access web-based discussions on the internet; Smith (2006) argues while this may not be any more reliable it would at least be more thought-provoking. It is striking that most of the literature on problems in peer review addresses bio-medicine; more work is needed to know whether the problems addressed by Smith, Healy, Rennie, and others are general to peer review or specific to the demands, pressures, regulatory framework and financial inducements of bio-medicine. The issue of ghost-written papers, for example, has not, to my knowledge, come up in academic geology (although one might imagine a situation in which the CATO Institute would ghost-write an article on climate change) perhaps because of the absence of a regulatory framework creating large inducements.
For a discussion of whether this should be the case, see Cartwright (2007).
A science studies scholar might note a problem in the whole notion of reviewing peer review: if we don’t know whether or not a scientific claim is correct, how can we judge whether reviewers were correct in accepting or rejecting it? We can judge fairness—e.g. whether papers submitted by male and female authors are treated equitably—and consistency—e.g. whether the same paper submitted and re-submitted, or submitted under a different author and title—receives consistent treatment. Thus most studies that examine peer review attempt to judge fairness, equity, or consistency, but not the ultimate ‘correctness’ of the decisions being made.
It seems to us an open question whether—or in what circumstances—extended peer review produces an epistemically (as opposed to politically) superior outcome; we see this as a fertile area for future research. On extended peer review, see Hisschemöller et al. (2001), Funtowicz and Ravetz (1993) and Pereira and Funtowicz (2005).
The potentially anti-democratic aspects of expertise has been raised especially by Jasanoff 1990, 2005 and 2010); cf. also Novotny et al. (2001) and Lentsch and Weingart (2011). While we recognise the potential for scientific expertise to operate in undemocratic ways, our work clearly outlines an equally, if not more distressing, pattern in which attacks on scientific expertise can undermine democracy by undermining the basis for informed decision-making. The question is not, we believe, whether scientific expertise is intrinsically democratic or anti-democratic, but under what conditions scientific expertise best serves democratic governance. Our views thus overlap in some but not all respects with Collins and Evans (2002), who called for a new approach in science studies more directly engaged in evaluating expertise. “Wave Three” of science studies, they suggested, should re-establish the distinction between experts and lay persons, while acknowledging the continuity between the wider scientific community and the public in all but specialists’ areas. This paper provoked a fairly negative response even from scholars whose own work might be viewed as engaged with such questions, presumably because distinctions often imply inequities, and these latter scholars are loathe to re-inscribe the superiority of scientific expertise as a source of cultural authority (cf. Jasanoff 2003; Wynne 2003). The latter position, in our view, makes the mistake of conflating equality with identity, or rather, difference with inequality (cf. also Epstein 1996 and Zammito 2004).
The exception to the rule involves nuclear winter. In 1984, when first debated, there was no scientific consensus on its reality or severity. However, we argue that the scientific process worked as it should during the next decade: research was done, claims were narrowed, and a consensus emerged that the problem was real, but probably not as severe as originally suggested (hence, ‘nuclear autumn’). Yet, the contrarians in our story did not contribute to this scientific process; they tried to undermine it. Moreover, the story supports a historical point that chronology matters: it is crucial to pursue diverse viewpoints in the early stages of research and debate, and not to shut down outlier voices prematurely, less potentially fertile lines of inquiry be missed. See, for example, Solomon (2001). However, we would argue that it becomes less valuable after decades of research (assuming that the early research was truly open), and at some point it becomes repetitive, uninformative, and potentially a waste of resources. Once a consensus has stabilised, new information is still important, of course, but re-hashing of old debates is rarely productive and more often a waste of time and printer ink. An important point in the debate over continental drift was that it was productively re-opened when scientists had new evidence to offer (cf. Oreskes 1999 and 2002).
Some readers will object that there are many cases where expert opinion was wrong. We would suggest that in many, perhaps most, of those examples, the scientific community was divided; there was not an expert consensus. Consider two cases we have studied carefully. In the 1920s, American geologists had a consensus that continental drift was not supported, but Europeans were not so sure (cf. Oreskes 1999). In the early 1990s, physical oceanographers insisted on the safety of long-range low frequency acoustic transmissions to detect climate change, but cetacean biologists disagreed (cf. Oreskes 2004). In both cases there were significant divides within the scientific community, or between different communities with relevant expertise. And it both cases debate (rightly) continued. Hence a key component of doubt-mongering campaigns is to deny consensus and insist that the debate continues. For if that were true, then it would be appropriate to defer judgment and resist premature closure. Their argument was not illogical; it was unfounded.
For explicit versions of this argument, particularly in defence of feminist epistemologies, see Longino (1990) and Solomon (2001). These feminist critiques raise the important questions of the limits of social empiricism—is science strengthened by letting 1000 flowers bloom, or only ones cultivated in certain ways? For a critique of Solomon on this question, see Oreskes (2008).
Some scholars involved in defending climate science have noted this—for example Washington and Cook (2011). And some scholars critiquing climate science have drawn on science-studies approaches (e.g. Hulme 2009; and van der Sluijs et al. 2010). Hulme’s claim that climate science has become ‘hegemonic’ seems hard to understand given the failure of climate scientists to effect policies to prevent “dangerous anthropogenic interference”, the specific goal their work laid out in the UNFCCC. Perhaps he means hegemonic within the expert community—but if so, then that is no more than to say that the reality of AGW is now the scientific paradigm, akin to plate tectonics or relativity—i.e. a successful scientific theory.
References
Abrevaya, J., and D.S. Hamermesh. 2010. Charity and favoritism in the field: are female economists nicer (to each other)? National Bureau of Economic Research, Working Paper No. 15972.
Alexander, L.V., and J.M. Arblaster. 2009. Assessing trends in observed and modelled climate extremes over Australia in relation to future projections. International Journal of Climatology 29: 417–435.
Ashmore, M. 1996. Ending up on the wrong side: Must the two forms of radicalism always be at war? Social Studies of Science 26: 305–322.
Balanyá, J., J.M. Oller, R.B. Huey, G.W. Gilchrist, and L. Serra. 2006. Global genetic change tracks global climate warming in Drosophila subobscura. Science 313: 1773–1775.
Bamber, J.L., R.E.M. Riva, B.L.A. Vermeersen, and A.M. LeBrocq. 2009. Reassessment of the potential sea-level rise from a collapse of the west Antarctic ice sheet. Science 324: 901–903.
Beck, U. 1992. Risk society: Towards a new modernity. London: Sage.
Bornmann, L., R. Mutz, and H.-D. Daniel. 2007. Gender difference in grant peer review: A meta-analysis. Journal of Informetrics 1: 226–228.
Canadell, J.G. et al. 2007. Contributions to accelerating atmospheric CO2 growth from economic activity, carbon intensity, and efficiency of natural sinks. Proceedings of the National Academy of Sciences 104: 18,866–18,870.
Cartwright, N. 2007. Are RCTs the gold standard? Biosocieties 2: 11–20.
CCSP. 2008. Weather and climate extremes in a changing climate. Regions of focus: North America, Hawaii, Caribbean, and U.S. Pacific Islands. A report by the U. S. climate change science program. Department of Commerce, NOAA’s National Climatic Data Center, Washington, DC, USA.
Church, J.A., and N.J. White. 2006. A 20th-century acceleration in global sea-level rise. Geophysical Research Letters 33: L01602.
Cole, S., J.R. Cole, and G.A. Simon. 1981. Chance and consensus in peer review. Science 214: 881–886.
Collins, H.M., and R. Evans. 2002. The third wave of science studies: Studies of expertise and experience. Social Studies of Science 32: 235–296.
Daston, L., and P. Galison. 2010. Objectivity. New York: Zone Books.
Dessler, A.E., et al. 2008. Water-vapor climate feedback inferred from climate fluctuations, 2003–2008. Geophysical Research Letters 35: L20704.
Easterling, D.R., and M.F. Wehner. 2009. Is the climate warming or cooling? Geophysical Research Letters 36: L08706.
Edmond, G., and D. Mercer. 2004. Daubert and the exclusionary ethos: The convergence of corporate and judicial attitudes towards the admissibility of expert evidence in tort litigation. Law and Policy 26: 231–257.
Epstein, S. 1996. Impure science: AIDS, activism and the politics of knowledge. California: The University of California Press.
Ezrahi, Y. 2004. Science and political imagination in contemporary democracies. In States of knowledge: The co-production of science and social order, ed. S. Jasanoff, 254–273. London: Routledge.
Fisher, M., S.B. Friedman, and B. Strauss. 1994. The effects of blinding on acceptance of research papers by peer review. Journal of the American Medical Association 272: 143–146.
Flanagin, A., et al. 1998. Prevalence of articles with honorary authors and ghost authors in peer-reviewed medical journals. Journal of the American Medical Association 280: 222–224.
Foster, K.R., and P.W. Huber. 1997. Judging Science: Scientific Knowledge and the Federal Courts. Cambridge: The MIT Press.
Funtowicz, S.O., and J.R. Ravetz. 1993. Science for the post-normal age. Futures 25: 739–755.
Furedi, F. 2009. Energizing the debate about climate change. Spiked, 20 March.
Gillett, N.P., and P.A. Stott. 2009. Attribution of anthropogenic influence on seasonal sea level pressure. Geophysical Research Letters 36: L23709.
Glassner, B. 1999. The culture of fear: Why Americans fear the wrong things. New York: Basic Books.
Healy, D. 2000. Good science or good business? Hastings Center Report 30: 19–23.
Healy, D. 2004. Let them eat Prozac. New York: New York University Press.
Hisschemöller, M., T. Hoppe, P. Groenewegen, and C.J.H. Midden. 2001. Knowledge use and political choice in Dutch environmental policy: A problem structuring perspective on real life experiments in extended peer review. In Knowledge, power, and participation in environmental policy, eds. Hisschememöller, M., R. Hoope, W.N. Dunn, and J.R. Ravetz, vol 12, 437–452. Policy Studies Review Annual.
Hobsbawm, E. 1994. Age of extremes: The short twentieth century 1914–1991. New York: Michael Joseph.
Hulme, M. 2009. Why we disagree about climate change: Understanding controversy, inaction and opportunity. Cambridge: Cambridge University Press.
IPCC. 2007. Climate change 2007: The physical science basis. Contribution of working group I to the fourth assessment report of the intergovernmental panel on climate change. eds. Solomon, S., D. Qin, M. Manning, Z. Chen, M. Marquis, K.B. Avery, M. Tignor, and H.L. Miller. Cambridge: Cambridge University Press.
Jasanoff, S. 1990. The fifth branch: science advisors as policy makers. Cambridge: Harvard University Press.
Jasanoff, S. 2003. Breaking the waves in science studies: Comment on H. M. Collins and Robert Evans, the third wave of science studies. Social Studies of Science 33: 389–400.
Jasanoff, S. 2005. Designs on nature: Science and democracy in Europe and the United States. Princeton: Princeton University Press.
Jasanoff, S. 2010. Beyond calculation: A democratic response to risk. In Disaster and the politics of intervention, ed. A. Lakoff, 14–40. New York: Columbia University Press.
Keeling, C.D. 1998. Rewards and penalties of monitoring the earth. Annual Reviews of Energy and the Environment 23: 25–82.
Kriegler, E., J. W. Hall, H. Held, R. Dawson, and H.-J. Schellnhuber. 2009. Imprecise probability assessment of tipping points in the climate system. Proceedings of the National Academy of Sciences 106: 5,041–5,046.
Krimsky, S. 2003. Science in the public interest. Lantham: Rowman and Littlefield.
Lamont, M. 2010. How professors think: Inside the curious world of academic judgment. Cambridge: Harvard University Press.
Latour, B., and S. Woolgar. 1986. Laboratory life: The construction of scientific fact. Princeton: Princeton University Press.
Lentsch, J., and P. Weingart (eds.). 2011. The politics of science advice: Institutional design for quality assurance. Cambridge: Cambridge University Press.
Longino, H. 1990. Science as social knowledge. Princeton: Princeton University Press.
Mercer, D. 2008. Science legitimacy and folk epistemology in medicine and law: Parallels between legal reforms to the admissibility of expert evidence and evidence based medicine. Social Epistemology 22: 405–423.
Michaels, D. 2008. Doubt is their product: How industry’s assault on science threatens your health. New York: Oxford University Press.
Mooney, C. 2005. The republican war on science. New York: Basic Books.
Moore, K. 2008. Disrupting science: Social movements, American scientists and the politics of the military, 1945–1975. Princeton and Oxford: Princeton University Press.
Newell, B.R., and A.J. Pitman. 2010. The psychology of global warming. Bulletin of the American Meteorological Society 91: 1003–1014.
Novotny, H., P. Scott, and M. Gibbons. 2001. Rethinking science: Knowledge and the public in an age of uncertainty. Cambridge: Polity Press.
Oreskes, N. 1999. The rejection of continental drift: Theory and method in American earth science. New York: Oxford University Press.
Oreskes, N. 2002. Gravity surveys in the ‘permanent’ ocean basins: An instrumental chink in a theoretical suit of armor. In Oceanographic history: The Pacific and beyond, ed. K.R. Benson, and P.F. Rehbock, 502–510. Seattle: University of Washington Press.
Oreskes, N. 2004. Science and public policy: What’s proof got to do with it? Environmental Science and Policy 7: 369–383.
Oreskes, N. 2008. The devil is in the (historical) details: Continental drift as a case of normatively appropriate consensus? Perspectives on Science 16: 253–264.
Pereira, Â.G., and S. Funtowicz. 2005. Quality assurance by extended peer review: tools to inform debates, dialogues and deliberations. Technikfolgenabschätzung Theorie und Praxis 2: 74–79.
Peters, D., and S. Ceci. 1983. Peer-review practices of psychological journals: The fate of submitted articles, submitted again. Behavior and Brain Science 5: 187–255.
Plimer, I. 2009. Heaven + Earth. Global warming: The missing science. Balan, Victoria: Connor Court Publishing.
Porter, T. 1996. Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.
Pritchard, H.D. and D.G. Vaughan. 2007. Widespread acceleration of tidewater glaciers on the Antarctic Peninsula. Journal of Geophysical Research 112: F03S29.
Rampton, S., and J. Stauber. 2002. Trust us, we’re experts: How industry manipulates science and gambles with your future. New York: Penguin.
Ravetz, J.R. 2006. The no-nonsense guide to science. Oxford: New Internationalist Books.
Rennie, D. 1986. Guarding the guardians: A conference on editorial peer review. Journal of the American Medical Association 256: 2,391–2,392.
Rennie, D. 1999. Editorial peer review: Its development and rationale. In Peer review in health sciences, ed. F. Godlee, and T. Jefferson. London: BMJ Books.
Roqueplo, P. 1994. Climats sous surveillance: Limites et conditions de l’expertise scientifique. Paris: Economica.
Rudwick, Martin.J.S. 1988. The great Devonian controversy: The shaping of scientific knowledge among gentlemanly specialists. Chicago: The University of Chicago Press.
Shapin, S. 1995. A social history of truth: Civility and science in seventeenth century England. Chicago: University of Chicago Press.
Smith, R. 2006. Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine 99: 178–182.
Solomon, M. 2001. Social empiricism. Cambridge: MIT Press.
Talent, J. 1989. The case of peripatetic fossils. Nature 338: 613–615.
van der Sluijs, J.P., R. van Est, and M. Riphagen. 2010. Beyond consensus: Reflections from a democratic perspective on the interaction between climate politics and science. Current Opinion in Environmental Sustainability 2: 409–415.
Van, G.F., S. Evans, R. Smith, and N. Black. 1998. Effect of blinding and unmasking on the quality of peer review: A randomized trial. Journal of the American Medical Association 280: 234–237.
Wallerstein, I. 2000. The essential Wallerstein. New York: New Press.
Washington, H.W., and J. Cook. 2011. Climate change denial: Heads in the sand. London: Earthscan Press.
Wenneras, C., and A. Wold. 1997. Sexism and nepotism in peer review. Nature 387: 341–343.
Willett, K.M., N.P. Gillett, P.D. Jones, and P.W. Thorne. 2007. Attribution of observed surface humidity changes to human influence. Nature 449: 710–712.
Wynne, B. 2003. Seasick on the third wave? Subverting the hegemony of propositionalism: Response to Collins and Evans. Social Studies of Science 33: 401–417.
Wynne, B. 2010. When doubt becomes a weapon. Nature 466: 441–442.
Yearley, S. 1997. The changing social authority of science. Science Studies 10: 65–75.
Zammito, J.H. 2004. A nice derangement of epistemes: Post-positivism in the study of science from Quine to Latour. Chicago: University of Chicago Press.
Zhang, X., F.W. Zwiers, G.C. Hegerl, F.H. Lambert, N.P. Gillett, S. Solomon, P.A. Stott, and T. Nozawa. 2007. Detection of human influence on twentieth-century precipitation trends. Nature 448: 461–465.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yearley, S., Mercer, D., Pitman, A. et al. Perspectives on global warming. Metascience 21, 531–559 (2012). https://doi.org/10.1007/s11016-011-9639-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11016-011-9639-9