Minerva

, Volume 50, Issue 1, pp 3–19 | Cite as

The Dedisciplining of Peer Review

Article

Abstract

The demand for greater public accountability is changing the nature of ex ante peer review at public science agencies worldwide. Based on a four year research project, this essay examines these changes through an analysis of the process of grant proposal review at two US public science agencies, the National Institutes of Health (NIH) and the National Science Foundation (NSF). Weaving historical and conceptual narratives with analytical accounts, we describe the ways in which these two agencies struggle with the question of incorporating considerations of societal impact into the process of peer review. We use this comparative analysis to draw two main conclusions. First, evaluation of broader societal impacts is not different in kind from evaluation of intellectual merit. Second, the scientific community may actually bolster its autonomy by taking a broader range of considerations into its peer review processes.

Keywords

Peer review Disciplinarity Societal impacts Scientific autonomy Interdisciplinarity National Science Foundation National Institutes of Health 

References

  1. Allen, Janice, 2010. ‘The NIH Scientific Review Process.’ CAPR Digital Repository, at http://csid-capr.unt.edu/fedora/repository/capr:1063
  2. Ben-David, Joseph. 1984. The Scientist’s Role in Society. Chicago: University of Chicago Press.Google Scholar
  3. Benos, Dale, et al, 2007. “The Ups and Downs of Peer Review.” Advances in Physiology Education, vol. 31, pp. 145–152.Google Scholar
  4. Beecher, Henry, 1966. “Ethics and Clinical Research.” New England Journal of Medicine, vol. 274, pp. 1354–60.Google Scholar
  5. Blanpied, William. 1999. “Science and Public Policy: The Steelman Report and the Politics of Post-World War II Science Policy.” AAAS Science and Technology Policy Yearbook, available at: http://www.aaas.org/spp/yearbook/chap29.htm.
  6. Bozeman, Barry, and Craig Boardman. 2009. ‘Broad Impacts and Narrow Perspectives: Passing the Buck on Science and Social Impacts.’ Special Issue, Social Epistemology: US National Science Foundation’s Broader Impacts Criterion. Vo. 23, No. 3–4, Dec. 2009, pp. 183–198.Google Scholar
  7. Briggle, Adam. 2005. “Institutional Review Boards,” in Encyclopedia of Science, Technology, and Ethics, Carl Mitcham (ed.), 4 vols. Farmington Hills, MI: Macmillan Reference, 2005, vol. 2, pp. 1024–26.Google Scholar
  8. Briggle, Adam. 2010. A Rich Bioethics: Public Policy, Biotechnology, and the Kass Council. Notre Dame, IN: University of Notre Dame Press.Google Scholar
  9. Carmichael, Mary, and Sharon Begley. 2010. “Desperately Seeking Cures.” Newsweek, May 15, 2010.Google Scholar
  10. Echegoyen, Luis. 2008. Dear Colleague Letter on Broader Impacts Proposal Requirements; at http://www.delta.wisc.edu/events/Workshops/workshop_docs/2008%20workshop%20docs/Dear_Colleague_NSF_Letter.pdf
  11. Fitzpatrick, Kathleen. 2010. ‘Peer-to-Peer Review and the Future of Scholarly Authority.’ Social Epistemology, vol. 24, no. 3, pp. 161–180.Google Scholar
  12. Frodeman, Robert, and J. Britt Holbrook. 2007. “Science’s Social Effects.” Issues in Science and Technology, Vol. 23, Issue 3 (Spring 2007), pp 28–30: http://www.issues.org/23.3/p_frodeman.html.
  13. Frodeman, Robert, and J. Britt Holbrook. 2011. “NSF’s Struggle to Articulate Societal Relevance.” Science, July 8, 2011.Google Scholar
  14. Frodeman, Robert, and J. Britt Holbrook, under review. “A Comparative Assessment of Metrics for Societal Impact.” Research Evaluation.Google Scholar
  15. Gieryn, Thomas. 1999. Cultural Boundaries of Science: Credibility on the Line. Chicago: University of Chicago Press.Google Scholar
  16. Guston, David. 2000. Between Politics and Science: Assuring the Integrity and Productivity of Research. Cambridge: Cambridge University Press.Google Scholar
  17. Holbrook, J. Britt. 2005. “Assessing the Science – Society Relation: The Case of the U.S. National Science Foundation’s Second Merit Review Criterion,” in Technology in Society, vol. 27, Issue 4, November 2005, pp. 437–451.Google Scholar
  18. Holbrook, J. Britt. 2010. “Peer Review,” in The Oxford Handbook of Interdisciplinarity, Robert Frodeman, (ed.). Oxford: Oxford University Press, 2010, pp. 321–32.Google Scholar
  19. Holbrook, J. Britt, and Robert Frodeman. 2011. “Peer Review and Societal Impacts: A Comparison of the US National Science Foundation and the European Commission’s Framework Programmes.” Research Evaluation, Volume 20, Number 3, September 2011, 239–246(8)Google Scholar
  20. Hollander, unpublished. “Ethics at the National Science Foundation (NSF): A Brief History of Bureaucratic Struggle.” Available at http://csi.ensmp.fr.
  21. Kaiser, Jocelyn. 2011. “Collins Sparks Furor with Proposed NIH Reshuffling.” Science, vol. 331, no. 6016 (28 January), p. 386.Google Scholar
  22. Kuhn, Thomas, 1970. “Logic of Discovery or Psychology of Research?” in Criticism and the Growth of Knowledge, edited by I. Lakatos and A. Musgrave (London: Cambridge University Press) pp. 1–23.Google Scholar
  23. Latour, Bruno. 1993. We Have Never Been Modern. Harvard University Press.Google Scholar
  24. Lightborne, James. 2010. Public Presentation, CAPR Midterm Workshop, Washington, DC, April 22, 2010.Google Scholar
  25. Mandel, Richard. 1996. A Half Century of Peer Review, 1946–1996. Bethesda, Md.: Division of Research Grants, National Institutes of Health.Google Scholar
  26. NAPA, 2001. National Academy of Public Administration Report on Implementation of new Grant Proposal Guide Requirements Related to the Broader Impacts Criterion. Available at http://www.nsf.gov/publications/pub_summ.jsp?ods_key=iin127.
  27. National Academies. 2010. ‘Federal Demonstration Partnership: STAR metrics working Group’. Available at http://sites.nationalacademies.org/PGA/fdp/PGA_057189
  28. Peer Review Process. 2011. NIH website, at http://grants.nih.gov/grants/peer_review_process.htm.
  29. Pielke, Jr., Roger A., and R. Byerly. 1998. ‘Beyond basic and applied’. Physics Today 51 (2) 42–46, Times Cited: 17, issn: 0031–9228.Google Scholar
  30. Reisch, George. 2005. How the Cold War Transformed Philosophy of Science: To the Icy Slopes of Logic. Cambridge University Press: Cambridge.Google Scholar
  31. Roberts, Melanie. 2009. “Realizing Societal Benefit from Academic Research: Analysis of the National Science Foundation’s Broader Impacts Criterion.” Special Issue, Social Epistemology: US National Science Foundation’s Broader Impacts Criterion. Vo. 23, No. 3-4, Dec. 2009, pp. 199–219.Google Scholar
  32. Rossiter, Margaret W. 1984. ‘The History and Philosophy of Science Program at the National Science Foundation.’ Isis, vol. 75, no. 1, pp. 95–104.Google Scholar
  33. Rothenberg, Marc. 2010. ‘Making Judgments about Grant Proposals: A Brief History of the Merit Review Criteria at the National Science Foundation.’ Technology and Innovation, Vol. 12, pp. 189–195.Google Scholar
  34. Sarewitz, Daniel. 1996. Frontiers of Illusion: Science, Technology, and the Politics of Progress. Temple University Press.Google Scholar
  35. Sarewitz, Daniel. 2000. “Science and Environmental Policy: An Excess of Objectivity,” in Earth Matters: The Earth Sciences, Philosophy, and the Claims of Community, edited by Robert Frodeman (Upper Saddle River, NJ: Prentice Hall) pp. 79–98.Google Scholar
  36. Sarewitz, Daniel. April 08, 2003. “Does Science Policy Exist, and If So, Does it Matter?: Some Observations on the U.S. R&D Budget.” Discussion paper. Columbia University’s Earth Institute’s Science, Technology and Global Development Seminar, New York, NY.Google Scholar
  37. Schienke, Erich, et al. 2009. “The Role of the National Science Foundation Broader Impacts Criterion in Enhancing Research Ethics Pedagogy.” Special Issue, Social Epistemology: US National Science Foundation’s Broader Impacts Criterion. Vo. 23, No. 3–4, Dec. 2009, pp. 317–336.Google Scholar
  38. Shapin, Steven, and Simon Schaffer. 1985. Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. Princeton, NJ: Princeton University Press.Google Scholar
  39. Spier, Ray. 2002. “The History of the Peer-review Process,” TRENDS in Biotechnology, vol. 20, no. 8, pp. 357–358.Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2012

Authors and Affiliations

  1. 1.Department of Philosophy and Religion Studies, Center for the Study of InterdisciplinarityUniversity of North TexasDentonUSA
  2. 2.Department of Philosophy and Religion StudiesUniversity of North TexasDentonUSA

Personalised recommendations