The Dedisciplining of Peer Review
Purchase on Springer.com
$39.95 / €34.95 / £29.95*
Rent the article at a discountRent now
* Final gross prices may vary according to local VAT.
The demand for greater public accountability is changing the nature of ex ante peer review at public science agencies worldwide. Based on a four year research project, this essay examines these changes through an analysis of the process of grant proposal review at two US public science agencies, the National Institutes of Health (NIH) and the National Science Foundation (NSF). Weaving historical and conceptual narratives with analytical accounts, we describe the ways in which these two agencies struggle with the question of incorporating considerations of societal impact into the process of peer review. We use this comparative analysis to draw two main conclusions. First, evaluation of broader societal impacts is not different in kind from evaluation of intellectual merit. Second, the scientific community may actually bolster its autonomy by taking a broader range of considerations into its peer review processes.
- Allen, Janice, 2010. ‘The NIH Scientific Review Process.’ CAPR Digital Repository, at http://csid-capr.unt.edu/fedora/repository/capr:1063
- Ben-David, Joseph. 1984. The Scientist’s Role in Society. Chicago: University of Chicago Press.
- Benos, Dale, et al, 2007. “The Ups and Downs of Peer Review.” Advances in Physiology Education, vol. 31, pp. 145–152.
- Beecher, Henry, 1966. “Ethics and Clinical Research.” New England Journal of Medicine, vol. 274, pp. 1354–60.
- Blanpied, William. 1999. “Science and Public Policy: The Steelman Report and the Politics of Post-World War II Science Policy.” AAAS Science and Technology Policy Yearbook, available at: http://www.aaas.org/spp/yearbook/chap29.htm.
- Bozeman, Barry, and Craig Boardman. 2009. ‘Broad Impacts and Narrow Perspectives: Passing the Buck on Science and Social Impacts.’ Special Issue, Social Epistemology: US National Science Foundation’s Broader Impacts Criterion. Vo. 23, No. 3–4, Dec. 2009, pp. 183–198.
- Briggle, Adam. 2005. “Institutional Review Boards,” in Encyclopedia of Science, Technology, and Ethics, Carl Mitcham (ed.), 4 vols. Farmington Hills, MI: Macmillan Reference, 2005, vol. 2, pp. 1024–26.
- Briggle, Adam. 2010. A Rich Bioethics: Public Policy, Biotechnology, and the Kass Council. Notre Dame, IN: University of Notre Dame Press.
- Carmichael, Mary, and Sharon Begley. 2010. “Desperately Seeking Cures.” Newsweek, May 15, 2010.
- Echegoyen, Luis. 2008. Dear Colleague Letter on Broader Impacts Proposal Requirements; at http://www.delta.wisc.edu/events/Workshops/workshop_docs/2008%20workshop%20docs/Dear_Colleague_NSF_Letter.pdf
- Fitzpatrick, Kathleen. 2010. ‘Peer-to-Peer Review and the Future of Scholarly Authority.’ Social Epistemology, vol. 24, no. 3, pp. 161–180.
- Frodeman, Robert, and J. Britt Holbrook. 2007. “Science’s Social Effects.” Issues in Science and Technology, Vol. 23, Issue 3 (Spring 2007), pp 28–30: http://www.issues.org/23.3/p_frodeman.html.
- Frodeman, Robert, and J. Britt Holbrook. 2011. “NSF’s Struggle to Articulate Societal Relevance.” Science, July 8, 2011.
- Frodeman, Robert, and J. Britt Holbrook, under review. “A Comparative Assessment of Metrics for Societal Impact.” Research Evaluation.
- Gieryn, Thomas. 1999. Cultural Boundaries of Science: Credibility on the Line. Chicago: University of Chicago Press.
- Guston, David. 2000. Between Politics and Science: Assuring the Integrity and Productivity of Research. Cambridge: Cambridge University Press.
- Holbrook, J. Britt. 2005. “Assessing the Science – Society Relation: The Case of the U.S. National Science Foundation’s Second Merit Review Criterion,” in Technology in Society, vol. 27, Issue 4, November 2005, pp. 437–451.
- Holbrook, J. Britt. 2010. “Peer Review,” in The Oxford Handbook of Interdisciplinarity, Robert Frodeman, (ed.). Oxford: Oxford University Press, 2010, pp. 321–32.
- Holbrook, J. Britt, and Robert Frodeman. 2011. “Peer Review and Societal Impacts: A Comparison of the US National Science Foundation and the European Commission’s Framework Programmes.” Research Evaluation, Volume 20, Number 3, September 2011, 239–246(8)
- Hollander, unpublished. “Ethics at the National Science Foundation (NSF): A Brief History of Bureaucratic Struggle.” Available at http://csi.ensmp.fr.
- Kaiser, Jocelyn. 2011. “Collins Sparks Furor with Proposed NIH Reshuffling.” Science, vol. 331, no. 6016 (28 January), p. 386.
- Kington, 2007, at http://enhancing-peer-review.nih.gov/meetings/102207-summary.html.
- Kuhn, Thomas, 1970. “Logic of Discovery or Psychology of Research?” in Criticism and the Growth of Knowledge, edited by I. Lakatos and A. Musgrave (London: Cambridge University Press) pp. 1–23.
- Latour, Bruno. 1993. We Have Never Been Modern. Harvard University Press.
- Lightborne, James. 2010. Public Presentation, CAPR Midterm Workshop, Washington, DC, April 22, 2010.
- Mandel, Richard. 1996. A Half Century of Peer Review, 1946–1996. Bethesda, Md.: Division of Research Grants, National Institutes of Health.
- NAPA, 2001. National Academy of Public Administration Report on Implementation of new Grant Proposal Guide Requirements Related to the Broader Impacts Criterion. Available at http://www.nsf.gov/publications/pub_summ.jsp?ods_key=iin127.
- National Academies. 2010. ‘Federal Demonstration Partnership: STAR metrics working Group’. Available at http://sites.nationalacademies.org/PGA/fdp/PGA_057189
- Peer Review Process. 2011. NIH website, at http://grants.nih.gov/grants/peer_review_process.htm.
- Pielke, Jr., Roger A., and R. Byerly. 1998. ‘Beyond basic and applied’. Physics Today 51 (2) 42–46, Times Cited: 17, issn: 0031–9228.
- Reisch, George. 2005. How the Cold War Transformed Philosophy of Science: To the Icy Slopes of Logic. Cambridge University Press: Cambridge.
- Roberts, Melanie. 2009. “Realizing Societal Benefit from Academic Research: Analysis of the National Science Foundation’s Broader Impacts Criterion.” Special Issue, Social Epistemology: US National Science Foundation’s Broader Impacts Criterion. Vo. 23, No. 3-4, Dec. 2009, pp. 199–219.
- Rossiter, Margaret W. 1984. ‘The History and Philosophy of Science Program at the National Science Foundation.’ Isis, vol. 75, no. 1, pp. 95–104.
- Rothenberg, Marc. 2010. ‘Making Judgments about Grant Proposals: A Brief History of the Merit Review Criteria at the National Science Foundation.’ Technology and Innovation, Vol. 12, pp. 189–195.
- Sarewitz, Daniel. 1996. Frontiers of Illusion: Science, Technology, and the Politics of Progress. Temple University Press.
- Sarewitz, Daniel. 2000. “Science and Environmental Policy: An Excess of Objectivity,” in Earth Matters: The Earth Sciences, Philosophy, and the Claims of Community, edited by Robert Frodeman (Upper Saddle River, NJ: Prentice Hall) pp. 79–98.
- Sarewitz, Daniel. April 08, 2003. “Does Science Policy Exist, and If So, Does it Matter?: Some Observations on the U.S. R&D Budget.” Discussion paper. Columbia University’s Earth Institute’s Science, Technology and Global Development Seminar, New York, NY.
- Schienke, Erich, et al. 2009. “The Role of the National Science Foundation Broader Impacts Criterion in Enhancing Research Ethics Pedagogy.” Special Issue, Social Epistemology: US National Science Foundation’s Broader Impacts Criterion. Vo. 23, No. 3–4, Dec. 2009, pp. 317–336.
- Shapin, Steven, and Simon Schaffer. 1985. Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. Princeton, NJ: Princeton University Press.
- Spier, Ray. 2002. “The History of the Peer-review Process,” TRENDS in Biotechnology, vol. 20, no. 8, pp. 357–358.
- The Dedisciplining of Peer Review
Volume 50, Issue 1 , pp 3-19
- Cover Date
- Print ISSN
- Online ISSN
- Springer Netherlands
- Additional Links
- Peer review
- Societal impacts
- Scientific autonomy
- National Science Foundation
- National Institutes of Health
- Industry Sectors
- Author Affiliations
- 1. Department of Philosophy and Religion Studies, Center for the Study of Interdisciplinarity, University of North Texas, 225 EESAT, Box 310920, Denton, TX, 76203-0920, USA
- 2. Department of Philosophy and Religion Studies, University of North Texas, 1155 Union Circle, #310920, Denton, TX, 76203-5017, USA