Advertisement

The Participation and Motivations of Grant Peer Reviewers: A Comprehensive Survey

  • Stephen A. GalloEmail author
  • Lisa A. Thompson
  • Karen B. Schmaling
  • Scott R. Glisson
Original Research/Scholarship
  • 7 Downloads

Abstract

Scientific peer reviewers play an integral role in the grant selection process, yet very little has been reported on the levels of participation or the motivations of scientists to take part in peer review. The American Institute of Biological Sciences (AIBS) developed a comprehensive peer review survey that examined the motivations and levels of participation of grant reviewers. The survey was disseminated to 13,091 scientists in AIBS’s proprietary database. Of the 874 respondents, 76% indicated they had reviewed grant applications in the last 3 years; however, the number of reviews was unevenly distributed across this sample. Higher review loads were associated with respondents who had submitted more grant proposals over this time period, some of whom were likely to be study section members for large funding agencies. The most prevalent reason to participate in a review was to give back to the scientific community (especially among frequent grant submitters) and the most common reason to decline an invitation to review was lack of time. Interestingly, few suggested that expectation from the funding agency was a motivation to review. Most felt that review participation positively influenced their careers through improving grantsmanship and exposure to new scientific ideas. Of those who reviewed, respondents reported dedicating 2–5% of their total annual work time to grant review and, based on their self-reported maximum review loads, it is estimated they are participating at 56–87% of their capacity, which may have important implications regarding the sustainability of the system. Overall, it is clear that participation in peer review is uneven and in some cases near capacity, and more needs to be done to create new motivations and incentives to increase the future pool of reviewers.

Keywords

Peer review Participation Sustainability Research funding Grant applications Motivation Survey 

Notes

Supplementary material

11948_2019_123_MOESM1_ESM.pdf (129 kb)
Supplementary material 1 (PDF 128 kb)

References

  1. Amero, S. A. (2015). Enhancing peer review: Expectation for service on NIH peer review and advisory groups. https://grants.nih.gov/grants/guide/notice-files/NOT-OD-10-089.html. Accessed November 2018.
  2. Arns, M. (2014). Open access is tiring out peer reviewers. Nature, 515(7528), 467–467.CrossRefGoogle Scholar
  3. CDMRP. (2018). CDMRP’s two-tiered review process. http://cdmrp.army.mil/about/2tierRevProcess. Accessed November 2018.
  4. DataUSA. (2016). Biochemistry, biophysics and molecular biology: Diversity. https://datausa.io/profile/cip/26/?compare=2602. Accessed November 2018.
  5. Gallo, S., Thompson, L., Schmaling, K., & Glisson, S. (2018). Risk evaluation in peer review of grant applications. Environment Systems and Decisions, 38, 216–229.CrossRefGoogle Scholar
  6. Gallo, S., Thompson, L., Schmaling, K., & Glisson, S. (2019). Grant reviewer perceptions of panel discussion in face-to-face and virtual formats: Lessons from team science? BioRxiv, 586685.Google Scholar
  7. Gropp, R., Glisson, S., Gallo, S., & Thompson, L. (2017). Peer review: A system under stress. BioScience, 67(5), 407–410.CrossRefGoogle Scholar
  8. Herbert, D., Barnett, A., Clarke, P., & Graves, N. (2013). On the time spent preparing grant proposals: An observational study of Australian researchers. British Medical Journal Open, 3(5), e002800.Google Scholar
  9. Irwin, D., Gallo, S., & Glisson, S. (2013). Opinion: Learning from peer review. The Scientist. http://www.the-scientist.com/?articles.view/articleNo/35608/title/Opinion–Learning-from-Peer-Review/. Accessed November 2018.
  10. Kovanis, M., Porcher, R., Ravaud, P., & Trinquart, L. (2016). The global burden of journal peer review in the biomedical literature: Strong imbalance in the collective enterprise. PLoS ONE, 11(11), e0166387.CrossRefGoogle Scholar
  11. Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
  12. Lauer, M. (2018). FY2017 by the numbers. https://nexus.od.nih.gov/all/2018/03/07/fy-2017-by-the-numbers/. Accessed November 2018.
  13. National Institutes of Health. (2008). 20072008 Peer review self-study final drafthttp://enhancing-peer-review.nih.gov/meetings/nihpeerreviewreportfinaldraft.pdf. Accessed November 2018.
  14. National Institute of Health. (2012). Enhancing Peer review survey results report_2012. https://enhancing-peer-review.nih.gov/docs/Enhancing_Peer_Review_Report_2012.pdf. Accessed November 2018.
  15. National Institute of Health. (2018). Become a reviewer: Benefits. https://public.csr.nih.gov/ForReviewers/BecomeAReviewer/ECR/Benefits. Accessed November 2018.
  16. National Science Foundation. (2015). Women, minorities and persons with disabilities in science and engineering. https://www.nsf.gov/statistics/2017/nsf17310/digest/occupation/women.cfm. Accessed November 2018.
  17. National Science Foundation. (2018). Funding rate by state and organization. https://dellweb.bfa.nsf.gov/awdfr3/default.asp. Accessed November 2018.
  18. Nobarany, S., Booth, K., & Hsieh, G. (2015). What motivate people to review articles? The case of the human–computer interaction community. Journal of the Association for Information Science and Technology, 67(6), 1358–1371.CrossRefGoogle Scholar
  19. Rennie, D. (2016). Let’s make peer review scientific. Nature, 535(7610), 31–33.CrossRefGoogle Scholar
  20. Rockey, S. (2015). Understanding the capacity of NIH’s peer review system. https://nexus.od.nih.gov/all/2015/07/30/understanding-capacity-peer-review/. Accessed November 2018.
  21. Schroter, S., Groves, T., & Højgaard, L. (2010). Surveys of current status in biomedical science grant review: funding organisations’ and grant reviewers’ perspectives. BMC Medicine, 8(1), 62.CrossRefGoogle Scholar
  22. Sense About Science. (2009). Peer review survey. http://archive.senseaboutscience.org/pages/peer-review-survey-2009.html. Accessed November 2018.
  23. Squazzoni, F., Bravo, G., & Takacs, K. (2013). Does incentive provision increase the quality of peer review? An experimental study. Research Policy, 42(1), 287–294.CrossRefGoogle Scholar
  24. Stahel, P., & Moore, E. (2014). Peer review for biomedical publications: We can improve the system? BMC Medicine, 12(1), 179.CrossRefGoogle Scholar
  25. Thorngate, W., Dawes, R., & Foddy, M. (2010). Judging merit. New York, NY: Taylor and Francis.CrossRefGoogle Scholar
  26. Wahls, W. (2018). Point of view: The NIH must reduce disparities in funding to maximize its return on investments from taxpayers. eLife, 7, e34965.CrossRefGoogle Scholar
  27. Ware, M. (2008). Peer review: Benefits, perceptions and alternatives. London: Publishing Research Consortium.Google Scholar
  28. Ware, M., & Monkman, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community—An international study. London: Publishing Research Consortium.Google Scholar
  29. Woolston, C. (2016). Salaries: Reality check. Nature, 537(7621), 573–576.CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Scientific Peer Advisory and Review ServicesAmerican Institute of Biological SciencesHerndonUSA
  2. 2.Washington State UniversityVancouverUSA

Personalised recommendations