Advertisement

Journal of General Internal Medicine

, Volume 34, Issue 4, pp 513–514 | Cite as

Laughter and the Chair: Social Pressures Influencing Scoring During Grant Peer Review Meetings

  • Elizabeth L. Pier
  • Joshua Raclaw
  • Molly CarnesEmail author
  • Cecilia E. Ford
  • Anna Kaatz
Concise Research Reports

INTRODUCTION

During NIH peer review meetings (“study sections”), scientists discuss and assign “priority scores” to grant applications that largely determine funding outcomes. Although the final priority score is an average of each panelist’s score, their individual score is anchored to the scores declared publicly by those scientists (usually three) assigned to review and report on the grant application in detail. We have identified “score calibration talk” (SCT), a discourse practice where a study section member discusses and interprets the scoring rather than the content of a grant application. We found two forms: self-initiated SCT, when a panelist provides commentary about their own scoring (e.g., “So I gave it a four, which was probably generous”); and other-initiated SCT,when a panelist challenges the scoring of an assigned reviewer (e.g., “Yeah, that was generous.”). Only other-initiated SCT correlated with changes from the initial to the final score among the assigned...

Notes

Acknowledgements

During the time this research was conducted, Elizabeth Pier was a postdoctoral fellow and Anna Kaatz was an Associate Scientist at the Center for Women’s Health Research at the University of Wisconsin-Madison.

Funding

The research reported in this paper was supported by the National Institute of General Medical Sciences of the National Institutes of Health under Award Numbers R01GM111002 and R35GM122557.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.

References

  1. 1.
    Pier EL, Raclaw J, Kaatz A, Brauer M, Carnes M, Nathan MJ, et al. ‘Your comments are meaner than your score’: score calibration talk influences intra- and inter-panel variability during scientific grant peer review. Res Eval. 2017;26(1):1–14.CrossRefGoogle Scholar
  2. 2.
    Kooistra B, Dijkman B, Einhorn TA, Bhandari M. How to design a good case series. J Bone Joint Surg Am. 2009;91 Suppl 3:21–6.CrossRefGoogle Scholar
  3. 3.
    Holt E. The last laugh: shared laughter and topic termination. J Pragmat. 2010;42 (6 ):1513–25.CrossRefGoogle Scholar
  4. 4.
    Glenn P. Interviewees volunteered laughter in employment interviews: a case of “nervous” laughter? In: Glenn P HE, ed. Studies of Laughter in Interaction. London: Bloomsbury Academic; 2013:255–75.Google Scholar
  5. 5.
    Raclaw J, Ford CE. Laughter and the management of divergent positions in peer review interactions. J Pragmat. 2017;113:1–15.CrossRefGoogle Scholar
  6. 6.
    Pier EL, Brauer M, Filut A, Kaatz A, Raclaw J, Nathan MJ, et al. Low agreement among reviewers evaluating the same NIH grant applications. Proc Natl Acad Sci U S A. 2018;115(12):2952–7.CrossRefGoogle Scholar

Copyright information

© Society of General Internal Medicine 2018

Authors and Affiliations

  • Elizabeth L. Pier
    • 1
  • Joshua Raclaw
    • 2
  • Molly Carnes
    • 3
    • 4
    • 5
    Email author
  • Cecilia E. Ford
    • 6
    • 7
  • Anna Kaatz
    • 8
  1. 1.Education AnalyticsMadisonUSA
  2. 2.Department of EnglishWest Chester UniversityWest ChesterUSA
  3. 3.Department of MedicineUniversity of Wisconsin-MadisonMadisonUSA
  4. 4.Department of PsychiatryUniversity of Wisconsin-MadisonMadisonUSA
  5. 5.Department of Industrial & Systems Engineering University of Wisconsin-MadisonMadisonUSA
  6. 6.Department of EnglishUniversity of Wisconsin-MadisonMadisonUSA
  7. 7.Department of SociologyUniversity of Wisconsin-MadisonMadisonUSA
  8. 8.Center for Women’s Health ResearchUniversity of Wisconsin-MadisonMadisonUSA

Personalised recommendations