Does mentoring new peer reviewers improve review quality? A randomized trial
- Debra HouryAffiliated withDepartment of Emergency Medicine, Emory University School of Medicine
- , Steven GreenAffiliated withDepartment of Emergency Medicine, Loma Linda School of Medicine
- , Michael CallahamAffiliated withDepartment of Emergency Medicine, University of California Email author
Prior efforts to train medical journal peer reviewers have not improved subsequent review quality, although such interventions were general and brief. We hypothesized that a manuscript-specific and more extended intervention pairing new reviewers with high-quality senior reviewers as mentors would improve subsequent review quality.
Over a four-year period we randomly assigned all new reviewers for Annals of Emergency Medicine to receive our standard written informational materials alone, or these materials plus a new mentoring intervention. For this program we paired new reviewers with a high-quality senior reviewer for each of their first three manuscript reviews, and asked mentees to discuss their review with their mentor by email or phone. We then compared the quality of subsequent reviews between the control and intervention groups, using linear mixed effects models of the slopes of review quality scores over time.
We studied 490 manuscript reviews, with similar baseline characteristics between the 24 mentees who completed the trial and the 22 control reviewers. Mean quality scores for the first 3 reviews on our 1 to 5 point scale were similar between control and mentee groups (3.4 versus 3.5), as were slopes of change of review scores (-0.229 versus -0.549) and all other secondary measures of reviewer performance.
A structured training intervention of pairing newly recruited medical journal peer reviewers with senior reviewer mentors did not improve the quality of their subsequent reviews.
KeywordsMentoring Peer review Scientific publication Critical analysis Journal peer reviewer
- Does mentoring new peer reviewers improve review quality? A randomized trial
- Open Access
- Available under Open Access This content is freely available online to anyone, anywhere at any time.
BMC Medical Education
- Online Date
- August 2012
- Online ISSN
- BioMed Central
- Additional Links
- Peer review
- Scientific publication
- Critical analysis
- Journal peer reviewer
- Author Affiliations
- 1. Department of Emergency Medicine, Emory University School of Medicine, Atlanta, GA, USA
- 2. Department of Emergency Medicine, Loma Linda School of Medicine, Los Angeles, CA, USA
- 3. Department of Emergency Medicine, University of California, San Francisco, CA, USA