Teaching Feedback to First-year Medical Students: Long-term Skill Retention and Accuracy of Student Self-assessment
- First Online:
- Cite this article as:
- Kruidering-Hall, M., O’Sullivan, P.S. & Chou, C.L. J GEN INTERN MED (2009) 24: 721. doi:10.1007/s11606-009-0983-z
- 852 Downloads
Giving and receiving feedback are critical skills and should be taught early in the process of medical education, yet few studies discuss the effect of feedback curricula for first-year medical students.
To study short-term and long-term skills and attitudes of first-year medical students after a multidisciplinary feedback curriculum.
Prospective pre- vs. post-course evaluation using mixed-methods data analysis.
First-year students at a public university medical school.
We collected anonymous student feedback to faculty before, immediately after, and 8 months after the curriculum and classified comments by recommendation (reinforcing/corrective) and specificity (global/specific). Students also self-rated their comfort with and quality of feedback. We assessed changes in comments (skills) and self-rated abilities (attitudes) across the three time points.
MEASUREMENTS AND MAIN RESULTS
Across the three time points, students’ evaluation contained more corrective specific comments per evaluation [pre-curriculum mean (SD) 0.48 (0.99); post-curriculum 1.20 (1.7); year-end 0.95 (1.5); p = 0.006]. Students reported increased skill and comfort in giving and receiving feedback and at providing constructive feedback (p < 0.001). However, the number of specific comments on year-end evaluations declined [pre 3.35 (2.0); post 3.49 (2.3); year-end 2.8 (2.1)]; p = 0.008], as did students’ self-rated ability to give specific comments.
Teaching feedback to early medical students resulted in improved skills of delivering corrective specific feedback and enhanced comfort with feedback. However, students’ overall ability to deliver specific feedback decreased over time.
KEY WORDSmedical educationcurriculum developmentfeedbackself-assessmentqualitative analysis
Giving feedback is a critical skill for effective teaching and learning, the “heart of medical education” 1–3. A recent review of the social science literature defined feedback in clinical education as “specific information about the comparison between a trainee’s performance and a standard, given with intent to improve the trainee’s performance” 4. Generally accepted characteristics of effective formative feedback include aspects of structure, content, and format. Structural requirements encompass location, time, and orientation of the learner to the process and to the goal of the process. Ideally, the provider and recipient are allies and operate in a culture of mutual respect, while feedback is co-constructed through loops of dialogue and information and reflective practice 1,5,6. As for the content of feedback, studies have validated that effective feedback is constructive, specific, and non-judgmental 4,7,8. Useful formats for feedback include oral, written, graphic, and video 9–11.
It is desirable for medical students to learn effective feedback skills early in their careers. Medical educators rely on feedback from learners to impel enhancement of educational programs 12. Moreover, the increased emphasis on assessment of professionalism underscores the need for feedback curricula; in a multicenter study, student focus groups identified that receiving training to give and receive effective feedback is necessary for success of a peer evaluation system of professionalism 13,14. However, there are few reported outcomes of curricula to teach feedback to early medical students. In published descriptions of such curricula for residents and faculty 15–19, there remain several open questions about the scope and effectiveness of these curricula. Few studies report improvements in the quality of feedback as a result of these courses, despite data showing that written feedback is quantifiable 9,19,20. We also found scarce data in the medical literature assessing the long-term effects of curricular interventions to teach feedback.
Our research objectives were: to objectively assess students’ feedback skills at three time points (before, immediately after, and 8 months after a curriculum to teach feedback) and to elicit students’ comfort and perceived efficacy at delivering feedback at the same time points.
Participants were the entire class of 141 first-year medical students at a single institution in 2006. All students received the same feedback curriculum. Institutional Review Board approval was obtained for all parts of the study, including student participation in surveys and review of anonymous student course evaluations. A consent form provided to all students explained the purpose and methods of the study, and students voluntarily participated in the study by submitting their surveys.
Setting and Curricular Intervention
At our School of Medicine, the inaugural course for first-year medical students is an interdisciplinary, integrated basic science and foundational clinical skills course called “Prologue/Foundations of Patient Care” (“PFPC”). The course spans 8 weeks.
To create a highly integrated setting where students could practice feedback skills relevant to their level of clinical expertise, we created role-play scenarios adapted to the students’ current clinical skills and anatomy knowledge. 6 These role-plays occurred in weeks 5 and 7 of PFPC, immediately after curricular modules on surface anatomy. Students worked in pairs: one student portrayed a patient and read a script of the patient’s concerns, while the other portrayed a physician (without a script) in a clinical vignette highlighting surface anatomy objectives that they had just learned. After the exercises, we asked “patients” to give feedback to “physicians” about their interviewing and physical examination techniques, while “physicians” were asked to self-reflect on their performance. The feedback was structured using a checklist and required a written comment about the encounter. No subsequent formal teaching or reinforcement of feedback principles occurred for the remainder of the school year, although small groups in medical interviewing with actual or standardized patients, coupled with peer feedback, continued on average every other week, for a total of 12 sessions.
Outcome Measures, Evaluation, and Analysis
Feedback Skill Assessment
Self-report of Feedback Skills
In October 2006, after the second series of surface anatomy role-play exercises in week 7 of PFPC, all students were invited to complete a brief self-report survey assessing their attitudes towards giving and receiving feedback using a retrospective pre- and post-intervention survey 21. Students self-reported their abilities on a single retrospective pre-/post-course survey using a five-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree (time points A and B; Figure 1). Students completed the same self-report survey assessing their attitudes towards giving and receiving feedback in June 2007, at the end of their first year (time point C). We calculated descriptive statistics and analyzed changes in self-rated feedback skills with a repeated-measures analysis of variance.
Feedback Coding Scheme: Direct Quotes from Student Evaluations to Illustrate Examples of Comments in each Category
Reinforcing global (rg)*
Her patient interviews were engaging
Extraordinary professor and lecturer
Had interesting things to say about all subjects
Some of the strongest lectures from the year
Thank you for all of your effort!
Reinforcing specific (rs)**
Very succinct about what is clinically important
Thorough and easy to follow
Always stuck to the objectives
She breaks down a complex area in a clean way
I really enjoy the interactive nature of the lectures and the simulations given in class to demonstrate concepts
Corrective global (cg)
The syllabus section was quite inadequate
It would be great if he did something to spice up his lectures
Students didn’t really seem to think her lectures were substantive enough
Some of the demos were not that useful
The powerpoint slides can be made more professional looking
Corrective specific (cs)
There are no bulleted key points that grasp my attention and focus my learning
Try to answer fewer questions and finish the lecture materials
Syllabus material could have been greatly enhanced with diagrams in addition to the text
The syllabus section was hard to follow and correlate with the lectures
Qualitative data from time point C were analyzed using open coding, refining the definitions of the initial coding scheme. Subsequently, coders independently classified comments from the two remaining time points using the refined coding scheme and identified and discussed each coding discrepancy until reaching consensus. The first dataset was then re-analyzed using the final coding scheme; discrepancies were discussed until reaching consensus. To ensure that the comments could be consistently categorized, another author (PSOS) was trained to use the coding scheme. She independently rated 32 comments representing a sampling across all faculty for whom written comments were made. The correlation with those scores and those of the primary raters were 0.76 for positive specific comments, 0.92 for negative specifics, 0.67 for global positives and 0.51 for global negatives. The latter score was low due to the very few negative global comments in the sample. This reliability study supports that another reviewer can obtain consistent ratings for counting comments.
The average number of comments per written student evaluation was calculated. Descriptive statistics were calculated using a one-way ANOVA, followed by a Student-Newman-Keuls post hoc analysis of differences between time points A, B, and C.
Feedback skill assessment
Response Rates and Number of Comments Coded at each Time Point
Number of faculty members evaluated
Number of students providing evaluations (% of invited students who provided evaluations)
Total number of evaluations from students
Total number of comments coded
48 (100 %)
Descriptive Statistics for Student-generated Evaluations: Average Number of Comments in Written Evaluation Provided to Lecturers (n = Number of Evaluations Analyzed per Time Point)
A Pre n = 69
B Post (8 weeks) n = 174
C End (1 year) n = 225
Total specific (sum of reinforcing and corrective specific)
Total global (sum of reinforcing and corrective global)
Self-report on Feedback
A Retrospective pre
B Post (8 weeks)
C End (1 year)
Survey item (number of respondents)
I feel comfortable giving feedback to my peers (n = 116)
I feel comfortable receiving feedback from peers (n = 117)
My feedback was specific (n = 117)
My feedback was constructive (n = 118)
Our study of the effects of an integrated curricular module on feedback for first-year medical students resulted in the following three findings. (1) Students’ skill at delivering corrective specific comments to faculty in an anonymous written venue increased immediately after the curricular module and remained high after 8 months, and this skill corresponded to their self-assessment that their feedback was more constructive at these time points. (2) The overall specificity of students’ feedback to faculty decreased immediately post-curriculum, contrary to students’ self-assessment; long term, students accurately self-assessed that the specificity of their feedback had declined. (3) Students felt significantly more comfortable giving and receiving peer feedback, both immediately and 8 months after the completion of the module. These findings are consistent with studies showing that teaching feedback to faculty results in short-term increases in specific comments 9,19,22.
We believe this analysis extends what is known about teaching feedback in several substantive ways. Most previous curricular descriptions assessed immediate post-curricular attitudes in small numbers of residents or faculty who chose to undertake specific courses in teaching 23,24; here, we objectively assessed both skills and attitudes in feedback. Our study population was a large class of first-year medical students unselected for their interest in teaching. Moreover, students knew that the feedback they provide to lecturers is anonymous, an important factor influencing the likelihood to participate in providing specific feedback 13,14. Finally, we studied long-term skill retention of our curriculum.
In our study, the observed fall in the overall specificity of students’ feedback appeared to apply wholly to their delivery of reinforcing comments, since their frequency of corrective specific comments remained statistically significantly higher than baseline. Interestingly, as the average number of reinforcing specific comments fell, there was a corresponding rise in reinforcing global comments. High levels of comfort giving and receiving feedback may reflect a high degree of unconditional support and praise for classmates. This supposition is consistent with a study reporting that the ratio between reinforcing and corrective comments provided to peers was four to one 25.
The increase in corrective specific comments is consistent with prior studies that noted short-term increases or no change in written specific feedback comments 9,19. Our study adds that students retained these skills for at least 8 months. The reduced number of reinforcing specific comments we observed could indicate that our curriculum emphasized corrective over supportive feedback or the development of “evaluation fatigue” rather than an actual decrement in skills. Indeed, compared with a 100% response rate on evaluations at time point B, only 74% of students completed evaluations at time point C. Alternatively, the anonymous nature of the evaluations may have increased critical appraisals of faculty 26. Specific appreciative comments have been shown to enhance significant cultural change at a medical school through participants’ heightened awareness of relational capacity 27; therefore, it may be desirable to change our curricular module to further highlight the utility and skill of reinforcing specific feedback, rather than merely corrective feedback.
There is considerable literature showing that self-assessments by physicians, residents, and medical students at all levels are inaccurate 25,28–30. Similar but limited data exist for self-assessment of feedback skills: general practitioners in training report being capable of giving feedback, while objective review indicates that their skills need improvement 31. Our first-year students erroneously assessed the overall change in specificity of their feedback by time point B, but accurately reported at time point C that their feedback was less specific and more constructive. Students may have favored the importance of corrective comments when asked to rate their own feedback skills; however, further studies are required to test this assumption. Additionally, accuracy in self-assessment may be related to students’ familiarity with the task at hand 32; they may have had more experience using specific language for constructive, rather than reinforcing, comments.
Our finding that students can show long-term improvement of feedback skills reflects prior data in faculty development. Thirty-two participants reported increased feedback skills as long as 14 years after completion of a faculty development program 23; this study did not provide data on feedback quality. In a recent review examining long-term retention of basic science knowledge, several principles emerged, including reinforcement, prolonged contact with a domain, and structured revisitation over time 33. It is possible that these factors also apply to retention of feedback skills: as our students progressed through the remainder of their first year of medical school, they continued to practice feedback in discussion groups and to complete online evaluations of their instructors.
Our study appears to be one of the first to objectively analyze the characteristics of students’ evaluations of instructors. One prior analysis of evaluations in a problem-based learning curriculum noted that faculty recipients of students’ written feedback valued the specific directions students offered for improvement 34. Other reports demonstrated that most faculty in health sciences and other disciplines use written feedback from students to improve the effectiveness of their teaching 35,36. Future studies will need to assess if and how instructors differentially change their teaching behaviors in response to the reinforcing/constructive and global/specific nature of students’ comments.
There are limitations to this study. First, analysis occurred in a single school, in a single class of students. Second, though our curriculum taught students to give both oral and written feedback to peers, we measured the specificity of their written anonymous feedback to faculty. We maintain these are the same or highly analogous skills: the ability to compose a feedback statement should meet the general principles in oral or written settings 8,37. Third, the generalizability of our results might be affected by sampling bias. We believe that our school’s evaluation system attempts to balance evaluation fatigue with self-selection bias; however, it is still possible that the students’ skill levels demonstrated in this study are not fully representative of the entire class. Finally, our study examined objective analysis of students’ skill in writing comments, without determining the impact of those comments on the intended recipient; therefore, we have measured and reported just one aspect of feedback, and not its ultimate efficacy.
In summary, we found that teaching feedback in an integrated curriculum to first-year medical students resulted in immediate and long-term improvement in comfort giving and receiving feedback, and skills of delivering corrective specific feedback. Moreover, students were able to conduct accurate long-term self-assessments of the constructiveness of their feedback. Further studies could address how to expand the acquisition and retention of feedback skills to include specific reinforcing feedback, and to determine factors that influence long-term retention of feedback skills. Ultimately, assessing the impact of student comments on the intended recipient will determine the true efficacy of such curricula.
We thank Joseph Crawford and Melissa Salonga for their invaluable support with survey data collection, Josephine Tan for her help with literature searches, and UCSF medical students for their cooperation.
Disclosure conflict of interest and funding sources
None of the authors has a conflict of interest, and no external or internal funding resources contributed to completion of this project.