Using parent feedback: A qualitative study of residents’ and physician-educators’ perspectives

Introduction Patients and family members can contribute to resident assessment in competency-based medical education. However, few studies have examined the use of patient/family member feedback generated from questionnaire-based assessments. To implement appropriate assessment strategies and optimize feedback use, we need to understand how residents and physician-educators would use feedback from these stakeholders. This study aimed to understand how paediatric residents and physician-educators would use parent feedback generated from questionnaire-based assessments. Methods This study was conducted at a paediatric academic health science centre. We held dyadic interviews with six residents and six physician-educators. A three-step approach was used to analyze the data: data reduction, data display, and conclusions/verifications. We developed an initial coding scheme, conducted an in-depth review of the data and coded it, finalized our coding scheme, and identified categories. Results Participants described that they would use parent feedback to: (a) provide additional direct observations of residents’ performances, (b) teach and coach residents, (c) assess residents’ overall performance and progression, and (d) encourage resident self-assessment and behaviour change. Discussion Parents directly observe residents as they interact with them and their children and, therefore, can provide feedback on residents’ performances. Residency programs should include parent feedback and promote and facilitate its use by residents and physician-educators. Conclusion This study provides an initial understanding of how paediatric residents and physician-educators would use parent feedback if they were to receive it. This information, combined with future research, can inform the development and implementation of parent feedback strategies in competency-based medical education.


Introduction
Patients and family members can meaningfully assess residents' professionalism, collaboration, interpersonal abilities, and communication skills [1,2]. Their involvement in resident assessment is important given the prevalence of patient-and family-centred care in health contexts as well as its impact on healthcare quality and health outcomes [3]. The Institute for Patient-and Family-Centered Care stresses that all aspects of professional practice should incorporate patient-and family-centred care, including medical education [4]. Educators and researchers further emphasize these stakeholders' roles in medical education and the need for leaners to learn with and from them [5,6]. Efforts are underway to involve patients and family members in learner assessment [7], including, for example, the assessment of learners' communication skills via questionnaires [8][9][10][11]. However, research is needed to understand how residents and physician-educators would use the feedback generated from questionnaires to improve clinical practices.
As educators and researchers work to involve patients and family members in medical education, others are working to transition from time-based to competencybased medical education (CBME) [12]. Of interest to this study, paediatrics residency programs in Canada are currently in the midst of transitioning to CBME [13] and therefore, educators and researchers are looking for new and innovative assessment strategies for residents [14]. To effectively improve residents' skills, direct observation of performance is the most accurate assessment method and allows for timely feedback [14,15]. Performance-based assessment, especially through direct observation in clinical settings, is essential but not well studied amidst the abovementioned curricular changes [16,17].
For assessment to be effective, residents must utilize the feedback generated from it. Many factors can influence the use of feedback, such as its timing, understanding what to do with it, and one's perceived need for and openness to receiving it [18][19][20]. Residents' perceptions of the assessors' credibility and the data collected are central to their use of feedback [21][22][23]. It is therefore important to understand how residents and physician-educators would incorporate it into their practices. This understanding would facilitate the implementation of appropriate assessment strategies within paediatric settings that adhere to or are transitioning to CBME and optimize feedback use. Within paediatrics, medical educators also need to be mindful of the important role that patients' parents (i. e., caregivers, guardians) can play in resident assessment and understand how residents and physician-educators would use parent feedback.
Parents are experts on residents' interactions with them and their children, as they directly witness residents' performance. Therefore, parents are considered important sources of feedback in multisource feedback [14,24], an assessment method that includes the perspectives and observations of various assessors [25,26]. That said, a review of assessment data found that only 34% of Accreditation Council for Graduate Medical Education specialties and subspecialty programs in the United States (US) included family members as assessors, but that this involvement was increasingly prevalent in primary care specialties, including paediatrics, where learners had multiple interactions with family members [27]. In the Canadian paediatric context, parents typ-ically provide feedback informally when physician-educators request it. This informal parent feedback is a potential reflection of the limited research on parent involvement in assessment processes and on the development of educational assessment tools to facilitate this involvement [28]. Few studies have examined residents' and physician-educators' use of parent feedback. As such, the aim of this study was to answer: How would paediatric residents and physician-educators use parent feedback generated from questionnaire-based assessments?

Methods
To fulfil the study aim, we used a basic interpretive qualitative approach [29], within a constructivist viewpoint. This approach seeks to understand a phenomenon from the perspective of those who experience it and provides a descriptive account of the findings. As our understanding of the potential use of parent feedback in this context is limited, this approach allowed us to describe this use from the perspective of paediatric residents and physician-educators. However, we recognize that our backgrounds as PhD-trained health professions education researchers and advocates for patient and family engagement in educational processes are also reflected throughout this study and our interpretation of the findings. We obtained approval from the study site's Research Ethics Board prior to study commencement.

Setting and participants
We conducted this study at a standalone, urban paediatric academic health science centre in Ontario, Canada. This centre is committed to patient-and family-centred care. The paediatrics residency program at this centre typically consists of 40 residents and is currently preparing to transition to CBME.
We used purposeful sampling [30] to recruit residents enrolled in the paediatrics residency program as well as physician-educators with resident supervision and assessment responsibilities. We sought to identify individuals with similar relationships to and experiences with resident training and assessment in order to pair them for participation in dyadic interviews. Specifically, we recruited and paired paediatric residents by postgraduate year and physician-educators by number of years of experience supervising and assessing residents in order create a safe interview space for participants to express their opinions and minimize any potential hierarchy issues within each dyad [31]. The research assistant emailed the information letter to paediatric residents on rotation and to physician-educators at the centre via their designated listservs. With this information letter, we invited interested residents and physician-educators to contact the research assistant for additional information and to schedule an interview.

Data collection
We [KE and KAM] individually conducted the dyadic interviews from October to November 2015 while the research assistant took notes. Dyadic interviews require two interviewees (i. e., two residents or two physician-educators) to interact with one another, engage in dialogue on the topic, and share ideas while answering open-ended questions posed by a moderator [32,33]. This data collection method allows participants to share their experiences and thoughts on a topic in an in-depth manner that is not always feasible in focus groups, while maintaining and encouraging the active interaction between participants, which is lost in one-on-one interviews. We used the published assessment literature to inform the development of the interview guide (see Tab. 1), which we piloted with three residents and three physician-educators who did not participate in this study. Written, informed consent was obtained prior to participation. During the interviews, we used in-vivo member checking to verify our understanding of the participants' discussion and responses. The dyadic interviews lasted 60 min, took place in a private room at the centre, were audio-recorded, and transcribed verbatim by a professional transcriptionist. Following transcription, we verified transcript accuracy by comparing the transcripts with the audio-recordings. We also shared the individual transcripts with the respective participants for their review and feedback, all of whom were accepting of them and did not provide any additional comments [34].

Data analysis
The transcribed interviews were entered into NVivo 11 to organize and manage the data [35]. We independently analyzed and coded the data, ensuring rigor and trustworthiness. A systematic, three-step approach to data analysis was used: data reduction, data display, and conclusions and verifications [36]. Our research question, the assessment literature, and our interview notes and summaries guided our analysis. First we reduced the data by developing an initial coding scheme. Next, we conducted an in-depth review of the transcripts and coded the data as per our initial coding scheme, while allowing new codes to emerge from the data. Then we met to compare and discuss our analysis and made minor revisions to our initial coding scheme. This allowed us to identify categories [37] across the participant groups and ensure that we fulfilled our study purpose. Using the final coding scheme, we created a matrix query in NVivo to display and review the data across the participant groups, allowing us to better understand the findings and draw conclusions. We also extracted supporting quotations for reporting purposes. We created an audit trail to document all coding decisions [34].

Results
Six paediatric residents and six physician-educators participated in this study. Half of the residents (n = 3) were female and two were from each year 1, 2, and 3 of residency. Four of the physician-educators were female. All had experience supervising and assessing residents: two had 1-5 years; two had 6-10 years; and two had 16-20 years of experience.
Paediatric residents and physician-educators believed that they would use parent feedback to: (a) provide additional direct observations of residents' performances, (b) teach and coach residents, (c) assess residents' overall performance and progression, and (d) encourage resident self-assessment and behaviour change.

To provide additional direct observations of residents' performances
Participants expressed that parent feedback could contribute to resident assessment by providing additional direct observations of their performances. Because physician-educators are not always able to observe residents directly, the physician-educators explained that they sometimes have to make assumptions about residents' interactions with parents: Likewise, the physician-educators conveyed how parent feedback could prompt them to work more closely with residents on certain skills: ... if there's a trend over time of parents saying similar comments or the same kind of ratings ... that could be a flag to you ... that this is someone that you need to go into the room with them and work together to figure out why the parents are getting that.

To assess residents' overall performance and progression
Participants believed that collecting feedback from a variety of parents over the course of a rotation would be valuable to provide a sense of residents' overall performance at the end of their rotations, as they advance in their training and develop their competencies. A resident expressed, 'I feel like, for me anyway, it would be more useful over the whole block, to see, okay, what were the general themes.' Another described a potential delay in receiving supervisor feedback, and how parent feedback, immediately following their rotations, could help them improve their performance while they wait:

To encourage resident self-assessment and behaviour change
Participants believed that parent feedback could inform their self-assessments, subsequently informing their behaviours and practices. They explained how as residents progress through their training, they make their own training decisions. A resident spoke to how parent feedback could contribute to this process: ... you get all these suggestions from the staff about 'this is how I do it ... so therefore, this is how you should do it'. But you have 10 different people who do it 10 different ways, so you kind of pick what you think is best. But it would be really nice from the ... receiver, the caregiver or the child who is getting ... what we're putting out for them, how they actually took it. Because I think that is actually the more important thing.
Another resident described the potential thought process of using parent feedback to inform their learning: Negative feedback is, thankfully, it's more uncommon, but when you do get it, there's a lot more thinking about negative feedback. And, you know, the process is usually first of all, do I consciously agree with that parent assessment of, or that negative feedback that I'm getting from them ... and if I do, well then we say, okay, well what led to that?
The physician-educators explained that providing residents with parent feedback would provide them with: '... feedback that's truly of value to them. That they're going to appreciate, that they're going to incorporate and use and not feel like it's just one more thing.' Reflecting from a resident perspective, a physician-educator further explained that '.

Discussion
This study sought to understand how paediatric residents and physician-educators would use parent feedback generated from questionnaire-based assessments. Assessment in CBME demands direct observations of authentic residentpatient-parent interactions. This performance-based assessment is critical for residents' professional growth to ensure that they can enact the skills they have learned in clinical settings [14,15,19,38]. Moreover, this type of assessment should be done regularly, rather than on a one-off basis, to properly reflect residents' skills and abilities [16]. We believe parents can provide timely feedback that can be regularly collected via a questionnaire following their interactions with a resident. The immediate use of parent feedback would benefit residents by facilitating appropriate resident recall and opportunity for immediate behaviour change [19]. However, to enable parents to provide feedback, we need to develop and use questionnaires designed specifically for parents in paediatric contexts [28].
Participants' descriptions of how they would use parent feedback suggest that they view parents as credible sources of feedback. This aligns with research from the US showing that the inclusion of family members as assessors is more prevalent in primary care specialties [27]. The participants noted that parents directly observe residents as they interact with them and their children, and therefore are directly familiar with residents' performance. These views are important to consider as the credibility of the assessor is a central factor in the use of feedback [21][22][23]. Further, the participants thought that parent feedback would be important and meaningful for residents' professional development. These perceptions reflect those of previous research that described learners' perceptions of service user involvement in health professions education as a positive and rewarding professional development experience [39].
In multisource feedback, an assessment method promoted within CBME [25], family members are viewed as a source of feedback for residents in addition to that from physician-educators, peers, and other health professionals [14,24,26]. Research has shown that feedback generated through this questionnaire-based method is generally accepted by learners and that they use it to reflect on and/or make changes to their practices [26]. The participants expressed desire for parent feedback in addition to that from physician-educators. That being said, in the Canadian paediatric context, feedback from family members is typically provided informally and on an ad hoc basis, most often if physician-educators ask for their feedback on residents who interacted with them. In fact, while multisource feedback is used within the paediatrics program in this study, parent feedback is not included in it.
The findings from this study suggest that there is value in incorporating parent feedback into assessment processes across all paediatric residency programs. This would contribute to well-rounded assessments and ensure consistency across programs [40,41]. Incorporating parent feedback would promote and facilitate the consistent use of this feedback in training environments, all of which are necessary factors in using feedback for self-assessment [22]. Self-assessment plays an integral role in medical education; reflective practice is encouraged and modelled so that residents involve themselves in their own learning and assessment. The literature also highlights that self-assessment should not be done in isolation, but informed by multiple sources of evidence [14,42]. The participants expressed that they would use parent feedback to inform resident self-assessment and in turn, inform their reflections on their behaviours and practices. Therefore, parent feedback can be a valuable contribution to this process.
This study presents certain limitations that should be considered. It included a limited group of paediatric residents from a single paediatric residency program and a limited group of physician-educators from a single paediatric academic health science centre. Further, this study described residents' and physician-educators' perceptions of how they would use parent feedback, and does not report descriptions of their actual use of this feedback.

Conclusion
Parents can play an important role in paediatric resident assessment. This study provides an initial understanding of how paediatric residents and physician-educators would use parent feedback. This information, combined with future research, can inform the development and implementation of parent feedback strategies. Importantly, it can help promote the use of parent feedback for paediatric resident assessment in CBME.
Conflict of interest K. Eady and K.A. Moreau declare that they have no competing interests.
Ethical standards Ethical approval was sought and obtained from the centre's Research Ethics Board. This study was conducted in accordance with the Declaration of Helsinki. There was no potential harm to participants and participants signed an informed consent form prior to participating. While anonymity could not be guaranteed because of the use of dyadic interviews, participants were asked to keep each other's identities as well as the information shared in the interviews confidential.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http:// creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.