Introduction

Finding a direct correlation between medical education assessments and training a successful clinician has been a challenge among educators and physician leaders. Over the last decade, Entrustable professional activities (EPAs) have become a popular framework for medical trainee assessment and a supplemental component in evaluating competencies. The concept of EPAs emerged in 2005 and was designed to facilitate assessment of competency and milestone achievement as it translates to medical practice [1,2,3]. This is measured by assessing a trainees’ progress on the basis of the level of supervision required to successfully carry out clinical duties within a specific EPA [3]. This assessment framework has gained attention throughout programs in North American medical education governing bodies. The Accreditation Council for Graduate Medical Education (ACGME) in the United States and the Canadian Medical Education Directives for Specialists (CanMEDS) framework in Canada have supported the incorporation of EPAs into residency and fellowship programs. This framework was designed to be utilized in tandem with the existing six-core competencies framework established by the ACGME which have been implemented across both undergraduate and graduate medical education programs [1, 2, 4].

The main distinction between clinical competency assessments and EPA frameworks is that the former focuses on conceptual domains to characterize a trainees’ performance while EPA domains are operational in nature, focusing on discrete clinical tasks, procedures, and medical conditions [See Table 1] [3]. A single EPA incorporates multiple competency domains into a singular task and functions as an adjunctive method of trainee evaluation. The competency domains include the following: patient care, medical knowledge, practice-based learning, interpersonal & communication skills, professionalism, systems-based practice, and personal & professional development. Problems within the competency based assessment framework are that since it is conceptually based, it is often difficult to find practical applications or assessments of trainees that would translate a trainee’s mastery of these assessments into everyday practice [5,6,7]. EPAs can address this weakness as they can be compiled into a set of discrete skills that are utilized.

Table 1 Comparison of entrustable professional activities (EPAs) and competency-based milestone assessments [4]
Full size table

We performed a metanarrative review to describe the current literature on EPAs and their potential efficacy, and shortcomings within pediatric cardiology. We chose this approach to distill the main concepts surrounding EPA development given the abundance of systematic reviews on EPA creation, assessment, and implementation. The goals of this study were to:

  1. 1.

    Review the history of EPAs and their role within the field of pediatrics [See Table 2]

  2. 2.

    Review the current state of medical education within pediatric cardiology and its challenges [See Table 3]

  3. 3.

    Review the role of EPAs within pediatric cardiology [See Table 4]

Table 2 Literature summary table of papers pertaining to entrustable professional activities as well as EPAs specific to pediatric subspecialties
Full size table
Table 3 Literature summary table of papers pertaining to medical education within pediatric cardiology
Full size table
Table 4 Literature summary table of papers pertaining to entrustable professional activities specific to pediatric cardiology
Full size table

The EPA framework is a promising assessment mechanism for pediatric cardiology trainees. From a post-positivist perspective, it is important to consider how medical education has changed over time with its use of EPAs and whether there is a translational component to clinical practice.

Review of EPA Development and Research

History of EPAs & Competency Assessments

The introduction of EPAs started within obstetrics and gynecology as a clinical assessment of trainees performing deliveries [3, 8, 9]. This EPA framework included multiple competencies that culminated into obtaining a statement of awarded responsibility (STAR) [9]. This goal ensured that residents who obtained a STAR would be approved to perform that specific task unsupervised. EPAs consist of various tasks that can be “entrusted” to a learner, with appropriate supervision by a trained professional [1]. While competencies describe an individual’s performance and abilities, the number of milestones and checklists can be overwhelming for the supervisor and the trainee. For example, the ACGME model outlines 28 competencies while CanMEDS outlines 28 key competencies with up to 126 enabling competencies [9]. EPAs would act as a supplement to the learner’s progress by identifying tasks the learner can master on a spectrum of supervision, which indirectly addresses multiple competencies at once.

The evaluation of EPAs for a specific trainee is variable as it is based on the personality traits of, and dynamics between, the learner and supervisor. This is a problem with observation-based assessments and would require supervisors to be in consensus on determining what is acceptable for trainees to progress through EPAs [10,11,12]. Five factors which dictate the clinical supervisor’s assessment of the learner include the following: (1) perceived learner features, (2) supervisor’s propensity to delegate responsibility, (3) complexity of the EPA, (4) the clinical context, and (5) the nature of relationship between the supervisor and learner [8]. The number of EPAs and their importance for graduation depend on the program and consensus of content experts. While the initial creation of EPA tools were done with expert consensus opinion, Kane’s framework on generalization would highlight the necessity for faculty supervisors to be consistent in their evaluations and minimize the subjectivity in assessing trainees [13].

Further EPA Development and Research

The field of pediatrics has been a burgeoning specialty with medical education research with its early adoption and modification of ACGME milestones and the incorporation of EPAs in residency and fellowship programs [2]. Since the inception of pediatrics milestones in 2009, they have been used to describe the progression of a learner’s progress from the basics as a medical student to a practicing physician [2]. This work has since been advanced a list of 17 EPAs which could be utilized within pediatrics training serves as a prototype which could be modified into other medical specialties. By 2013, EPAs were designed for pediatric subspecialties with the assistance of the American Board of Pediatrics (ABP) and Council of Pediatric Subspecialties (CoPS). This led to the creation of 7 common subspecialty EPAs which were to be supplemented by 3–7 additional EPAs by each pediatric subspecialty.

The Subspecialty Pediatrics Investigator Network (SPIN) created supervision scales through consensus expert opinion for 6 of the 7 EPAs that are used across all pediatric subspecialties [14]. In addition, 6 additional EPAs were uniquely created within pediatric cardiology. These were introduced in 2015 and ABP and CoPS to be incorporated into pediatric cardiology programs [15, 16]. The specific EPAs are as follows: (1) diagnosis and management of patients with arrhythmias and conduction abnormalities; (2) caring for patients who require catheter-based interventions; (3) diagnosis, initial management, and referral of children with advanced or end-stage heart failure or pulmonary hypertension for medical therapy, extracorporeal membrane oxygenator, ventricular assist devices, or cardiac transplantation; (4) diagnosis and management of patients with congenital or acquired cardiac issues; (5) diagnosis and management of patients with acute congenital or acquired cardiac issues requiring intensive care; and (6) application of imaging skills required for all aspects of pediatric and congenital cardiology care [15].

Since the creation of this initiative, EPAs have been utilized in varying degrees in North American programs and data collection is ongoing [15, 17]. Currently, there are no mandates that require programs to adopt EPAs into their assessment models. EPA frameworks have begun to gain interest from medical programs outside of North America as other international centers are looking toward standardizing their fellowship assessment models. Research is ongoing in examining the translational component of assessment frameworks to clinical practice within various validity frameworks (i.e., Kane’s framework for validation) [13, 18]. While these were successfully created through a consensus expert panel of physicians and educators, EPAs have not demonstrated any notable validation within Kane’s framework beyond the generalization phase [19]. There has been criticism around the adoption of multiple assessment frameworks without knowing their efficacy as they can generate more work for both the trainer and trainees.

Prior literature reviews examined the vast numbers of studies performed on EPAs across different medical specialties and levels of education [17, 20,21,22]. Several programs which explored EPA use had prior experience with other assessment frameworks such as competency milestones. Successful EPA incorporation has been demonstrated in programs where faculty are trained and prepared to utilize it as a formal evaluation tool [22].

EPAs are in a position to become a standard assessment framework across all levels of medical education. It is important that EPAs are kept to a certain standard and that they hold construct validity to ensure that they can be a bridge for competency frameworks and clinical practice [23]. The work led to the development of the Quality of EPA (QUEPA) tool, an instrument assessing EPA quality with the following seven attributes: 1. having focus, 2. being observable, 3. having clear intention, 4. being realistic, 5. articulating trustworthiness, 6. being generalizable across rotations, and 7. integrating multiple competencies [23]. The natural progression of EPA research has now moved away from discussing the creation of EPAs and is focusing on the practical aspects of the implementation and assessment of EPAs in medical education [24]. The other concern remains whether EPAs as programs demonstrate superior training and transition to clinical practice compared to traditional methods [24].

Review of Medical Education within Pediatric Cardiology

Challenges with Medical Education in Pediatric Cardiology

The practice of pediatric cardiology has changed drastically in response to the advent of new technologies, improved surgical techniques, and an overall growth of understanding within the field. There are questions surrounding how best to train cardiology fellows to become capable, independent, practitioners while being familiar with the latest practices and knowledge [25]. Therefore, competency-based training must evolve in order to meet the demands of the specialty as there are new technical skills and standards of care when considering medical and surgical management. One example is the recent change from ACGME programs requiring a specific number of echocardiograms, electrophysiology studies, and cardiac catheterizations for fellow graduation. Due to increasing sub-specialization and concerns for duty-hour violations, these measures were re-examined and are no longer a requirement. The pace of change in our field will inevitably leave trainees insufficiently prepared without constant evolution and re-evaluation of how we train them [26]. One way addressing this feeling of insufficiency has been to build the trainee’s confidence and competence through entrustment with various tasks [27]. This sentiment aligns with EPA frameworks and therefore is a potential benefit.

The validity of theoretical and esoteric models of competency-based education and their applicability in mastering patient-centered care in a few years is a concern for trainees [28]. An additional aspect of training that has garnered attention in pediatric cardiology programs is the teaching of communication skills and the provision of ‘customer service’ [28, 29]. As consultants, the unique expertise of pediatric cardiologists is important to physicians and patient families with the highly emotive information they can provide. Therefore, training fellows to communicate effectively rely on sensitivity and respect [28]. Whether these skills can be effectively taught and transferred to trainees through the monitoring of EPAs has yet to be seen. Examples such as these have been a part of the paradigm shift in the role of subspecialists and therefore should be included in trainee assessments.

Pediatric cardiology is unique as it requires broad medical knowledge, exemplary communication skills, and technical mastery. Competency-based assessment frameworks within pediatric cardiology have revolved around medical knowledge and technical exposure rather than prowess. An overview of recommendations published by the American Academy of Pediatrics, American Heart Association, and American College of Cardiology (AAP/AHA/ACC) provided a framework for fellowship programs to determine the amount of time within specific inpatient/outpatient experiences and procedure volume requirements to graduate a categorical fellow [30]. These recommendations summate to mechanical aspects of training and do not provide guidance in developing finesse of the skills or knowledge required. As EPAs are gradually adopted into programs, educators are interested in determining if a new assessment framework would lead to successfully training pediatric cardiologists [25]. European pediatric cardiology programs vary significantly from their training practices and requirements and have expressed interest in the creation of a standardized approach to training and certifying pediatric cardiologists [31].

Measuring technical competencies in pediatric cardiology is challenging. Two-dimensional echocardiography is an example that even with basic knowledge of imaging windows, positioning, and machine optimization, technical mastery requires prowess combined with the comprehensive understanding of complex congenital heart disease. Reaching that level of proficiency takes time and is often the reason pediatric cardiology fellowship is described as an apprenticeship. One program developed an assessment tool for both the performance and clinical interpretation of transthoracic echocardiography [32]. Fellows who were assessed felt that intermittent observation from faculty is inadequate and that the volume of echocardiographic studies performed is a poor surrogate for technical competency [32]. In addition to the steep knowledge and technical learning curves, fellows attributed disorganized learning structures, burnout from excessive duty hours, imposter syndrome, and a feeling of inadequacy to anxiety and uncertainty after graduation [26, 33]. One way programs have taken initiative with these issues is the creation of “boot camps” for first-year fellows. These short-term, intensive training periods function as a primer in pediatric cardiology and provide a space for fellows to discuss their anxieties for fellowship [34].

Pediatric Cardiology Education During the COVID-19 Pandemic

The content and delivery of education have been challenged the last two years during the COVID-19 pandemic. Pediatric cardiology was no exception to its impact on the educational curriculum, necessitating training programs to adjust accordingly [25]. Some programs created an online lecture series for fellows nationally, while others (i.e., Boston Children’s Hospital) developed online platforms to create a curriculum that incorporated both synchronous and asynchronous learning [35, 36]. These approaches utilized assigned learning teams in which each fellow was given specific tasks to complete, including cardiac lesion flow diagrams, creating educational content for other trainees, and developing mock board exam questions [35, 36].

Heart University is another example of an online educational forum that provides an asynchronous learning environment with training modules and provides direct access to publications including White papers, Guideline papers, and Landmark papers [37]. These platforms, while initially developed out of necessity during a pandemic, have demonstrated educational innovation and potential to change the way programs provide instructional materials to fellows [25]. While the impact of these platforms are unknown beyond their general favorability among trainees, there is no indication that these methods of education will disappear in the near future. EPAs and competency frameworks may be adversely affected from online education due to the clinical nature of EPAs and the lack of in-person entrustment.

The Relevance of EPAs within Pediatric Cardiology

Since the introduction of pediatric cardiology EPAs, there have been relatively few studies evaluating their adoption into programs and their overall efficacy. While the framework is thorough, it is time- and resource-intensive to track EPAs for every fellow during their training [15]. There has also been the creation of EPAs pertaining to pediatric cardiology subspecialties, such as cardiac critical care [38]. The inherent flexibility of the EPA framework allows program directors to determine how they wish to utilize EPAs in their fellowship program and therefore limits assessing their efficacy and utility [4, 14].

Congenital cardiothoracic surgery requires an extremely high level of technical expertise and it is understood that training and mentorship continue for our surgical counterparts following completion of formal training in order to excel professionally [39]. For example, few surgical trainees would be entrusted to perform a complex surgical procedure independently within the first year following completion of training. Likewise, it may be appropriate to reconsider certain EPAs specific to pediatric cardiologists which unrealistically expect training to independency within a three-year training period (or any subsequent year(s) of subspecialty training), but may require continued mentorship and evaluation [14]. If EPA thresholds for graduation are consensus-based assessments from programs, then there should be standardized evidence to justify the progression of trainees through fellowship.

The opinion of medical learners is an important factor to consider with the implementation of EPAs. Medical student perspectives on EPAs revealed that the most important aspect to their learning was the reciprocated interest in the supervisor–trainee relationship and building trust through feedback [40]. While this has not been examined among pediatric cardiology fellows, previous studies have shown that there is a strong preference from fellows for structured learning and progression throughout their training [33]. The culture of learning and mentorship is at the heart of continued medical education and reinforces the desirability of EPAs as an assessment model.

Entrustment is a key component within a field as complex as pediatric cardiology. Whether EPAs can objectively measure this in both technically and medically intensive tasks is important to understand [41]. These discussions are not unique to this subspecialty as other pediatric subspecialties have noted trainees are requiring supervision beyond graduation even with successful completion of their EPAs throughout fellowship [42, 43]. This trend may indicate that independent practice is not the goal but instead highlighting that entrustment is not a linear process in spite of a trainee’s growth in competency through the years [44].

To date, there have not been any studies examining patient/family perception and their input on EPAs. One study reported on a small group of pediatric fellows who performed self-assessments on EPAs and revealed that trainees continue to desire supervision and guidance through all three years of training although the self-assessments may have been affected by the trainees’ lack of familiarity with EPAs [45].

Conclusion

Pediatric cardiology is a unique in that there is an element of craftsmanship that lends itself to an apprenticeship model that is pivotal to the education and success of aspiring residents and fellows. Cardiology fellows have emphasized the steep learning curve as a point of uncertainty and anxiety in training [26, 33]. While multiple studies outlined the criteria to successfully train a pediatric cardiologist, it is unclear whether these approaches are in the best interest of the trainee’s professional development.

Evaluating an EPA framework requires a significant amount of time for programs to implement robust EPAs and follow their trainees into their early years of clinical practice. It is worthwhile to utilize a common framework to evaluate and optimize EPAs subspecialties (i.e., QUEPA) to ensure there is consistency when evaluating trainees [23]. We anticipate the next phases of EPA research should begin assessing EPAs and its efficacy in carrying the mission it was designed for initially [46].

The creation of EPAs provided a novel framework to create a continuous assessment model through a trainee’s medical career. At this juncture, EPAs have not been examined with respect to their ability to translate training to high performance in everyday practice. It is essential to question whether the addition of EPAs will be a benefit for programs or if this is merely performative. Future studies should be directed at evaluating pediatric cardiology fellowship programs and their progress in EPA implementation and trainee outcomes.