The overall self-assessed confidence in trauma care competency improved for both general and specific skills, as a result of attending the combined masterclass DSTC™ or DATC™. In the follow-up period, we found that (with exception of the scrub nurses) the improvements in self-assessed confidence over baseline were maintained for the duration of the study. When we extrapolate our data to, respectively, 3, 4 and 5 years, we see a downward trend in self-assessed confidence in performing general and specific skills for cases that are rarely seen in the Netherlands. In accordance with the Advanced Trauma Life Support® (ATLS®), we would therefore advise that the course should be repeated every 4 years. Increase in self-perceived competency is likely when participants have gained experience in their practice [16, 17]. Several participants indeed reported the use of knowledge and skills learned during the trauma course in actual trauma cases, which was confirmed by instructors working in the same institution. However, despite practical experience, we found participants with more than 10 years of experience still had a significant increase in their self-assessed skills over baseline, underlining the usefulness of this masterclass even for experienced consultants. The foregoing is possible the result of the fact that a part of the cases practiced is not often seen in the Netherlands. This in combination with extrapolation of our data makes it useful to timely repeat this course.
Studies on self-assessment have been performed in the past, but not on this scale [13, 18, 19]. In our study, the self-assessment of confidence in specific surgical skills increased more than in specific anesthetic skills, which may be attributed to the fact that the used anesthetic cases are more resembling common anesthetic practice. Other contributing might be the larger group of surgical participants and loss of follow-up after 3 months. In addition, improvements over baseline in the self-assessed confidence of surgical participants were maintained for the 2 years of the study. In contrast, anesthetic participants were noted to have a trend to decline in self-assessed confidence after 3 months of follow-up, but this decline was to a level still significantly above the pre-course baseline. We found that residents had a lower beginning level of self-assessed confidence when compared to consultants, likely explained by the result of work experience. We found no statistical significant differences for participants from academic versus peripheral hospitals or by gender. While the course was largely the same during the study period, attempts were made each year to improve the course based on participant and faculty feedback. Participants rated the course as very good during the years. In spite of minor adjustments in the course and changing some of the faculty year to year, there were no differences found in participants’ self-assessment based on the year of the course.
The increase in self-assessed competency we saw with the live porcine laboratory was an expected finding and has been reported by others [6, 8]. This can be attributed to the realistic nature of a live tissue laboratory and the ability to replicate the actual skills required in practice. The unique addition of other members of the team (anesthetist and operating room nurses) to our course may have contributed to the improvements seen in self-assessed crew resource management and team communication skills. Other simulation-based critical care courses, with emphasis on team performance, also had a positive effect on general and specific surgical competencies in surgical trainees [20].
Residents seem to benefit more from a masterclass than senior consultants and show a steeper learning curve [21, 22]. Similar results were found after a laparoscopic training course, in which trainees had comparable performance data at the end as surgeons [23]. The possible lack of variability and novelty in the anesthetic part of the exercises compared to the surgical part may also explain the difference in specific competence increase. Notably, reported technical and non-technical competences were rated relatively high before the masterclass in the anesthetic group.
Self-reporting is an important tool to uncover strengths and weaknesses and enable further development effectively. Physicians, however, the least experienced in particular, often fail to recognize their impairments and overrate knowledge and skills [18]. Concerning technical skills, some studies show inaccurate self-reports of physicians when comparing their self-reports to objective measurements (like for instance the Objective Structured Assessment of Technical Skills) [24, 25] while other studies show accurate self-reports [26] and an improvement in accuracy with increasing experience [18, 27, 28]. Concerning non-technical skills, surgeons proved not to be very accurate in their self-reports [18, 26]. Novices tend to overrate while seniors tend to underestimate their non-technical skills [26]. On the other hand, senior residents tend to an unwarranted confidence in skills which was not comparable with reality [29].
Our study aimed to minimize socially desirable answers by making the questionnaires anonymous particularly because most participants and instructors were acquainted. This approach was also used in a comparable study regarding advanced trauma operative management [30]. However, as the data were anonymous, data could only be viewed in terms of means (medians of total means) and no comparison at the individual level could be made. This also hampered the analysis of the three-month follow-up results, particularly because response rate had dropped and the study lacked information on participant’s experience with trauma cases after the course. Results from an ongoing German study, with less participants but similar outcomes are pending [19].
Strength and limitations
With 214 individual respondents, this study is the largest determining self-assessed confidence for skills as a result of a trauma course. Research started in 2013 and therefore only data from four course years are obtained. The faculty board had remained largely the same, the course format changed only minimally, which makes it possible to compare data from different years.
The study has few shortcomings. Our conclusion that self-assessed skills were maintained up to 2 years is based upon a response of 20% selection of all participants. It is not unlikely that there was a response bias in this group when responders represent those who reported sustained self-assessed confidence and those who did not respond. This limitation could be addressed by increasing the response rate, for example, by making the evaluation mandatory. Though al course participants were closely monitored by the faculty and discussed each day at the faculty meeting, it is not clear whether the self-assessed improvements in general technical and specific non-technical skills are the result of attending this trauma masterclass or may be a feature of ongoing clinical training and gained experience. To access this problem, it is needed to compare a group of non-attenders with attenders of the masterclass. Therefore, the study group is planning a future study with focus on expert-based observations at subsequent intervals. This in combination with the assessed self-confidence in performing technical and non-technical skills should demonstrate the added value of this masterclass.
The self-reported questionnaire was not validated as such. It was based on questionnaires used by Gaarder et al. [9] in a comparable Norwegian trauma course and adjusted to the content of both DSTC™ and DATC™.
The original goal was to achieve a five-year follow-up. However, the response for 3, 4 and 5 years after the course was so low that these data were not included in the analysis <11% (see Table 1) Possible explanation is that e-mail addresses are outdated or that participants have started working elsewhere. Another common idea is that participants simply are no longer able to adequately self-assess themselves. In a future follow-up study, these all can be prevented by having the faculty or an independent expert assess these technical and non-technical skills with validated scoring lists. This study only looked at technical and non-technical skills. The effect on participants’ self-assessed knowledge that is gathered by attending lectures, case discussions and practical sessions is not evaluated as such and is an important topic for further research.
This is the largest study to date to evaluate the long-term self-assessed confidence to perform skills following a trauma masterclass. We demonstrated that this three-day multidisciplinary trauma curriculum resulted in sustained self-assessed confidence and was of benefit to all participants, regardless of specialty, seniority or experience. A result of these findings increased emphasis on crew resource management was implemented in especially the practical sessions of this multidisciplinary masterclass. Dedicated training curricula such as the DSTC™ and DATC™ course contribute positively to self-assessed confidence to perform the skills taught. Studies such as this can assist in determining the retention of skills and help determine the interval between re-training. In order to retain advanced skills for cases that are not widely seen in the Netherlands, it is advised to timely refresh these skills. In accordance with ATLS®, this is advised after 4 years. The complex question of whether self-assessed confidence translates into improved outcomes could not be addressed in this study and is an important target of future study.