Background

Each year a masterclass on trauma care for surgeons or final-year residents in surgery under the auspices of the International Association for Trauma Surgery and Intensive Care (IATSIC) and the Dutch Trauma Society (NVT) is organized in the Netherlands. The course consists of a Definitive Surgical Trauma Care (DSTC™) course and a Definitive Anesthetic Trauma Care (DATC™) course. In the Netherlands, both DSTC™ and DATC™ are combined and integrated into a local multidisciplinair variation of the masterclass, in accordance with the core format of the DSTC™ and DATC™ curriculum (DSATC). This masterclass is based on well-established principals and objectives of trauma management described by the IATSIC and is intended to provide surgeons, anesthesiologists and scrub nurses with knowledge and skills needed to safely care for the critically injured trauma patient. Despite the fact that severe traumatic injury with an injury severity score >16 affects only 5% of the Dutch trauma patient population in contrast to 22% in the USA, participants must be skilled in all facets of trauma care [1, 2]. This course offers the possibility to practice cases that are rarely seen, such as a prehospital emergency thoracotomy (Netherlands n = 33 in 5 years) [3] (London n = 71 in 15 years) [4] (Japan n = 91 in 5 years) [5].

The three-day course consists of a theoretical part with key lectures, case presentations and case discussions and hands-on workshops with operative procedures on fresh human cadavers and a live porcine model. Apart from technical skills, crew resource management principles such as communication within the team, leadership and team work are practiced in the workshops. The live porcine operative laboratory includes all members of the team (surgeons, anesthetists, operative and anesthesia nurses, and a staffed laboratory), realistically reinforcing the concepts of crew resource management.

Information on short- and long-term learning effects of a definitive trauma course is scarce, and available data only regard the change in theoretical knowledge [6,7,8,9,10]. A change in technical skills, communication and team skills has not been documented most likely because these are more difficult to assess objectively. Smaller studies have already shown that a short trauma masterclass resulted in significant improvement in participants’ knowledge [11, 12]. However, because there is no standardization, it is difficult to draw a conclusion about effectiveness [13]. Therefore, we have started research on evaluating the DSTC™ and DATC™ with focus on self-assessment of learners’ confidence in their ability to perform the desired skills as a result of this trauma training curriculum. As part of this research, we investigated the change in and retention of technical skills and non-technical skills by means of a self-reporting questionnaire regarding skills to manage the critically injured trauma patient.

Methods

All data were collected from an online questionnaire (Appendix) that was sent to the participants (in masterclasses organized annually from 2013 to 2016) before the course started, directly after and at subsequent intervals of 3 months, 1, 2, 3, 4 and 5 years after the course. Each course consisted on average of 24 surgical participants, 24 anesthetic participants and 12 scrub nurses. Scrub nurses were also included because this group was intimately involved in performing the technical and non-technical skills and were also part of the dedicated trauma team. All participants were requested to anonymously complete the questionnaire. The follow-up questionnaire was sent by e-mail with one reminder 4 weeks later. The questionnaire consisted of six items regarding general skills (for instance; non-technical skills such as questions related to leadership and communication within the team, and general technical skills such as triage and treatment of mass casualties) and 22 DSTC™ or 18 DATC™ items, regarding specific technical skills. An example of a question regarding general skills was: “How competent do you feel with being the leader of the team?”. An example of a question regarding specific technical skills was: “Do you feel competent in handling a patient with a pelvic trauma?”. The items were derived from the course content and objectives. Each item was scored on a 1–5 Likert scale, with a Likert response of 1 indicating completely incompetent and a response of 5 indicating fully competent [14]. The following participant characteristics were collected for descriptive purposes: gender, training phase (resident or consultant), hospital setting (academic or peripheral hospital) and the number of years of experience as a consultant. The questionnaire remained identical during follow-up before (pre), at the end (post) and 3 months, 1 year and 2 years after the masterclass (follow-up).

We compared pre- and post-scores with post- and follow-up scores. The changes in these scores were compared between DSTC™ and DATC™ participants. The general and specific competencies were analyzed separately and compared between DSTC™ and DATC™ participants.

Statistics

All respondents were labeled with a random number which remained the same during the whole follow-up period. Data from each questionnaire were automatically collected and annually arranged in a data file.

For the comparison between the two groups, medians and interquartile ranges (IQR) of total mean Likert scale scores were calculated. Each questionnaire consisted of a general and specific part with the variables pooled for each subgroup. Total scores were analyzed per profession. A mixed model using factor time with random factor participants was used to deal with the correlation of the total score within a participant group. Comparisons between time points were adjusted for multiple testing using the method of Sidak [15]. The significance of each correlation coefficient from the mixed-model analyses as well as the random effect for predicting each outcome measure was assessed at p ≤ 0.05. All analyses were carried out using SPSS software (version 25, SPSS, Inc., Chicago, IL).

Results

Descriptive analysis

This study included questionnaire results obtained from participants in Dutch DSATC masterclasses conducted between 2013 and 2016. This made it possible to analyze four complete cohorts of participants with a two-year follow-up. The response rate of the 265 participants who were sent invitations to fill in the questionnaire is summarized in Table 1. A total of 214 individual participants were identified. A total of 101 were in the surgical group (47%), 61 were in the anesthetic group (29%) and 52 (24%) were scrub nurses. Fifty-nine percent (n = 126) of the participants were male. The descriptive statistics of the participants are summarized in Table 2.

Table 1 Distribution of participants during the courses (2013–2016)
Full size table
Table 2 Descriptive statistics of the participants
Full size table

Statistical analysis

Overall analysis contained the surgical, the anesthetic and the scrub-nurse subgroups (Fig. 1). This analysis was further divided into general skills (communication, teamwork) for both groups and leadership for the surgical group. In addition to general skills, specific skills were also analyzed. For a detailed overview of the results, see supplementary materials (Tables 1–5).

Fig. 1
figure 1

Overall analysis for general skills for the surgical, the anesthetic and the scrub-nurse subgroups

Full size image

General skills

For surgical and anesthetic participants, the total mean scores for self-assessed general skills significantly increased and remained above the pre-course level during 2 years of follow-up (Fig. 1). For scrub nurses, there was an significant increase after 3 months.

Specific skills

For all the subgroups, the total mean scores for specific skills increased significantly (Fig. 2). For surgical and anesthetic participants, these scores remained statistically significant above the pre-course level at interval follow-up. For scrub nurses, significance was lost after 2 years. Specific skills in the anesthetic group contained more variables in comparison with the surgical group but also increased significantly. Sub-analysis showed an increase in self-assessed confidence on communication and teamwork skills. These scores also remained above the pre-course level, but showed a decrease after 3 months.

Fig. 2
figure 2

Overall analysis for specific skills for the surgical, the anesthetic and the scrub-nurse subgroups

Full size image

Sub-analysis

Additional analysis where performed for (1) course year (2013, 2014, 2015, 2016) and subdivided into residents and consultants, (2) experience level (resident, specialist with <10 years’ experience, specialist with >10 years’ experience), (3) gender and (4) type of hospital (academic, peripheral, other institutions).

Course year

No major differences were found between the course years. Residents showed statistical increase in general skills in the course year 2014 and for specific skills in all years. Consultants showed significant increase in general skills in 2013 and for specific skills in all 4 years.

Experience level

For both residents and consultants, self-assessed confidence in both general and specific skills raised significantly. The pre-course level of experienced participants was significantly higher in comparison with residents. A significant raise was found for specific skills, but lost significance after 1 year.

Gender

Gender was further divided in surgical general and specific skills, anesthetic general and specific skills and scrub nurses general and specific skills. There were no significant differences in baseline scores based on gender. Male participants in both surgical and anesthetic groups had significant increases in self-assessed confidence in both general and specific skills. Female surgical participants had significant improvement in specific skills over baseline, while female anesthetic providers had significant increases in both general and specific skills.

Type of hospital

Besides statistical significant increase in both general and specific skills, no differences were found between academic versus peripheral hospitals. For both groups, these scores remained significantly above the pre-course level at two-year follow-up.

Discussion

The overall self-assessed confidence in trauma care competency improved for both general and specific skills, as a result of attending the combined masterclass DSTC™ or DATC™. In the follow-up period, we found that (with exception of the scrub nurses) the improvements in self-assessed confidence over baseline were maintained for the duration of the study. When we extrapolate our data to, respectively, 3, 4 and 5 years, we see a downward trend in self-assessed confidence in performing general and specific skills for cases that are rarely seen in the Netherlands. In accordance with the Advanced Trauma Life Support® (ATLS®), we would therefore advise that the course should be repeated every 4 years. Increase in self-perceived competency is likely when participants have gained experience in their practice [16, 17]. Several participants indeed reported the use of knowledge and skills learned during the trauma course in actual trauma cases, which was confirmed by instructors working in the same institution. However, despite practical experience, we found participants with more than 10 years of experience still had a significant increase in their self-assessed skills over baseline, underlining the usefulness of this masterclass even for experienced consultants. The foregoing is possible the result of the fact that a part of the cases practiced is not often seen in the Netherlands. This in combination with extrapolation of our data makes it useful to timely repeat this course.

Studies on self-assessment have been performed in the past, but not on this scale [13, 18, 19]. In our study, the self-assessment of confidence in specific surgical skills increased more than in specific anesthetic skills, which may be attributed to the fact that the used anesthetic cases are more resembling common anesthetic practice. Other contributing might be the larger group of surgical participants and loss of follow-up after 3 months. In addition, improvements over baseline in the self-assessed confidence of surgical participants were maintained for the 2 years of the study. In contrast, anesthetic participants were noted to have a trend to decline in self-assessed confidence after 3 months of follow-up, but this decline was to a level still significantly above the pre-course baseline. We found that residents had a lower beginning level of self-assessed confidence when compared to consultants, likely explained by the result of work experience. We found no statistical significant differences for participants from academic versus peripheral hospitals or by gender. While the course was largely the same during the study period, attempts were made each year to improve the course based on participant and faculty feedback. Participants rated the course as very good during the years. In spite of minor adjustments in the course and changing some of the faculty year to year, there were no differences found in participants’ self-assessment based on the year of the course.

The increase in self-assessed competency we saw with the live porcine laboratory was an expected finding and has been reported by others [6, 8]. This can be attributed to the realistic nature of a live tissue laboratory and the ability to replicate the actual skills required in practice. The unique addition of other members of the team (anesthetist and operating room nurses) to our course may have contributed to the improvements seen in self-assessed crew resource management and team communication skills. Other simulation-based critical care courses, with emphasis on team performance, also had a positive effect on general and specific surgical competencies in surgical trainees [20].

Residents seem to benefit more from a masterclass than senior consultants and show a steeper learning curve [21, 22]. Similar results were found after a laparoscopic training course, in which trainees had comparable performance data at the end as surgeons [23]. The possible lack of variability and novelty in the anesthetic part of the exercises compared to the surgical part may also explain the difference in specific competence increase. Notably, reported technical and non-technical competences were rated relatively high before the masterclass in the anesthetic group.

Self-reporting is an important tool to uncover strengths and weaknesses and enable further development effectively. Physicians, however, the least experienced in particular, often fail to recognize their impairments and overrate knowledge and skills [18]. Concerning technical skills, some studies show inaccurate self-reports of physicians when comparing their self-reports to objective measurements (like for instance the Objective Structured Assessment of Technical Skills) [24, 25] while other studies show accurate self-reports [26] and an improvement in accuracy with increasing experience [18, 27, 28]. Concerning non-technical skills, surgeons proved not to be very accurate in their self-reports [18, 26]. Novices tend to overrate while seniors tend to underestimate their non-technical skills [26]. On the other hand, senior residents tend to an unwarranted confidence in skills which was not comparable with reality [29].

Our study aimed to minimize socially desirable answers by making the questionnaires anonymous particularly because most participants and instructors were acquainted. This approach was also used in a comparable study regarding advanced trauma operative management [30]. However, as the data were anonymous, data could only be viewed in terms of means (medians of total means) and no comparison at the individual level could be made. This also hampered the analysis of the three-month follow-up results, particularly because response rate had dropped and the study lacked information on participant’s experience with trauma cases after the course. Results from an ongoing German study, with less participants but similar outcomes are pending [19].

Strength and limitations

With 214 individual respondents, this study is the largest determining self-assessed confidence for skills as a result of a trauma course. Research started in 2013 and therefore only data from four course years are obtained. The faculty board had remained largely the same, the course format changed only minimally, which makes it possible to compare data from different years.

The study has few shortcomings. Our conclusion that self-assessed skills were maintained up to 2 years is based upon a response of 20% selection of all participants. It is not unlikely that there was a response bias in this group when responders represent those who reported sustained self-assessed confidence and those who did not respond. This limitation could be addressed by increasing the response rate, for example, by making the evaluation mandatory. Though al course participants were closely monitored by the faculty and discussed each day at the faculty meeting, it is not clear whether the self-assessed improvements in general technical and specific non-technical skills are the result of attending this trauma masterclass or may be a feature of ongoing clinical training and gained experience. To access this problem, it is needed to compare a group of non-attenders with attenders of the masterclass. Therefore, the study group is planning a future study with focus on expert-based observations at subsequent intervals. This in combination with the assessed self-confidence in performing technical and non-technical skills should demonstrate the added value of this masterclass.

The self-reported questionnaire was not validated as such. It was based on questionnaires used by Gaarder et al. [9] in a comparable Norwegian trauma course and adjusted to the content of both DSTC™ and DATC™.

The original goal was to achieve a five-year follow-up. However, the response for 3, 4 and 5 years after the course was so low that these data were not included in the analysis <11% (see Table 1) Possible explanation is that e-mail addresses are outdated or that participants have started working elsewhere. Another common idea is that participants simply are no longer able to adequately self-assess themselves. In a future follow-up study, these all can be prevented by having the faculty or an independent expert assess these technical and non-technical skills with validated scoring lists. This study only looked at technical and non-technical skills. The effect on participants’ self-assessed knowledge that is gathered by attending lectures, case discussions and practical sessions is not evaluated as such and is an important topic for further research.

This is the largest study to date to evaluate the long-term self-assessed confidence to perform skills following a trauma masterclass. We demonstrated that this three-day multidisciplinary trauma curriculum resulted in sustained self-assessed confidence and was of benefit to all participants, regardless of specialty, seniority or experience. A result of these findings increased emphasis on crew resource management was implemented in especially the practical sessions of this multidisciplinary masterclass. Dedicated training curricula such as the DSTC™ and DATC™ course contribute positively to self-assessed confidence to perform the skills taught. Studies such as this can assist in determining the retention of skills and help determine the interval between re-training. In order to retain advanced skills for cases that are not widely seen in the Netherlands, it is advised to timely refresh these skills. In accordance with ATLS®, this is advised after 4 years. The complex question of whether self-assessed confidence translates into improved outcomes could not be addressed in this study and is an important target of future study.