Globally, over 1 billion people suffer traumatic injuries, being responsible for 6 million deaths annually [1]. Mortality is concentrated in the first hours and is often caused by massive bleeding from big vessels or highly irrigated organs; creating fatal outcomes that are preventable with prompt and effective surgical care [2,3,4,5].

In rural areas and low-income countries, young general surgeons with little to no experience in independent practice are often on call at emergency services during non-business hours, which is when the highest number of accidents occur. However, nationally and internationally, it has been described that general surgeons do not feel qualified to take on this task [6]. In North America, up to 40% of surgical residents do not feel confident enough for their independent practice even though they spend 5 years training as residents [6, 7]. Regulations limiting residents’ training hours and the advent of non-operative management have been among the main reasons to explain this situation [8, 9]. In the past year, the COVID-19 pandemic has aggravated this to an unprecedented extent [10]. A significant reduction in surgical volume alongside severe shortages of personnel requiring residents to be redistributed to roles that involve taking care of non-surgical patients has drastically diminished residents’ caseload exposure [11, 12]. Even though programs have implemented different strategies to cope with these difficulties, the level at which it has hindered the quality of surgical training has yet to be seen. In this context, and considering that simulation-based teaching tools have proven to be effective in a variety of settings, the incorporation of simulation scenarios is even more crucial than before [13]. However, elevated costs can affect simulation tools’ availability. Therefore, easy-to-access and easy-to-implement alternatives are especially relevant. In previous work, our team developed a low-cost Objective Structured Clinical Examination (OSCE) scenario for the evaluation of surgical skills in trauma surgery [14]. In this article we implemented the OSCE as a training tool for residents, measuring the improvement of skills, and evaluating the impact it had on self-perception of competence.

Methods

An observational study was designed. Participants were invited to join in a three-day trauma surgical skills boot camp. Pre- and post-training performance were assessed alongside self-confidence to independently perform the procedures.

Recruiting

Surgical residents at different levels of training from 15 different programs across the country and recently graduated general surgeons from several regions were invited to participate. A minimum sample size of 11 participants per group was established using a power of 0.8 and alpha of 0.05.

Therefore, 75 participants were recruited and distributed into five groups of 15 trainees each to be enrolled in a 3-day training program. Traveling and hosting expenses were covered.

Assessment and training

Our team previously designed and tested under the Messick validity framework an Objective Structured Clinical Examination scenario (OSCE) for evaluation of surgical skills in trauma [14, 15]. The OSCE was now used to assess participants' performance and guide their training. It consisted of 5 simulation-based stations: bowel anastomosis (hand-sewn side- to-side entero-enterostomy on a bovine bowel), vascular anastomosis (end-to-end hand-sewn anastomosis in a perfused 5 mm porcine carotid or iliac artery), laparoscopic suturing (laparoscopic suturing of gastric perforation in an ex vivo porcine stomach model), penetrating lung injury (hand-sewn tractotomy in ex vivo ventilated porcine lungs), and penetrating cardiac injury (bleeding control and suturing of three standardized injuries: right atrium, right ventricle away from the coronary artery, and left ventricle in the vicinity of the coronary artery in a perfused ex vivo porcine model). The cardiac model was perfused using a pulsatile roller pump connected to the aorta, delivering up to 5 lt/min of fake blood, with a pumping heart effect of 120–140 beats per minute [14, 16,17,18].

These procedures were selected due to their reported frequency in Chilean Hospitals [19, 20] and guidelines developed by the Chilean Society of Surgeons regarding the minimal competencies needed to be a general surgeon in Chile. This is due to the reality observed in most hospitals in Chile, which do not have specialized surgical trauma departments, or on-call teams for surgical specialists (vascular, cardiothoracic, or gastrointestinal surgeons). Therefore, most trauma-related surgeries in our country are performed by general surgeons.

Participants had 20 min to complete each station.

For pre- and post-training assessments, each procedure was recorded in high definition and blindly evaluated by experts in an asynchronous manner through a novel technology-enhanced learning platform [21, 22]. Participants were evaluated using a modified OSATS (Objective Structured Assessment of Technical Skills), a widely used global rating scale designed to assess proficiency in practical surgical skills [23]. The total score is based on subscores in five domains: respect for tissue, time and motion, instrument handling, the flow of operation and forward planning, and knowledge of the specific procedure. Each domain is assessed on a scale from 1 to 5; the maximum possible score for each station was 25 points. In addition, a task-specific rating scale (SRS) was developed according to experts’ opinions on essential aspects that had to be included. This “checklist” type of assessment was specific to each station, with varying maximum scores.

A trauma surgery boot camp was designed, with training sessions guided by expert trauma surgeons from our institution. Participants were divided into groups to deliberately practice on the models while receiving direct feedback from experts. The training sessions lasted 2 h for each station.

Self-perception of skills

A self-assessment survey was designed and applied before and after training. The survey contained Likert-type [24] questions regarding participants’ self-perception of competence performing different trauma procedures as the leading surgeon [20].

Statistical analysis

Data were analyzed with RStudio® (2019 v1.2.5001 on base R, Boston, US) using nonparametric statistics. Pre- and post-training performance and self-confidence were compared using a paired, two-tailed Wilcoxon signed-rank test with an alpha set at 0.05 and a power of 80%. To compare scores among groups, the Kruskal–Wallis test for multiple comparisons was used to assess whether there were significant differences among the groups in terms of scores and procedural times. The Dunn test for pairwise comparisons was used to determine in which groups the differences were statistically significant.

Results

Pre-training OSATS results showed statistically significant differences among the previous levels of expertise for the cardiac injury [postgraduate year 1 (PGY1) vs recently graduated surgeons (RGS), p = 0.0282], bowel anastomosis [postgraduate year 2 (PGY2) vs postgraduate year 3 (PGY3) p = 0.0189], and vascular anastomosis [PGY1 vs PGY3, p = 0.0423] stations (Table 1). In the cardiac injury station, first-year residents got the lowest score of all stations, with a median of 15 points.

Table 1 OSATS pre-training
Full size table

In terms of the SRS performance before training, bowel and vascular anastomosis stations showed significant differences in performance. For the bowel anastomosis station, Median pre-training scores were very similar for all groups, however, there was 1 participant in the PGY2 group that got only 3 points. A statistically significant difference was found between that group when compared to PGY3 and RGS, which correlates with what was obtained with OSATS (Table 2).

Table 2 SRS pre-training
Full size table

In the vascular anastomosis station, statistically significant differences were also found, even though all groups got very similar pre-training scores.

As it is shown in Table 3, in terms of individual results, participants’ achieved a statistically significant improvement after the boot camp in the majority of the stations, with p values < 0.0001 in both grading scales for the cardiac, lung, bowel, and vascular stations. For the laparoscopic suturing station, p values of 0.0002 and 0.0006 were obtained for OSATS and SRS, respectively.

Table 3 SRS post-training
Full size table

Regarding the time to complete the procedure, statistically, significant differences were found in all stations. In the pulmonary tractotomy, bowel anastomosis and vascular anastomosis stations’ time to complete the procedure increased after training. The biggest increase was found in the lung injury station with a median increase of 6.11 min after training.

In the cardiac injury station, to better assess participants’ performance timing was subdivided into bleeding control, ventricular repair, and atrial repair. Participants showed a statistically significant improvement in bleeding control and ventricular repair, diminishing the median time 35 and 27 s, respectively (Table 4).

Table 4 Cardiac injury station time to complete the procedure
Full size table

Regarding the self-perception of competence survey applied before and after training, all participants reported an improvement in self-confidence after training (Fig. 3, Table 5). In the bowel anastomosis station, the PGY1, PGY2, and PGY3 groups showed statistically significant improvements. The PGY1 group was more heterogeneous after training but showed the highest median improvement among all groups, while the PGY2 and PGY3 groups improved median scores and reduced interquartile range. Even though non-statistically significant, the RGS started with the highest median level of confidence, and after training the group dispersion was also reduced.

Table 5 Assessment of self-confidence to perform the procedure
Full size table

For the Vascular injury and lung Injury station, all groups improved median scores with similar levels of dispersion.

In the cardiac injury station, all groups improved their median level of confidence with the PGY-1 showing almost no dispersion before training, since most participants rated their confidence level as 1, and only 3 as 2 (the lowest confidence level among all stations). The highest heterogeneity is shown in the PGY-3 group, with results ranging from 1 to 5. Finally, in the laparoscopic suturing station, all groups improved their median level of confidence, with the highest dispersion shown in the first 2 years of residency.

Discussion

As it was previously stated, the importance of developing a low-cost, easy-to-access trauma surgery skills training system is crucial to impact the global burden of trauma injuries, which is particularly important in middle to low-income countries. OSCE scenarios have previously shown to be effective assessment tools for practical skills in medicine, which aligns with previous work developed by our team in which we designed and tested an OSCE scenario for the evaluation of trauma surgical skills [14]. This OSCE type of training system has considerably lower costs than other training courses available in our country, such as the American College of Surgeons Advanced Trauma Operative Management (ATOM) that cost up to 1.000 USD in our country [25]. In comparison, the system that we propose has an estimated cost of 330 USD per participant (Table 6).

Table 6 Costs associated with the development of the boot camp training
Full size table

In this article we aimed to evaluate the OSCE stations as a training system, testing it in participants with different backgrounds and levels of training. The assessment tools used for this OSCE contemplated the widely validated global rating scale OSATS, and task-specific rating scales (SRS) for each procedure. The SRS were developed based on the opinion of experts at our institution.

The results showed participants significantly improved their performance after a training session with the models (Figs. 1 and 2), improvements that were particularly noticeable in the cardiac, pulmonary, bowel, and vascular stations. When analyzing procedural times (Table 7), it is interesting to notice that in three stations (pulmonary, bowel, and vascular anastomosis) time significantly increased after training. This could be a reflection of a deficiency in participants’ previous knowledge of the technique, meaning they had to “learn” how to perform those procedures in the boot camp instead of practicing a previously acquired skill. These findings align with previous work done by Vela et al. that showed a very low level of preparedness in general surgeons to deal with complex trauma cases [20]. Particularly noticeable were the results obtained in the pulmonary tractotomy station, which is a procedure not often performed by general surgery residents, and a median increase of 6.11 min was observed. The cardiac injury and laparoscopic suturing stations showed significant reductions in procedural times, which may indicate participants had a previous notion on how to perform those procedures and used their training time to become more proficient. It is particularly significant from a clinical perspective a 35-s reduction in time to control myocardial bleeding.

Fig. 1
figure 1

OSATS individual scores before and after training. *p-value < 0.01, **p-value < 0.001

Full size image
Fig. 2
figure 2

SRS individual scores before and after training. *p-value < 0.01, **p-value < 0.001

Full size image
Table 7 Comparison of scores and time to complete the procedure before and after training
Full size table

In terms of participants’ scores according to the previous level of expertise (Table 8), significant differences in performance were found among the 4 groups in the cardiac injury, bowel anastomosis, and vascular anastomosis stations. Those differences disappeared after training, except for the cardiac injury station, where statistically significant differences among groups persisted.

Table 8 OSATS post-training
Full size table

Regarding self-perception of competence, overall participants reported an improvement in self-confidence (Table 6, Fig. 3). In the bowel anastomosis station, residents improved their confidence level alongside reducing the group’s dispersion, except for first-year residents, which was also the group with the biggest median improvement and higher dispersion. A possible explanation for this could be that as residents move forward on their training, they tend to level up in this technique, as they possess more tools to reach a successful learning curve. Something similar happened in the laparoscopic suturing station, where higher heterogeneity was observed during the first two years of residency. On the other hand, in the cardiac injury station, the highest level of dispersion was shown in the PGY3 group which alongside extremely low pre-training results obtained by PGY1 might reflect that abilities required for a successful learning curve in this procedure are acquired later during the course of training.

Fig. 3
figure 3

A–E Confidence level results per station. PGY1 postgraduate year 1, PGY2 postgraduate year 2, PGY3 postgraduate year 3, RGS recently graduated surgeons

Full size image

Limitations

The number of participants was limited because of geographical constraints. In some cases, trainees had to travel over a thousand miles to attend the boot camp. In addition, our country has a limited number of general surgery residents. Since the participants of this study came from 15 certified general surgery residency programs available in our country, we believe this sample is heterogeneous and representative of our reality.

On secondhand, we acknowledge a weakness in one of the scales used for assessment, the task-specific rating scales did not undergo a specific validation process.

Finally, this study contemplates assessment in simulated procedures, showing acquisition of technical skills in this scenario. Further studies are required to evaluate predictive validity with a transfer of skills from this boot camp training to the real scenario, the operating room.

Conclusion

An improvement in the objective scores, time to perform the procedure, and self-perception of surgical skills were shown, proving that this training system is effective and can be used as an effective training strategy for the acquisition of surgical skills in trauma surgery. Extremely low objective scores and level of confidence at the cardiopulmonary and lung stations, show a deficit in the training program and should be addressed. If simulation-based programs like this one are included in the first years of residency, general surgeons could have better skills and confidence when facing these extremely severe trauma scenarios.