Introduction

Endotracheal intubation (ETI) is a frequently used life-saving procedure for critically ill patients in emergency departments (EDs). Thus, ETI is a core aspect of the emergency medicine (EM) training curriculum. However, it is a complex psychomotor skill that carries a significant risk of procedure-related complications. Therefore, substantial practice is required to become proficient with this technique.

The Accreditation Council of Graduate Medical Education (ACGME) classified ETI as a key index procedure essential for the independent practice of EM1. As is the case with any manual technique, ETI skill has a learning curve. Setting goals for the minimum experience required to be proficient with this technique is critical when planning an appropriate EM training program. The ACGME requires EM residents to perform at least 35 ETIs during their training1. Although some studies have been conducted on the learning curve of ETI for EM trainees, there is insufficient data to support this minimum experience requirement2,3. Additionally, the learning curves for ETI differed between studies based on the statistical methods, study population, devices used, study setting, and the proficiency definition used2,3,4,5,6,7,8,9.

In the field of EM, a universally accepted definition of proficiency for psychomotor skills, such as ETI, is lacking. In prior studies that examined the learning curve for ETI, researchers employed different criteria to describe proficiency2,3,10. A common indicator of one's proficiency for ETI is the first attempt success (FAS). In current practice, FAS at emergency ETI is prioritized, as several studies have shown that an increased number of ETI attempts is associated with an increased risk of adverse outcomes. In a previous systematic review and meta-analysis that included a total of 42,081 ETIs from 83 institutions in 10 countries, the FAS in the ED was 84.1% (95% confidence interval [CI] 80.1–87.4])11. Given the findings of the previous study and the relevance of FAS rates to patient safety, it may be reasonable to set the proficiency criterion for EM physicians at ≥ 85%.

Our study aimed to determine the number of ETI cases required for a novice EM trainee to achieve a proficiency level, defined as achieving an FAS rate of ≥ 85%.

Methods

Institutional review board statement

The study was conducted in accordance with the Declaration of Helsinki. The Samsung Medical Center Institutional Review Board approved this study and waived the requirement for informed consent due to the anonymity and retrospective nature of the study.

Study design and setting

This single-center retrospective study was conducted at the ED of the Samsung Medical Center (a tertiary referral hospital in an urban area, with an annual ED census of > 70,000 patients). The study data were retrieved from the institutional airway registry between April 2014 and March 2021, and all the ETI attempts that the EM trainees who began their residency program between 2014 and 2018 made throughout their entire training period was evaluated.

This ED is affiliated with a residency training program in EM and generally accepts four or five trainees. Thirty-four residents began their first year of EM residency during the study period. In South Korea, a newly licensed medical doctor who has completed medical school is required to undergo a one-year internship before beginning residency training in each specialty field, including EM. During the internship course, they receive basic medical training while rotating across several departments, typically for a month. As a high-level technique, ETI is rarely assigned to interns, and only a few departments, such as the anesthesia department, give restricted opportunities for ETI. Thus, EM trainees who have completed their internships usually have a very limited experience with ETI when they enter the residency program in South Korea. The residency program in EM requires 4 years of training, which begins in March each year.

ETI is performed on approximately 300–400 adult patients annually, and EM physicians perform the majority of ETIs in the ED. EM trainees commonly engage in ETIs to gain proficiency with the technique. Trainees should be required to perform ETIs under the supervision of EM faculty. EM trainees are generally allowed up to two ETI attempts per patient to minimize the risk of compromising patient safety. However, this limit could be waived depending on the supervisor's judgment or patient's condition. If the trainee cannot successfully complete the ETI, the senior resident or faculty member takes responsibility for the laryngoscope. Adult patients who underwent at least one ETI attempt by an EM trainee who began their residency program between 2014 and 2018 were included in this study. ETIs that were performed exclusively by EM specialists, EM residents who began their first year of residency prior to 2014 or after 2018, or by non-EM physicians were excluded from the study.

Data collection and outcome measures

Every ETI performed on adult patients in the ED during the study period was recorded with detailed patient and procedural data in the institutional airway registry, regardless of the time of day. No ETI attempt in the ED was omitted during the study period. The entire procedure of each ETI was monitored and recorded in real-time by an independent staff member using a standardized data entry format to minimize recall and reporting bias12. The intubator and faculty members reviewed and completed the data after the procedure. The faculty responsible for the registry regularly monitored the data for quality control.

We extracted the following data from our institutional registry and electronic medical records: baseline characteristics of the patients, including age, sex, and body mass index (BMI); the presence of difficult airway characteristics; reason for ETI; intubating devices (conventional Macintosh-type direct laryngoscope [DL], C-MAC video laryngoscope [C-MAC VL, Karl Storz Endoskope, Tuttlingen, Germany], and Pentax Airwayscope [PAS]); and number of ETI attempts.

An ETI attempt was defined as insertion of the laryngoscope blade into the mouth, regardless of successful endotracheal tube placement into the trachea. Changing operators without removing the intubating device or endotracheal tube from the patient's mouth was also counted as an attempt. The monitoring staff strictly documented the number of attempts, because even a simple manipulation by the blade could harm the patients. Successful ETI was defined as proper endotracheal tube placement confirmed by various methods, such as ultrasonography, end-tidal carbon dioxide measurement, and auscultation. FAS was defined as successful ETI on the first attempt. An ETI case was defined as an ETI attempt by an individual trainee on a patient. In the case of multiple attempts on a single patient by two or more trainees, if the first trainee failed but the second trainee successfully intubated the same patient, it was counted as one case for each trainee. We classified the patient’s BMI into the following four categories according to the categories specified by the World Health Organization: underweight (< 18.5 kg/m2), normal weight (18.5–24.9 kg/m2), preobese (25.0–29.9 kg/m2), and obese (≥ 30.0 kg/m2)13.

Anticipated difficult airway was determined by an intubator, if one or more of the following characteristics were observed in the patient during the evaluation prior to ETI: external appearance such as a short neck, facial trauma, small mandible, or obesity; Mallampati class ≥ 3; airway obstruction, including airway edema or tracheal stenosis; cervical immobilization; and limited mouth opening of < 3 cm. Due to the lack of patient participation and time constraints, it might be challenging to apply complex criteria with precise measurements in an emergency situation. Thus, although some criteria were vulnerable to subjectivity, we employed a list of difficult airway features that were feasible for the intubator to assess prior to ETI using a quick and straightforward evaluation tool. The device was selected at the discretion of the intubator or the on-duty EM faculty.

We used an FAS rate of ≥ 85% as a proxy for ETI proficiency. We also evaluated the minimum number of ETI attempts required to achieve an FAS of over 80% and 90%, respectively, and a success rate of ≥ 90% within two ETI attempts.

Data analysis

Continuous data were presented as medians with interquartile ranges (IQR, 1st quartile–3rd quartile) and categorical data as frequencies with percentages. The variables were compared using either the Mann–Whitney U test or the chi-square test, as applicable. For univariable and multivariable analyses, a generalized linear mixed model (GLMM) with a random intercept and a logit link function was used to analyze the association between FAS and cumulative ETI cases and to establish the equation for predicted probability of FAS based on the cumulative number of ETI cases.

The coefficient and intercept of the cumulative number of ETI cases for FAS were estimated using GLMMs with a random intercept, and they were used to predict the probability of FAS. The specific predictor value was multiplied by the coefficient of the model corresponding to the cumulative number of ETI cases and then added to the intercept value of the model. The following formula was used to determine the predicted FAS probability:

$$\mathrm{Predicted} \, \mathrm{ probability } \, \mathrm{ of } \, \mathrm{ FAS }=\frac{{e}^{0.95321 + 0.006586*\mathrm{cumulative} \, \mathrm{ ETI } \, \mathrm{ case}}}{1+ {e}^{0.95321 + 0.006586*\mathrm{cumulative} \, \mathrm{ ETI } \, \mathrm{ case}}}$$

Using this prediction equation, the number of cumulative ETI cases in which the FAS rate is at least 85% was determined inversely.

Variables with a p-value < 0.1 from univariable analyses were selected for the multivariable model. The following variables were included in the multivariable model: cumulative number of ETI attempts, device used for ETI (DL vs. VL), indication for ETI (cardiac arrest vs. non-cardiac arrest), and difficult airway (difficult vs. non-difficult airway). The results were presented as adjusted odds ratios (aOR) with 95% CIs. A statistical significance was defined as a p-value < 0.05. Statistical analysis was performed using R 4.1.0 (Vienna, Austria; http://www.R-project.org/).

Results

All trainees had fewer than five previous ETI experiences, except for one who had < 10 experiences.

Baseline characteristics

Of the 3075 adult patients who underwent ETI during the research period, 1096 were excluded because they met the exclusion criteria, leaving 1979 patients for analysis. The evaluation of 1979 patients yielded a total of 2077 ETI cases (Fig. 1). The baseline characteristics of the patients are shown in Table 1. The median age was 67 years (IQR 55–77), and 62.8% of the patients were male (n = 1243/1979). A total of 25.5% (n = 504/1979) of the patients were classified as having a difficult airway. Cardiac arrest (37.2%, n = 737/1979) was the most common indication for ETI, followed by respiratory failure (30.4%, n = 602/1979) and airway protection (20.5%, n = 406/1979). The overall complication rate was 11.9% (n = 236/1979).

Figure 1
figure 1

Study flowchart. ETI endotracheal intubation, ED emergency department, EM emergency medicine, FAS, first attempt success. *An ETI case was defined as an ETI attempt by an individual trainee on a patient.

Full size image
Table 1 Baseline characteristics of the patients.
Full size table

EM trainees performed a median of 82.5 ETI cases (IQR 69–123). The ETI case-related variables are listed in Table 2 (See also Supplementary Table S1). C-MAC VL (61.3%, n = 1274/2077) was the most frequently used device in the first attempt at ETI, followed by DL (34.1%, n = 709/2077) and PAS (4.5%, n = 94/2077). Overall, the FAS rate was 78.6% (n = 1632/2077) and the success rate within two attempt was 85.4% (n = 1774/2077). The combination of devices used for the first and second ETI attempts, as well as the success rate for the second attempt, are described in Supplementary Table S1.

Table 2 Device used for first attempt and ETI success rate.
Full size table

Relationship between FAS and cumulative intubation experience

Figure 2 shows the relationship between the cumulative number of ETI cases and predicted probability of FAS, showing an increase in the predicted FAS probability with cumulative number of ETI cases. At least 119 ETI cases were needed to achieve an FAS probability of ≥ 85%. In addition, at least 66 and 189 cumulative ETI cases were required to achieve FAS probabilities of 80% and 90%, respectively. The probabilities of FAS based on intubating device, indication for ETI, and the presence of difficult airway characteristics are presented in Supplementary Fig. S1. At least 88 cumulative cases were required to achieve a success rate of > 90% within two ETI attempts (see Supplementary Fig. S2).

Figure 2
figure 2

Relation between the cumulative number of ETI cases and predicted probability of first attempt success. Generalized linear mixed models with random intercept using the logit link function were used to estimate the probability of FAS according to the cumulative number of ETI cases. FAS first attempt success, ETI endotracheal intubation.

Full size image

The results of the univariable and multivariable analyses to identify the association between FAS and cumulative intubation experience are presented in Table 3 (See also Supplementary Tables S2–S4). The cumulative number of ETI cases was associated with increased FAS (odds ratio, 1.007 per additional ETI case; 95% CI 1.007–1.010; p < 0.001). This association remained significant after adjusting for ETI device, indication for ETI, and presence of difficult airway (aOR 1.010 per additional ETI case; 95% CI 1.006–1.013; p < 0.001). Compared with DL use, VL use was significantly associated with increased FAS (aOR 1.890; 95% CI 1.506–2.370; p < 0.001). Conversely, ETI in cardiac arrest (vs. non-cardiac arrest; aOR 0.677; 95% CI 0.538–0.852; p < 0.001) and the presence of a difficult airway (vs. non-difficult airway; aOR 0.324; 95% CI 0.254–0.412; p < 0.001) were significantly associated with decreased FAS.

Table 3 Univariable and multivariable analyses to identify the association between FAS and cumulative ETI cases.
Full size table

Post-hoc power analysis

Power analyses were performed using a coefficient value that was less than the estimated regression coefficient to explore whether the sample size used in our retrospective analysis was sufficient or not14. The power to detect a regression coefficient (e.g., 0.005) smaller than the estimated regression coefficient (= 0.006586) of the cumulative number of ETI cases in the prediction model of FAS was calculated over a range of sample sizes using 1000 Monte Carlo simulations (see Supplementary Fig. S3). The range of possible values for the x-axis, representing the cumulative number of ETI cases, ranged from 50 (n = 1006) to 204 (n = 2077). The power of a minimum cumulative number of ETI cases of 119, which was necessary to achieve an FAS probability of 85%, was 83%.

Discussion

Several studies have been performed to determine the ETI learning curve, with varying results, presumably owing to the differences in the statistical methodologies, expertise of the study subjects, devices used, study setting, and definition of proficiency. In this study, we sought to define the learning curve of ETI for EM trainees in the ED. The cumulative number of ETI cases was significantly associated with an increased FAS in the multivariable model. A minimum of 119 ETI cases were required for EM trainees to achieve an FAS probability of ≥ 85% in our study. We believe that our study is clinically relevant. As ETI has the potential to save lives, EM physicians are expected to be skilled in it. Thus, evidence quantifying the number of ETI cases required to achieve a level of proficiency may serve as a better guide for EM trainee education.

The ACGME has deemed ETI an essential key index procedure for the independent practice of EM, mandating that EM residents perform at least 35 ETIs during their training1. According to the anesthetic literature, between 51 and 75 ETIs are necessary to obtain an overall success rate of ≥ 90% when utilizing a DL9. A previous meta-analysis on the learning curve of ETI employing a DL found that a minimum of 50 ETIs in an elective setting is required to obtain a success rate of ≥ 90% within two ETI attempts10. According to a study conducted by Je et al. for EM residents in the ED, a 90% success rate was not achieved despite having performed 114 ETI procedures2. In our study, EM trainees were required to perform a minimum of 119 ETI cases to achieve a FAS probability of ≥ 85%. Even if the criteria for proficiency were based on a FAS rate of ≥ 80% or a success rate of ≥ 90% within two attempts, at least 66 and 88 cumulative ETI cases, respectively, would be necessary. Considering evidence from previous literatures as well as our study findings, the 35 ETI cases may not be sufficient for establishing the proficiency of EM trainees. ETI in the ED is typically challenging due to the uncontrolled environment, critically-ill patients with compromised physiology, and frequently encountered difficult ETIs, leading to a variety of life-threatening complications. Thus, additional training for EM trainees might be necessary to be proficient in ETI.

Our study found that managing a substantial number of cases is necessary to develop proficiency in ETI. One possible explanation is that we used an FAS rate of ≥ 85%, which might be stricter compared to the criteria in other studies, as a proxy for ETI proficiency. The more difficult it was to achieve this level of expertise, the more experience was necessary. Second, it has been well established that incidences of a difficult or failed ETI are much higher in emergent settings than in elective settings. Patients with difficult airway characteristics and those who had suffered a cardiac arrest, both of which might have a detrimental effect on the FAS rate, accounted for 25.5% and 37.2% of the patients in our study, respectively. Kim et al. reported that 137 and 243 ETI cases were necessary to achieve a 90% success rate without complications within 60 and 30 s, respectively, during cardiopulmonary resuscitation3. Finally, the approach used to collect the data was different from that used in other studies. Our ETI attempts and successes were captured in real time by dedicated observers, ensuring objectivity of the data12. Studies that rely on recalled or self-reported data may be prone to bias in ETI attempt numbers and success rates.

In this study, we used a FAS rate of ≥ 85% as an indicator of proficiency. To date, there has been no consensus over the definition of ETI proficiency. Previous research evaluating the learning curve for ETI has employed various definitions of competency, such as a success rate of at least 90% in less than 30 s or a success rate of ≥ 90% within two attempts4,10. Current practice now focuses, as a priority, on successful ETI on the first attempt, as several studies have shown that increased ETI attempts are associated with a higher risk of adverse events (AEs). According to Sackles et al.’s study that analyzed 1828 ETI in the ED, the incidence of AEs was 14.2% in patients with FAS, but it increased to 47.2% and then to 63.6% as the number of attempts increased, reaching 70.6% in cases with more than four attempts15. In a study involving non-traumatic out-of-hospital cardiac arrest patients, Murphy et al. also showed that the likelihood of achieving a favorable neurologic outcome decreased with each successive attempt: 11% with one attempt, 4% with two attempts, 3% with three attempts, and 2% with four or more attempts16. These data suggest that the goal of emergency ETI should be to maximize the FAS, emphasizing the importance of future educational initiatives aimed at increasing the FAS rate. Although FAS is essential for patient safety, there are no universally accepted specific values for measuring proficiency. Several studies conducted in the ED have indicated that the FAS rate is often around 80–90%11,17,18,19,20,21,22. Park et al. conducted a systematic meta-analysis on data collected from 42,081 patients intubated in the EDs of 83 institutions located across ten counties11. This data showed that the FAS rate in the ED was 84.1% (95% CI 80.1–87.4]). Based on the findings of previous studies, as well as the importance of the FAS for patient safety, it may be reasonable to establish a FAS target of ≥ 85% for EM trainees; hence, this criterion was selected as the primary outcome of this study.

Simulation-based training for ETI enables guided experiences in safe environments, thereby possibly facilitating the attainment of the level of proficiency required to safely operate in an actual clinical setting. Despite the fact that studies on the effectiveness of simulation-based training have yielded varying degrees of success based on study methodology, subjects, and outcomes of interest, several studies have shown favorable findings for the efficacy of simulation-based training. Mosier et al. implemented a 3-year training program including an intensive simulation-based curriculum that emphasized the recognition and preparation for potentially difficult ETI cases and procedural skills23. This curriculum increased the FAS rate from 74 to 82% (P = 0.006) and reduced the occurrence of complications, such as desaturation rates, from 26 to 17% (P = 0.002). Our study did not assess the effect of simulation-based training. However, because EM physicians need to manage airways in a variety of scenarios, continuous and comprehensive simulation-based training that includes other components of airway management, such as preparation and patient evaluation, in addition to training to acquire the technical skills of ETI, may be beneficial in reducing the number of ETI required to achieve proficiency in ETI.

There are several limitations in our study, which should be noted when interpreting the results. First, because this study was conducted in a single academic ED, its findings may not be generalizable to other settings. In particular, because the majority of ETIs were performed using conventional DL and C-MAC VL, there may be limits to extrapolating our findings when alternative devices are employed. Second, this was an observational retrospective study conducted using an institutional registry. As a result, the number of ETIs could not be controlled for each trainee, resulting in a wide range of ETI cases per trainee. Additionally, selection bias may exist in the choice of airway device based on the intubator’s preference or the presence of difficult airway characteristics. It was challenging to design a study in which airway equipment was randomly assigned for research purposes in clinical practice. It is reasonable for the intubator to choose the device that most effectively secures the airway in each scenario. Third, we analyzed only ETIs conducted in the ED, but not those performed during rotations in other departments, such as the intensive care unit. In addition, we did not consider prior ETI experiences before starting the residency program. However, this is likely to have minimal effect because the majority of trainees did not have prior ETI experience. Taken together, the number of ETI cases for trainees in this study was equivalent to or less than the actual number of ETIs performed. Thus, our findings may have been influenced by the uncounted ETI cases. Fourth, the duration of the ETI attempt, another relevant indicator for determining proficiencies, could not be presented because the airway registry lack information on the attempt duration for some patients. Fifth, repeated laryngoscope insertion may increase the incidence of airway obstruction and reduce the success rate of subsequent attempts. Consequently, following the failure of the initial ETI, successive intubations would have been performed under more challenging conditions by other EM trainee. However, only a small number of patients underwent ETI by two or more EM trainees; thus, the impact of this limitation might not be significant. Sixth, the evaluation of difficult airway characteristics was highly susceptible to subjectivity and interobserver variability, especially the assessment of “external appearance,” which includes ambiguous phrases like “short neck” and “small mandible”. Seventh, obesity, a well-known risk factor for ETI failure, was not statistically significant for FAS in our study. This finding may be attributed to a decrease in statistical power resulting from our study's relatively small sample size of obese patients. Finally, we documented all ETI-related complications that occurred during the procedure. However, these complications could not be linked to the individual intubator.

In conclusion, the cumulative number of ETI cases was associated with an increased FAS rate. To achieve an FAS rate of ≥ 85%, at least 119 ETI cases were required for EM trainees in the ED. These findings may have implications for the minimum training number of ETI cases required for EM training.