Pulmonary aspiration of gastric content is considered a major complication in anesthesia practice with resulting morbidity and mortality.1-4 Certain factors have been linked to the severity of patient outcomes, including the volume, nature, and acidity of the aspirate.5-8 The preoperative assessment of the risk of pulmonary aspiration relies essentially on the patient’s history, and the clinical management typically adheres to fasting recommendations of current guidelines.9 The ultimate assessment of the nature and volume of gastric content at bedside remains inaccessible to the anesthesiologist.

Recent studies have shown that bedside ultrasonography can provide reliable information about the nature (clear fluid, solid, or none) and volume of gastric content.10-14 In these studies, either a certified sonographer or a single anesthesiologist with no specified previous training performed all the examinations.11-13,15-17 The process of knowledge translation demands further research to confirm generalizability of these findings. The amount of training required to achieve competence in the performance of gastric ultrasonography as a bedside clinical tool remains unknown. The aim of this study was to determine the amount of training an anesthesiologist would need to achieve competence in the bedside ultrasound technique for qualitative assessment of gastric content in healthy volunteers.

Methods

Following approval by the Research Ethics Board (ref: 11-0237-E) of Mount Sinai Hospital (Toronto, Canada) on October 14, 2011, we conducted this observational cohort study of a group of anesthesiologists learning qualitative ultrasound assessment of gastric content in healthy volunteers along with the reference standard of an expert sonographer. The study was carried out from October 2011 to December 2011. Written informed consent was obtained from all participants (anesthesiologists, volunteers, and expert sonographer). We followed the guidelines for reporting and the applicable checklist from the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) Statement.18

Eligible participants were advanced level trainees (anesthesia fellows during one year of subspecialty training) or staff anesthesiologists. Participants had previous experience in ultrasound-assisted/guided procedures, but not in gastric ultrasound.

Inclusion criteria for volunteers were: age 18 yr or older, weight 50-120 kg, and height 150 cm or greater. Volunteers with a pre-existing abnormal anatomy of the upper gastrointestinal tract (previous esophageal, gastric, or upper abdomen surgery and hiatus hernia) were excluded. A certified sonographer experienced in diagnostic abdominal ultrasonography with more than 500 examinations of gastric content was considered the reference standard for the ultrasonographic assessments. The cohort of anesthesiologists underwent a teaching intervention which was followed by a formative assessment process in order to construct learning curves.

Teaching intervention

Initially, the cohort of anesthesiologists received didactic teaching in the form of reading material, educational video/picture library, and a 30-min lecture on the topic. The reading material included research studies11-14 and one educational article with a description of the technique.19 Subsequently, the teaching proceeded with an interactive three-hour session consisting of a hands-on workshop and a live demonstration performed by the expert sonographer. Five volunteers were scanned after fasting for at least eight hours, and they were later scanned after ingesting either clear fluids (300 mL of apple juice) or a standardized solid meal (muffin and apple juice). Each anesthesiologist received one-on-one feedback on ten exams that included every prandial status (fasted, clear fluids, and solid meal). During the workshop, the anesthesiologists were instructed on how to perform a systematic ultrasound assessment in order to make a qualitative diagnosis of three distinct gastric content categories (empty, clear fluid, solid).

The standardized technique was performed with a portable ultrasound system equipped with a 5-2 MHz curved array transducer (M-Turbo™, SonoSite® Canada Inc., Markham, ON, Canada). Volunteers were first placed in a 45° semi-recumbent position and then in a right lateral decubitus position. In either of these positions, fluid or semi-fluid content gravitates preferentially to the antrum, and air or gas is displaced proximally towards the body or fundus, thus facilitating antral sonography.11 Both positions are part of the same continuous diagnostic process, and we did not consider them two different diagnostic modalities (Fig. 1). A detailed description of the ultrasound technique and ultrasonography characteristics of gastric antrum content has been reported recently.19

Fig. 1
figure 1

Sonographic images of the epigastric area (parasagittal plane). L = liver; A = antrum; P = pancreas; IVC = inferior vena cava; Ao = aorta. (a) Empty gastric antrum. The arrowheads denote an empty gastric antrum. The “bull’s eye” appearance is typical if the antrum is contracted. (b) Gastric antrum after 300 mL of oral fluid intake. The “starry night appearance” is caused by air mixed with fluid. (c) Gastric antrum with a “frosted glass appearance” secondary to air after recent ingestion of a solid meal. (d) Gastric antrum with heterogeneous granular appearance of ongoing digestion

Full size image

Assessment process

The formative assessment process consisted of a series of ultrasound examinations performed by each anesthesiologist followed by confirmation and feedback from the expert sonographer. We scheduled six sessions with five volunteers per session, with 30 separate sets of data per anesthesiologist. For each evaluation session, five study volunteers were randomly allocated to one prandial status similar to that previously described in the hands-on workshop (eight-hour fasting, clear fluids, and solid meal). Randomization allocation was in two blocks of 15. Each block of 15 included the three gastric content categories: five empty, five clear fluids, and five solid. The allocation concealment comprised of sequentially numbered and sealed opaque envelopes prepared by an independent research assistant. At each session, the ultrasound examinations were started five minutes after the ingestion of liquid or solid, and the assessment session was paused every 15 min to provide a top-up of clear fluids (100 mL) to the fluid group, which was also blinded. The purpose of the top-up was to attempt to maintain equal examination conditions due to possible rapid gastric emptying of clear fluids in a fasted volunteer. The expert sonographer and the anesthesiologists were blinded to the volunteer prandial status, and they examined all five volunteers in a random sequence. Additionally, each of the six sessions was scheduled one week apart (six-week assessment period), and each volunteer was scanned only once per session by each anesthesiologist. The prandial status was disclosed only when the examination of the five volunteers was completed by all anesthesiologists and the sonographer. The expert sonographer then conducted a 50-min feedback session, including review of images and discussion and reinforcement of the technical aspects. In order to maintain uniform learning progress for the entire group, we did not allow participants to practice the ultrasound technique between sessions; however, they were encouraged to review the didactic teaching material.

Learning curves

The aim of the qualitative ultrasound examination was to diagnose three distinct gastric content categories (empty, clear fluid, or solid). Success was defined as a correct diagnosis, which was recorded over time. Although the diagnosis from the expert sonographer was the reference standard by study design, we also followed her performance, as she was also blinded to the volunteers’ prandial status. In this way, we were able to show the construct validity of the technique. We aimed to construct learning curves for a series of ultrasound examinations through the cumulative sum method (CUSUM).20

Study outcomes

The primary outcome was the number of ultrasound examinations (cases) an anesthesiologist required to achieve competence in the ultrasonography qualitative assessment of gastric content. Competence was defined as 90% success rate in a series of ultrasound examinations.20-22 The secondary outcomes included the overall success rate of correct ultrasound diagnoses, the specific success rate per gastric content category, and characteristics of the series of ultrasound examinations.

Statistical analysis

In order to estimate the primary outcome for each anesthesiologist, individual learning curves were constructed using the CUSUM graphical method (Appendix 1). The CUSUM graphs display the cumulative differences plotted in sequence, allowing detection of deviations from a predetermined standard.20-26 A negative trend of the individual CUSUM line indicates success, whereas a positive trend indicates failure. We considered the anesthesiologist competent at a 10% failure rate if the graph passed through two decision lines from above. In this way, early failures did not penalize the anesthesiologist for later increases in accuracy.25,26

We calculated a minimum sample size of 13-18 consecutive cases as necessary to determine competence at an acceptable failure rate of 10% and an unacceptable failure rate of 30%, respectively (Appendix 1). We increased the sample to 30 consecutive cases per anesthesiologist (twice the average of calculated cases) to allow an adequate assessment process.

Furthermore, we used the entire cohort data in the construction of a predicted learning curve based on Bush and Mosteller’s mathematical learning model, which estimates the average number of cases to achieve a 95% success rate (Appendix 2).23,27 In this way, we were able to determine an average CUSUM score and construct a learning curve for the average of all participants, allowing interpretations for the entire cohort rather than for each individual alone.

The assessments were also compared with respect to a correct or incorrect diagnosis. We examined the effect of the gastric content category on the correct ultrasound diagnosis. For the comparisons, we used the generalized regression models with a generalized estimating equation approach accounting for the correlated data. Further evaluation of the effect of the gastric content category and the duration of the ultrasound examination was conducted using multiple logistic regression models for correlated data.

Descriptive statistical methods were used to describe the study population. The statistical analyses were performed using SAS® 9.2 (SAS Institute Inc., Cary, NC, USA), R 10.2 (http://www.r-project.org/), and STATA® for Macintosh, Release 12.1 (StataCorp, College Station, TX, USA). A two-sided significance level of < 0.05 was used without multiple comparison adjustment.

Results

The cohort of anesthesiologists consisted of four anesthesia fellows and two anesthesia staff. All participants completed the teaching and evaluation sessions. The group of healthy male volunteers had a mean (standard deviation = SD) age of 25 (2) yr, weight of 71 (12) kg, height of 173 (4) cm, and body mass index of 25 (3) kg·m−2.

Learning curves

Each of the six anesthesiologists performed 30 ultrasound examinations for a total of 180 assessments, and the blinded sonographer made the correct diagnosis in every case. The analysis of learning curves through CUSUM showed that three of the six anesthesiologists (two staff and one anesthesia fellow) achieved competence (90% success rate) with a median number of 24 cases (15, 24, and 28 cases) (Fig. 2). The average CUSUM curves were obtained for those who achieved and those who did not achieve competence in the series of 30 cases (Fig. 3A). The Bush and Mosteller’s learning model predicted 33 as the average number of cases required to achieve a 95% success rate (Fig. 3B).

Fig. 2
figure 2

Learning curves by CUSUM (cumulative sum) graphs. Connected plotted lines represent learning curves for individual anesthesiologists. Upward and downward trend indicates failure and success, respectively. Horizontal red dashed lines represent decision limits. (a) Learning curves of all anesthesiologists. (b) Learning curves of anesthesiologists achieving competence

Full size image
Fig. 3
figure 3

(a) Average CUSUM (cumulative sum) learning curves for those achieving competence (red cross, connected-plotted line) and those not achieving competence (green triangle, connected-plotted line). (b) The observed success rates (red connected line) and the predicted success rates (green line) at each case using the Bush and Mosteller’s learning model. The predicted 95% (blue dotted line) success rate is attained after an average of 33 cases

Full size image

Secondary outcomes

The overall success rate (correct diagnosis) was 79% (142/180) and the failure rate (incorrect diagnosis) was 21% (38/180). The relative proportions of the gastric content categories in the groups of correct diagnosis (success) and incorrect diagnosis (failure) are presented in Table 1. When comparing the three gastric content categories (Table 2), the “clear fluid” group appeared with a significantly lower failure rate than “empty” (P < 0.05), while the “solid” group showed a lower failure rate compared with both “empty” and “clear fluid”. The duration of the examination for the “clear fluid” was shorter than for the “empty” and “solid” groups (P < 0.05).

Table 1 Incidence of correct (Success) and incorrect diagnosis (Failure) according to the gastric content category and the duration of the ultrasound examination
Full size table
Table 2 Diagnostic failure among the three gastric content categories and duration of the ultrasound examination
Full size table

Furthermore, when comparing success (correct diagnosis) using multivariable analyses, the odds of success for “solid” was 2.6 times that for “empty”, and 1.7 times that for “clear fluid”. Finally, the odds of success decreased for any one-minute increase in the ultrasound examination (Table 3).

Table 3 Multivariable analysis: odds of success according to gastric content categories
Full size table

Discussion

This study determined the learning curves for the qualitative ultrasonography assessment of gastric contents performed by a cohort of anesthesiologists. Under the study conditions, CUSUM analysis revealed that 24 consecutive ultrasound examinations were required to achieve competence with a 90% success rate, and at least 33 examinations would be required for a 95% success rate based on a predicted model. The fact that the expert sonographer correctly diagnosed the gastric content in every volunteer confirms the construct validity of the technique. Nevertheless, according to CUSUM analysis, only three of six novice participants achieved competence in the series of 30 examinations. This may indicate a medium level of complexity in learning the technical and cognitive skills necessary to perform this ultrasound diagnostic procedure.

The CUSUM method has been used in many studies investigating the acquisition of competence or assessment of performance in anesthesia technical skills,22,25,28-31 and it has been used more recently in ultrasound-guided procedures.21,23,24,26,32 It can be used to show proficiency in a newly learned technical skill or measure quality once a technical skill has been mastered.33 In the context of medical education, it is crucial to understand CUSUM as a statistical method that looks at the outcome rather than the process of performing procedural skills.20,28 Although we focused on the formative assessment (training, feedback, discussion) aspect in our study, achieving and declaring competence still reflects the goal of summative assessment.34-36 Defining competence is a complex task. Acceptable rates of success can be determined by institutional rates or expert consensus, but they also depend on the definition of success.20,30 Currently, there is no standard for this new diagnostic skill in the anesthesia setting. In some studies on newly acquired procedural technical skills in anesthesia, 90% success rates have been used based on previous training studies21 or departmental consensus using a modified Delphi approach.22 Therefore, for this study, we decided to define competence as a 90% success rate as determined by consensus of opinion among the investigators who had performed previous research on ultrasound assessment of gastric content.

If we had followed the method of pooled cumulative success rate described by Kopacz et al.,37 all participants could have been assessed by their actual overall success rate, which was greater than 70% for all six anesthesiologists and corresponds to the predefined cut-off point of 30% unacceptable failure rate. Nevertheless, the CUSUM method tends to benefit consistency over a sequence of cases, and its value comes from providing statistical inference while testing the failure rate against a minimum standard. Furthermore, if we assume that all anesthesiologists had the same baseline experience, we can use the summated changing success rates to develop a learning curve for the average of all participants until a certain target rate is achieved in this outcome-based task. In this way, the predicted model facilitates making interpretations for a population rather than for each individual alone, providing potential application for curriculum development, practice rotations, and mentor evaluations.38

We planned a study protocol with very distinct gastric content categories to enhance the early stages of learning. Among the three gastric content categories, study results suggest that “solid” content appears to be more easily diagnosed by novice sonographers, followed by “clear fluid”, and then the “empty” category. Additionally, the “clear fluid” is more readily appreciated. We managed to optimize the clear fluid content group in order to avoid the possible confounding factor of rapid gastric emptying. The antrum filled with clear fluid is more obvious to the observer, appearing round and filled with hypoechoic/anechoic content (Fig. 1B). Nevertheless, the “solid” content group appeared to have been diagnosed more easily than both the “clear fluid” and the “empty” groups. This may be related to the relatively larger size of the antrum and the characteristic hyperechoic mucosal-air interface in cases of recent ingestion of solid food that depicts a “frosted glass” appearance, impairing visualization of deeper structures (Fig. 1C).19 Further research is warranted to determine the particular challenges and difficulty in the diagnosis of various gastric contents.

Our study has some limitations. First, the quantitative dimension (volume assessment) was beyond the scope of this study. This aspect of the ultrasound assessment is not required in the diagnosis of “empty stomach”. Nevertheless, if the gastric sonogram should show clear fluid content, a quantitative volume assessment can help differentiate a low-volume status (similar to baseline physiologic gastric content) from a high-volume status (higher than baseline gastric volume).11-13 We concentrated instead on using a focused qualitative exam as an approach to a screening test. Second, although we assessed quite distinct categories of gastric content and performed the ultrasound assessments at fairly constant time points, scanning conditions for the individual participants could have varied during the course of the session. Third, our five volunteers were healthy young males with a normal body mass index; while we believe that this was essential for reliable and valid training, results may not be fully applicable to other populations. Although we had a limited number of volunteers and bias was a concern, we planned to minimize such bias by carrying out each of the six sessions one week apart (six-week assessment period), and each volunteer was scanned only once per session by each anesthesiologist. Fourth, some of the predefined statistical variables of CUSUM in the study design may have limited participants’ achievement of competence. Specifically, the learning curves were defined by the level to declare competence, the number of participants, and the individual attempts per participant. Finally, this cohort of anesthesiologists had previous experience in other applications of ultrasound for diagnostic and interventional procedures in anesthesia. This baseline condition could have facilitated the development of their learning curves and could also make conclusions more suitable for this particular population.

The use of ultrasound by anesthesiologists as a perioperative point-of-care resource has dramatically increased over the last ten years, and many current best practices entail ultrasound to guide clinical decision-making and procedures. A growing body of evidence shows the benefits of this change in practice.39 In this context, bedside ultrasound could potentially become a clinically useful noninvasive tool for accurate determination of gastric content and volume. It could help to establish the risk of perioperative aspiration more precisely, both in a given individual and in different patient populations.9,12,14,40 Practice guidelines are subject to revision as warranted by the evolution of medical knowledge, technology, and practice.9,41 Factors such as current literature, expert and practitioner opinion, open forum commentary, and clinical feasibility data support these recommendations. Presently, it may be early and premature to anticipate how this new technology could affect practice, although we believe it is a promising clinical skill to pursue.

In conclusion, we have described the feasibility of appropriate training of anesthesiologists to reach a level of competence in performing bedside qualitative ultrasound assessment of gastric content. Individual learning curves and a predictive model suggest that 24 and 33 are the average numbers of cases required to achieve 90% and 95% success rates, respectively.