1 Introduction

Patient-ventilator asynchrony (PVA) is a well-recognized problem in mechanically ventilated patients in the intensive care unit. Recent studies have shown that severe asynchrony (i.e., asynchrony index > 10%) is associated with higher mortality [1, 2]. Ineffective triggering (i.e., inspiratory efforts during expiration that fail to trigger ventilator-assisted breaths) is a common type of PVA.

Many studies to quantify PVA depend on manual and time-consuming annotations of ventilation waveforms at the bedside. Furthermore, the sensitivity of manual detection of asynchrony based on visual analysis of ventilator waveforms has been reported to be extremely low (16–28%) on a breath-by-breath basis [3]. There is an opportunity for highly sensitive real-time automated PVA detection systems to enable effective patient monitoring and respiratory management for patients undergoing mechanical ventilation.

In this paper, we compare the sensitivity and specificity of clinician visual waveform analysis against an automated system’s waveform analysis in detecting ineffective triggering when compared against a reference label set based upon analysis of electrical activity of the diaphragm (EAdi) or esophageal (Pes) or transdiaphragmatic pressure (Pdi).

2 Materials and methods

2.1 Subjects

Ventilation data of seven adult patients were selected from a database storing patient data from several studies, all approved by the local ethical committee and conducted in the last 15 years in the surgical intensive care unit of Policlinico Gemelli of Rome, Italy. For each study, informed consent was obtained from all patients. For the purpose of this study, we selected patient tracings with high incidence of ineffective triggering.

2.2 Ventilation data

All patients were ventilated in pressure support ventilation (PSV) mode with a Puritan Bennett 840 (Medtronic, Dublin, Ireland) or a Servo-i (Getinge, Solna, Sweden) ventilator to obtain a tidal volume of 7–8 mL/kg and a positive end expiratory pressure (PEEP) value set to obtain peripheral oxygen saturation above 92% with fraction of inspired oxygen (FiO2) between 0.35 and 0.5. Airway flow (Qaw) was measured at the Y-piece of the ventilator circuit with a pneumotachograph (Fleish No. 2, Metabo, Epalinges, Switzerland). Pressure at the airway opening (Paw) was measured distally to the pneumotachograph with a differential pressure transducer (Digima Clic-1, KleisTEK, ICU-Lab System, Bari, Italy). Esophageal and gastric pressures were measured with balloon-tipped catheters. All signals were sampled at 200 Hz and recorded on a dedicated personal computer. EAdi signal was acquired through a RS232 interface (sampling rate 100 Hz) and recorded with a dedicated software (NAVA Tracker V. 2.0, Maquet, Solna, Sweden). The data obtained were further analyzed by using a specific function of the ICU Lab 2.3 (Nava Merger, ICU Lab 2.3, KleisTEK Advanced Electronic System, Italy).

2.3 Ineffective triggering classification

EAdi or Pes or Pdi waveforms were available to a single clinician for the generation of a reference label set indicating the ground truth, that is, presence or absence of ineffective triggering, on a breath-by-breath basis. The presence of an ineffective trigger was defined as a patient effort not followed by a mechanical breath during exhalation. It can be identified as a positive swing in EAdi or Pdi from their baseline to their peak, or a negative swing in Pes from the baseline to its nadir, which fails to trigger the mechanical support as previously described [4, 5]. A breath cycle is labeled as having infective triggering if at least one ineffective triggering is detected.

Pressure and flow versus time tracings were made available to (i) a group of three clinicians with 7–21 years of experience as a critical care respiratory therapist; and (ii) the automated Syncron-E™ system capable of detecting PVA in real-time, in order to obtain breath-by-breath labels to detect the presence or absence of ineffective triggering. The clinicians and the automated system (used for clinical investigation) did not have access to other waveforms such as Pes, Pdi or EAdi. The final breath-by-breath labels from the group of clinicians were combined by a majority vote for the presence or absence of ineffective triggering in each breath.

Paw and Qaw waveforms were analyzed in real-time by the automated system, where the tracings were played back in real-time using a computer. Various aspects of the automated PVA detection framework were reported previously [6]. Briefly, the automated system generates an internal signal referred to as the delta waveform defined as the synchronous pairwise differences between normalized Paw and Qaw waveforms, thereby effectively reducing the dimensionality to a single channel for its pattern recognition algorithms.

2.4 Statistical analysis

The detection of ineffective triggering by clinicians and the automated system were compared against the reference label set from the independent clinician who had access to further invasive measurements (i.e., EAdi, Pes, or Pdi). Related-samples Cochran’s Q-test was used to determine if there were statistically significant differences in the proportions of ineffective triggering detections between the three approaches (i.e., independent clinician with invasive measurements, group of clinicians, and automated system). This omnibus test was used to first establish whether the three approaches were equally effective without the inflation of Type I error that occurs with multiple comparison testing. If the null-hypothesis of the Q-test was rejected, post hoc McNemar pairwise comparisons were made with continuity corrections and the Bonferroni correction for multiple tests applied, as appropriate. Cohen’s Kappa coefficients (K) were used to assess the magnitude of agreement in detections of ineffective triggering. Sensitivity and specificity were calculated to summarize detection performances. Statistical significance was defined as α < 0.05.

3 Results

In total, 926 breaths were analyzed across the seven patients. Each patient contributed 3.7–10.5 min of continuous ventilation recordings (total duration of 42 min). The prevalence of ineffective triggering in the reference label set was 10.3% (95/926), which is comparable with the prevalence reported in a previous study [7]. Specificity for clinicians and the automated system were high (see Table 1). The automated system had a significantly higher sensitivity (83.2%; true positives = 79, true negatives = 819, false positives = 12, false negatives = 16) compared to clinicians (41.1%; true positives = 39, true negatives = 825, false positives = 6, false negatives = 56).

Table 1 Comparison of performance between clinicians and the automated system compared against a common reference (EAdi/Pes/Pdi analysis)
Full size table

Cochran’s Q-test determined a statistically significant difference in proportions of breaths with ineffective triggering detected between the clinicians, automated system, and reference label set generated by the independent clinician, χ2(2) = 64.33, p < 0.001. Exact post hoc McNemar’s test determined statistically significant differences (with Bonferroni corrections) in proportions detected for clinicians vs. reference label (χ2 = 38.726, p-adjusted < 0.001) and clinicians vs. automated system (χ2 = 37.5, p-adjusted < 0.001), and no significant differences for automated system vs. reference labels (χ2 = 0.321, p-adjusted = 1.000).

Cohen’s K showed moderate agreements between clinicians and reference labels (K = 0.53, p < 0.001) and clinicians and automated system (K = 0.58, p < 0.001), and substantial agreement between the automated system and reference labels (K = 0.83, p < 0.001).

4 Discussion

In this paper, we demonstrate that ineffective triggering detected by the automated Syncron-E™ system, which has access only to airway pressure and flow versus time tracings, is in substantial agreement with a reference detection derived from analysis of invasively measured patient effort waveforms. Conversely, clinicians with access only to the same pressure- and flow-versus-time recordings as the automated system had a significantly lower sensitivity and only a moderate agreement with the detected ineffective efforts in the reference set.

This is consistent with previous reports [8,9,10] that computerized algorithms can automate the detection of ineffective triggering with acceptable sensitivity and specificity. However, this study goes further by presenting the performance of the automated system in the context of a direct comparison with clinicians, where both the automated system and clinicians had access only to the same pressure- and flow-versus-time tracings. Specifically, it was demonstrated that although specificity is high and comparable between the automated system and clinicians when benchmarked against a common reference (derived from analysis of invasive recordings), the automated system is twice as sensitive in detecting such events. Improving sensitivity without jeopardizing specificity is particularly important to minimize “alarm fatigue” that may occur due to false positives. Unnecessarily high false positive rates may desensitize clinicians to alarms, thereby decreasing the value of a highly sensitive automated system for monitoring ventilated patients. Most importantly, the automated system employs a framework that does not rely on external input making it suitable for a real-time waveform analysis system.

The authors in [7] use one dataset to compare their algorithm against clinician waveform analysis and a different dataset to compare their algorithm against diaphragm activity. In addition, their presented framework is not suitable for real-time analysis as it relies on external input to define the profile of a “normal” expiratory flow curve. The use of different data for the comparisons preclude a clear demonstration of potential performance benefits of the algorithm over clinicians. Other reports of automated ineffective triggering detection systems also lack direct comparisons with clinician waveform analysis to demonstrate a clear performance benefit [8,9,10].

In another study, the authors reported low prevalence of ineffective triggering in their sample of patients and induced artificial ineffective triggering in each patient by ventilating in pressure control ventilation mode with an intentionally low ventilation rate [11]. These artificially “exaggerated” ineffective triggers may have contributed importantly to the high performance of their system as the authors later discussed in a follow-up report [8], thereby making it difficult to generalize their reported performance on naturally occurring ineffective triggers. The performance of Syncron-E™ in this report is based upon naturally occurring ineffective triggers that include many mild disturbances to the flow waveform (see Fig. 1) and reports a twofold increase in sensitivity compared to a panel of three clinicians. This marked improvement in sensitivity addresses the reported low sensitivity (16–28%) but high specificity (88–93%) of breath-by-breath visual waveform analysis by clinicians [3].

Fig. 1
figure 1

Typical example of a clinician-adjudicated case of a mild ineffective triggering showing the Syncron-E™ system’s internally calculated delta waveform (top), airway pressure versus time tracing (middle), and airway flow versus time tracing (bottom). The shaded grey region of the delta waveform, which captures simultaneous perturbations to pressure and flow as a result of ineffective triggering, is a pattern for which the automated system searches to detect occurrence of ineffective triggering

Full size image

Our study should be interpreted in the context of its limitations. The present study was conducted on a limited sample of a selected group of patients receiving PSV mode of ventilation. Future studies should validate the automated system’s ineffective triggering detection on a wider range of ventilation modes including time-cycled ventilation (e.g., pressure control ventilation (PCV) and volume control ventilation (VCV)). We note, however, that since the occurrence of ineffective triggering follows from spontaneous activity of a patient, PSV as a mode indicated for patients with spontaneous efforts is a particularly important mode to validate the system for this particular type of asynchrony; time-cycled ventilation modes may be more commonly used for passive patients without significant spontaneous efforts which may result in fewer incidences of ineffective triggering. Indeed it was reported in [1] that ineffective efforts during expiration occurred significantly more often in PSV than in the time-cycled modes (PCV and VCV).

The present study’s limited sample size of seven patients is another limitation. A larger and more clinically diverse sample of patients should be studied to further validate the Syncron-E™ system’s performance to cover the significant variabilities of breathing patterns that exist in practice. Such studies may also help establish the most appropriate indications for use of the automated system. Finally, the reference label set was generated by a single clinician with access to EAdi or Pes or Pdi waveforms. Although the clinician in this study had extensive experience in the interpretation of such waveforms (e.g., see [5]), involving multiple clinicians in the generation of the reference label set and its effect on the results should be investigated in a future study.

5 Conclusion

This study demonstrates a clear performance benefit of an automated system for detecting ineffective triggering over waveform analysis by a group of clinicians. A major advantage for an automated system is indeed in automation—a platform for continuously monitoring each patient at the bedside without requiring intense human resources to detect PVA. Further research applying the automated system will better clarify the true prevalence of PVA and any potential causal relationship with patient outcome.