Sleep and Breathing

, Volume 10, Issue 2, pp 62–69

Nasal pressure recordings to detect obstructive sleep apnea

  • Fernanda Ribeiro de Almeida
  • Najib T. Ayas
  • Ryo Otsuka
  • Hiroshi Ueda
  • Peter Hamilton
  • Frank C. Ryan
  • Alan A. Lowe
Original Article

DOI: 10.1007/s11325-005-0042-x

Cite this article as:
de Almeida, F.R., Ayas, N.T., Otsuka, R. et al. Sleep Breath (2006) 10: 62. doi:10.1007/s11325-005-0042-x

Abstract

Obstructive sleep apnea (OSA) is a common disease. Given the costs of in-laboratory polysomnography (PSG), alternative ambulatory methods for accurate diagnosis are desirable. The objective of this study was to evaluate the performance of a simple device (SleepCheck) to identify patients with sleep apnea. A total of 30 consecutive patients with suspected OSA syndrome referred to the sleep clinic were prospectively evaluated with standard PSG and SleepCheck simultaneously during an in-laboratory, supervised full-night diagnostic study. The PSG apnea and hypopnea index (AHI) was evaluated according to standard criteria, and SleepCheck assessed the respiratory disturbance index (RDI) based on nasal cannula pressure fluctuations. Compared to the full-night PSG, SleepCheck systematically overscored respiratory events (the mean difference between SleepCheck RDI and PSG AHI was 27.4±13.3 events per hour). This overscoring was in part related to normal physiologic decreases in flow during rapid eye movement sleep or after an arousal. However, there was reasonable correlation between AHI and RDI (r=0.805). Receiver operating characteristic curves with threshold values of AHI of 10 and 20/h demonstrated areas under the curves (AUCs) of 0.915 and 0.910, respectively. Optimum combinations of sensitivity and specificity for these thresholds were calculated as 86.4/75.0 and 88.9/81.0, respectively. Overall, the SleepCheck substantially overscored apneas and hypopneas in patients with suspected OSA. However, after correction of the bias, the SleepCheck had reasonable accuracy with an AUC, sensitivity, and specificity similar to other ambulatory type 4 devices currently available.

Keywords

Sleep apnea Nasal cannula Screening Diagnosis Home monitoring 

Introduction

Obstructive sleep apnea (OSA) is a commonly under-diagnosed disorder characterized by repetitive collapse of the upper airway during sleep leading to nocturnal desaturation, sleep fragmentation, daytime sleepiness, an increased risk of motor vehicle crashes and cardiovascular morbidity [1, 2]. Therapy for sleep apnea effectively treats many of these adverse consequences, thus stressing the importance of prompt diagnosis of the disease. In-laboratory overnight polysomnography (PSG) is the standard method to confirm the presence of OSA and assess its severity [3]. PSG can also discriminate OSA from other sleep disorders that might have similar clinical symptoms such as central apneas or periodic limb movement disorder [4]. PSG is the gold standard diagnostic test for OSA; however, it is costly and cumbersome. In some countries, waiting lists to obtain a PSG can be over a year, leading to delays in diagnosis and treatment. A clear need exists for less expensive ambulatory devices that can accurately diagnose OSA.

A variety of ambulatory devices to diagnose OSA have been developed. There have been several studies showing reasonable accuracy of portable monitors classified as type 4. Type 4 devices record one or two physiological signals and generally use only one signal, usually either SaO2 or airflow, to define a sleep-disordered breathing event. No electroencephalography signals are monitored. Many of these devices use different methodologies to evaluate sleep apnea severity, which include oximetry alone; [5, 6, 7] oximetry and nasal airflow [8, 9, 10, 11]; peripheral arterial tonometry, oximetry, and actigraphy [12, 13]; airflow, body position, wrist actimetry, pulse rate, and oxygen desaturation [14]; oral and nasal thermistor [15]; microphone and nasal airflow [16]; or tracheal sound analysis [17]. These modalities are feasible for an ambulatory setting in which physicians treat sleep-related breathing disorders. Most of them require a computer to analyze the data, and while some are disposable, they still involve a high degree of complexity in the interpretation of the results and are expensive. Devices even simpler than these would thus be useful.

A recently developed simple device to detect sleep apnea is the SleepCheck. SleepCheck (IM Systems Inc., Baltimore) is a type 4 unattended ambulatory device which records one physiological signal. SleepCheck measures the airflow through an oro-nasal cannula, runs on one 1.5 V AAA battery, has a size similar to a pager and is user-friendly. It uses an algorithm that relies on nasal pressure fluctuations to detect apneas and hypopneas. The liquid crystal display (LCD) on the SleepCheck indicates the total number of events, the number of events for each hour of sleep, and the total number of events during the entire recording period.

Although the device has been approved by the FDA, prior studies with SleepCheck were done with a small number of patients and published in abstract form only [18, 19, 20]. The primary objective of this study was to prospectively evaluate the accuracy of the SleepCheck to evaluate OSA as compared to the gold standard method, the in-laboratory PSG.

Materials and methods

Consecutive patients referred for a full-night, in-laboratory PSG with suspected sleep-related breathing disorders were invited to participate in this study. To reduce possible bias, all consecutive individuals that had suspected sleep apnea and/or OSA-related symptoms and had not undergone previous treatment were eligible for the present study. At the University of British Columbia (UBC) sleep laboratory, there is one bedroom reserved for diagnosis only; patients who had their sleep study in this designated bedroom of the sleep laboratory were eligible. Therefore, patients that were referred for a split-night study with continuous positive airway pressure (CPAP) or a follow-up study using any type of therapy were excluded from the study.

Full-night diagnostic in-laboratory PSG was conducted for each patient. Standard measurements included electroencephalography (EEG), electrooculography (EOG), submental and tibial electromyography (EMG), electrocardiography (ECG), chest and abdominal inductance plethysmography (Respitrace; Ambulatory Monitoring Systems, Ardsley, NY), arterial oxygen saturation (pulse oximeter built into Sandman Sleep Diagnostic System), and airflow with an oro-nasal cannula attached to a “Y” tube, with one of the tube ends connected to the PSG through a pressure transducer (PTAF2, Protech Services, WA) and the other end connected to the SleepCheck.

The SleepCheck has a solid-state air pressure transducer that measures the nasal flow signal which is digitized and traced continuously. It monitors the airflow signal and starts a 10 s timer when the signal decreases by 50% of baseline. The baseline is defined as the average signal value of the last eight inhale–exhale cycles. Apneas and hypopneas are then scored when the signal amplitude drops below the baseline. Custom-made for this study, SleepCheck had a DC output cable connected to an accessory channel of the PSG; it exported a signal of 0 V for normal breathing and a 1 V signal lasting 1 s for apnea and hypopnea events recorded on one PSG channel. Concomitantly, the SleepCheck recorded in its memory the total number of events (total of apneas and hypopneas), number of events in each hour, and the respiratory disturbance index (RDI) as the total apneas and hypopneas divided by the total period; these measurements were shown in an LCD. If the SleepCheck recorded 4 or more consecutive minutes of low signal, it was interpreted by its built-in software as a low signal error (LSE). If less than three LSEs, or if less than 12 total minutes of LSE occurred, the data for the hour were considered valid. If there were more than three LSEs, or greater than 12 min of LSE, the data for the hour were invalid and that hour was not included in the RDI calculation. In the PSG marker, there was no indication of LSEs. The display readings were always completed by one of the authors (F.R.A.).

Sleep staging and respiratory events were scored manually by the same technologist, following the guidelines of Rechtschaffen and Kales [21] and the American Academy of Sleep Medicine [4]. The sleep technologist (P.H.) who scored the data was blinded to the SleepCheck marker channel and to the display readings. First, we compared the RDI calculated by the SleepCheck algorithm (LCD reading) to the PSG apnea and hypopnea index (AHI). Then we manually counted the RDI as all apneas and hypopneas marked according to the SleepCheck signal in the PSG divided by the total recording period. For this evaluation, one of the authors (F.R.A.) analyzed the PSG in epochs of 120 s blinded the apneas and sleep staging.

We also performed a more in-depth analysis for the first 15 patients with the assessment of sensitivity and positive predictive value (PPV) for individual events. The PSG is the gold standard, and a true positive was identified if there was temporal agreement of less than 5 s with the SleepCheck. Discordances between the SleepCheck and PSG were considered as false positives or false negatives. The same author (F.R.A.) assessed the apnea and hypopnea scorings of both methods in epochs of 30 s. The duration of the apneas was not measured by the SleepCheck; if there was an apnea event that lasted 30 s and the SleepCheck marked two events, one of them was considered a false positive. Apneas and hypopneas were not discriminated and marked equally.

For comparative purposes, respiratory events from the PSG were scored from flow signals and oximetry only, with the technician blinded to EEG, EOG, EMG, EKG, SleepCheck marker, and previous scoring. This analysis was performed by the same technician (P.H.) 1 month after the initial full PSG scoring. This scoring followed the same criteria described by the American Academy of Sleep Medicine [4], but without sleep staging and arousals. Patients were included in the study only if they signed a consent form. This study was approved by the Ethics Committee at UBC and by the Vancouver Coastal Health Authority.

Statistical analysis

The results were analyzed with SPSS software (Chicago, IL). The correlation between the SleepCheck RDI and the PSG AHI was assessed using the Pearson correlation coefficient and linear regression equation. A Bland–Altman plot [22] was used to compare PSG and SleepCheck. To assess sensitivity and specificity of the SleepCheck for the diagnosis of sleep apnea, a receiver operating characteristics (ROC) curve was plotted and the area under the curve (AUC) was calculated.

Results

Of 35 consecutive patients, 30 agreed to participate in the study. For the comparison of the SleepCheck RDI (LCD reading) and the RDI manually counted from the SleepCheck marker in the PSG (total number of events/total recording time), we found that the SleepCheck algorithm and error detection slightly improved the detection of the apneas and hypopneas. Because of these results, the display data on the 30 patients were used to evaluate this portable device.

There was a wide distribution of OSA severity, with 18 patients with an AHI less than 30/h and 12 patients with an AHI greater than 30/h (see Table 1). Using a proposed value of an AHI >5 events per hour, the prevalence of OSA in the study population was 73.3%. Within the patients without significant sleep apnea, the mean difference of SleepCheck RDI minus PSG AHI was 25.4 events per hour. In the mild sleep apnea group, the RDI/AHI difference was 34.2 events per hour. Six patients were classified as moderate; their mean difference (RDI−AHI) was 30.5 events per hour. For patients with an AHI >30 events per hour, the mean (RDI−AHI) was 15.5 events per hour. Figure 1 shows the scatter plot that illustrates the significant correlation between SleepCheck RDI and PSG AHI (r=0.803, p<0.001, n=30) and illustrates the tendency of the SleepCheck to overscore apneas since the best fit line crosses the y-axis at the 33.0 index.
Table 1

Characteristics of the subjects within each sleep apnea severity group and the entire sample, expressed as mean±SD

AHI

Number of subjects

F/M

PSG AHI

SC RDI

Age

BMI

<5

8

0F/8M

1.7±1.5

27.2±12.4

34.4±8.7

26.2±2.8

5–15

10

4F/6M

7.8±2.3

42.0±12.7

47.5±6.2

31.7±10.7

15–30

6

2F/4M

22.4±4.4

52.8±11.6

44.5±19.2

34.5±8.3

>30

6

1F/5M

55.1±22.5

70.7±10.6

51.3±9.9

33.5±4.5

ALL

30

7F/23M

18.6±22.1

45.9±19.2

44.2±12.3

31.1±8.0

F Female, M male, PSG AHI polysomnography apnea and hypopnea index, SC RDI SleepCheck respiratory disturbance index, BMI body mass index (kilograms per square meter)

Fig. 1

Scatter plot of the PSG AHI vs SleepCheck RDI. Pearson correlation coefficient was r=0.805, p<0.001, n=30, and the linear regression equation was calculated as y=0.6961x+33.002

A Bland–Altman plot of the differences between the SleepCheck RDI and the PSG AHI vs. the average of the PSG AHI and SleepCheck RDI is provided in Fig. 2. The SleepCheck showed a systematic overscoring tendency by an average of 27.4 events per hour. Therefore, as suggested by Bland and Altman [22], we adjusted the mean difference by transforming the SleepCheck RDI values by minus 27.4. After the adjustment, the limits of agreement were 26.6 and −26.6 events per hour. After this adjustment, the SleepCheck RDI showed a tendency to underscore the events in nonapneic and severe patients while overscoring mild and moderate sleep apnea subjects. With the aim of assessing the sensitivity and specificity of the SleepCheck, we established different ROC curves, with threshold AHIs of 5, 10, 15, and 20 events per hour; the respective optimum combinations of sensitivity and specificity were 86.4/75.0, 85.7/87.5, 83.5/83.5, and 88.9/81.0. Corresponding AUCs were 0.886, 0.915, 0.898, and 0.910. Figure 3 shows the ROC curves for the AHI cutoff points of 10 and 20 events per hour. For the analysis of the PSG scored with flow and oximetry only, there was a correlation of r=0.94 with the full PSG. When this method of RDI was compared to the SleepCheck RDI, the correlation was r=0.814 (p<0.001).
Fig. 2

Bland–Altman plot illustrating the agreement between SleepCheck (SC) RDI and PSG AHI after adjusting for the systematic overscoring bias of 27.4. Limits of agreement were 26.6 and −26.6

Fig. 3

Representative ROC curves for PSG AHI threshold of greater than 10 and greater than 20 events per hour

The reasons for the substantial difference between PSG AHI and SleepCheck RDI were evaluated to determine potential discrepancies. The PSG and SleepCheck apnea and hypopnea scoring from four subjects (one nonapneic, one mild, one moderate, and one severe) were intensively reviewed in a qualitative manner. We found that there was a marked overscoring of the respiratory events in rapid eye movement (REM) sleep as illustrated in Fig. 4. Because there was no desaturation or arousal, the decrease in flow detected by the SleepCheck is considered normal respiratory physiology in REM sleep. Another example of systematic overscoring is illustrated in Fig. 5, where SleepCheck scored events that occurred after arousals and periodic limb movements. This represents the commonly seen period of hyperventilation during an arousal, followed by hypoventilation after the arousal once sleep is reestablished. Since patients normally present an irregularity of breathing after such kinds of events, this physiological breath recovery was incorrectly scored by the SleepCheck. In some epochs, as illustrated in Fig. 6, we found a decrease in flow followed by an arousal that was scored by the SleepCheck but not by the sleep technician. These events may represent respiratory effort-related arousal that does not meet standard criteria for hypopnea. Whether these events are clinically important is open to speculation.
Fig. 4

Example of a 30 s PSG epoch with the SleepCheck signal which illustrates a variation in ventilation that commonly occurs during REM sleep and that was scored as an apnea by SleepCheck (in circle)

Fig. 5

Example of a 30 s PSG. An event scored by the SleepCheck (circle) related to a normal decrease in flow after arousal from a leg movement

Fig. 6

Example of a 30 s PSG. The SleepCheck scored an event with paradoxical rib cage/abdominal movement followed by an arousal. This may represent a respiratory effort related arousal

For the sensitivity and specificity analysis of individual apneas and hypopneas, one of the authors (F.R.A.) evaluated the apnea and hypopnea markers of the SleepCheck and compared that to the apneas and hypopneas in the PSG. The display RDI and the LSE were not included. The sensitivity and PPV of the SleepCheck marker in detecting apneas and hypopneas in each of the 15 patients assessed for this purpose are shown in Table 2. In this evaluation, there were two nonapneics, four mild, five moderate, and four severe sleep apnea patients. Sensitivity ranged from 0.27 to 1.00, with a mean value of 0.81. The PPV ranged from 0.01 to 0.80, with a mean value of 0.32.
Table 2

Assessment of sensitivity and PPV of individual respiratory events during the entire night for 15 patients

Patient no.

PSG AHI

SC RDI

True positive

False positive

False negative

Sensitivity

PPV

1

0.1

22.6

1

182

0

1.00

0.01

2

3.3

44.2

21

333

5

0.81

0.06

3

6.4

36.4

35

225

8

0.81

0.13

4

7.4

35.7

13

254

36

0.27

0.05

5

9.5

32.3

41

201

5

0.89

0.17

6

11.9

68.6

62

328

9

0.87

0.16

7

18.9

63.3

140

336

1

0.99

0.29

8

24.6

38.4

123

144

19

0.87

0.46

9

26.1

63.6

133

338

54

0.71

0.28

10

28

42.2

126

137

41

0.75

0.48

11

29.2

57.1

158

211

20

0.89

0.43

12

30.3

34.3

101

176

109

0.48

0.36

13

45.9

63.4

225

203

51

0.82

0.53

14

47.9

74.8

378

248

7

0.98

0.60

15

75.7

75.5

468

117

5

0.99

0.80

Mean

24.3

50.2

135.0

228.9

24.7

0.81

0.32

Sensitivity=[true positive/(true positive + false negative)]; PPV=[true positive/(true positive + false positive)]

PSG AHI Polysomnography apnea and hypopnea index, SC RDI SleepCheck respiratory disturbance index, PPV positive predictive value

Discussion

SleepCheck provided an RDI that was comparable to the AHI obtained from the PSG. There was a significant correlation between the SleepCheck RDI and the PSG AHI, with an r value of 0.80. The Bland–Altman plot limits of agreement were calculated as ±26.6 events per hour after the adjustment of a consistent bias of 27.4 events per hour (interpreted as a consistent overscoring of the SleepCheck). As illustrated by the ROC curves, we found an AUC of 0.915 and 0.910 for thresholds of an AHI of 10 and 20/h, respectively. For the assessment of individual respiratory events, the mean sensitivity and PPV were determined as 0.81 and 0.32, respectively. This overscoring bias was reported to the manufacturer and, as a consequence, they increased the sensitivity of the SleepCheck by increasing the gain of the nasal cannula flow analysis. This new version was not tested in the present study but may perform better in future studies.

SleepCheck showed a large overscoring and some disagreement with the PSG. The automatic scoring of the apneas and hypopneas could be compromised since the airflow measured might vary according to patient breathing patterns, such as intermittent oral breathing and nasal anatomy [23, 24]. Furthermore, we found that decreases in flow related to REM sleep and decreases of flow following periodic limb movements and arousals were incorrectly scored as respiratory events by SleepCheck. This is a limitation of such a single-channel device that relies on flow, as flow can vary accordingly in different stages of sleep and in response to different physiological stimuli. Such events can only be accurately scored with simultaneous oximetry and EEG. However, we found that SleepCheck did detect several events that were not scored in the PSG as decreases of flow preceding an arousal. These could be interpreted as respiratory effort-related arousals, which may have clinical significance.

Previous correlations between the SleepCheck RDI and PSG AHI were found as 0.94 and 0.99 with samples of seven and six patients, respectively [18, 20]. We found a lower correlation of 0.80 that could be related to a larger sample size with a greater diversity of patients and apnea severity. Gorny and collaborators [19] found a high correlation (r=0.98) of SleepCheck RDI measured in the sleep laboratory and at home, demonstrating a high night-to-night consistency. However, the three previous studies with SleepCheck were only published as abstracts and were conducted at the IM Systems site.

Although this study took place in a supervised setting, the monitor utilized in this study is fairly simple to use, and we predict that the rate of lost data in an unsupervised environment should be minimal. Training patients for the correct placement of the cannula and to turn on the machine prior to the testing are the only requirements for this technique. SleepCheck is a small device, and it is clipped to the patient’s pajamas through out the night, avoiding the displacement of the cannula or restricting patients from their preferred sleep position. Since it does not register EEG or any actual measurement of sleep [10] (for example, actigraphy), a diary of the night is probably useful and thus certain hours of recording could be excluded and the index recalculated.

An advantage of the present study was that SleepCheck data were recorded simultaneously with the PSG and used the same methodology to record airflow with a nasal pressure transducer. Studies that compare PSG to home monitoring could lead to misinterpretation of the data because of night-to-night variability of the AHI [25], and also a comparison of in-laboratory with home assessment does induce an error based on patient sleep position, reported as being more supine at the sleep lab [24]. Previous studies showed loss rates of 10 to 14% in studies using a nasal cannula associated with oximetry [26], but there is still a need to evaluate the performance of the present monitor in an unattended setting.

The sensitivity to detect individual events of the SleepCheck in 15 patients was 0.81, but the sensitivity and PPVs were small, confirming the overestimate bias of about 27 events per hour. The low signal event algorithm is built in to the SleepCheck and therefore these errors were not visible in the evaluation of individual respiratory events. Some of the false positive events could then be excluded with the algorithm of SleepCheck, increasing the specificity of the device. However, this is one of the few studies on type 4 portable monitors [10] to carefully evaluate the synchronism of the respiratory events scored by the portable devices compared to PSG. The comparison of this type of analysis with the literature is therefore compromised. Nevertheless, continuous efforts should persist to get a better algorithm associated with this machine that could reduce the false positive detection of apneas and hypopneas. We could confirm that apneas and hypopneas were detected without differences, which was expected by the nature of the nasal pressure analysis [24].

A possible use of the SleepCheck might be in the evaluation of different OSA treatment approaches, such as oral appliance therapy. Although it is still not fully understood, there is some evidence that the amount of mandibular protrusion correlates with the efficacy of the treatment [27, 28, 29, 30, 31, 32]. There have been some attempts to titrate OA’s during one-night PSG [28, 31, 33], but this is not clinically available at the present time. Because OA titration is still dependent mainly on subjective evaluation such as sleepiness and bed-partners’ reports of snoring and witnessed apneas [34], patients often undergo more than two PSGs in order to find the therapeutic mandibular position. There can be significant delay in the diagnosis of inadequate mandibular advancement and subsequent treatment failures. Repeated PSG recordings are the ideal method to properly titrate an OA, but these may be too complex and expensive [30]. Fleury and colleagues [32] found that the combination of oximetry and clinical evaluation improved the OA effectiveness in the follow-up PSG. An important role of the SleepCheck or other types of portable monitors could be to more rapidly assess treatment efficacy and to improve the oral appliance clinical titration protocols.

This study has several limitations such as being an in-laboratory setting without ambulatory assessment. Although there is still a need for in-home assessment, we doubt, because of the simplicity of the device, that its performance in an ambulatory setting would be substantially different. The small sample size is also a limitation, but because the data were collected on consecutive patients, there was less bias, and patients were distributed fairly equally according to OSA severity. Our sample consisted of patients referred to the sleep lab for assessment of OSA. As such, it is unclear whether the performance of the device would be similar in other patient populations (e.g., general clinics, screening of asymptomatic individuals).

The SleepCheck substantially overscored apneas and hypopneas in patients with suspected OSA. This was partially due to the scoring of normal physiological variations in airflow that occur after nonrespiratory arousals and REM sleep. This overscoring was greatest in patients with mild to moderate OSA. However, after correction of bias, the SleepCheck exhibited reasonable accuracy with an AUC, sensitivity, and specificity similar to other ambulatory type 4 devices currently in the market. Nevertheless, we believe that further studies in ambulatory settings and on a broader range of patients need to be performed before this device can be recommended.

Acknowledgements

The authors wish to thank Mrs. Ingrid Ellis for her editorial assistance in the final preparation of the manuscript. We are grateful for the statistical analysis provided by Ms. Mary Wong. The present study was supported by CNPq (Brazilian Government), who provided a scholarship to the first author (F.R.A.). One of the authors (N.T.A.) is supported by a Michael Smith Foundation for Health Research Scholar Award, a CIMR/BCLA new investigator award, and a departmental scholar award at UBC. In addition, SleepCheck was provided by IM Systems Inc., Baltimore.

Copyright information

© Springer-Verlag 2006

Authors and Affiliations

  • Fernanda Ribeiro de Almeida
    • 1
  • Najib T. Ayas
    • 2
  • Ryo Otsuka
    • 3
  • Hiroshi Ueda
    • 1
  • Peter Hamilton
    • 4
  • Frank C. Ryan
    • 5
  • Alan A. Lowe
    • 1
  1. 1.Division of Orthodontics, Department of Oral Health Sciences, Faculty of DentistryThe University of British ColumbiaVancouverCanada
  2. 2.Centre for Clinical Epidemiology and EvaluationVancouver Coastal Health Research InstituteVancouverCanada
  3. 3.Maxillofacial Orthognathics, Maxillofacial/Neck ReconstructionGraduate School of Tokyo Medical and Dental UniversityTokyoJapan
  4. 4.Sleep Laboratory, Vancouver General HospitalVancouverCanada
  5. 5.Department of Medicine, Division of Respiratory Medicine, Faculty of MedicineThe University of British ColumbiaVancouverCanada

Personalised recommendations