Data–Driven Multimodal Sleep Apnea Events Detection

Synchrosquezing Transform Processing and Riemannian Geometry Classification Approaches
Transactional Processing Systems
Part of the following topical collections:
  1. New Technologies and Bio-inspired Approaches for Medical Data Analysis and Semantic Interpretation


A novel multimodal and bio–inspired approach to biomedical signal processing and classification is presented in the paper. This approach allows for an automatic semantic labeling (interpretation) of sleep apnea events based the proposed data–driven biomedical signal processing and classification. The presented signal processing and classification methods have been already successfully applied to real–time unimodal brainwaves (EEG only) decoding in brain–computer interfaces developed by the author. In the current project the very encouraging results are obtained using multimodal biomedical (brainwaves and peripheral physiological) signals in a unified processing approach allowing for the automatic semantic data description. The results thus support a hypothesis of the data–driven and bio–inspired signal processing approach validity for medical data semantic interpretation based on the sleep apnea events machine–learning–related classification.


EEG Sleep apnea semantic interpretation Data–driven biomedical data processing Bio–inspired data processing Semantic biomedical data interpretation 


In recent years new technologies and bio–inspired approaches for medical data analysis and subsequent semantic interpretation (qualitative analysis) are gaining a research momentum [10, 15]. The automatic semantic interpretation of biomedical datasets allows for home–based medical symptoms interpretations and preliminary diagnosis before a subsequent clinical evaluation. In the current study an extension of the previously developed by the author method [19] to the multimodal physiological signals such as: EEG, ECG, EOG, EMG, oro–nasal airflow, ribcage and abdomen movements, oxygen saturation (pulse–oximetery), acoustic snoring, and body position is proposed in an unified approach leading to successful automatic classification (semantic interpretation) of sleep apnea events. The employed synchrosqueezing transform (SST) method [2, 3, 4, 9, 19] for the multimodal physiological signals outperforms the classical time-frequency analysis techniques of the non–linear and non–stationary signals [16, 17, 18] such as multimodal sleep physiological datasets. This not only allows for examination of the spectral contents of each signal as well as the interdependence between the multimodal data streams, but also complies with what clinical experts use in their visual judgments of EEG and body peripheral physiological signals used in sleep studies. For clinically, as well as home–user oriented sleep support applications, the plug & play operation is nowadays considered as a minimum requirement for any consumer devices. In the classification step, thanks to information geometry approach it is possible to define a metric enjoying the sought adaptive (data–driven) properties. The information geometry is a field of information theory where the probability distributions are taken as point of a Riemannian manifold [6]. This field has been popularized by S. Amari [5].

We present successful application of adaptive sleep apnea events classification approach leading to the semantic data interpretation, which employs the above mentioned data–driven filtering and information geometry–based classification techniques. The paper from now on is organized as follows. In the next section methods used in the paper are introduced and explained. Discussion of results follows. Finally conclusions with future research directions summarize the paper.


Offline sleep data processing and classification experiments have been conducted within an institutional review board permission from The Ethical Committee of the Faculty of Engineering, Information and Systems at the University of Tsukuba, Tsukuba, Japan. The developed data–driven and filtering methods are tested on the freely available for research purpose St. Vincent’s University Hospital / University College Dublin Sleep Apnea Database [1], which contains 25 full overnight polysomnograms. Twenty five adult subjects with suspected sleep–disordered breathing were recorded within the project. The subjects were randomly selected from patients referred to the sleep disorders clinic. They were candidates for a possible diagnosis of the obstructive sleep apnea (OSA), the central sleep apnea (CSA) or a primary snoring. All subjects were of legal age in Europe (above 18 years old). They had no known cardiac disease, autonomic dysfunction, and not on any medication known to interfere with a heart rate. The selected for the study subjects were 4 females and 21 males. Their average age was of 50 ± 10 years old. The following signals were recorded:
  • bipolar EEG derivations C3 − A2 and C4 − A1;

  • bipolar EOG of left and right eyes;

  • submentas EMG;

  • ECG of a modified lead V2

  • the oro–nasal airflow using a thermistor;

  • the ribcage and abdomen movements;

  • the oxygen saturation from a finger pulse–oximeter;

  • the snoring from a tracheal microphone;

  • the body position.

Onset times and durations of respiratory events were annotated by the same professional sleep technologist from St. Vincent’s University Hospital / University College Dublin Sleep Apnea Database [1]. The above annotations were used for data segmentation and classifier evaluation. The following nine classes of sleep evens were used:
  1. 1.

    obstructive hypopnea;

  2. 2.

    central hypopnea;

  3. 3.

    mixed hypopnea;

  4. 4.

    obstructive apnea;

  5. 5.

    central apnea;

  6. 6.

    mixed apnea;

  7. 7.

    periodic breathing;

  8. 8.

    possible apnea or hypopnea;

  9. 9.

    normal sleep.

The chance level in conducted classification experiments was of 1/9 = 11.11 %.

Synchrosquezing transform processing of multimodal biomedical signals

An empirical mode decomposition (EMD) class of algorithms is a technique that decomposes a given univariate [11] or multivariate [16] signal into its building block functions. Those functions are the superposition of a limited number of extracted (sifted from the original signal) components. The EMD–based techniques have been already applied successfully to artifacts removal from EEG [12, 13, 14, 17, 19]. Unfortunately the original EMD algorithm due to its iterative decomposition nature is difficult to apply for online EEG processing due to its computational complexity [19]. A recently proposed solution for this problem is the synchrosqueezing transform (SST) method proposed by [9]. The SST method is a combination of wavelet analysis and reallocation methods [2, 3, 4, 19].

Before a discussion of the proposed implementation of the SST technique to the multimodal sleep datasets, let’s review briefly the method applied by the author previously to brain–computer interface (BCI) online brainwave decoding [19]. Given the originally recorded single channel i of a multimodal sleep signal s(t) (EEG, ECG, EOG, EMG, oro–nasal airflow, ribcage and abdomen movements, oxygen saturation, acoustic snoring, and body position), its classical wavelet transform Ws, i(a, t) is obtained as [19],
$$ W_{s,i}(a,t) = \frac{1}{\sqrt{a}}{\int}_{-\infty}^{\infty} s_{i}(u) \overline{\psi\left( \frac{u-t}{a}\right)du}, $$
where a sets the scale and ψ(u) is the chosen wavelet function (Morlet wavelet in this project). The wavelet transform fortunately is an information preserving operation, so the original signal is reconstructed as,
$$ s_{i}(u)=D_{\psi}^{-1}{\int}_{-\infty}^{\infty} dt {\int}_{-\infty}^{\infty} \psi\left( \frac{u-t}{a}\right) W_{s,i}(a,t)\frac{da}{a^{2}}, $$
with the constant Dψ determined as \(D_{\psi } = {\int }_{0}^{\infty } |{\Psi }(\xi )|\frac {d\xi }{\xi }\) with Ψ(ξ) denoting a Fourier transform of the wavelet function used in the original decomposition (1). In usual signal processing cases, both the time and frequency take the discrete values. Time tk = kδt = k/fs where fs is the sampling frequency of the original, multimodal sleep recording in this case, signal. The recently proposed SST method by [9] allows for providing a time–frequency representation with more precise oscillatory frequency tracking and time resolutions at the same time leading to more accurate signal decompositions [19]. The above mentioned concept is based an identification of the frequencies f(a, t) for which the phase of the wavelet coefficient grows for each scale and time, at the first processing step [9]:
$$ f(a,t) = \frac{1}{2\pi}\frac{\delta}{\delta t} \arg\left( W_{s,i}(a,t)\right), $$
where arg(⋅) stands for the phase of the complex coefficient and the multiplier 1/2π converts frequency between circular and normal units. Determination of the f(a, t) from the analyzed signal leads identification of fi frequency bins as \([f_{i}^{-},f_{i}^{+}].\) The SST decomposition is calculated next as
$$ T_{s}(f_{i},t) = C_{\psi}^{-1} \sum\limits_{j:f_{i}^{-}<f(a_{j},t) \leq f_{i}^{+}} W_{s,i}(a_{j},t) a_{j}^{-3/2} {\Delta} a_{j}, $$
where Δaj are the distances between the adjacent scales. The constant Cψ representing the amplitude is defined as,
$$ C_{\psi} = \frac{1}{2} {\int}_{0}^{\infty} \overline{\Psi(\xi)}\frac{d\xi}{\xi}, $$
The single channel of the multimodal sleep signals after transformation to the SST frequency domain as in equation (4) are bandpass filtered only in the frequency range of 0.1 ∼ 30 Hz, which allows us to keep the majority sleep–related oscillations, while removing unnecessary noise.
The original signals could be reconstructed from SST decompositions to its time domain form simply [9] as
$$\begin{array}{@{}rcl@{}} s_{i}(t) &=& \text{real}\left( \sum\limits_{i} T_{s}(f_{i},t)\right)\\ &=& \left|\sum\limits_{i} T_{s}(f_{i},t)\right| \cos \left( \arg \sum\limits_{i} T_{s}(f_{i},t)\right). \end{array} $$
The convenient SST transform, from the point of view of non–linear and non–stationary signals, and its inverse as in equations (4) and (6) allows for the very precise data–driven and flexible bandpass filtering of the non–stationary and non–linear multimodal sleep signals. The very encouraging results of sleep apnea classification have been summarized in the Fig. 1. The proposed SST–based preprocessing steps have been compared with univariate [18, 21] and multivariate [20] EMD approaches with no significantly different results yet with an increased computational complexity (a lower computation speed) as also previously reported in [19].

Riemannian geometry feature extraction for multimodal biomedical signals

A task of the automatic sleep apnea events detection from physiological recordings is to classify various length multimodal data samples. First of all a generic model for the observed data shall be specified. Let \(s(t)\in \mathbb {\mathbb {}R}^{n}\) be the zero–mean multimodal data vector for N sensors captured at discrete time sample t and let \(\mathbf {S}_{k}\in \mathbb {\mathbb {}R}^{NK}\) be a single event record comprised of K samples belonging to a sleep event k ∈ {1,…,K}. The single record data shall be always assumed having a zero mean as result of the data–driven filtering procedures (effectively band–limited spectral decompositions) discussed in Section “Synchrosquezing transform processing of multimodal biomedical signals”. Thus, the sleep record sample covariance matrix of a given trial Sk belonging to a class z is given by
$$ \mathbf{C}_{z} = \frac{1}{K-1}\mathbf{S}_{z} \mathbf{S}_{z}^{T}. $$
Assuming a multivariate Gaussian distribution the covariance matrix is the only unique parameter of each class. We shall describe a classification algorithm that can be applied to the covariance matrices capturing the relevant information of sleep events.
Fig. 1

Apnea classification accuracy results obtained in four different mean covariance matrix and classification distance metrics using arithmetic, Riemanian, a mixture of Riemannian for means and geodesic for classification, and finally geodesic only cases. For each of the metric case three different sleep sensors subsets were evaluated, namely “all”, “face” and “body” area subsets. The plots depict distribution shapes with color coded numbers of results in each bin. Medians and quantiles are marked with red lines. Numbers of perfect (100 %) were the highest for Riemmanian distance cases. Results of statistical significance analysis are depicted in Fig. 2

A classification task consists of assigning to an unlabeled sleep event data Si, of which the covariance matrix Ci is computed, to one of the K classes. From the collected previously training records a geometric mean covariance matrix Mi representing each one of the K classes is computed. The covariance matrices are symmetric and positive definite, which implies that they can be diagonalized by rotation. They have also all positive eigenvalues. In order to measure a distance of the new sleep data record to the above mentioned class means an appropriate metric shall be employed. We propose to utilize a Riemannian geometry framework [5, 6]. On the Riemannian manifold each of the symmetric positive definite matrices is represented by a point. In such a space case a geodesic between two points C1 and C2 is a curve passing the two points having a minimum length. Additionally, such connecting curve is unique for a given metric. The length of the geodesic connecting two points is equal to a distance between them. The Riemannian distance between two covariance matrices C1 and C2 is computed as follows [6],
$$ \delta_{R} = \left|\left|ln(C_{1}^{-1}C_{2})\right|\right|_{F} = \sqrt{\sum\limits_{n}[ln(w_{n})]^{2}}, $$
where the symbol || ⋅ || denotes Frobenius norm and w1,…,wn the eigenvalues of \(C_{1}^{-1}C_{2}\), respectively. The geometric mean of m covariance matrices using the above defined Riemannian distance (8) is computed as follows [7]
$$ D(C_{1},\cdots,C_{m}) = \arg\min_{\sum\in P(n)}{\delta_{R}^{2}}(C,C_{i}). $$
The geodesic, defined as the shortest curve between two covariance matrices (points) of the manifold, according to the Riemannian metric is given by [6]
$$ {\Gamma}(C_{1},C_{2},t) = C_{1}^{\frac{1}{2}}\left( C_{1}^{-\frac{1}{2}}C_{2}C_{1}^{-\frac{1}{2}}\right)^{t}C_{1}^{\frac{1}{2}}, $$
with t ∈ [0:1] being a scalar. It is worth to note, that the geometric mean of two points is equal to the geodesic in the middle at Γ(C1,C2,0.5).

Sleep apnea classification approaches

A very natural for Riemmanian geometry–based features generic classification algorithm, as proposed in [8], shall be based on fining a minimum distance of the newly arriving sleep data record from the mean covariance matrices, representing the desired classes K, using the proposed framework. A minimum distance to mean (MDM) classifier [8] meets the above criterium and it has been chosen in this project. This very generic approach is applied in four distance evaluation metric methods taking into account arithmetic; Riemannian; a mixture of Riemanian for mean covariance and geodesic for subsequent distance from mean calculation; and finally for geodesic only distances. The MDM classifier is also compared with a K−means unsupervised [6, 22]; tangent space linear discriminant analysis (LDA) [6, 22]; and tangent space linear support vector machine (SVM) [6, 23] methods.


As a result of data–driven filtering and classification methods, discussed in Sections “Synchrosquezing transform processing of multimodal biomedical signals” and “Riemannian geometry feature extraction for multimodal biomedical signals”, very encouraging results of automatic sleep apnea detection resulted. The task was to automatically classify the events into one of nine classes comprising of six apnea cases, possible (uncertain) occurrences and normal (non–obstructive) periods. The proposed minimum distance to mean classifier application within the Riemannian geometry approach was tested in leave–one–out setting. The obtained results of 25 subject datasets from St. Vincent’s University Hospital / University College Dublin Sleep Apnea Database [1] are summarized in Fig. 1.

The best result were obtained for Riemannian geometry distance application to mean covariance matrices calculation (8) and final the minimum distance to mean (MDM) classifier applications. Those results were statistically significant in the majority of comparisons as tested with non–parametric Wilcoxon rank sum pairwise tests for all sensors in Riemmanian case as summarized in Fig. 2. The 100 % accuracies were also at a high number for the Riemmanian distance cases as reported in Fig. 1.
Fig. 2

Scatter plot of p–values obtained from non–parametric Wilcoxon rank sum pairwise tests of statistically significant differences of classification accuracies depicted in Figure 1. The tested classifiers are marked by lower case abbreviations: mdm for minimum distance to mean; km for K−means unsupervised; lda for tangent space multi–class linear discriminant analysis; svm for tangent space multi–class linear support vector machine classifiers, respectively. The distance metrics are encoded by: A for arithmetic; R for Riemannian; RG for Riemanian and geodesic mixture; and G for geodesic only, respectively. Finally the utilized sensor sets are decoded similarly as in Fig. 1 by all; face and body abbreviations. All K−means unsupervised classification results were significantly lower comparing to the remaining methods since the obtained median results were on a chance level as shown in Fig. 1. The best and in majority better at p<0.05 statistical significant level accuracies were obtained for the minimum distance to mean (MDM) classifier utilizing Riemannian distance for all sensors (mdmR(all)) case


This study demonstrated automatic apnea classification results from 25 subjects suffering from various levels of sleep problems. The study confirmed also the validity of the data–driven based and frequency domain filtering in SST space. It allowed for successful classification (semantic interpretation) of the six apnea classes, possible (uncertain) occurrences, periodic breathing and normal (non–obstructive) periods.

The results presented offer a step forward in the development of novel and very much expected to improve life of sleep obstructive condition suffering patients using brainwave decoding neurotechnology applications. Since the results obtained from the tested sleep database did not yield maximum possible accuracies, the current methods would obviously still need improvements and modifications. These needs determine the major lines of study for future research. However, even in its current form, the proposed automatic apnea detections method can be regarded as a practical solution for sleep obstruction suffering patients.


  1. 1.
    St. Vincent’s University Hospital / University College Dublin sleep apnea database (2011). doi:
  2. 2.
    Ahrabian, A., Looney, D., Stanković, L., and Mandic, D. P., Synchrosqueezing-based time-frequency analysis of multivariate data. Signal Process. 106:331–341, 2015.CrossRefGoogle Scholar
  3. 3.
    Ahrabian, A., and Mandic, D. P., A class of multivariate denoising algorithms based on synchrosqueezing. IEEE Trans. Signal Process. 63(9):2196–2208, 2015.Google Scholar
  4. 4.
    Ahrabian, A., and Mandic, D.P., Selective time-frequency reassignment based on synchrosqueezing. IEEE Signal Process. Lett. 22(11):2039–2043, 2015.CrossRefGoogle Scholar
  5. 5.
    Amari, S., and Wu, S.: An information-geometrical method for improving the performance of support vector machine classifiers. In: Ninth International Conference on Artificial Neural Networks, ICANN 99. (Conf. Publ. No. 470), Vol. 1, pp. 85–90. IET (1999)Google Scholar
  6. 6.
    Barachant, A., Bonnet, S., Congedo, M., and Jutten, C., Multiclass brain–computer interface classification by riemannian geometry. IEEE Trans. Biomed. Eng. 59(4):920–928, 2012.CrossRefPubMedGoogle Scholar
  7. 7.
    Barachant, A., and Congedo, M.: A plug&play P300 BCI using information geometry. arXiv preprint arXiv:1409.0107 (2014)
  8. 8.
    Congedo, M., Barachant, A., and Andreev, A.: A new generation of brain-computer interface based on Riemannian geometry. arXiv preprint arXiv:1310.8115 (2013)
  9. 9.
    Daubechies, I., Lu, J., and Wu, H. T., Synchrosqueezed wavelet transforms: An empirical mode decomposition-like tool. Appl. Comput. Harmonic Anal. 30(2):243–261, 2011. doi:10.1016/j.acha.2010.08.002. Scholar
  10. 10.
    Hachaj, T., and Ogiela, M. R., Cad system for automatic analysis of ct perfusion maps. Opto-Electron. Rev. 19(1):95–103 , 2011.CrossRefGoogle Scholar
  11. 11.
    Huang, N., Shen, Z., Long, S., Wu, M., Shih, H., Zheng, Q., Yen, N. C., Tung, C., and Liu, H., The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. R. Soc. A: Math. Phys. Eng. Sci. 454(1971):903–995, 1998.CrossRefGoogle Scholar
  12. 12.
    Mandic, D., Rehman, N., Wu, Z., and Huang, N., Empirical mode decomposition-based time-frequency analysis of multivariate signals: The power of adaptive data analysis. IEEE Signal Process. Mag. 30(6):74–86, 2013. doi:10.1109/MSP.2013.2267931.CrossRefGoogle Scholar
  13. 13.
    Molla, M., Tanaka, T., Rutkowski, T., and Tanaka, K.: Phase synchronization analysis of eeg channels using bivariate empirical mode decomposition. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1182–1186. doi:10.1109/ICASSP.2013.6637837 (2013)
  14. 14.
    Molla, M. K. I., Tanaka, T., and Rutkowski, T.: Multivariate emd based approach to eog artifacts separation from eeg. In: 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 653–656. doi:10.1109/ICASSP.2012.6287968 (2012)
  15. 15.
    Ogiela, L., and Ogiela, M. R.: Semantic analysis processes in advanced pattern understanding systems. In: Advanced Computer Science and Information Technology, pp. 26–30. Springer (2011)Google Scholar
  16. 16.
    Rehman, N., and Mandic, D. P., Multivariate empirical mode decomposition. Proc. R. Soc. A: Math. Phys. Eng. Sci. 466(2117):1291–1302, 2010. doi:10.1098/rspa.2009.0502. Scholar
  17. 17.
    Rutkowski, T., Cichocki, A., Tanaka, T., Mandic, D., Cao, J., and Ralescu, A.: Multichannel spectral pattern separation - an eeg processing application. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2009. ICASSP 2009. pp. 373–376. doi:10.1109/ICASSP.2009.4959598 (2009)
  18. 18.
    Rutkowski, T. M., Cichocki, A., Tanaka, T., Ralescu, A.L., and Mandic, D. P.: Clustering of spectral patterns based on emd components of eeg channels with applications to neurophysiological signals separation. In: Advances in Neuro-Information Processing, pp. 453–460. Springer (2008)Google Scholar
  19. 19.
    Rutkowski, T. M., and Mori, H., Tactile and bone-conduction auditory brain computer interface for vision and hearing impaired users. J. Neurosci. Methods 244:45–51, 2015. doi:10.1016/j.jneumeth.2014.04.010. Scholar
  20. 20.
    Rutkowski, T. M., Struzik, Z. R., Mandic, D. P.: EEG epileptic seizures separation with multivariate empirical mode decomposition for diagnostic purposes. In: 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 7128–7131. IEEE Engineering in Medicine and Biology Society, IEEE Press, Osaka. Japan. doi:10.1109/EMBC.2013.6611201 (2013)
  21. 21.
    Rutkowski, T. M., Vialatte, F., Cichocki, A., Mandic, D., Barros, A. K., Knowledge-Based Intelligent Information and Engineering Systems, Lecture Notes in Artificial Intelligence, vol. 4253, chap. Auditory Feedback for Brain Computer Interface Management - An EEG Data Sonification Approach, pp. 1232–1239. Berlin & Heidelberg: Springer, 2006.Google Scholar
  22. 22.
    Theodoridis, S., and Koutroumbas, K.: Pattern Recognition, 4th edn. Acedemic Press (2009)Google Scholar
  23. 23.
    Vapnik, V.: The Nature of Statistical Learnig Theory. Springer Verlag (1995)Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  1. 1.Life Science Center of TARA University of TsukubaTsukuba-shiJapan

Personalised recommendations