Feature Fusion for the Detection of Microsleep Events

  • Martin Golz
  • David Sommer
  • Mo Chen
  • Udo Trutschel
  • Danilo Mandic


A combination of linear and nonlinear methods for feature fusion is introduced and the performance of this methodology is illustrated on a real-world problem: the detection of sudden and non-anticipated lapses of attention in car drivers due to drowsiness. To achieve this, signals coming from heterogeneous sources are processed, namely the brain electric activity, variation in the pupil size, and eye and eyelid movements. For all the signals considered, the features are extracted both in the spectral domain and in state space. Linear features are obtained by the modified periodogram, whereas the nonlinear features are based on the recently introduced method of delay vector variance (DVV). The decision process based on such fused features is achieved by support vector machines (SVM) and learning vector quantization (LVQ) neural networks. For the latter also methods of metrics adaptation in the input space are applied. The parameters of all utilized algorithms are optimized empirically in order to gain maximal classification accuracy. It is also shown that metrics adaptation by weighting the input features can improve the classification accuracy, but only to a limited extent. Limited improvements are also obtained when fusing features of selected signals, but highest improvements are gained by fusion of features of all available signals. In this case test errors are reduced down to 9% in the mean, which clearly illustrates the potential of our methodology to establish a reference standard of drowsiness and microsleep detection devices for future online driver monitoring.


feature fusion microsleep events delay vector variance support vector machines learning vector quantization automatic relevance determination genetic algorithms 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    H. Kantz and T. Schreiber, “Nonlinear Time Series Analysis,” 2nd edn. Cambridge University Press, 2004.Google Scholar
  2. 2.
    T.-P. Jung, M. Stensmo, T. Sejnowski, S. Makeig, “Estimating Alertness from the EEG Power Spectrum,”. IEEE Trans. Biomed. Eng., vol. 44, 1997, pp. 60–69.CrossRefGoogle Scholar
  3. 3.
    M. Golz, D. Sommer, A. Seyfarth, U. Trutschel, M. Moore-Ede, “Application of Vector-Based Neural Networks for the Recognition of Beginning Microsleep Episodes with an Eyetracking System.” In Comput Intell: Methods & Applic, L.I. Kuncheva (Ed.), 2001, pp. 130–134.Google Scholar
  4. 4.
    U. Trutschel, R. Guttkuhn, C. Ramsthaler, M. Golz, M. Moore-Ede, “Automatic Detection of Microsleep Events Using a Neuro-Fuzzy Hybrid System,” Proc. 6th Europ. Congr. Intellig. Techn. Soft. Comput. (EUFIT98), vol. 3, 1998, pp. 1762–66.Google Scholar
  5. 5.
    B.V. Dasarathy, “Decision Fusion,” IEEE Computer Society Press, Los Alamitos, 1994. ISBN 0-8186-4452-4.Google Scholar
  6. 6.
    D.J.C. MacKay, “Probable Networks and Plausible Predictions—A Review of Practical Bayesian Methods for Supervised Neural Networks,” Network Comp. Neural Syst., vol. 6, 1995, pp. 469–505.CrossRefzbMATHGoogle Scholar
  7. 7.
    Y. Bengio, O. Delalleau, N. Le Roux, “The Curse of Dimensionality for Local Kernel Machines, Techn. Rep. 1258,” Université de Montréal, 2005.Google Scholar
  8. 8.
    N. Galley, G. Andrés, C. Reitter, “Driver Fatigue as Identified by Saccadic and Blink Indicators,” in Vision in Vehicles–VII, A. Gale (ed.); Elsevier, Amsterdam, 1999, pp. 49–59.Google Scholar
  9. 9.
    D.B. Percival and A.T. Walden, “Spectral Analysis for Physical Applications,” University Press, Cambridge, 1993.CrossRefzbMATHGoogle Scholar
  10. 10.
    T. Gautama, D.P. Mandic, M.M. Van Hulle, “The Delay Vector Variance Method for Detecting Determinism and Nonlinearity in Time Series,” Physica D, vol. 190, 2004, pp. 167–176.MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    T. Kohonen, “Self-Organizing Maps, 3rd ed.”, Springer, Berlin, 2001.CrossRefzbMATHGoogle Scholar
  12. 12.
    C. Cortes, V.N. Vapnik, “Support Vector Networks,” Mach. Learn. vol. 20, 1995, pp. 273–297.zbMATHGoogle Scholar
  13. 13.
    B. Hammer, T. Villmann, “Generalized Relevance Learning Vector Quantization,” Neural Netw, vol. 15, nos. 8–9, 2002, pp. 1059–1068.CrossRefGoogle Scholar
  14. 14.
    D. Sommer, M. Golz, Trutschel U, Mandic D. “Fusion of State Space and Frequency–Domain Features for Improved Microsleep Detection,” in Int Conf Artificial Neural Networks (ICANN 2005), W. Duch et al. (Eds.), LNCS3697, Springer, Berlin, 2005, pp. 753–759.Google Scholar
  15. 15.
    Joachims T. “Learning to Classify Text Using Support Vector Machines.” Kluwer, Boston, 2002.CrossRefGoogle Scholar
  16. 16.
    T. Gautama, D.P. Mandic, M.M. Van Hulle, “A Novel Method for Determining the Nature of Time Series,” IEEE Trans. Biomed. Eng., vol. 51, 2004, pp. 728–736.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2007

Authors and Affiliations

  • Martin Golz
    • 1
  • David Sommer
    • 1
  • Mo Chen
    • 2
  • Udo Trutschel
    • 3
  • Danilo Mandic
    • 2
  1. 1.Department of Computer ScienceUniversity of Applied SciencesSchmalkaldenGermany
  2. 2.Department of Electrical and Electronic EngineeringImperial CollegeLondonUK
  3. 3.Circadian Technologies, Inc.StonehamUSA

Personalised recommendations