Automatic Knowledge Extraction: Fusion of Human Expert Ratings and Biosignal Features for Fatigue Monitoring Applications

  • Martin Golz
  • David Sommer

A framework for automatic relevance determination based on artificial neural networks and evolution strategy is presented. It is applied for an important problem in biomedicine, namely the detection of unintentional episodes of sleep during sustained operations of subjects, so-called microsleep episodes. Human expert ratings based on video and biosignal recordings are necessary to judge microsleep episodes. Ratings are fused together with linear and nonlinear features which are extracted from three types of biosignals: electroencephalography, electrooculography, and eyetracking. Changes in signal modality due to nonlinearity and stochasticity are quantified by the ‘delay vector variance’ method. Results show large inter-individual variability. Though the framework is outperformed by support vector machines in terms of classi- fication accuracy, the estimated relevance values provide knowledge of signal characteristics during microsleep episodes.


Support Vector Machine Power Spectral Density Target Variance Learn Vector Quantization Feature Fusion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Akerstedt, T., Kecklund, G., Knutsson, A.: Manifest sleepiness and the spectral content of the EEG during shift work. Sleep 14, 221–225 (1991)Google Scholar
  2. 2.
    Cajochen, C.: Power density in theta/alpha frequencies of the waking EEG progressively increases during sustained wakefulness. Sleep 18, 890–894 (1995)Google Scholar
  3. 3.
    Cortes, C., Vapnik, V.: Support vector networks. Mach Learn 20, 273–297 (1995)zbMATHGoogle Scholar
  4. 4.
    Dinges, D.: An overview of sleepiness and accidents. J Sleep Res. 4(Suppl 2), 4–14 (1995)CrossRefGoogle Scholar
  5. 5.
    Gautama, T., Mandic, D., VanHulle, M.: The delay vector variance method for detecting determinism and nonlinearity in time series. Physica D 190, 167–176 (2004)zbMATHCrossRefMathSciNetGoogle Scholar
  6. 6.
    Golz, M., Sommer, D.: The performance of LVQ based automatic relevance determination applied to spontaneous biosignals. In: Gabrys et al. (eds.) Proceedings of the 10th International Conference KES-2006, LNAI, vol. 4253, pp. 1256–1263. Springer, Berlin Heidelberg New York (2006)Google Scholar
  7. 7.
    Golz, M., Sommer, D., Chen, M., Trutschel, U., Mandic, D.: Feature fusion for the detection of microsleep events. J VLSI Signal Proc 49, 329–342 (2007)CrossRefGoogle Scholar
  8. 8.
    Golz, M., Sommer, D., Seyfarth, A., Trutschel, U., Moore-Ede, M.: Application of vector-based neural networks for the recognition of beginning microsleep episodes with an eyetracking system. In: Kuncheva et al. (eds.) Computational Intelligence: Methods and Applications, pp. 130–134. ICSC Academic, Canada (2001)Google Scholar
  9. 9.
    Horne, J., Baulk, S.: Awareness of sleepiness when driving. Psychophysiology 41, 161–165 (2004)CrossRefGoogle Scholar
  10. 10.
    Ingre, M., Åkerstedt, T., Peters, Anund, Kecklund: Subjective sleepiness, simulated driving performance and blink duration. J Sleep Res. 15, 47–53 (2006)CrossRefGoogle Scholar
  11. 11.
    Knipling, R.: Individual differences in commercial driver fatigue susceptibility: Evidence and implications. In: Proceedings of the International Conference on Fatigue Management in Transportation Operations. Seattle (2005)Google Scholar
  12. 12.
    Kohonen, T.: Self-Organizing Maps, 3rd edn. Springer, Berlin Heidelberg New York (2001)zbMATHGoogle Scholar
  13. 13.
    Lal, S., Craig, A.: Driver fatigue: Electroencephalography and psychological assessment. Psychophysiology 39, 313–321 (2002)CrossRefGoogle Scholar
  14. 14.
    Langwieder, K., Sporner, A., Hell, W.: Einschlafen am Steuer: Hauptursache schwerer Verkehrsunfälle. Wien Klin Wochenschr 145, 473 (1995)Google Scholar
  15. 15.
    Makeig, S., Jung, T.: Changes in alertness are a principal component of variance in the EEG spectrum. Neuroreport 7, 213–216 (1995)Google Scholar
  16. 16.
    Makeig, S., Jung, T., Sejnowski, T.: Awareness during drowsiness: Dynamics and electrophysiological correlates. Can J Exp Psychol 54, 266–273 (2000)Google Scholar
  17. 17.
    Maulsby, R., et al.: The normative electroencephalographic data reference library. Final report, National Aeronautics and Space Administration (1968)Google Scholar
  18. 18.
    Santamaria, J., Chiappa, K.: The EEG of drowsiness in normal adults. J Clin Neurophysiol 4, 327–382 (1987)CrossRefGoogle Scholar
  19. 19.
    Schleicher, R., Galley, N., Briest, S., Galley, L.: Looking tired? Blinks and saccades as indicators of fatigue. Ergonomics, in press (2008)Google Scholar
  20. 20.
    Schwefel, H., Rudolph, G.: Contemporary evolution strategies. In: Proceedings of the 3rd European Conference on Artificial Life, vol. 929, pp. 893–907. Springer, Berlin (1995)Google Scholar
  21. 21.
    Sommer, D., Golz, M., Trutschel, U., Mandic, D.: Fusion of state space and frequency-domain features for improved microsleep detection. In: W. Duch, et al. (eds.) Proceedings of the ICANN, LNCS, vol. 3697, pp. 753–759. Springer, Berlin Heidelberg New York (2005)Google Scholar
  22. 22.
    Sommer, D., Hink, T., Golz, M.: Application of learning vector quantization to detect drivers dozing-off. In: Proceedings of the 6th World Multiconf Systemics, Cybernetics and Informatics (ISAS-SCI2002), vol. XI, pp. 95–98. Orlando, USA (2002)Google Scholar
  23. 23.
    Summala, H., Häkkänen, H., Mikkola, T., Sinkkonen, J.: Task effects on fatigue symptoms in overnight driving. Ergonomics 42, 798–806 (1999)CrossRefGoogle Scholar
  24. 24.
    Trutschel, U., Guttkuhn, R., Ramsthaler, C., Golz, M., Moore-Ede, M.: Automatic detection of microsleep events using a neuro-fuzzy hybrid system. In: Proceedings of the EUFIT-98, vol. 3, pp. 1762–1766. Aachen, Germany (1998)Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Martin Golz
    • 1
  • David Sommer
    • 1
  1. 1.University of Applied SciencesSchmalkaldenGermany

Personalised recommendations