The Prediction of Fatigue Using Speech as a Biosignal

  • Khan Baykaner
  • Mark Huckvale
  • Iya Whiteley
  • Oleg Ryumin
  • Svetlana Andreeva
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9449)


Automatic systems for estimating operator fatigue have application in safety-critical environments. We develop and evaluate a system to detect fatigue from speech recordings collected from speakers kept awake over a 60-hour period. A binary classification system (fatigued/not-fatigued) based on time spent awake showed good discrimination, with 80 % unweighted accuracy using raw features, and 90 % with speaker-normalized features. We describe the data collection, feature analysis, machine learning and cross-validation used in the study. Results are promising for real-world applications in domains such as aerospace, transportation and mining where operators are in regular verbal communication as part of their normal working activities.


Fatigue Speech Computational paralinguistics 



The authors would like to acknowledge the European Space Agency and University College of London who are jointly responsible for funding this work.


  1. 1.
    Dobbie, K.: Fatigue-related crashes: an analysis of fatigue-related crashes on Australian roads using an operational definition of fatigue. Australian transport safety bureau (OR23), (2002)Google Scholar
  2. 2.
    FMCSA: Regulatory impact analysis – hours of service final rule. Federal motor carrier safety administration, December 2011Google Scholar
  3. 3.
    Åkerstedt, T.: Consensus statement: Fatigue and accidents in transport operations. J. Sleep Res. 9, 395 (2000)CrossRefGoogle Scholar
  4. 4.
    Rosekind, M., Gander, P., Miller, D., Gregory, K., Smith, R., Weldon, K., Co, E., McNally, K., Lebacqz, J.: Fatigue in operational settings: examples from the aviation environment. J. Hum. Factors. 36, 327–338 (1994)CrossRefGoogle Scholar
  5. 5.
    Barr, L., Howarth, H., Popkin, S., Carroll, R.: A review and evaluation of emerging driver fatigue detection measures and technologies. John A. Volpe National Transportation Systems Center (2005)Google Scholar
  6. 6.
    Begum, S.: Intelligent driver monitoring systems based on physiological sensor signals: a review. In: IEEE Annual Conference on Intelligent Transportation Systems (ITSC) (2013)Google Scholar
  7. 7.
    Vogel, A., Fletcher, J., Maruff, P.: Acoustic analysis of the effects of sustained wakefulness on speech. J. Acoust. Soc. Am. 128, 3747–3756 (2010)CrossRefGoogle Scholar
  8. 8.
    Schuller, B., Batliner, A., Steidl, S., Schiel, F., Zrajewski, F.: The Interspeech 2011 Speaker state challenge. In: Proceedings of Interspeech 2011, pp. 2301–2304 (2011)Google Scholar
  9. 9.
    Huang, D., Ge, S., Zhang, Z.: Speaker State Classification Based on Fusion of Asymmetric SIMPLS and Support Vector Machines. In: Proceedings of the Interspeech 2011, pp. 3301–3304 (2011)Google Scholar
  10. 10.
    Krajewski, J., Batliner, A., Golz, M.: Acoustic sleepiness detection: Framework and validation of a speech-adapted pattern recognition approach. Behav. Res. Methods 41, 795–804 (2009)CrossRefGoogle Scholar
  11. 11.
    Kaida, K., Takahashi, M., Akerstedt, T., Nakata, A., Otsuka, Y., Haratani, T., Fukasawa, K.: Validation of the Karolinska sleepiness scale against performance and EEG variables. Clin. Neurophysiol. 117, 1574–1581 (2006)CrossRefGoogle Scholar
  12. 12.
    Gillberg, M., Kecklund, G., Akerstedt, T.: Relations between performance and subjective ratings of sleepiness during a night awake. J. Sleep 17, 236–241 (1994)CrossRefGoogle Scholar
  13. 13.
    Åhsberg, G., Kecklund, G., Åkerstedt, T., Gamberale, F.: Shiftwork and different dimensions of fatigue. Int. J. Ind. Ergon. 26, 457–465 (2000)CrossRefGoogle Scholar
  14. 14.
    Chawla, N., Bowyer, K., Hall, L., Kegelmeyer, W.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)zbMATHGoogle Scholar
  15. 15.
    Hsu, C., Chang, C., Lin, C.: A practical guide to support vector classification. Department of computer science technical report, National Taiwan University (2010)Google Scholar
  16. 16.
    Holte, R.: Very simple classification rules perform well. J. Mach. Learn. 11, 63–91 (1993)CrossRefzbMATHGoogle Scholar
  17. 17.
    Chen, S., Gopinath, R.: Gaussianization. In: Proceedings of the NIPS 2000, Denver Colorado (2000)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Khan Baykaner
    • 1
  • Mark Huckvale
    • 1
  • Iya Whiteley
    • 2
  • Oleg Ryumin
    • 3
  • Svetlana Andreeva
    • 3
  1. 1.Speech Hearing and Phonetic Sciences, UCLLondonUK
  2. 2.Centre for Space Medicine, UCLDorkingUK
  3. 3.Gagarin Cosmonaut Training CentreStar CityRussia

Personalised recommendations