Skip to main content
SpringerLink
Log in

You Sound Relaxed Now – Measuring Restorative Effects from Speech Signals

  • Conference paper
  • First Online:
Human-Computer Interaction – INTERACT 2021 (INTERACT 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12933))

Included in the following conference series:

  • 2668 Accesses

Abstract

The recently proposed restorative environments have the potential to restore attention and help against fatigue, but how can these effects be verified? We present a novel measurement method which can analyze participants’ speech signals in a study before and after a relaxing experience. Compared to other measurements such as attention scales or response tests, speech signal analysis is both less obtrusive and more accessible. In our study, we found that certain time- and frequency- domain speech features such as short-time energy and Mel Frequency Cepstral Coefficients (MFCC) are correlated with the attentional capacity measured by traditional ratings. We thus argue that speech signal analysis can provide a valid measure for attention and its restoration. We describe a practically feasible method for such a speech signal analysis along with some preliminary results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://www.youtube.com/watch?v=GlCazmVBUMg.

References

  1. Benesty, J., Chen, J., Huang, Y., Cohen, I.: Pearson correlation coefficient. In: Noise Reduction in Speech Processing, pp. 1–4. Springer, Vienna (2009). https://doi.org/10.1007/978-3-211-89836-9_1025

  2. Biesmans, W., Das, N., Francart, T., Bertrand, A.: Auditory-inspired speech envelope extraction methods for improved EEG-based auditory attention detection in a cocktail party scenario. IEEE Trans. Neural Syst. Rehabil. Eng. 25(5), 402–412 (2016)

    Article  Google Scholar 

  3. Braun, M., Weiser, S., Pfleging, B., Alt, F.: A comparison of emotion elicitation methods for affective driving studies. Presented at the (2018)

    Google Scholar 

  4. Carreiras, C., Lourenço, A., Aidos, H., da Silva, H.P., Fred, A.L.N.: Unsupervised analysis of morphological ECG features for attention detection. In: Madani, K., Dourado, A., Rosa, A., Filipe, J., Kacprzyk, J. (eds.) Computational Intelligence. SCI, vol. 613, pp. 437–453. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-23392-5_24

    Chapter  Google Scholar 

  5. Cimprich, B., Visovatti, M., Ronis, D.L.: The attentional function index-a self-report cognitive measure. Psychooncology 20(2), 194–202 (2011)

    Article  Google Scholar 

  6. Dhupati, L.S., Kar, S., Rajaguru, A., Routray, A.: A novel drowsiness detection scheme based on speech analysis with validation using simultaneous EEG recordings, pp. 917–921. IEEE (2010)

    Google Scholar 

  7. Franěk, M., Šefara, D., Petružálek, J., Cabal, J., Myška, K.: Differences in eye movements while viewing images with various levels of restorativeness. J. Environ. Psychol. 57, 10–16 (2018)

    Article  Google Scholar 

  8. Gao, B., Woo, W.L.: Wearable audio monitoring: content-based processing methodology and implementation. IEEE Trans. Hum. Mach. Syst. 44(2), 222–233 (2014)

    Article  Google Scholar 

  9. Gunn, S.R., et al.: Support vector machines for classification and regression. ISIS Technical Report 14(1), 5–16 (1998)

    Google Scholar 

  10. Guo, G., Wang, H., Bell, D., Bi, Y., Greer, K.: KNN model-based approach in classification. In: Meersman, R., Tari, Z., Schmidt, D.C. (eds.) OTM 2003. LNCS, vol. 2888, pp. 986–996. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39964-3_62

    Chapter  Google Scholar 

  11. Hogg, A.O., Evers, C., Naylor, P.A.: Speaker change detection using fundamental frequency with application to multi-talker segmentation, pp. 5826–5830. IEEE (2019)

    Google Scholar 

  12. Jalil, M., Butt, F.A., Malik, A.: Short-time energy, magnitude, zero crossing rate and autocorrelation measurement for discriminating voiced and unvoiced segments of speech signals, pp. 208–212. IEEE (2013)

    Google Scholar 

  13. Jiang, D., Hu, B., Chen, Y., Xue, Y., Li, W., Liang, Z.: Recognizing the human attention state using cardiac pulse from the noncontact and automatic-based measurements. Soft. Comput. 22(12), 3937–3949 (2018)

    Article  Google Scholar 

  14. Jingyi, L., Yong, M., Puzhen, L., Andreas, B.: A journey through nature: exploring virtual restorative environments as a means to relax in confined spaces. Association for Computing Machinery, New York, NY, USA (2021)

    Google Scholar 

  15. Joshi, D.D., Zalte, M.: Speech emotion recognition: a review. IOSR J. Electron. Commun. Eng. (IOSR-JECE) 4(4) (2013)

    Google Scholar 

  16. Lam, K.Y., et al.: Smartmood: toward pervasive mood tracking and analysis for manic episode detection. IEEE Trans. Hum. Mach. Syst. 45(1), 126–131 (2014)

    Article  Google Scholar 

  17. Li, X., Tan, N., Wang, T., Su, S.: Detecting driver fatigue based on nonlinear speech processing and fuzzy SVM, pp. 510–515. IEEE (2014)

    Google Scholar 

  18. Maurya, A., Kumar, D., Agarwal, R.: Speaker recognition for Hindi speech signal using MFCC-GMM approach. Procedia Comput. Sci. 125, 880–887 (2018)

    Article  Google Scholar 

  19. Narayanan, A.M., Bertrand, A.: Analysis of miniaturization effects and channel selection strategies for EEG sensor networks with application to auditory attention detection. IEEE Trans. Biomed. Eng. 67(1), 234–244 (2019)

    Article  Google Scholar 

  20. Ohly, H., et al.: Attention restoration theory: a systematic review of the attention restoration potential of exposure to natural environments. J. Toxicol. Environ. Health, Part B 19(7), 305–343 (2016)

    Google Scholar 

  21. Popov, D., Gapochkin, A., Nekrasov, A.: An algorithm of Daubechies wavelet transform in the final field when processing speech signals. Electronics 7(7), 120 (2018)

    Google Scholar 

  22. Rocha, R.B., Freire, V.V., Alencar, M.S.: Voice segmentation system based on energy estimation, pp. 860–864. IEEE (2014)

    Google Scholar 

  23. Schuller, B.W.: Intelligent Audio Analysis. Signals and Communication Technology, Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36806-6

    Book  Google Scholar 

  24. Stevenson, M.P., Schilhab, T., Bentsen, P.: Attention restoration theory ii: a systematic review to clarify attention processes affected by exposure to natural environments. J. Toxicol. Environ. Health Part B 21(4), 227–268 (2018)

    Google Scholar 

  25. Stone, R., Small, C., Knight, J., Qian, C., Shingari, V.: Virtual natural environments for restoration and rehabilitation in healthcare. Virtual Augment. Real. Ser. Games Healthc. 1, 497–521 (2014)

    Google Scholar 

  26. Sun, Y.X., Ma, Y., Shi, K.B., Hu, J.P., Zhao, Y.Y., Zhang, Y.P.: Unsupervised speaker segmentation framework based on sparse correlation feature, pp. 3058–3063. IEEE (2017)

    Google Scholar 

  27. Visuri, A., van Berkel, N.: Attention computing: overview of mobile sensing applied to measuring attention. Presented at the (2019)

    Google Scholar 

  28. Wieland, B., Urban, K., Funken, S.: Speech signal noise reduction with wavelets. Verlag nicht ermittelbar, Ph.D. thesis (2009)

    Google Scholar 

  29. Wilcoxon, F.: Individual comparisons by ranking methods. In: Kotz, S., Johnson, N.L. (eds.) Breakthroughs in Statistics, pp. 196–202. Springer, New York (1992). https://doi.org/10.1007/978-1-4612-4380-9_16

    Chapter  Google Scholar 

  30. Woods, D.L., et al.: Improving digit span assessment of short-term verbal memory. J. Clin. Exp. Neuropsychol. 33(1), 101–111 (2011)

    Article  Google Scholar 

  31. Yang, Y., et al.: Kullback-Leibler divergence frequency warping scale for acoustic scene classification using convolutional neural network, pp. 840–844. IEEE (2019)

    Google Scholar 

  32. Yella, S.H., Stolcke, A., Slaney, M.: Artificial neural network features for speaker diarization, pp. 402–406. IEEE (2014)

    Google Scholar 

Download references

Acknowledgements

We thank all study participants for their time and effort, as well as our anonymous reviewers for their valuable feedback. Y.M.’s contributions were funded by the China Scholarship Council (CSC), grant number 201706070119.

Author information

Authors and Affiliations

  1. LMU Munich, Munich, Germany

    Yong Ma, Jingyi Li, Heiko Drewes & Andreas Butz

Authors
  1. Yong Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jingyi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Heiko Drewes
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andreas Butz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Ma .

Editor information

Editors and Affiliations

  1. Department of Electrical and Information Engineering, Polytechnic University of Bari, Bari, Italy

    Carmelo Ardito

  2. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Rosa Lanzilotti

  3. Computer Science Department, University of Pisa, Pisa, Italy

    Alessio Malizia

  4. Department of Computer Science, University of York, York, UK

    Helen Petrie

  5. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Antonio Piccinno

  6. Computer Science Department, University of Bari Aldo Moro, Bari, Italy

    Giuseppe Desolda

  7. Microsoft Research, Redmond, WA, USA

    Kori Inkpen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ma, Y., Li, J., Drewes, H., Butz, A. (2021). You Sound Relaxed Now – Measuring Restorative Effects from Speech Signals. In: Ardito, C., et al. Human-Computer Interaction – INTERACT 2021. INTERACT 2021. Lecture Notes in Computer Science(), vol 12933. Springer, Cham. https://doi.org/10.1007/978-3-030-85616-8_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85616-8_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85615-1

  • Online ISBN: 978-3-030-85616-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation