Skip to main content
SpringerLink
Log in

Mono- and multi-lingual depression prediction based on speech processing

International Journal of Speech Technology volume 20, pages 919–935 (2017)Cite this article

Abstract

In this paper a mono- and multi-lingual study is presented about the depressed speech detection possibilities. Beck Depression Inventory questionnaires were used for the description of severity of depression of speakers for all languages. In the mono-lingual experiment a detailed speech parameter selection is shown, and the analysis of the connection between the severity of the depression and the calculated parameters is presented. The goal was to select the most relevant input feature vectors from a preselected set of vectors for the detection and prediction methods of depression. A detailed examination was carried out where and how to measure these features in continuous speech. After parameter selection, classification experiments were conducted on a Hungarian speech database. The overall accuracies of the classification experiments were 86%. The second part of this study concerns a multi-lingual automatic depression detection and prediction method, where three European languages were tested: German, Hungarian and Italian. With the selected quasi language-independent parameters, Support Vector Regression experiments were conducted on German, Hungarian and Italian speech databases, separately for both sexes. It was found that depression prediction based on speech signals can be achieved in a multi-lingual way. Our method is even capable of predicting the severity of depression in the case of a language not used during the training of the automatic prediction model. The experiments clearly show that multi-lingual depression recognition can be achieved, and it should be possible to construct an automated diagnostic tool for detecting depression, or for patient monitoring, in a multi-lingual way.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  • Abela, J. R., & D’Alessandro, D. U. (2002). Beck’s cognitive theory of depression: A test of the diathesis-stress and causal mediation components. British Journal of Clinical Psychology, 41(2), 111–128.

    Article  Google Scholar 

  • Alghowinem, S., Goecke, R., Epps, J., Wagner, M., & Cohn, J. (2016). Cross-cultural depression recognition from vocal biomarkers. In Interspeech 2016 (pp. 1943–1947).

  • Alghowinem, S., Goecke, R., Wagner, M., Epps, J., Breakspear, M., & Parker, G. (2013). Detecting depression: A comparison between spontaneous and read speech. In 2013 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 7547–7551).

  • Boersma, P. P. G. (2002). Praat, a system for doing phonetics by computer. Glot International, 5(9/10), 341–345.

    Google Scholar 

  • Chang, C. C., & Lin, C. J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.

    Google Scholar 

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.

    MATH  Google Scholar 

  • Cummins, N., Scherer, S., Krajewski, J., Schnieder, S., Epps, J., & Quatieri, T. F. (2015a). A review of depression and suicide risk assessment using speech analysis. Speech Communication, 71, 10–49.

    Article  Google Scholar 

  • Cummins, N., Sethu, V., Epps, J., Schnieder, S., & Krajewski, J. (2015b). Analysis of acoustic space variability in speech affected by depression. Speech Communication, 75, 27–49.

    Article  Google Scholar 

  • Cummins, N., Sethu, V., Epps, J., & Krajewski, J. (2015c). Relevance vector machine for depression prediction. In Sixteenth annual conference of the international speech communication association (pp. 110–114).

  • France, D. J., Shiavi, R. G., Silverman, S., Silverman, M., & Wilkes, M. (2000). Acoustical properties of speech as indicators of depression and suicidal risk. IEEE Transactions on Biomedical Engineering, 47(7), 829–837.

    Article  Google Scholar 

  • Hawton, K., i Comabella, C. C., Haw, C., & Saunders, K. (2013). Risk factors for suicide in individuals with depression: A systematic review. Journal of Affective Disorders, 147(1), 17–28.

    Article  Google Scholar 

  • Helfer, B. S., Quatieri, T. F., Williamson, J. R., Mehta, D. D., Horwitz, R., & Yu, B. (2013). Classification of depression state based on articulatory precision. In Interspeech 2013 (pp. 2172–2176).

  • Jiang, H., Hu, B., Liu, Z., Yan, L., Wang, T., Liu, F., Kang, H., & Li, X. (2017). Investigation of different speech types and emotions for detecting depression using different classifiers. Speech Communication, 90, 39–46.

    Article  Google Scholar 

  • Kiss, G., Sztahó, D., & Vicsi, K. (2013). Language independent automatic speech segmentation into phoneme-like units on the base of acoustic distinctive features. In 2013 IEEE 4th international conference on cognitive infocommunications (CogInfoCom) (pp. 579–582). IEEE.

  • Kiss, G., Tulics, M. G., Sztahó, D., Esposito, A., & Vicsi, K. (2016). Language independent detection possibilities of depression by speech. In A. Esposito, M. Faundez-Zanuy, A. M. Esposito, G. Cordasco, T. Drugman, J. Solé-Casals, F. C. Morabito (Eds.), Recent advances in nonlinear speech processing (pp. 103–114). New York: Springer.

    Chapter  Google Scholar 

  • Kiss, G., & Vicsi, K. (2014). Physiological and cognitive status monitoring on the base of acoustic-phonetic speech parameters. In International conference on statistical language and speech processing (pp. 120–131). New York: Springer.

  • Kiss, G., & Vicsi, K. (2015). Seasonal affective disorder speech detection on the base of acoustic phonetic speech parameters. Acta Universitatis Sapientiae Electrical and Mechanical Engineering, 7, 62–79.

    Google Scholar 

  • Kotti, M., & Paternò, F. (2012). Speaker-independent emotion recognition exploiting a psychologically-inspired binary cascade classification schema. International Journal of Speech Technology, 15(2), 131–150.

    Article  Google Scholar 

  • Kraepelin, E. (1921). Manic depressive insanity and paranoia. The Journal of Nervous and Mental Disease, 53(4), 350.

    Article  Google Scholar 

  • Lépine, J. P., & Briley, M. (2011). The increasing burden of depression. Neuropsychiatric Disease and Treatment, 7(Suppl 1), 3.

    Google Scholar 

  • Liu, Z., Hu, B., Yan, L., Wang, T., Liu, F., Li, X., & Kang, H. (2015). Detection of depression in speech. In 2015 international conference on affective computing and intelligent interaction (ACII) (pp. 743–747), IEEE.

  • Low, L. S. A., Maddage, N. C., Lech, M., Sheeber, L. B., & Allen, N. B. (2011). Detection of clinical depression in adolescents’ speech during family interactions. IEEE Transactions on Biomedical Engineering, 58(3), 574–586.

    Article  Google Scholar 

  • Marcus, M., Yasamy, M. T., van Ommeren, M., Chisholm, D., & Saxena, S. (2012). Depression: A global public health concern. WHO Department of Mental Health and Substance Abuse, 1, 6–8.

    Google Scholar 

  • Mathers, C. D., & Loncar, D. (2006). Projections of global mortality and burden of disease from 2002 to 2030. PLoS Medicine, 3(11), e442.

    Article  Google Scholar 

  • Mundt, J. C., Snyder, P. J., Cannizzaro, M. S., Chappie, K., & Geralts, D. S. (2007). Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology. Journal of Neurolinguistics, 20(1), 50–64.

    Article  Google Scholar 

  • Mundt, J. C., Vogel, A. P., Feltner, D. E., & Lenderking, W. R. (2012). Vocal acoustic biomarkers of depression severity and treatment response. Biological Psychiatry, 72(7), 580–587.

    Article  Google Scholar 

  • Scherer, K. R. (1986). Vocal affect expression: A review and a model for future research. Psychological Bulletin, 99(2), 143.

    Article  Google Scholar 

  • Smola, A., & Vapnik, V. (1997). Support vector regression machines. Advances in neural information processing systems, 9, 155–161.

    Google Scholar 

  • Sztahó, D., Imre, V., & Vicsi, K. (2011). Automatic classification of emotions in spontaneous speech. Analysis of verbal and nonverbal communication and enactment. The processing issues (pp. 229–239). Berlin Heidelberg: Springer.

    Google Scholar 

  • Sztahó, D., Kiss, G., Czap, L., & Vicsi, K. (2014). A computer-assisted prosody pronunciation teaching system. In WOCCI 2014 satellite workshop of interspeech singapore, paper Sztaho14-ACP.

  • Valstar, M., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., & Pantic, M. (2013). AVEC 2013: the continuous audio/visual emotion and depression recognition challenge. In Proceedings of the 3rd ACM international workshop on audio/visual emotion challenge (pp. 3–10). ACM.

  • Williamson, J. R., Quatieri, T. F., Helfer, B. S., Horwitz, R., Yu, B., & Mehta, D. D. (2013). Vocal biomarkers of depression based on motor incoordination. In Proceedings of the 3rd ACM international workshop on audio/visual emotion challenge (pp. 41–48). ACM.

Download references

Acknowledgements

We would like to thank Björn Schuller and his co-workers Jarek Krajewski and Sonja-Dana Roelena for sharing with us the database of AVEC 2013 for research purposes. They gave us the possibility to extend our multi-lingual research. We also thank Anna Esposito for the Italian depressed speech database. The research was supported by European Space Agency COALA project: Psychological Status Monitoring by Computerised Analysis of Language phenomena (COALA) (AO-11-Concordia).

Author information

Authors and Affiliations

  1. Department of Telecommunications and Media Informatics, Faculty of Electrical Engineering and Informatics, Budapest University of Technology and Economics, Budapest, Hungary

    Gabor Kiss & Klara Vicsi

Authors
  1. Gabor Kiss
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Klara Vicsi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gabor Kiss.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kiss, G., Vicsi, K. Mono- and multi-lingual depression prediction based on speech processing. Int J Speech Technol 20, 919–935 (2017). https://doi.org/10.1007/s10772-017-9455-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10772-017-9455-8

Keywords

search

Navigation