In this paper a mono- and multi-lingual study is presented about the depressed speech detection possibilities. Beck Depression Inventory questionnaires were used for the description of severity of depression of speakers for all languages. In the mono-lingual experiment a detailed speech parameter selection is shown, and the analysis of the connection between the severity of the depression and the calculated parameters is presented. The goal was to select the most relevant input feature vectors from a preselected set of vectors for the detection and prediction methods of depression. A detailed examination was carried out where and how to measure these features in continuous speech. After parameter selection, classification experiments were conducted on a Hungarian speech database. The overall accuracies of the classification experiments were 86%. The second part of this study concerns a multi-lingual automatic depression detection and prediction method, where three European languages were tested: German, Hungarian and Italian. With the selected quasi language-independent parameters, Support Vector Regression experiments were conducted on German, Hungarian and Italian speech databases, separately for both sexes. It was found that depression prediction based on speech signals can be achieved in a multi-lingual way. Our method is even capable of predicting the severity of depression in the case of a language not used during the training of the automatic prediction model. The experiments clearly show that multi-lingual depression recognition can be achieved, and it should be possible to construct an automated diagnostic tool for detecting depression, or for patient monitoring, in a multi-lingual way.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Price excludes VAT (USA)
Tax calculation will be finalised during checkout.
Abela, J. R., & D’Alessandro, D. U. (2002). Beck’s cognitive theory of depression: A test of the diathesis-stress and causal mediation components. British Journal of Clinical Psychology, 41(2), 111–128.
Alghowinem, S., Goecke, R., Epps, J., Wagner, M., & Cohn, J. (2016). Cross-cultural depression recognition from vocal biomarkers. In Interspeech 2016 (pp. 1943–1947).
Alghowinem, S., Goecke, R., Wagner, M., Epps, J., Breakspear, M., & Parker, G. (2013). Detecting depression: A comparison between spontaneous and read speech. In 2013 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 7547–7551).
Boersma, P. P. G. (2002). Praat, a system for doing phonetics by computer. Glot International, 5(9/10), 341–345.
Chang, C. C., & Lin, C. J. (2011). LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.
Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.
Cummins, N., Scherer, S., Krajewski, J., Schnieder, S., Epps, J., & Quatieri, T. F. (2015a). A review of depression and suicide risk assessment using speech analysis. Speech Communication, 71, 10–49.
Cummins, N., Sethu, V., Epps, J., Schnieder, S., & Krajewski, J. (2015b). Analysis of acoustic space variability in speech affected by depression. Speech Communication, 75, 27–49.
Cummins, N., Sethu, V., Epps, J., & Krajewski, J. (2015c). Relevance vector machine for depression prediction. In Sixteenth annual conference of the international speech communication association (pp. 110–114).
France, D. J., Shiavi, R. G., Silverman, S., Silverman, M., & Wilkes, M. (2000). Acoustical properties of speech as indicators of depression and suicidal risk. IEEE Transactions on Biomedical Engineering, 47(7), 829–837.
Hawton, K., i Comabella, C. C., Haw, C., & Saunders, K. (2013). Risk factors for suicide in individuals with depression: A systematic review. Journal of Affective Disorders, 147(1), 17–28.
Helfer, B. S., Quatieri, T. F., Williamson, J. R., Mehta, D. D., Horwitz, R., & Yu, B. (2013). Classification of depression state based on articulatory precision. In Interspeech 2013 (pp. 2172–2176).
Jiang, H., Hu, B., Liu, Z., Yan, L., Wang, T., Liu, F., Kang, H., & Li, X. (2017). Investigation of different speech types and emotions for detecting depression using different classifiers. Speech Communication, 90, 39–46.
Kiss, G., Sztahó, D., & Vicsi, K. (2013). Language independent automatic speech segmentation into phoneme-like units on the base of acoustic distinctive features. In 2013 IEEE 4th international conference on cognitive infocommunications (CogInfoCom) (pp. 579–582). IEEE.
Kiss, G., Tulics, M. G., Sztahó, D., Esposito, A., & Vicsi, K. (2016). Language independent detection possibilities of depression by speech. In A. Esposito, M. Faundez-Zanuy, A. M. Esposito, G. Cordasco, T. Drugman, J. Solé-Casals, F. C. Morabito (Eds.), Recent advances in nonlinear speech processing (pp. 103–114). New York: Springer.
Kiss, G., & Vicsi, K. (2014). Physiological and cognitive status monitoring on the base of acoustic-phonetic speech parameters. In International conference on statistical language and speech processing (pp. 120–131). New York: Springer.
Kiss, G., & Vicsi, K. (2015). Seasonal affective disorder speech detection on the base of acoustic phonetic speech parameters. Acta Universitatis Sapientiae Electrical and Mechanical Engineering, 7, 62–79.
Kotti, M., & Paternò, F. (2012). Speaker-independent emotion recognition exploiting a psychologically-inspired binary cascade classification schema. International Journal of Speech Technology, 15(2), 131–150.
Kraepelin, E. (1921). Manic depressive insanity and paranoia. The Journal of Nervous and Mental Disease, 53(4), 350.
Lépine, J. P., & Briley, M. (2011). The increasing burden of depression. Neuropsychiatric Disease and Treatment, 7(Suppl 1), 3.
Liu, Z., Hu, B., Yan, L., Wang, T., Liu, F., Li, X., & Kang, H. (2015). Detection of depression in speech. In 2015 international conference on affective computing and intelligent interaction (ACII) (pp. 743–747), IEEE.
Low, L. S. A., Maddage, N. C., Lech, M., Sheeber, L. B., & Allen, N. B. (2011). Detection of clinical depression in adolescents’ speech during family interactions. IEEE Transactions on Biomedical Engineering, 58(3), 574–586.
Marcus, M., Yasamy, M. T., van Ommeren, M., Chisholm, D., & Saxena, S. (2012). Depression: A global public health concern. WHO Department of Mental Health and Substance Abuse, 1, 6–8.
Mathers, C. D., & Loncar, D. (2006). Projections of global mortality and burden of disease from 2002 to 2030. PLoS Medicine, 3(11), e442.
Mundt, J. C., Snyder, P. J., Cannizzaro, M. S., Chappie, K., & Geralts, D. S. (2007). Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology. Journal of Neurolinguistics, 20(1), 50–64.
Mundt, J. C., Vogel, A. P., Feltner, D. E., & Lenderking, W. R. (2012). Vocal acoustic biomarkers of depression severity and treatment response. Biological Psychiatry, 72(7), 580–587.
Scherer, K. R. (1986). Vocal affect expression: A review and a model for future research. Psychological Bulletin, 99(2), 143.
Smola, A., & Vapnik, V. (1997). Support vector regression machines. Advances in neural information processing systems, 9, 155–161.
Sztahó, D., Imre, V., & Vicsi, K. (2011). Automatic classification of emotions in spontaneous speech. Analysis of verbal and nonverbal communication and enactment. The processing issues (pp. 229–239). Berlin Heidelberg: Springer.
Sztahó, D., Kiss, G., Czap, L., & Vicsi, K. (2014). A computer-assisted prosody pronunciation teaching system. In WOCCI 2014 satellite workshop of interspeech singapore, paper Sztaho14-ACP.
Valstar, M., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., & Pantic, M. (2013). AVEC 2013: the continuous audio/visual emotion and depression recognition challenge. In Proceedings of the 3rd ACM international workshop on audio/visual emotion challenge (pp. 3–10). ACM.
Williamson, J. R., Quatieri, T. F., Helfer, B. S., Horwitz, R., Yu, B., & Mehta, D. D. (2013). Vocal biomarkers of depression based on motor incoordination. In Proceedings of the 3rd ACM international workshop on audio/visual emotion challenge (pp. 41–48). ACM.
We would like to thank Björn Schuller and his co-workers Jarek Krajewski and Sonja-Dana Roelena for sharing with us the database of AVEC 2013 for research purposes. They gave us the possibility to extend our multi-lingual research. We also thank Anna Esposito for the Italian depressed speech database. The research was supported by European Space Agency COALA project: Psychological Status Monitoring by Computerised Analysis of Language phenomena (COALA) (AO-11-Concordia).
About this article
Cite this article
Kiss, G., Vicsi, K. Mono- and multi-lingual depression prediction based on speech processing. Int J Speech Technol 20, 919–935 (2017). https://doi.org/10.1007/s10772-017-9455-8