Abstract
Depressive symptoms in young people may persist into adulthood and develop into depression. Early screening of depressive tendencies in university students helps to reduce the number and intensity of their depressive episodes. Automatic prediction of depression from visual clues has gained increasing interest in recent years. This study proposes a deep learning model for depressive tendencies prediction, which extracts and fuses the dynamic facial features when participants were uttering positive text materials. The effectiveness of the proposed method is validated on our original multimodal behavioral dataset of Chinese university students. The results demonstrated that dynamic facial expression features can potentially reveal depressive tendencies. The average detection accuracy of individually uttered sentence is 67.32%, increasing to 71.4% when multiple sentences were fused.
Jia-Qing Liu and Yue Huang contributed equally to this work and they are co-first authors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Lei, X.Y., Xiao, L.M., Liu, Y.N., Li, Y.M.: Prevalence of depression among chinese university students: a meta-analysis. PLoS ONE 11(4), e0153454 (2016)
Bhui, K.: Depression and dementia in old age: improving effective care and prevention. Br. J. Psych. 214(4), 244–244 (2019)
Jan, A., Meng, H., Gaus, Y. FB.A., Zhang, F.: Artificial intelligent system for automatic depression level analysis through visual and vocal expressions. IEEE Trans. Cogn. Dev. Syst. 99, 1–1 (2017)
Dibeklioğlu, H., Hammal, Z., Cohn, J.F.: Dynamic multimodal measurement of depression severity using deep autoencoding. IEEE J. Biomed. Health Inf. 22(2), 525–536 (2018)
Jan, A., Meng, H., Gaus, Y. F.A., Zhang, F., Turabzadeh, S.: Automatic depression scale prediction using facial expression dynamics and regression. In: Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge, pp. 73–80. ACM (2014, Nov)
Girard, J.M., Cohn, J.F., Mahoor, M.H., Mavadati, S., Rosenwald, D.P.: Social risk and depression: Evidence from manual and automatic facial expression analysis. In: 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp. 1–8. IEEE (2013, April)
Yang, L., Jiang, D., Xia, X., Pei, E., Oveneke, M.C., Sahli, H.: Multimodal measurement of depression using deep learning models. In: Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, pp 53–59. ACM (2017, Oct)
Berenbaum, H.: Posed facial expressions of emotion in schizophrenia and depression. Psychol. Med. 22(4), 929–937 (1992)
Pampouchidou, A., Simos, P., Marias, K., Meriaudeau, F., Yang, F., Pediaditis, M., Tsiknakis, M.: Automatic assessment of depression based on visual cues: a systematic review. IEEE Trans. Affect. Comput. (2017)
Parameswaran, N.S., Venkataraman, D.: A computer vision based image processing system for depression detection among students for counseling. Indones. J. Electr. Eng. Comput. Sci. 14(1), 503–512 (2019)
Parkhi, OM., Vedlda, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference 2015, Section 3, pp. 41.1–41.12 (2015)
Afouras, T., Chung, JS., Senior, A., Vinyals, O., Zisserman, A.: Deep audio-visual speech recognition. In: IEEE Transactions on Pattern Analysis and Machine Intelligence (2018)
Chung, J.S., Zisserman, A.: Lip reading in the wild. In: Proceedings of the Asian Conference on Computer Vision (2016)
Yuan, J., Liberman, M.: Speaker identification on the SCOTUS corpus. J. Acoust. Soc. Am. 123(5), 3878 (2008)
Dlib: http://dlib.net/
Liu, J.Q., Huang, Y., Huang, X.Y., Xia, X.T., Niu, X.X., Chen, Y.W.: Multimodal behavioral dataset of depressive symptoms in chinese college students–preliminary study. In: Chen, Y.W. et. al. (eds.) Innovation in Medicine and Healthcare Systems, and Multimedia. Smart Innovation, Systems and Technologies, vol. 145, pp. 79–190. Springer, Singapore (2019)
Pan, W., Wang, J.Y., Liu, T.L., Liu, X.Q., Liu, M.M., Hu, B., Zhu, T.S.: Speech-based depression recognition. Sci. Bull. 63(20), 2081–2092 (2018)
Beck, A.T., Steer, R.A., Brown, G.K.: Beck depression inventory-II. San Antonio 78(2), 490–498 (1996)
Radloff, L.S.: The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Meas. 1(3), 385–401 (1977)
Keras: https://keras.io/
Acknowledgments
This work is supported in part by the Japan Society of Promotion of Science (JSPS) under Grant No. 20J13009.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Liu, JQ. et al. (2020). Dynamic Facial Features in Positive-Emotional Speech for Identification of Depressive Tendencies. In: Chen, YW., Tanaka, S., Howlett, R., Jain, L. (eds) Innovation in Medicine and Healthcare. Smart Innovation, Systems and Technologies, vol 192. Springer, Singapore. https://doi.org/10.1007/978-981-15-5852-8_12
Download citation
DOI: https://doi.org/10.1007/978-981-15-5852-8_12
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-5851-1
Online ISBN: 978-981-15-5852-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)