Skip to main content
SpringerLink
Log in

Data augmentation for depression detection using skeleton-based gait information

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

In recent years, the incidence of depression is rising rapidly worldwide, but large-scale depression screening is still challenging. Gait analysis provides a non-contact, low-cost, and efficient early screening method for depression. However, the early screening of depression based on gait analysis lacks sufficient effective sample data. In this paper, we propose a skeleton data augmentation method for assessing the risk of depression. First, we propose five techniques to augment skeleton data and apply them to depression and emotion datasets. Then, we divide augmentation methods into two types (non-noise augmentation and noise augmentation) based on the mutual information and the classification accuracy. Finally, we explore which augmentation strategies can capture the characteristics of human skeleton data more effectively. Experimental results show that the augmented training dataset that retains more of the raw skeleton data properties determines the performance of the detection model. Specifically, rotation augmentation and channel mask augmentation make the depression detection accuracy reach 92.15% and 91.34%, respectively.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Microsoft kinect. https://msdn.microsoft.com, accessed on 8 March 2022

  2. National health commission. http://www.chinanews.com, accessed on 19 March 2022

  3. World health organization. https://www.who.int/news-room/fact-sheets/detail/depression, accessed on 11 March 2022

  4. Bau D, Zhu JY, Strobelt H, Zhou B, Tenenbaum JB, Freeman WT, Torralba A (2018) Gan dissection: Visualizing and understanding generative adversarial networks. arXiv preprint arXiv:181110597

  5. Beck AT, Alford BA (2009) Depression: Causes and treatment. University of Pennsylvania Press

  6. Bhattacharya U, Mittal T, Chandra R, Randhavane T, Bera A, Manocha D (2020) Step: Spatial temporal graph convolutional networks for emotion perception from gaits. Proceedings of the AAAI Conference on Artificial Intelligence 34:1342–1350

    Article  Google Scholar 

  7. Branco L, Cotrena C, Ponsoni A, Salvador-Silva R, Vasconcellos S, Fonseca R (2018) Identification and perceived intensity of facial expressions of emotion in bipolar disorder and major depression. Arch Clin Neuropsychol 33(4):491–501

    Article  CAS  Google Scholar 

  8. Chen P, Liu S, Zhao H, Jia J (2020) Gridmask data augmentation

  9. Crane E, Gross M (2007) Motion capture and emotion: Affect detection in whole body movement. In: Paiva ACR, Prada R, Picard RW (eds) Affective Computing and Intelligent Interaction. Springer, Berlin Heidelberg, Berlin, Heidelberg, pp 95–101

    Chapter  Google Scholar 

  10. Cubuk ED, Zoph B, Mane D, Vasudevan V, Le QV (2018) Autoaugment: Learning augmentation policies from data

  11. Doersch C (2016) Tutorial on variational autoencoders. arXiv preprint arXiv:160605908

  12. Fang J, Wang T, Li C, Hu X, Ngai E, Seet BC, Cheng J, Guo Y, Jiang X (2019) Depression prevalence in postgraduate students and its association with gait abnormality. IEEE Access 7:174425–174437

    Article  Google Scholar 

  13. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  14. Gross MM, Crane EA, Fredrickson BL (2012) Effort-shape and kinematic assessment of bodily expression of emotion during gait. Hum Mov Sci 31(1):202–221

    Article  Google Scholar 

  15. Gwosdek P, Grewenig S, Bruhn A, Weickert J (2011) Theoretical foundations of gaussian convolution by extended box filtering. In: International Conference on Scale Space and Variational Methods in Computer Vision, Springer, pp 447–458

  16. Harris EJ, Khoo IH, Demircan E (2022) A survey of human gait-based artificial intelligence applications. Frontiers in Robotics and AI 8

  17. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  CAS  Google Scholar 

  18. Hussain Z, Gimenez F, Yi D, Rubin D (2017a) Differential data augmentation techniques for medical imaging classification tasks. In: AMIA Annual Symposium Proceedings, American Medical Informatics Association, vol 2017, p 979

  19. Hussain Z, Gimenez F, Yi D, Rubin D (2017b) Differential data augmentation techniques for medical imaging classification tasks

  20. Kang GE, Gross MM (2016) The effect of emotion on movement smoothness during gait in healthy young adults. Journal of Biomechanics p S0021929016311459

  21. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 4401–4410

  22. Kingma D, Ba J (2014) Adam: A method for stochastic optimization. Computer Science

  23. Kingma DP, Welling M (2013) Auto-encoding variational bayes. arXiv preprint arXiv:13126114

  24. Korf J (2015) Qualia in a contemporary neurobiological perspective. Dialogues in Philosophy Mental & Neuro Sciences

  25. Kroenke K, Spitzer RL, Williams JB (2001) The phq-9: validity of a brief depression severity measure. J Gen Intern Med 16(9):606–613

    Article  CAS  Google Scholar 

  26. Lea C, Vidal R, Reiter A, Hager GD (2016) Temporal convolutional networks: A unified approach to action segmentation. In: European Conference on Computer Vision, Springer, pp 47–54

  27. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. nature 521(7553):436–444

    CAS  PubMed  Google Scholar 

  28. Lemke MR, Wendorff T, Mieth B, Buhl K, Linnemann M (2000) Spatiotemporal gait patterns during over ground locomotion in major depression compared with healthy controls. J Psychiatr Res 34(4–5):277–283

    Article  CAS  Google Scholar 

  29. Lu H, Shao W, Ngai E, Hu X, Hu B (2021) A new skeletal representation based on gait for depression detection. In: 2020 IEEE International Conference on E-health Networking, Application Services (HEALTHCOM), pp 1–6

  30. Lu H, Xu S, Hu X, Ngai E, Guo Y, Wang W, Hu B (2022) Postgraduate student depression assessment by multimedia gait analysis. IEEE MultiMedia pp 1

  31. Michalak J, Troje NF, Fischer J, Vollmar P, Heidenreich T, Schulte D (2009) Embodiment of sadness and depression-gait patterns associated with dysphoric mood. Psychosom Med 71(5):580–587

    Article  Google Scholar 

  32. Moreno-Barea FJ, Strazzera F, Jerez JM, Urda D, Franco L (2018) Forward noise adjustment scheme for data augmentation. In: 2018 IEEE Symposium Series on Computational Intelligence (SSCI), IEEE, pp 728–734

  33. Pessoa Luiz (2017) A network model of the emotional brain. Trends Cogn Sci 21(5):357

    Article  Google Scholar 

  34. Ringeval F, Schuller B, Valstar M, Cummins N, Cowie R, Tavabi L, Schmitt M, Alisamir S, Amiriparian S, Messner EM, et al. (2019) Avec 2019 workshop and challenge: state-of-mind, detecting depression with ai, and cross-cultural affect recognition. In: Proceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop, pp 3–12

  35. Sanders JB, Bremmer MA, Comijs HC, Deeg DJ, Beekman AT (2016) Gait speed and the natural course of depressive symptoms in late life; an independent association with chronicity? J Am Med Dir Assoc 17(4):331–335

    Article  Google Scholar 

  36. Sanders RD, Gillig PM (2010) Gait and its assessment in psychiatry. Psychiatry (Edgmont) 7(7):38

    Google Scholar 

  37. Sapiński T, Kamińska D, Pelikant A, Ozcinar C, Avots E, Anbarjafari G (2018) Multimodal database of emotional speech, video and gestures. In: International Conference on Pattern Recognition, Springer, pp 153–163

  38. Shang Y, Pan Y, Jiang X, Shao Z, Guo G, Liu T, Ding H (2021) Lqgdnet: A local quaternion and global deep network for facial depression recognition. IEEE Transactions on Affective Computing pp 1

  39. Shao W, You Z, Liang L, Hu X, Li C, Wang W, Hu B (2021) A multi-modal gait analysis-based depression detection system. IEEE Journal of Biomedical and Health Informatics pp 1

  40. Shen J, Zhang X, Huang X, Wu M, Gao J, Lu D, Ding Z, Hu B (2021) An optimal channel selection for eeg-based depression detection via kernel-target alignment. IEEE J Biomed Health Inform 25(7):2545–2556

    Article  Google Scholar 

  41. Shorten C, Khoshgoftaar TM (2019a) A survey on image data augmentation for deep learning. Journal of Big Data 6(1):1–48

  42. Shorten C, Khoshgoftaar TM (2019b) A survey on image data augmentation for deep learning. Journal of Big Data 6(1):1–48

  43. Steuer R, Kurths J, Daub CO, Weise J, Selbig J (2002) The mutual information: detecting and evaluating dependencies between variables. Bioinformatics 18(suppl\_2):S231–S240

  44. Wang T, Li C, Wu C, Zhao C, Sun J, Peng H, Hu X, Hu B (2020) A gait assessment framework for depression detection using kinect sensors. IEEE Sens J 21(3):3260–3270

    Article  Google Scholar 

  45. Wong SC, Gatt A, Stamatescu V, McDonnell MD (2016) Understanding data augmentation for classification: when to warp? In: 2016 international conference on digital image computing: techniques and applications (DICTA), IEEE, pp 1–6

  46. Yang L, Jiang D, Sahli H (2021) Integrating deep and shallow models for multi-modal depression analysis-hybrid architectures. IEEE Trans Affect Comput 12(1):239–253

    Article  Google Scholar 

  47. Yu S, Tan D, Tan T (2006) A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. In: 18th International Conference on Pattern Recognition (ICPR’06), IEEE, vol 4, pp 441–444

  48. Yuan Y, Li B, Wang N, Ye Q, Liu Y, Zhu T (2018) Depression identification from gait spectrum features based on hilbert-huang transform. In: International Conference on Human Centered Computing, Springer, pp 503–515

  49. Zhao N, Zhang Z, Wang Y, Wang J, Li B, Zhu T, Xiang Y (2019) See your mental state from your walk: Recognizing anxiety and depression through kinect-recorded gait data. PLoS ONE 14(5):e0216591

  50. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2017) Random erasing data augmentation

  51. Zhong Z, Zheng L, Kang G, Li S, Yang Y (2020) Random erasing data augmentation. Proceedings of the AAAI Conference on Artificial Intelligence 34:13001–13008

    Article  Google Scholar 

  52. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2223–2232

  53. Zung WW (1965) A self-rating depression scale. Arch Gen Psychiatry 12(1):63–70

    Article  CAS  Google Scholar 

Download references

Author information

Author notes

  1. Jingjing Yang and Haifeng Lu contributed equally to this work.

Authors and Affiliations

  1. School of information Science and Engineering, Lanzhou University, Lanzhou, China

    Jingjing Yang, Haifeng Lu, Xiping Hu & Bin Hu

  2. School of Intelligent Systems Engineering, Sun Yat-sen University, Shenzhen, China

    Chengming Li & Xiping Hu

  3. Institute of Engineering Medicine, Beijing Institute of Technology, Beijing, China

    Bin Hu

Authors
  1. Jingjing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haifeng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chengming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiping Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Chengming Li, Xiping Hu or Bin Hu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, J., Lu, H., Li, C. et al. Data augmentation for depression detection using skeleton-based gait information. Med Biol Eng Comput 60, 2665–2679 (2022). https://doi.org/10.1007/s11517-022-02595-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02595-z

Keywords

Search

Navigation