Abstract
Biofeedback therapy is mainly based on the analysis of physiological features to improve an individual’s affective state. There are insufficient objective indicators to assess symptom improvement after biofeedback. In addition to psychological and physiological features, speech features can precisely convey information about emotions. The use of speech features can improve the objectivity of psychiatric assessments. Therefore, biofeedback based on subjective symptom scales, objective speech, and physiological features to evaluate efficacy provides a new approach for early screening and treatment of emotional problems in college students. A 4-week, randomized, controlled, parallel biofeedback therapy study was conducted with college students with symptoms of anxiety or depression. Speech samples, physiological samples, and clinical symptoms were collected at baseline and at the end of treatment, and the extracted speech features and physiological features were used for between-group comparisons and correlation analyses between the biofeedback and wait-list groups. Based on the speech features with differences between the biofeedback intervention and wait-list groups, an artificial neural network was used to predict the therapeutic effect and response after biofeedback therapy. Through biofeedback therapy, improvements in depression (p = 0.001), anxiety (p = 0.001), insomnia (p = 0.013), and stress (p = 0.004) severity were observed in college-going students (n = 52). The speech and physiological features in the biofeedback group also changed significantly compared to the waitlist group (n = 52) and were related to the change in symptoms. The energy parameters and Mel-Frequency Cepstral Coefficients (MFCC) of speech features can predict whether biofeedback intervention effectively improves anxiety and insomnia symptoms and treatment response. The accuracy of the classification model built using the artificial neural network (ANN) for treatment response and non-response was approximately 60%. The results of this study provide valuable information about biofeedback in improving the mental health of college-going students. The study identified speech features, such as the energy parameters, and MFCC as more accurate and objective indicators for tracking biofeedback therapy response and predicting efficacy. Trial Registration ClinicalTrials.gov ChiCTR2100045542.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albuquerque, L., Valente, A. R. S., Figueiredo, D., Teixeira, A., Sa-Couto, P., & Oliveira, C. (2021). Association between acoustic speech features and non-severe levels of anxiety and depression symptoms across lifespan. PloS One, 16(4), e0248842. https://doi.org/10.1371/journal.pone.0248842
Alneyadi, M., Drissi, N., & Almeqbaali, M. (2021). Biofeedback-based connected mental health interventions for anxiety: Systematic literature review. JMIR mHealth and uHealth, 9(4), e26038. https://doi.org/10.2196/26038
Auerbach, R. P., Alonso, J., Axinn, W. G., Cuijpers, P., Ebert, D. D., Green, J. G., Hwang, I., Kessler, R. C., Liu, H., & Mortier, P. (2016). Mental disorders among college students in the world health organization world mental health surveys. Psychological Medicine, 1(14), 2955–2970. https://doi.org/10.1017/S0033291716001665
Auerbach, R. P., Mortier, P., Bruffaerts, R., Alonso, J., Benjet, C., Cuijpers, P., Demyttenaere, K., Ebert, D. D., Green, J. G., Hasking, P., Murray, E., Nock, M. K., Pinder-Amaker, S., Sampson, N. A., Stein, D. J., Vilagut, G., Zaslavsky, A. M., Kessler, R. C., WHO, WMH-ICS, & Collaborators. (2018). WHO world mental health surveys international college student project: prevalence and distribution of mental disorders. Journal of Abnormal Psychology, 127(7), 623–38. https://doi.org/10.1037/abn0000362
Beiter, R., Nash, R., Mccrady, M., Rhoades, D., Linscomb, M., Clarahan, M., & Sammut, S. (2015). The prevalence and correlates of depression, anxiety, and stress in a sample of college students. Journal of Affective Disorders, 173, 90–96. https://doi.org/10.1016/j.jad.2014.10.054
Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157–166. https://doi.org/10.1109/72.279181.
Bernard, J. A., & Mittal, V. A. (2015). Updating the research domain criteria: The utility of a motor dimension. Psychological Medicine, 45(13), 2685–2689. https://doi.org/10.1017/S0033291715000872
Cannizzaro, M., Harel, B., Reilly, N., Chappell, P., & Peter, J. S. (2004). Voice acoustical measurement of the severity of major depression. Brain and Cognition, 56(1), 30–35. https://doi.org/10.1016/j.bandc.2004.05.003
Cummins, N., Baird, A., & Björn, W. S. (2018). Speech analysis for health: Current state-of-the-art and the increasing impact of deep learning. Methods (San Diego Calif), 151, 41–54. https://doi.org/10.1016/j.ymeth.2018.07.007
da Maynart, W. H., de Albuquerque, M. C. D. S., Santos, R. C. S., de Albuquerque Sarmento, P., da Silva, J. J., Costa, C. D. S. G., da Silva Santos, J. D., de Oliveira Pontes, C., dos Santos Barros, M. C., Belo, F. M. P., & Costa, C. R. B. (2021). The use of biofeedback intervention in the improvement of depression levels: A randomised trial. Acta Neuropsychiatrica, 33(3), 126–133. https://doi.org/10.1017/neu.2020.46
de Girolamo, G., Dagani, J., Purcell, R., Cocchi, A., & McGorry, P. D. (2012). Age of onset of mental disorders and use of mental health services: Needs, opportunities and obstacles. Epidemiology and Psychiatric Sciences, 21(1), 47–57. https://doi.org/10.1017/s2045796011000746
Delk, K. K., Gevirtz, R., Hicks, D. A., & Carden, F. (1994). The effects of biofeedback assisted breathing retraining on lung functions in patients with cystic fibrosis. Chest, 105(1), 23–28. https://doi.org/10.1378/chest.105.1.23
Di, Y., Wang, J., Li, W., & Zhu, T. (2021). Using I-Vectors from voice features to identify major depressive disorder. Journal of Affective Disorders, 288, 161–166. https://doi.org/10.1016/j.jad.2021.04.004
Dormal, V., Vermeulen, N., & Mejias, S. (2021). Is heart rate variability biofeedback useful in children and adolescents? A systematic review. Journalof Child Psychology and Psychiatry. https://doi.org/10.1111/jcpp.13463
Du, N., Yu, K., Ye, Y., & Chen, S. (2017). Validity study of patient health questionnaire-9 items for internet screening in depression among Chinese university students. Asia–Pacific Psychiatry: Official Journal of the Pacific Rim College of Psychiatrists. https://doi.org/10.1111/appy.12266
Ellgring, H., & Scherer, K. R. (1996). Vocal indicators of mood change in depression. Journal of Nonverbal Behavior, 20(2), 83–110. https://doi.org/10.1007/BF02253071
France, D. J., Shiavi, R. G., Silverman, S., Silverman, M., & Wilkes, D. M. (2000). Acoustical properties of speech as indicators of depression and suicidal risk. IEEE Transactions on Bio-Medical Engineering, 47(7), 829–837. https://doi.org/10.1109/10.846676
Freeman, D., Sheaves, B., Goodwin, G. M., Yu, L. M., Nickless, A., Harrison, P. J., Emsley, R., Luik, A. I., Foster, R. G., Wadekar, V., Hinds, C., Gumley, A., Jones, R., Lightman, S., Jones, S., Bentall, R., Kinderman, P., Rowse, G., Brugha, T., … Espie, C. A. (2017). The effects of improving sleep on mental health (OASIS): A randomised controlled trial with mediation analysis. The Lancet Psychiatry, 4(10), 749–758. https://doi.org/10.1016/S2215-0366(17)30328-0
Gaggioli, A., Pallavicini, F., Morganti, L., & Serino, S. (2014). Experiential virtual scenarios with real-time monitoring (interreality) for the management of psychological stress: A block randomized controlled trial. Journal of Medical Internet Research, 16(7), e167. https://doi.org/10.2196/jmir.3235
Giggins, O. M., Persson, U. M. C., & Caulfield, B. (2013). Biofeedback in rehabilitation. Journal of Neuroengineering and Rehabilitation, 10, 60. https://doi.org/10.1186/1743-0003-10-60
Hasson, U., Nastase, S. A., & Goldstein, A. (2020). Direct fit to nature: An evolutionary perspective on biological and artificial neural networks. Neuron, 105(3), 416–434. https://doi.org/10.1016/j.neuron.2019.12.002
Hoyer, P., & Graf, S. (2019). Adjustment of the vocal tract shape via biofeedback: A case study. Journal of Voice: Official Journal of the Voice Foundation, 33(4), 482–489. https://doi.org/10.1016/j.jvoice.2018.01.018
Kacur, J., Puterka, B., & Pavlovicova, J. (2021). On the speech properties and feature extraction methods in speech emotion recognition. Sensors (Basel Switzerland), 21(5), 1888. https://doi.org/10.3390/s21051888
Kessler, R. C., Amminger, G. P., Aguilar-Gaxiola, S., Alonso, J., Lee, S., & Üstün, T. B. (2007). Age of onset of mental disorders: a review of recent literature. Current Opinion in Psychiatry, 20(4), 359–364. https://doi.org/10.1097/YCO.0b013e32816ebc8c
König, A., Mallick, E., Tröger, J., Linz, N., Zeghari, R., & Manera, V. (2021). Measuring neuropsychiatric symptoms in patients with early cognitive decline using speech analysis. European Psychiatry: The Journal of the Association of European Psychiatrists, 64(1), e64. https://doi.org/10.1192/j.eurpsy.2021.2236
Kotwas, I., Mcgonigal, A., Khalfa, S., Bastien-Toniazzo, M., & Bartolomei, F. (2017). A case-control study of skin conductance biofeedback on seizure frequency and emotion regulation in drug-resistant temporal lobe epilepsy. International Journal of Psychophysiology. https://doi.org/10.1016/j.ijpsycho.2017.10.005
Kraepelin, E. (1921). Manic depressive insanity and paranoia. The Journal of Nervous and Mental Disease, 53(4), 350. https://doi.org/10.1097/00005053-192104000-00057
Krusemark, E. A., & Wen, L. (2012). Enhanced olfactory sensory perception of threat in anxiety: An event-related FMRI study. Chemosensory Perception, 5(1), 37–45. https://doi.org/10.1007/s12078-011-9111-7
Kuny, S., & Stassen, H. H. (1993). Speaking behavior and voice sound characteristics in depressive patients during recovery. Journal of Psychiatric Research, 27(3), 289–307. https://doi.org/10.1016/0022-3956(93)90040-9
Lee, S., Suh, S. W., Kim, T., Kim, K., Lee, K. H., Lee, J. R., Han, G., Hong, J. W., Han, J. W., Lee, K., & Kim, K. W. (2021). Screening major depressive disorder using vocal acoustic features in the elderly by sex. Journal of Affective Disorders, 291, 15–23. https://doi.org/10.1016/j.jad.2021.04.098
Little, B., Alshabrawy, O., Stow, D., Nicol Ferrier, I., McNaney, R., Jackson, D. G., Ladha, K., Ladha, C., Ploetz, T., Bacardit, J., Olivier, P., Gallagher, P., & O’Brien, J. T. (2021). Deep learning-based automated speech detection as a marker of social functioning in late-life depression. Psychological Medicine, 51(9), 1441–1450. https://doi.org/10.1017/S0033291719003994
Low, D. M., Bentley, K. H., & Ghosh, S. S. (2020). Automated assessment of psychiatric disorders using speech: A systematic review. Laryngoscope Investigative Otolaryngology, 5(1), 96–116. https://doi.org/10.1002/lio2.354
Mather, M. (2021). Is there a maximum desirable heart rate variability? Neuroscience & Biobehavioral Reviews, 128, 87–89. https://doi.org/10.1016/j.neubiorev.2021.06.015
McFee, Brian, Vincent Lostanlen, Alexandros Metsai, Matt McVicar, Stefan Balke, Carl Thomé, Colin Raffel, Frank Zalkow, Ayoub Malek, Dana, Kyungyun Lee, Oriol Nieto, Jack Mason, Dan Ellis, Eric Battenberg, Scott Seyfarth, Ryuichi Yamamoto, Keunwoo Choi, viktorandreevichmorozov, Josh Moore, Rachel Bittner, Shunsuke Hidaka, Ziyao Wei, nullmightybofo, Darío Hereñú, Fabian-Robert Stöter, Pius Friesch, Adam Weiss, Matt Vollrath, and Taewoon Kim. 2020. “Librosa/Librosa: 0.8.0.
Meng, R., Li, J., Wang, Z., Zhang, D., Liu, B., & Luo, Y. (2020). The Chinese version of the perceived stress questionnaire: Development and validation amongst medical students and workers. Health and Quality of Life Outcomes, 18(1), 70. https://doi.org/10.1186/s12955-020-01307-1
Mundt, J. C., Adam, P., Vogel, D. E., Feltner, & Lenderking, W. R. (2012). Vocal acoustic biomarkers of depression severity and treatment response. Biological Psychiatry, 72(7), 580–587. https://doi.org/10.1016/j.biopsych.2012.03.015
Muzammel, M., Salam, H., & Othmani, A. (2021). End-to-end multimodal clinical depression recognition using deep neural networks: A comparative analysis. Computer Methods and Programs in Biomedicine, 211, 106433. https://doi.org/10.1016/j.cmpb.2021.106433
Oezseven, T., Duegenci, M., Doruk, A., & Hilal, I. K. (2018). Voice traces of anxiety: Acoustic parameters affected by anxiety disorder. Archives of Acoustics, 43(4), 625–636. https://doi.org/10.24425/aoa.2018.125156.
Ozdas, A., Shiavi, R. G., Silverman, S. E., Silverman, M. K., & Mitchell Wilkes, D. (2004). Investigation of vocal jitter and glottal flow spectrum as possible cues for depression and near-term suicidal risk. IEEE Transactions on Bio-Medical Engineering, 51(9), 1530–1540. https://doi.org/10.1109/TBME.2004.827544
Predatu, R., David, O. A., Ovidiu David, D., & Maffei, A. (2021). Emotion regulation abilities as a predictor of anxiety, positive emotions, heart rate, and alpha asymmetry during an impromptu speech task. Anxiety, Stress, & Coping, 34(6), 719–733. https://doi.org/10.1080/10615806.2020.1867717
Salekin, A., Eberle, J. W., Glenn, J. J., Teachman, B. A., & Stankovic, J. A. (2018). A weakly supervised learning framework for detecting social anxiety and depression. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 2(2), 81. https://doi.org/10.1145/3214284
Schleifer, L. M., Spalding, T. W., Kerick, S. E., Cram, J. R., & Ley, R. (2008). Mental stress and trapezius muscle activation under psychomotor challenge: A focus on EMG gaps during computer work. Psychophysiology, 45(3), 356–365. https://doi.org/10.1111/j.1469-8986.2008.00645.x
Schoenberg, P., & David, A. S. (2014). Biofeedback for psychiatric disorders: A systematic review. Applied Psychophysiology and Biofeedback, 39(2), 109. https://doi.org/10.1007/s10484-014-9246-9
Schultebraucks, K., Yadav, V., Shalev, A. Y., Bonanno, G. A., & Galatzer-Levy, I. R. (2020). Deep learning-based classification of posttraumatic stress disorder and depression following trauma utilizing visual and auditory markers of arousal and mood. Psychological Medicine, 52(5), 957–967. https://doi.org/10.1017/S0033291720002718
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-CAM: Visual explanations from deep networks via griadient-based localization. in IEEE international conference on computer vision.
Sheaves, B., Porcheret, K., Tsanas, A., Espie, C. A., Foster, R. G., Harrison, D. F. P. J., & Wulff, K. (2016). Insomnia, nightmares, and chronotype as markers of risk for severe mental illness: Results from a student population. Sleep, 39(1), 173–181. https://doi.org/10.5665/sleep.5342
Silber-Varod, V., Kreiner, H., Lovett, R., Levi-Belz, Y., & Amir, N. (2016). Do social anxiety individuals hesitate more? The Prosodic profile of hesitation disfluencies in social anxiety disorder individuals. in Speech Prosody, Boston 2016.
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013). Deep inside convolutional networks: Visualising image classification models and saliency maps. Computer Ence. https://doi.org/10.48550/arXiv.1312.6034
Squarcina, L., Villa, F. M., Nobile, M., & Grisan, E. (2021). Deep learning for the prediction of treatment response in depression. Journal of Affective Disorders, 281, 618–622. https://doi.org/10.1016/j.jad.2020.11.104
Sverdlov, A., Curcic, J., Hannesdottir, K., De Luca, V., Ambrosetti, F., Zhang, B., Praestgaard, J., Vallejo, V., Dolman, A., Gomez-Mancilla, B., & Biliouris, K. (2021). A study of new exploratory tools, digital technologies and biomarkers to characterize depression. Frontiers in Psychiatry. https://doi.org/10.3389/fpsyt.2021.640741
Taguchi, T., Tachikawa, H., Nemoto, K., Suzuki, M., Nagano, T., Tachibana, R., & Nishimura, M. (2018). Major depressive disorder discrimination using vocal acoustic features. Journal of Affective Disorders, 225, 214–220. https://doi.org/10.1016/j.jad.2017.08.038
Taha, A., Yang, X., Shrivastava, A., & Davis, L. (2020). A generic visualization approach for convolutional neural networks. https://doi.org/10.1007/978-3-030-58520-4_43.
Tolkmitt, F., Helfrich, H., Standke, R., & Scherer, K. R. (1982). Vocal indicators of psychiatric treatment effects in depressives and schizophrenics. Journal of Communication Disorders, 15(3), 209–222. https://doi.org/10.1016/0021-9924(82)90034-x
Tong, X., An, D., McGonigal, A., & Park, S. P. (2016). Validation of the generalized anxiety disorder-7 (GAD-7) among Chinese people with epilepsy. Epilepsy Research, 120, 31–36. https://doi.org/10.1016/j.eplepsyres.2015.11.019
Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM. https://doi.org/10.1145/1968.1972
Vincent, P., Larochelle, H., Lajoie, I., & Bengio, Y., and Pierre Antoine Manzagol (2010). Stacked Denoising autoencoders: Learning useful representations in a Deep Network with a local Denoising Criterion. Journal of Machine Learning Research, 11(12), 3371–3408. https://doi.org/10.1016/j.mechatronics.2010.09.004.
Wang, J., Zhang, L., Liu, T., & Wei Pan, B. H. (2019). Acoustic differences between healthy and depressed people: A cross-situation study. Bmc Psychiatry, 19(1), 300. https://doi.org/10.1186/s12888-019-2300-7
Weerdmeester, J., van Rooij, M. M., & Engels, R. C. (2020). An integrative model for the effectiveness of biofeedback interventions for anxiety regulation: Viewpoint. Journal of Medical Internet Research, 22(7), e14958. https://doi.org/10.2196/14958
Yann, L., Bottou, L., Bengio, Y., & Haffner, P. n.d. Gradientbased learning applied to document recognition.
Yu, D. S. F. (2010). Insomnia severity index: Psychometric properties with Chinese community-dwelling older people. Journal of Advanced Nursing, 66(10), 2350–2359. https://doi.org/10.1111/j.1365-2648.2010.05394.x
Zeiler, M. D., & Fergus, R. (2013). Visualizing and understanding convolutional neural networks. Springer International Publishing. https://doi.org/10.1007/978-3-319-10590-1_53.
Zhang, Q. S., Zhu, S. C., & California, U. O. (2018). Visual interpretability for deep learning: A survey. Frontiers of Information Technology & Electronic Engineering. https://doi.org/10.1631/FITEE.1700808
Zhu, L., Chen, L., & Zhao, D. (2017). Emotion recognition from chinese speech for smart affective services using a combination of SVM and DBN. Sensors (Basel Switzerland), 17(7), e1694. https://doi.org/10.3390/s17071694
Acknowledgements
The authors thank all participants who took part in this study. The authors thank all participants who took part in this study. The support to recruitment by personnel in Xinxiang Medical University is acknowledged. The authors thank Xinxiang Medical University and The Affiliated Brain Hospital of Nanjing Medical University for excellent research assistance.
Funding
This study was funded by National Science Fund for Distinguished Young Scholars (81725005 to Fei Wang), National Natural Science Foundation Regional Innovation and Development Joint Fund (U20A6005 to Fei Wang), Jiangsu Provincial Key Research and Development Program (BE2021617), National Natural Science Foundation of China (62176129 to Xizhe Zhang) and Key Project supported by Medical Science and Technology Development Foundation, Jiangsu Commission of Health (ZD2021026 to Zhu Rongxin).
Author information
Ethics declarations
Conflict of interest
The authors declare that the authors have no conflict of interest. The authors have full control of all primary data and that agree to allow the journal to review their data if requested
Additional information
Publisher’s Note
Springer nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wang, L., Liu, R., Wang, Y. et al. Effectiveness of a Biofeedback Intervention Targeting Mental and Physical Health Among College Students Through Speech and Physiology as Biomarkers Using Machine Learning: A Randomized Controlled Trial. Appl Psychophysiol Biofeedback 49, 71–83 (2024). https://doi.org/10.1007/s10484-023-09612-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10484-023-09612-3