Abstract
Objective
Depression involves deficits in emotional flexibility. To date, the varied and dynamic nature of emotional processes during therapy has mostly been measured at discrete time intervals using clients’ subjective reports. Because emotions tend to fluctuate and change from moment to moment, the understanding of emotional processes in the treatment of depression depends to a great extent on the existence of sensitive, continuous, and objectively codified measures of emotional expression. In this observational study, we used computerized measures to analyze high-resolution time-series facial expression data as well as self-reports to examine the association between emotional flexibility and depressive symptoms at the client as well as at the session levels.
Method
Video recordings from 283 therapy sessions of 58 clients who underwent 16 sessions of manualized psychodynamic psychotherapy for depression were analyzed. Data was collected as part of routine practice in a university clinic that provides treatments to the community. Emotional flexibility was measured in each session using an automated facial expression emotion recognition system. The clients’ depression level was assessed at the beginning of each session using the Beck Depression Inventory-II (Beck et al., 1996).
Results
Higher emotional flexibility was associated with lower depressive symptoms at the treatment as well as at the session levels.
Conclusion
These findings highlight the centrality of emotional flexibility both as a trait-like as well as a state-like characteristic of depression. The results also demonstrate the usefulness of computerized measures to capture key emotional processes in the treatment of depression at a high scale and specificity.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The random effect of the clients’ average BDI scores was also significant, indicating significant between-client variability.
References
Baldwin, S. A., Imel, Z. E., Braithwaite, S. R., & Atkins, D. C. (2014). Analyzing multiple outcomes in clinical research using multivariate multilevel models. Journal of Consulting and Clinical Psychology, 82(5), 920.
Bar-Kalifa, E., & Atzil-Slonim, D. (2020). Intrapersonal and interpersonal emotional networks and their associations with treatment outcome. Journal of Counseling Psychology, 67(5), 580.
Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck depression inventory.
Bonanno, G. A., Papa, A., Lalande, K., Westphal, M., & Coifman, K. (2004). The importance of being flexible: The ability to both enhance and suppress emotional expression predicts long-term adjustment. Psychological Science, 15(7), 482–487.
Brose, A., Schmiedek, F., Koval, P., & Kuppens, P. (2015). Emotional inertia contributes to depressive symptoms beyond perseverative thinking. Cognition and Emotion, 29(3), 527–538.
Cohn, J. F., Ambadar, Z., & Ekman, P. (2007). Observer-based measurement of facial expression with the Facial Action Coding System. Handbook of emotion elicitation and assessment (pp. 203–221). Oxford University Press.
Cohn, J. F., & De la Torre, F. (2015). Automated face analysis for affective computing
Coifman, K. G., & Summers, C. B. (2019). Understanding emotion inflexibility in risk for affective disease: Integrating current research and finding a path forward. Frontiers in Psychology, 10, 392. https://doi.org/10.3389/fpsyg.2019.00392.
Ekman, P., & Friesen, W. V. (1976). Measuring facial movement. Environmental Psychology and Nonverbal Behavior, 1(1), 56–75. https://doi.org/10.1007/BF01115465.
Fosha, D. (2001). The dyadic regulation of affect. Journal of Clinical Psychology, 57(2), 227–242.
Freitas-Magalhães, A. (2012). Microexpression and macroexpression. Encyclopedia of Human Behavior, 2, 173–183.
Gavrilescu, M., & Vizireanu, N. (2019). Predicting depression, anxiety, and stress levels from videos using the facial action coding system. Sensors (Basel, Switzerland), 19(17), 3693.
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S., & Rosenwald, D. P. (2013). Social risk and depression: Evidence from manual and automatic facial expression analysis. 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 1–8. https://doi.org/10.1109/FG.2013.6553748.
Greenberg, L. S. (2012). Emotions, the great captains of our lives: Their role in the process of change in psychotherapy. American Psychologist, 67(8), 697.
Gupta, S., & Bonanno, G. A. (2011). Complicated grief and deficits in emotional expressive flexibility. Journal of Abnormal Psychology, 120(3), 635.
Hamilton, M. (1960). A rating scale for depression. Journal of Neurology Neurosurgery and Psychiatry, 23(1), 56.
Harati, S., Crowell, A., Huang, Y., Mayberg, H., & Nemati, S. (2019). Classifying depression severity in recovery from major depressive disorder via dynamic facial features. IEEE Journal of Biomedical and Health Informatics, 24(3), 815–824.
Herrmann, I. R., Greenberg, L. S., & Auszra, L. (2016). Emotion categories and patterns of change in experiential therapy for depression. Psychotherapy Research, 26(2), 178–195.
Höfling, T. T. A., Gerdes, A. B. M., Föhl, U., & Alpers, G. W. (2020). Read my face: Automatic facial coding versus psychophysiological indicators of emotional valence and arousal. Frontiers in Psychology, 11. https://doi.org/10.3389/fpsyg.2020.01388. https://www.frontiersin.org/article/
Imel, Z. E., Caperton, D. D., Tanana, M., & Atkins, D. C. (2017). Technology-enhanced human interaction in psychotherapy. Journal of Counseling Psychology, 64(4), 385.
Kashdan, T. B., & Rottenberg, J. (2010). Psychological flexibility as a fundamental aspect of health. Clinical Psychology Review, 30(7), 865–878. https://doi.org/10.1016/j.cpr.2010.03.001.
Koval, P., Kuppens, P., Allen, N. B., & Sheeber, L. (2012). Getting stuck in depression: The roles of rumination and emotional inertia. Cognition and Emotion, 26(8), 1412–1427. https://doi.org/10.1080/02699931.2012.667392.
Kramer, U., Pascual-Leone, A., Despland, J. N., & de Roten, Y. (2015). One minute of grief: Emotional processing in short-term dynamic psychotherapy for adjustment disorder. Journal of Consulting and Clinical Psychology, 83(1), 187. https://doi.org/10.1037/a0037979.
Kuppens, P., Allen, N. B., & Sheeber, L. B. (2010). Emotional inertia and psychological maladjustment. Psychological Science, 21(7), 984–991. https://doi.org/10.1177/0956797610372634.
Lewinski, P., Uyl, T., & Butler, C. (2014). Automated facial coding: Validation of Basic Emotions and FACS AUs in FaceReader. Journal of Neuroscience Psychology and Economics, 7, 227–236. https://doi.org/10.1037/npe0000028.
Luborsky, L., & Mark, D. (1991). Short-term supportive-expressive psychoanalytic psychotherapy. In Handbook of Short-Term Dynamic Psychotherapy, 110–136.
Luborsky, L., Mark, D., Hole, A. V., Popp, C., Goldsmith, B., & Cacciola, J. (1995). Supportive-expressive dynamic psychotherapy of depression: A time-limited version.
McCullough, L., & Magill, M. (2009). Affect-focused short-term dynamic therapy. In Handbook of evidence-based psychodynamic psychotherapy (pp. 249–277). Springer.
Nasser, S. A., Hashim, I. A., & Ali, W. H. (2020). A review on depression detection and diagnoses based on visual facial cues. 2020 3rd International Conference on Engineering Technology and Its Applications (IICETA), 35–40.
Noldus (2022). Facial expression analysis—FaceReader. Facial Expression Analysis - FaceReader. https://www.noldus.com/facereader/facial-expression-analysis.
Ogbaselase, F. A., Mancini, K. J., & Luebbe, A. M. (2020). Indirect effect of family climate on adolescent depression through emotion regulatory processes. Emotion.
Pampouchidou, A., Simos, P. G., Marias, K., Meriaudeau, F., Yang, F., Pediaditis, M., & Tsiknakis, M. (2017). Automatic assessment of depression based on visual cues: A systematic review. IEEE Transactions on Affective Computing, 10(4), 445–470.
Pascual-Leone, A., & Greenberg, L. S. (2007). Emotional processing in experiential therapy: Why the only way out is through. Journal of Consulting and Clinical Psychology, 75(6), 875.
Pe, M. L., Kircanski, K., Thompson, R. J., Bringmann, L. F., Tuerlinckx, F., Mestdagh, M., Mata, J., Jaeggi, S. M., Buschkuehl, M., & Jonides, J. (2015). Emotion-network density in major depressive disorder. Clinical Psychological Science, 3(2), 292–300.
Pos, A. E., Greenberg, L. S., & Warwar, S. H. (2009). Testing a model of change in the experiential treatment of depression. Journal of Consulting and Clinical Psychology, 77(6), 1055.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). sage.
Rottenberg, J. (2017). Emotions in depression: What do we really know. Annual Review of Clinical Psychology, 13(1), 241–263.
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161.
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., Hergueta, T., Baker, R., & Dunbar, G. C. (1998). The mini-international neuropsychiatric interview (MINI): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59(20), 22–33.
Skiendziel, T., Rösch, A. G., & Schultheiss, O. C. (2019). Assessing the convergent validity between the automated emotion recognition software Noldus FaceReader 7 and Facial Action Coding System Scoring. PloS One, 14(10), e0223905.
Subica, A. M., Fowler, J. C., Elhai, J. D., Frueh, B. C., Sharp, C., Kelly, E. L., & Allen, J. G. (2014). Factor structure and diagnostic validity of the Beck Depression Inventory–II with adult clinical inpatients: Comparison to a gold-standard diagnostic interview. Psychological Assessment, 26(4), 1106.
Terhürne, P., Schwartz, B., Baur, T., Schiller, D., Eberhardt, S. T., André, E., & Lutz, W. (2022). Validation and application of the non-verbal behavior analyzer: An automated tool to assess non-verbal emotional expressions in psychotherapy. Frontiers in Psychiatry, 13, 1026015. https://doi.org/10.3389/fpsyt.2022.1026015
Wang, Q., Yang, H., & Yu, Y. (2018). Facial expression video analysis for depression detection in chinese patients. Journal of Visual Communication and Image Representation, 57, 228–233. https://doi.org/10.1016/j.jvcir.2018.11.003.
Wichers, M., Wigman, J. T. W., & Myin-Germeys, I. (2015). Micro-level affect dynamics in psychopathology viewed from complex dynamical system theory. Emotion Review, 7(4), 362–367.
Williams, J. B. (1988). A structured interview guide for the Hamilton Depression Rating Scale. Archives of General Psychiatry, 45(8), 742–747.
Zilcha-Mano, S. (2021). Toward personalized psychotherapy: The importance of the trait-like/state-like distinction for understanding therapeutic change. American Psychologist, 76(3), 516.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
The authors have no relevant financial or non-financial interests to disclose.
The authors have no competing interests to declare that are relevant to the content of this article.
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.
The authors have no financial or proprietary interests in any material discussed in this article.
The study was approved by Bar-Ilan University Review Board (Num. AS/01507/2019) and was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments or comparable ethical standards.
The study followed guidelines from The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD).
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Slonim, D.A., Yehezkel, I., Paz, A. et al. Facing Change: Using Automated Facial Expression Analysis to Examine Emotional Flexibility in the Treatment of Depression. Adm Policy Ment Health (2023). https://doi.org/10.1007/s10488-023-01310-w
Accepted:
Published:
DOI: https://doi.org/10.1007/s10488-023-01310-w