Abstract
Ecological momentary assessment (EMA) is a burgeoning area of research, and several clinical applications of the resulting data have been identified by researchers, suggesting potential benefit to psychotherapy practice. However, practitioners often do not use traditional empirically-supported tools for diagnosis and outcome monitoring (e.g., validated interview measures and questionnaires). Thus, it is not clear how readily practitioners will take up newer technology-enhanced assessment methods, despite current enthusiasm among researchers. The current study aimed to explore the perceived usefulness of EMA-based tools for clinical assessment and outcome monitoring of difficult psychotherapy cases, as well as to identify correlates of attitudes about the usefulness of these tools. Clinical psychologists in active therapy practice with adults (n = 375) completed an internet survey including the Attitudes toward Standardized Assessment scale and the Attitudes toward Standardized Assessment Scales-Monitoring and Feedback. Respondents characterized their current diagnostic and outcome monitoring practices and rated how helpful they would find several assessment and outcome monitoring resources for a difficult case, including both traditional instruments and EMA-based methods. EMA-based tools had lower perceived usefulness than existing instruments. Attitudes toward standardized assessment and outcome monitoring predicted the perceived utility of these methods, as did several professional variables. Practicing psychologists may not adopt EMA for clinical assessment more readily than traditional assessment tools. Recommendations for facilitating the uptake of new technologies by psychotherapists are offered.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
Visual inspection of Predicted Probability plots and scatterplots of the predicted values and residuals suggested that the assumptions of normality and homoscedasticity were not violated for any of the regression analyses.
References
Becker, C. B., Stice, E., Shaw, H., & Woda, S. (2009). Use of empirically-supported interventions for psychopathology: Can the participatory approach move us beyond the research-to-practice gap? Behaviour Research and Therapy, 47(4), 265–274. https://doi.org/10.1016/j.brat.2009.02.007
Ben-Zeev, D., McHugo, G. J., Xie, H., Dobbins, K., & Young, M. A. (2012). Comparing retrospective reports to real-time/real-place mobile assessments in individuals with schizophrenia and a nonclinical comparison group. Schizophrenia Bulletin, 38(3), 396–404. https://doi.org/10.1093/schbul/sbr171
Ben-Zeev, D., Young, M. A., & Madsen, J. W. (2009). Retrospective recall of affect in clinically depressed individuals and controls. Cognition and Emotion, 23(5), 1021–1040. https://doi.org/10.1080/02699930802607937
Bos, F. M., Snippe, E., Bruggeman, R., Wichers, M., & van der Krieke, L. (2019). Insights of patients and clinicians on the promise of the experience sampling method for psychiatric care. Psychiatric Services, 70(11), 983–991. https://doi.org/10.1176/appi.ps.201900050
Brown, C. L., Bosley, H. G., Kenyon, A. D., Chen, K.-H., & Levenson, R. W. (2019). An idiographic statistical approach to clinical hypothesis testing for routine psychotherapy: A case study. Behaviour Research and Therapy, 118, 43–53. https://doi.org/10.1016/j.brat.2019.03.014
Bruchmüller, K., Margraf, J., Suppiger, A., & Schneider, S. (2011). Popular or unpopular? Therapists’ use of structured interviews and their estimation of patient acceptance. Behavior Therapy, 42, 634–643.
David, S. J., Marshall, A. J., Evanovich, E. K., & Mumma, G. H. (2018). Intraindividual dynamic network analysis: Implications for clinical assessment. Journal of Psychopathology and Behavioral Assessment, 40, 235–248. https://doi.org/10.1007/s10862-017-9632-8
Ellison, W. D., Trahan, A. C., Pinzon, J. C., Gillespie, M. E., Simmons, L. M., & King, K. Y. (2020). For whom, and for what, is experience sampling more accurate than retrospective report? Personality and Individual Differences. https://doi.org/10.1016/j.paid.2020.110071
Epskamp, S., van Borkulo, C. D., van der Veen, D. C., Servaas, M. N., Isvoranu, A.-M., Riese, H., & Cramer, A. O. J. (2018). Personalized network modeling in psychopathology: The importance of contemporaneous and temporal connections. Clinical Psychological Science, 6(3), 416–427. https://doi.org/10.1177/2167702617744325
First, M. B., Bhat, V., Adler, D., Dixon, L., Goldman, B., Koh, S., et al. (2018). How do clinicians actually use the Diagnostic and Statistical Manual of Mental Disorders in clinical practice and why we need to know more. Journal of Nervous and Mental Disease, 202, 841–844. https://doi.org/10.1097/NMD.0000000000000210
Fisher, A. J. (2015). Toward a dynamic model of psychological assessment: Implications for personalized care. Journal of Consulting and Clinical Psychology, 83(4), 825–836. https://doi.org/10.1037/ccp0000026
Fisher, A. J., Bosley, H. G., Fernandez, K. C., Reeves, J. W., Soyster, P. D., Diamond, A. E., & Barkin, J. (2019). Open trial of a personalized modular treatment for mood and anxiety. Behaviour Research and Therapy, 116, 69–79. https://doi.org/10.1016/j.brat.2019.01.010
Fisher, A. J., & Boswell, J. F. (2016). Enhancing the personalization of psychotherapy with dynamic assessment and modeling. Assessment, 23(4), 496–506. https://doi.org/10.1177/1073191116638735
Fisher, A. J., & Newman, M. G. (2016). Reductions in the diurnal rigidity of anxiety predict treatment outcome in cognitive behavioral therapy for generalized anxiety disorder. Behaviour Research and Therapy, 79, 46–55. https://doi.org/10.1016/j.brat.2016.02.006
Fisher, A. J., Reeves, J. W., Lawyer, G., Medaglia, J. D., & Rubel, J. A. (2017). Exploring the idiographic dynamics of mood and anxiety via network analysis. Journal of Abnormal Psychology, 126(8), 1044–1056. https://doi.org/10.1037/abn0000311
Fisher, C. E., & Appelbaum, P. S. (2017). Beyond Googling: The ethics of using patients’ electronic footprints in psychiatric practice. Harvard Review of Psychiatry, 25(4), 1–10. https://doi.org/10.1097/HRP.0000000000000145
Frumkin, M. R., Piccirillo, M. L., Beck, E. D., Grossman, J. T., & Rodebaugh, T. L. (2020). Feasibility and utility of idiographic models in the clinic: A pilot study. Psychotherapy Research. https://doi.org/10.1080/10503307.2020.1805133
Gallo, K. P., & Barlow, D. H. (2012). Factors involved in clinician adoption and nonadoption of evidence-based interventions in mental health. Clinical Psychology: Science and Practice, 19, 93–106.
Gaspar-Barba, E., Calati, R., Cruz-Fuentes, C. S., Ontiveros-Uribe, M. P., Natale, V., De Ronchi, D., & Serretti, A. (2009). Depressive symptomatology is influenced by chronotypes. Journal of Affective Disorders, 119, 100–106.
Gilbody, S. M., House, A. O., & Sheldon, T. A. (2002). Psychiatrists in the UK do not use outcomes measures: National survey. British Journal of Psychiatry, 180, 101–103.
Gloster, A. T., Meyer, A. H., Witthauer, C., Lieb, R., & Mata, J. (2017). ‘I feel better when…’: An analysis of the memory-experience gap for peoples’ estimates of the relationship between health behaviors and experiences. Psychology & Health, 32(9), 1152–1166. https://doi.org/10.1080/08870446.2017.1327586
Gloster, A. T., Richard, D. C. S., Himle, J., Koch, E., Anson, H., Lokers, L., & Thornton, J. (2008). Accuracy of retrospective memory and covariation estimation in patients with obsessive–compulsive disorder. Behaviour Research and Therapy, 46, 642–655. https://doi.org/10.1016/j.brat.2008.02.010
Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35(5), 485–491. https://doi.org/10.1037/0735-7028.35.5.485
Hatfield, D. R., & Ogles, B. M. (2007). Why some clinicians use outcome measures and others do not. Administration and Policy in Mental Health and Mental Health Services Research, 34, 283–291. https://doi.org/10.1007/s10488-006-0110-y
Hayes, A. M., Yasinski, C., Barnes, J. B., & Bockting, C. L. H. (2015). Network destabilization and transition in depression: New methods for studying the dynamics of therapeutic change. Clinical Psychology Review, 41, 27–39. https://doi.org/10.1016/j.cpr.2015.06.007
Haynes, S. N., Mumma, G. H., & Pinson, C. (2009). Idiographic assessment: Conceptual and psychometric foundations of individualized behavioral assessment. Clinical Psychology Review, 29, 179–191. https://doi.org/10.1016/j.cpr.2008.12.003
Hopwood, C. J., Thomas, K. M., Luo, X., Bernard, N., Lin, Y., & Levendosky, A. A. (2016). Implementing dynamic assessments in psychotherapy. Assessment, 23(4), 507–517. https://doi.org/10.1177/1073191116649658
Hunsley, J., & Mash, E. J. (2009). Evidence-based assessment. Annual Review of Clinical Psychology, 3, 29–51. https://doi.org/10.1146/annurev.clinpsy.3.022806.091419
Jensen-Doss, A., & Hawley, K. M. (2010). Understanding barriers to evidence-based assessment: Clinician attitudes toward standardized assessment tools. Journal of Clinical Child & Adolescent Psychology, 39(6), 885–896. https://doi.org/10.1080/15374416.2010.517169
Jensen-Doss, A., Becker Haimes, E. M., Smith, A. M., Lyon, A. R., Lewis, C. C., Stanick, C. F., & Hawley, K. M. (2018). Monitoring treatment progress and providing feedback is viewed favorably but rarely used in practice. Administration and Policy in Mental Health, 45, 48–61. https://doi.org/10.1007/s10488-016-0763-0
Kelly, J. M., Kertz, S. J., Simpson, R., Bloch, M. H., & Pittenger, C. (2019). Exploring retrospective biases in obsessive-compulsive disorder: An experience-sampling study. Journal of Technology in Behavioral Science, 4, 297–302. https://doi.org/10.1007/s41347-018-0078-y
Koerner, K., & Castonguay, L. G. (2015). Practice-oriented research: What it takes to do collaborative research in private practice. Psychotherapy Research, 25(1), 67–83. https://doi.org/10.1080/10503307.2014.939119
Kroeze, R., van der Veen, D. C., Servaas, M. N., Bastiaansen, J. A., Oude Voshaar, R. C., Borsboom, D., et al. (2017). Personalized feedback on symptom dynamics of psychopathology: A proof-of-principle study. Journal for Person-Oriented Research, 3(1), 1–10. https://doi.org/10.17505/jpor.2017.01
Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018). Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy, 55(4), 520–537. https://doi.org/10.1037/pst0000167
Lewis, K. C., Ridenour, J. M., Pitman, S., & Roche, M. (2020). Evaluating stable and situational expressions of passive-aggressive personality disorder: A multimethod experience sampling case study. Journal of Personality Assessment. https://doi.org/10.1080/00223891.2020.1818572
Lushin, V., Becker-Haimes, E. M., Mandell, D., Conrad, J., Kaploun, V., Bailey, S., et al. (2019). What motivates mental health clinicians-in-training to implement evidence-based assessment? A survey of social work trainees. Administration and Policy in Mental Health and Mental Health Services Research, 46, 411–424. https://doi.org/10.1007/s10488-019-00923-4
Miller, G. (2012). The smartphone psychology manifesto. Perspectives on Psychological Science, 7(3), 221–237. https://doi.org/10.1177/1745691612441215
Miller, M. A., Rothenberger, S. D., Hasler, B. P., Donofry, S. D., Wong, P. M., Manuck, S. B., et al. (2015). Chronotype predicts positive affect rhythms measured by ecological momentary assessment. Chronobiology International, 32(3), 376–384. https://doi.org/10.3109/07420528.2014.983602
Molenaar, P. C. M., & Campbell, C. G. (2009). The new person-specific paradigm in psychology. Current Directions in Psychological Science, 18(2), 112–117.
Mumma, G. H., Marshall, A. J., & Mauer, C. (2018). Person-specific validation and testing of functional relations in cognitive-behavioural case formulation: Guidelines and options. Clinical Psychology and Psychotherapy, 25, 672–691. https://doi.org/10.1002/cpp.2298
Nelson, B., McGorry, P. D., Wichers, M., Wigman, J. T. W., & Hartmann, J. A. (2017). Moving from static to dynamic models of the onset of mental disorder: A review. JAMA Psychiatry, 74(5), 528–534. https://doi.org/10.1001/jamapsychiatry.2017.0001
Olthof, M., Hasselman, F., Strunk, G., Aas, B., Schiepek, G., & Lichtwarck-Aschoff, A. (2020). Destabilization in self-ratings of the psychotherapeutic process is associated with better treatment outcome in patients with mood disorders. Psychotherapy Research, 30(4), 520–531. https://doi.org/10.1080/10503307.2019.1633484
Olthof, M., Hasselman, F., Strunk, G., van Rooij, M., Aas, B., Helmich, M. A., et al. (2020). Critical fluctuations as an early-warning signal for sudden gains and losses in patients receiving psychotherapy for mood disorders. Clinical Psychological Science, 8(1), 25–35. https://doi.org/10.1177/21677026198659
Pew Research Center. (2019, June). Mobile technology and home broadband 2019. https://www.pewresearch.org/internet/2019/06/13/mobile-technology-and-home-broadband-2019/.
Piccirillo, M. L., Beck, E. D., & Rodebaugh, T. L. (2019). A clinician’s primer for idiographic research: Considerations and recommendations. Behavior Therapy, 50, 938–951.
Piccirillo, M. L., & Rodebaugh, T. L. (2019). Foundations of idiographic methods in psychology and applications for psychotherapy. Clinical Psychology Review, 71, 90–100. https://doi.org/10.1016/j.cpr.2019.01.002
Poston, J. M., & Hanson, W. E. (2010). Meta-analysis of psychological assessment as a therapeutic intervention. Psychological Assessment, 22(2), 203–212. https://doi.org/10.1037/a0018679
Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health, 36, 24–34. https://doi.org/10.1007/s10488-008-0197-4
Roche, M. J., & Pincus, A. L. (2016). Precision assessment: An individualized and temporally dynamic approach to understanding patients in their daily lives. In U. Kumar (Ed.), The Wiley handbook of personality assessment (pp. 192–204). Wiley Blackwell: Hoboken, NJ.
Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York: Free Press.
Rubel, J. A., Fisher, A. J., Husen, K., & Lutz, W. (2018). Translating person-specific network models into personalized treatments: Development and demonstration of the Dynamic Assessment Treatment Algorithm for Individual Networks (DATA-IN). Psychotherapy and Psychosomatics, 87(4), 249–251. https://doi.org/10.1159/000487769
Schiepek, G. K., Stöger-Schmidinger, B., Aichhorn, W., Schöller, H., & Aas, B. (2016). Systemic case formulation, individualized process monitoring, and state dynamics in a case of dissociative identity disorder. Frontiers in Psychology, 7, 1545. https://doi.org/10.3389/fpsyg.2016.01545
Sebo, P., Maisonneuve, H., Cerutti, B., Fournier, J. P., Senn, N., & Haller, D. M. (2017). Rates, delays, and completeness of general practitioners’ responses to a postal versus web-based survey: A randomized trial. Journal of Medical Internet Research, 19(3), e83. https://doi.org/10.2196/jmir.6308
Soyster, P. D., & Fisher, A. J. (2019). Involving stakeholders in the design of ecological momentary assessment research: An example from smoking cessation. PLoS ONE. https://doi.org/10.1371/journal.pone.0217150
Trull, T. J., & Ebner-Priemer, U. (2013). Ambulatory assessment. Annual Review of Clinical Psychology, 9, 151–176. https://doi.org/10.1146/annurev-clinpsy-050212-185510
Urban, E. J., Charles, S. T., Levine, L. J., & Almeida, D. M. (2018). Depression history and memory bias for specific daily emotions. PLoS ONE. https://doi.org/10.1371/journal.pone.0203574
van de Leemput, I. A., Wichers, M., Cramer, A. O. J., Borsboom, D., Tuerlinckx, F., Kuppens, P., et al. (2014). Critical slowing down as early warning for the onset and termination of depression. Proceedings of National and Academic Science USA, 111(1), 87–92. https://doi.org/10.1073/pnas.1312114110
van Os, J., Delespaul, P., Wigman, J., Myin-Germeys, I., & Wichers, M. (2013). Beyond DSM and ICD: Introducing “precision diagnosis” for psychiatry using momentary assessment technology. World Psychiatry, 12, 113–117. https://doi.org/10.1002/wps.20046
Wenze, S. J., Gunthert, K. C., & German, R. E. (2012). Biases in affective forecasting and recall in individuals with depression and anxiety symptoms. Personality and Social Psychology Bulletin, 38(7), 895–906. https://doi.org/10.1177/0146167212447242
Wichers, M., Groot, P. C., Psychosystems, ESM Group, & EWS Group. (2016). Critical slowing down as a personalized early warning signal for depression. Psychotherapy and Psychosomatics, 85, 114–116.
Wichers, M., Smit, A. C., & Snippe, E. (2020). Early warning signals based on momentary affect dynamics can expose nearby transitions in depression: A confirmatory single-subject time-series study. Journal for Person Oriented Research, 6(1), 1–15. https://doi.org/10.17505/jpor.2020.22042
Wright, A. G. C., & Woods, W. C. (2020). Personalized models of psychopathology. Annual Review of Clinical Psychology, 16, 49–74. https://doi.org/10.1146/annurev-clinpsy-102419-125032
Youn, S. J., Valentine, S. E., Patrick, K. A., Baldwin, M., Chablani-Medley, A., Aguilar Silvan, Y., et al. (2019). Practical solutions for sustaining long-term academic-community partnerships. Psychotherapy, 56(1), 115–125. https://doi.org/10.1037/pst0000188
Zander, E., Wyder, L., Holtforth, M., Schnyder, U., Hepp, U., & Stulz, N. (2018). Validity of routine clinical diagnoses in acute psychiatric inpatients. Psychiatry Research, 259, 482–487. https://doi.org/10.1016/j.psychres.2017.11.004
Zimmerman, M. (2003). What should the standard of care for psychiatric diagnostic evaluations be? Journal of Nervous and Mental Disease, 191(5), 281–286.
Zimmermann, J., Woods, W. C., Ritter, S., Happel, M., Masuhr, O., Jaeger, U., et al. (2019). Integrating structure and dynamics in personality assessment: First steps toward the development and validation of a personality dynamics diary. Psychological Assessment, 31(4), 516–531. https://doi.org/10.1037/pas0000625
Acknowledgements
The work was funded by faculty startup funds provided to the author. The author thanks Megan E. Gillespie for her assistance developing study materials.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The author has no potential conflicts of interest to disclose.
Ethical Approval
All procedures performed in the study were in accordance with the ethical standards of the Trinity University Institutional Review Board (IRB #00001449) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Informed Consent
Participants provided informed consent electronically before beginning the study
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Ellison, W.D. An Initial Study of Practicing Psychologists’ Views of the Utility of Ecological Momentary Assessment for Difficult Psychotherapy Cases. Adm Policy Ment Health 48, 597–607 (2021). https://doi.org/10.1007/s10488-020-01093-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10488-020-01093-4