Mood Mirroring with an Embodied Virtual Agent: A Pilot Study on the Relationship Between Personalized Visual Feedback and Adherence

  • Simon ProvoostEmail author
  • Jeroen Ruwaard
  • Koen Neijenhuijs
  • Tibor Bosse
  • Heleen Riper
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 887)


Human support is thought to increase adherence to internet-based interventions for common mental health disorders, but can be costly and reduce treatment accessibility. Embodied virtual agents may be used to deliver automated support, but while many solutions have been shown to be feasible, there is still little controlled research that empirically validates their clinical effectiveness in this context. This study uses a controlled and randomized paradigm to investigate whether feedback from an embodied virtual agent can increase adherence. In a three-week ecological momentary assessment smartphone study, 68 participants were asked to report their mood three times a day. An embodied virtual agent could mirror participant-reported mood states when thanking them for their answers. A two-stage randomization into a text and personalized visual feedback group, versus a text-only control group, was applied to control for individual differences (study onset) and feedback history (after two weeks). Results indicate that while personalized visual feedback did not increase adherence, it did manage to keep adherence constant over a three-week period, whereas fluctuations in adherence could be observed in the text-only control group. Although this was a pilot study, and its results should be interpreted with some caution, this paper shows how virtual agent feedback may have a stabilizing effect on adherence, how controlled experiments on the relationship between virtual agent support and clinically relevant measures such as adherence can be conducted, and how results may be analyzed.


Ecological momentary assessment Virtual agent Feedback Adherence 


  1. 1.
    Andersson, G., Cuijpers, P., Carlbring, P., Riper, H., Hedman, E.: Guided internet-based vs. face-to-face cognitive behavior therapy for psychiatric and somatic disorders: a systematic review and meta-analysis. World Psychiatry 13, 288–295 (2014)CrossRefGoogle Scholar
  2. 2.
    Richards, D., Richardson, T.: Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin. Psychol. Rev. 32, 329–342 (2012)CrossRefGoogle Scholar
  3. 3.
    Schueller, S.M., Tomasino, K.N., Mohr, D.C.: Integrating human support into behavioral intervention technologies: the efficiency model of support. Clin. Psychol. Sci. Pract. 24, 27–45 (2017)CrossRefGoogle Scholar
  4. 4.
    Mohr, D.C., Cuijpers, P., Lehman, K.: Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions. J. Med. Internet Res. 13, e30 (2011)CrossRefGoogle Scholar
  5. 5.
    Van Ballegooijen, W., Cuijpers, P., Van Straten, A., Karyotaki, E., Andersson, G., Smit, J.H., Riper, H.: Adherence to internet-based and face-to-face cognitive behavioural therapy for depression: a meta-analysis. PLoS ONE 9, e100674 (2014)CrossRefGoogle Scholar
  6. 6.
    Donkin, L., Christensen, H., Naismith, S.L., Neal, B., Hickie, I.B., Glozier, N.: A systematic review of the impact of adherence on the effectiveness of e-therapies. J. Med. Internet Res. 13, e52 (2011)CrossRefGoogle Scholar
  7. 7.
    Provoost, S., Lau, H.M., Ruwaard, J., Riper, H.: Embodied conversational agents in clinical psychology: a scoping review. J. Med. Internet Res. 19, e151 (2017)CrossRefGoogle Scholar
  8. 8.
    Shiffman, S., Stone, A.A., Hufford, M.R.: Ecological momentary assessment. Annu. Rev. Clin. Psychol. 4, 1–32 (2008)CrossRefGoogle Scholar
  9. 9.
    Wenze, S.J., Miller, I.W.: Use of ecological momentary assessment in mood disorders research. Clin. Psychol. Rev. 30, 794–804 (2010)CrossRefGoogle Scholar
  10. 10.
    Isbister, K., Doyle, P.: The blind men and the elephant revisited. In: Ruttkay, Z., Pelachaud, C. (eds.) From Brows to Trust. HIS, vol. 7, pp. 3–26. Springer, Dordrecht (2004). Scholar
  11. 11.
    Bickmore, T., Schulman, D., Shaw, G.: DTask and LiteBody: open source, standards-based tools for building web-deployed embodied conversational agents. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS (LNAI), vol. 5773, pp. 425–431. Springer, Heidelberg (2009). Scholar
  12. 12.
    Gratch, J., Hartholt, A.: Virtual humans: a new toolkit for cognitive science research. Proc. Cogn. Sci. Soc. 35, 41–42 (2013)Google Scholar
  13. 13.
    Mumm, J., Mutlu, B.: Designing motivational agents: the role of praise, social comparison, and embodiment in computer feedback. Comput. Hum. Behav. 27, 1643–1650 (2011)CrossRefGoogle Scholar
  14. 14.
    Bickmore, T., Schulman, D., Sidner, C.L.: Issues in Designing Agents for Long-Term Behavior Change. In: CHI Workshop on Engagement by Design, pp. 1–5 (2009)Google Scholar
  15. 15.
    movisensXS, Version 0.7.4162 (movisens GmbH, Karlsruhe, Germany).
  16. 16.
    Russell, J.A.: A circumplex model of affect. J. Pers. Soc. Psychol. 39, 1161–1178 (1980)CrossRefGoogle Scholar
  17. 17.
    Asselbergs, J., Ruwaard, J., Ejdys, M., Schrader, N., Sijbrandij, M., Riper, H.: Mobile phone-based unobtrusive ecological momentary assessment of day-to-day mood: an explorative study. J. Med. Internet Res. 18, e72 (2016)CrossRefGoogle Scholar
  18. 18.
    Desmet, P.M.A., Vastenburg, M.H., Van Bel, D., Romero, N.: Pick-a-mood development and application of a pictorial mood-reporting instrument. In: Proceedings of the 8th International Conference on Design and Emotion, London 2012, Central Saint Martins College, Art and Design, 11–14 September 2012 (2012)Google Scholar
  19. 19.
    Oinas-Kukkonen, H., Harjumaa, M.: Persuasive systems design: key issues, process model, and system features. Commun. Assoc. Inf. Syst. 24, 485–500 (2009)Google Scholar
  20. 20.
    Kelders, M.S., Kok, N.R., Ossebaard, C.H., Van Gemert-Pijnen, E.W.C.J.: Persuasive system design does matter: a systematic review of adherence to web-based interventions. J Med Internet Res. 14, e152 (2014)CrossRefGoogle Scholar
  21. 21.
    Broderick, J.E., Schwartz, J.E., Shiffman, S., Hufford, M.R., Stone, A.A.: Signaling does not adequately improve diary compliance. Ann. Behav. Med. 26, 139–148 (2003)CrossRefGoogle Scholar
  22. 22.
    Bates, D., Mächler, M., Bolker, B., Walker, S.: Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015)CrossRefGoogle Scholar
  23. 23.
    R Core Team: R: A Language and Environment for Statistical Computing (2016).
  24. 24.
    Pagliari, C., Burton, C., McKinstry, B., Szentatotai, A., David, D., Serrano Blanco, A., Ferrini, L., Albertini, S., Castro, J.C., Estevez, S., Wolters, M.: Psychosocial implications of avatar use in supporting therapy for depression. Stud. Health Technol. Inform. 181, 329–333 (2012)Google Scholar
  25. 25.
    Burleson, W.: Affective learning companions: strategies for empathetic agents with real-time multimodal affective sensing to foster meta-cognitive and meta-affective approaches to learning, motivation, and perseverance (2006).
  26. 26.
    van Ballegooijen, W., Ruwaard, J., Karyotaki, E., Ebert, D.D., Smit, J.H., Riper, H.: Reactivity to smartphone-based ecological momentary assessment of depressive symptoms (MoodMonitor): protocol of a randomised controlled trial. BMC Psychiatry 16, 359 (2016)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Simon Provoost
    • 1
    • 2
    Email author
  • Jeroen Ruwaard
    • 1
    • 2
    • 3
  • Koen Neijenhuijs
    • 1
    • 2
  • Tibor Bosse
    • 4
  • Heleen Riper
    • 1
    • 2
    • 3
    • 5
  1. 1.Section of Clinical PsychologyVrije Universiteit AmsterdamAmsterdamNetherlands
  2. 2.Amsterdam Public HealthAmsterdamNetherlands
  3. 3.GGZ inGeestAmsterdamNetherlands
  4. 4.Department of Computer ScienceVrije Universiteit AmsterdamAmsterdamNetherlands
  5. 5.University of Southern DenmarkOdenseDenmark

Personalised recommendations