Skip to main content
SpringerLink
Log in

Usability of an Intelligent Virtual Assistant for Promoting Behavior Change and Self-Care in Older People with Type 2 Diabetes

  • Mobile & Wireless Health
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

In the context of the VASelfCare project, we developed an application prototype of an intelligent anthropomorphic virtual assistant. Designed as a relational agent, the virtual assistant has the role of supporting older people with Type 2 Diabetes Mellitus (T2D) in medication adherence and lifestyle changes. Our paper has two goals: describing the essentials of this prototype, and reporting on usability evaluation. We describe the general architecture of the prototype, including the graphical component, and focus on its main feature: the incorporation, in the way the dialogue flows, of Behavior Change Techniques, identified through a theoretical framework, the Behaviour Change Wheel. Usability was experimentally evaluated in field tests in a purposive sample of 20 participants (11 older adults with T2D and 9 experts). The Portuguese version of the System Usability Scale was employed, supplemented with qualitative data from open questions, diaries, digital notes and telephone follow-ups. The aggregated mean SUS score was 73,75 (SD 13,31), which corresponds to a borderline rating of excellent. Textual data were content analyzed and will be prioritized to further improve usability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. https://www.daz3d.com

  2. https://unity.com/

  3. https://harposoftware.com

  4. Administração Regional de Saúde de Lisboa e Vale do Tejo (Regional Health Administration of Lisbon and Tagus Valley)

References

  1. op den Akker R., Klaassen R., Lavrysen T., Geleijnse G., van Halteren A., Schwietert H., Van der Hout M.: A Personal Context-Aware Multi-Device Coaching Service That Supports a Healthy Lifestyle.. In: Proceedings of the 25th BCS Conference on Human-Computer Interaction, BCS-HCI ’11, pp 443-448, BCS Learning & Development Ltd., Swindon, GBR, 2011

  2. (2019). American Diabetes Association: 12. older adults: Standards of medical care in diabetes—2019. Diabetes Care 42 (Supplement 1), S139–S147. https://doi.org/10.2337/dc19-S012. https://care.diabetesjournals.org/content/42/Supplement_1/S139

  3. (2019). American Diabetes Association: 5. lifestyle management: Standards of medical care in diabetes—2019. Diabetes Care 42 (Supplement 1), S46–S60. https://doi.org/10.2337/dc19-S005. https://care.diabetesjournals.org/content/42/Supplement_1/S46

  4. Arnhold M., Quade M., Kirch W.: Mobile Applications for Diabetics : A Systematic Review and Expert-Based Usability Evaluation Considering the Special Requirements of Diabetes Patients Age 50 Years or Older Corresponding Author : Journal of Medical Internet Research 16 (4): 1–18, 2014. https://doi.org/10.2196/jmir.2968

    Google Scholar 

  5. Balsa J., Neves P., Félix I.B., Guerreiro M.P., Alves P., Carmo M.B., Marques D., Dias A., Henriques A., Cláudio A.P.: Intelligent Virtual Assistant for Promoting Behaviour Change in Older People with T2D. In: (P.M. Oliveira, P. Novais, L.P. Reis, Eds.) Progress in Artificial Intelligence - 19th EPIA Conference on Artificial Intelligence, EPIA 2019, Vila Real, Portugal, September 3-6, 2019, Proceedings, Part I, Lecture Notes in Computer Science, vol. 11804, pp. 372–383. Springer, 2019, https://doi.org/10.1007/978-3-030-30241-2_32

  6. Beinema T., op den Akker H., Hermens H.: Creating an artificial coaching engine for multi-domain conversational coaches in eHealth applications. In: (E. André, T.W. Bickmore, S. Vrochidis, L. Wanner, Eds.) Proceedings of the AAMAS Workshop on Intelligent Conversation Agents in Home and Geriatric Care Applications co-located with the Federated AI Meeting (FAIM 2018), ICAHGCA@AAMAS 2018, Stockholm, Sweden, July 15, 2018, CEUR Workshop Proceedings, vol. 2338, pp. 35–39. CEUR-WS.org, 2018. http://ceur-ws.org/Vol-2338/paper5.pdf

  7. Bhuyan S.S., Lu N., Chandak A., Kim H., Wyant D., Bhatt J., Kedia S., Chang C.F. (2016) Use of Mobile Health Applications for Health-Seeking Behavior Among US Adults, vol 40. https://doi.org/10.1007/s10916-016-0492-7

  8. Bickmore T. W., Caruso L., Clough-Gorr K.: Heeren, T.: ‘It’s just like you talk to a friend’ relational agents for older adults. Interacting with Computers 17 (6): 711–735, 2005. https://doi.org/10.1016/j.intcom.2005.09.002

    Google Scholar 

  9. Bickmore T.W., Puskar K., Schlenk E.A., Pfeifer L.M., Sereika S.M.: Maintaining reality: Relational agents for antipsychotic medication adherence. Interacting with Computers 22 (4): 276–288, 2010. https://doi.org/10.1016/j.intcom.2010.02.001

    Google Scholar 

  10. Bickmore T. W., Silliman R. A., Nelson K., Cheng D. M., Winter M., Henault L., Paasche-Orlow M. K.: A randomized controlled trial of an automated exercise coach for older adults. Journal of the American Geriatrics Society 61 (10): 1676–1683, 2013. https://doi.org/10.1111/jgs.12449

    PubMed  Google Scholar 

  11. Boye J.: Dialogue management for automatic troubleshooting and other problem-solving applications.. In: Proceedings of the 8th SIGDial workshop on discourse and dialogue, pp. 247–255., 2007. https://sigdial.org/files/workshops/workshop8/Proceedings/SIGdial45.pdf

  12. Brooke J.: SUS: A retrospective. Journal of Usability Studies 8 (2): 29–40, 2013

    Google Scholar 

  13. Buinhas S., Cláudio A.P., Carmo M.B., Balsa J., Cavaco A., Mendes A., Félix I., Pimenta N., Guerreiro M.P.: Virtual Assistant to Improve Self-care of Older People with Type 2 Diabetes: First Prototype.. In: Gerontechnology: First International Workshop, IWoG 2018, pp. 236–248. Springer, Cham., 2019, https://doi.org/10.1007/978-3-030-16028-9_21. http://link.springer.com/10.1007/978-3-030-16028-9_21

  14. Bunt H., Petukhova V., Malchanau A., Fang A., Wijnhoven K.: The dialogbank: dialogues with interoperable annotations. Language Resources and Evaluation 53 (2): 213–249, 2018. https://doi.org/10.1007/s10579-018-9436-9

    Google Scholar 

  15. Burgan D. (2017) Dialogue Systems & Dialogue Management. Tech. rep., National Security & ISR Division Defence Science and Technology Group

  16. collab=Centers for Disease Control and Prevention (2017) C.D.C.: National diabetes statistics report: estimates of diabetes and its burden in the united states. Atlanta GA: centers for disease control and prevention; 2017

  17. Clȧudio A. P., Carmo M. B., Pinto V., Cavaco A., Guerreiro M. P.: Virtual humans for training and assessment of self-medication consultation skills in pharmacy students.. In: 10Th international conference on computer science and education, pp. 175–180. ICCSE, 2015

  18. Cui M., Wu X., Mao J., Wang X., Nie M.: T2DM self-management via smartphone applications: a systematic review and meta-analysis. PLoS ONE 11 (11): 1–15, 2016

    CAS  Google Scholar 

  19. Das K., Beinema T., Akker H.O.D., Hermens. H.: Generation of multi-party dialogues among embodied conversational agents to promote active living and healthy diet for subjects suffering from type 2 diabetes.. In: Proceedings of the 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health - Volume 1: ICT4AWE, 2019, pp 297–304, INSTICC, SciTePress, https://doi.org/10.5220/0007750602970304

  20. Donsa K., Spat S., Beck P., Pieber T. R., Holzinger A.: towards personalization of diabetes therapy using computerized decision support and machine learning: some open problems and challenges.. In: Smart health, pp. 237–260. Springer international publishing, 2015, https://doi.org/10.1007/978-3-319-16226-3_10

  21. Fundação Francisco Manuel dos Santos Pordata: base de dados Portugal contemporaneô. Accessed: 2020-03-10. https://www.pordata.pt/DB/Ambiente+de+Consulta/Tabela

  22. Garci̇a-Pėrez L. E., Ȧlvarez M., Dilla T., Gil-Guillėn V., Orozco-Beltrȧn D.: Adherence to therapies in patients with type 2 diabetes. Diabetes Therapy 4 (2): 175–194, 2013. https://doi.org/10.1007/s13300-013-0034-y

    PubMed  PubMed Central  Google Scholar 

  23. Hou C., Carter B., Hewitt J., Francisa T., Mayor S.: Do Mobile Phone Applications Improve Glycemic Control (HbA1c) in the Self-management of Diabetes? A Systematic Review, Meta-analysis, and GRADE of 14 Randomized Trials. Diabetes Care 39 (11): 2089–2095, 2016

    PubMed  Google Scholar 

  24. (2017). International Diabetes Federation : IDF Diabetes Atlas, 8th edn. International Diabetes Federation, Brussels, Belgium

  25. Kebede M. M., Liedtke T. P., Mȯllers T., Pischke C. R. (2017) Characterizing Active Ingredients of eHealth Interventions Targeting Perso ns With Poorly Controlled Type 2 Diabetes Mellitus Using the Behavior Change Techniques Taxonomy: Scoping Review, vol 19. https://www.jmir.org/2017/10/e348/

  26. Kekȧlȧinen A., Kaikkonen A., Kankainen A., Cankar M., Kallio T.: Usability testing of mobile applications: a comparison between laboratory and field testing. Journal of Usability Studies 1 (1): 4–17, 2005

    Google Scholar 

  27. Lewis J.R.: Sample sizes for usability tests. Interactions 13 (6): 29, 2006. https://doi.org/10.1145/1167948.1167973

    Google Scholar 

  28. Lian J.X., McGhee S.M., Chau J., Wong C.K.H., Lam C.L.K., Wong W.C.W.: Systematic review on the cost-effectiveness of self-management education programme for type 2 diabetes mellitus. Diabetes Research and Clinical Practice 127: 21–34, 2017. https://doi.org/10.1016/j.diabres.2017.02.021

    CAS  PubMed  Google Scholar 

  29. Lison P.: A hybrid approach to dialogue management based on probabilistic rules. Computer Speech &, Language 34 (1): 232–255, 2015

    Google Scholar 

  30. Lorig K.R., Holman H.R.: Self-management education: History, definition, outcomes, and mechanisms. Annals of Behavioral Medicine 26 (1): 1–7, 2003. https://doi.org/10.1207/S15324796ABM2601_01

    PubMed  Google Scholar 

  31. Maramba I., Chatterjee A., Newman C.: Methods of usability testing in the development of eHealth applications: A scoping review. International Journal of Medical Informatics 126: 95–104, 2019. https://doi.org/10.1016/j.ijmedinf.2019.03.018. http://www.sciencedirect.com/science/article/pii/S1386505618313182

    PubMed  Google Scholar 

  32. Martins A.I., Rosa A.F., Queirós A., Silva A., Rocha N.P.: European portuguese validation of the system usability scale (sus). Procedia Computer Science 67: 293–300, 2015. https://doi.org/10.1016/j.procs.2015.09.273. Proceedings of the 6th International Conference on Software Development and Technologies for Enhancing Accessibility and Fighting Info-exclusion

    Google Scholar 

  33. Merdivan E., Singh D., Hanke S., Kropf J., Holzinger A., Geist M.: Human annotated dialogues dataset for natural conversational agents. Applied Sciences 10 (3): 762, 2020. https://doi.org/10.3390/app10030762

    Google Scholar 

  34. Michie S., Atkins L., West R. (2014) The behaviour change wheel : a guide to designing interventions Silverback Publishing

  35. Michie S., Richardson M., Johnston M., Abraham C., Francis J., Hardeman W., Eccles M. P., Cane J., Wood C.E.: The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: Building an international consensus for the reporting of behavior change interventions. Annals of Behavioral Medicine 46 (1): 81–95, 2013. https://doi.org/10.1007/s12160-013-9486-6

    PubMed  Google Scholar 

  36. Michie S., Wood C.E., Johnston M., Abraham C., Francis J.J., Hardeman W.: Behaviour change techniques: The development and evaluation of a taxonomic method for reporting and describing behaviour change interventions (a suite of five studies involving consensus methods, randomised controlled trials and analysis of qualitative da. Health Technology Assessment 19 (99): 1–187, 2015. https://doi.org/10.3310/hta19990

    PubMed  Google Scholar 

  37. Monkaresi H., Calvo R. A., Pardo A., Chow K., Mullan B., Lam M., M.Twigg S., Cook D.I.: Intelligent diabetes lifestyle coach.. In: Proceedings of the Fifth International Workshop on Smart Healthcare and Wellness Applications (SmartHealth’13), pp. 1–4, 2013

  38. Morrissey E.C., Corbett T.K., Walsh J.C., Molloy G.J.: Behavior change techniques in apps for medication adherence: A content analysis. American Journal of Preventive Medicine 50 (5): e143–e146, 2016. https://doi.org/10.1016/j.amepre.2015.09.034

    PubMed  Google Scholar 

  39. Moumane K., Idri A., Abran A.: Usability evaluation of mobile applications using iso 9241 and iso 25062 standards. SpringerPlus 5 (1): 548, 2016. https://doi.org/10.1186/s40064-016-2171-z

    PubMed  PubMed Central  Google Scholar 

  40. (2018). https://doi.org/10.1787/health_glance_eur-2018-en

  41. Pal K., Eastwood S. V., Michie S., Farmer A. J., Barnard M. L., Peacock R., Wood B., Inniss J. D., Murray E. Computer-based diabetes self-management interventions for adults with type 2 diabetes mellitus. Cochrane Database of Systematic Reviews 2013, Issue 3. Art. No.: CD008776. DOI: 10.1002/14651858.CD008776.pub2.

  42. (2018). PRO-STEP Project Consortium: PRO-STEP project: Promoting Self-management for chronic diseases in Europe - Final Report

  43. Sauro J. (2011) A practical guide to the system usability scale: background, benchmarks & best practices CreateSpace Independent Publishing Platform

  44. Sauro J., Lewis J. R. (2012) Quantifying the user experience : practical statistics for user research Elsevier/Morgan Kaufmann

  45. Schaarup C., Hartvigsen G., Larsen L.B., Tan Z., Årsand E., Hejlesen O.K.: Assessing the potential use of eye-tracking triangulation for evaluating the usability of an online diabetes exercise system. In: (I.N. Sarkar, A. Georgiou, P.M. de Azevedo Marques, Eds.) MEDINFO 2015: eHealth-enabled Health - Proceedings of the 15th World Congress on Health and Biomedical Informatics, São Paulo, Brazil, 19-23 August 2015, Studies in Health Technology and Informatics, vol. 216, pp. 84–88. IOS Press, 2015, https://doi.org/10.3233/978-1-61499-564-7-84

  46. Smith C., Crook N., Dobnik S., Charlton D., Boye J., Pulman S., de la Camara R. S., Turunen M., Benyon D., Bradley J., Gambȧck B., Hansen P., Mival O., Webb N., Cavazza M.: Interaction strategies for an affective conversational agent. Presence:, Teleoperators and Virtual Environments 20 (5): 395–411, 2011

    Google Scholar 

  47. Van Vugt M., De Wit M., Cleijne W.H., Snoek F.J.: Use of behavioral change techniques in web-based selfmanagement programs for type 2 diabetes patients: systematic review. J Med Internet Res 15 (12): e279, 2013. https://doi.org/10.2196/jmir.2800

    PubMed  PubMed Central  Google Scholar 

  48. Wildenbos G., Peute L., Jaspers M.: Aging barriers influencing mobile health usability for older adults: A literature based framework (mold-us). International Journal of Medical Informatics 114: 66–75, 2018. https://doi.org/10.1016/j.ijmedinf.2018.03.012. http://www.sciencedirect.com/science/article/pii/S1386505618302454 http://www.sciencedirect.com/science/article/pii/S1386505618302454

    CAS  PubMed  Google Scholar 

  49. (2018). World Health Organization: Diabetes. https://www.who.int/news-room/fact-sheets/detail/diabetes

  50. Zapata B.C., Fernández-Alemán J.L., Idri A., Toval A.: Empirical studies on usability of mhealth apps: A systematic literature review. Journal of Medical Systems 39 (2): 1, 2015. https://doi.org/10.1007/s10916-014-0182-2

    PubMed  Google Scholar 

Download references

Acknowledgments

The authors are indebted to other VASelfCare team members for their contribution to the software development (http://vaselfcare.rd.ciencias.ulisboa.pt/); Pedro Alves’ and Pedro Neves’ contributions were instrumental for the work herein reported.

Funding

This work was supported by FCT and Compete 2020 (grant number LISBOA-01-0145-FEDER-024250). It is also supported by UID/MULTI /04046/2019 Research Unit grant from FCT, Portugal (to BioISI).

Author information

Authors and Affiliations

  1. Faculdade de Ciências da Universidade de Lisboa, Lisbon, Portugal

    João Balsa

  2. Unidade de Investigação e Desenvolvimento em Enfermagem (ui&de), Escola Superior de Enfermagem de Lisboa, Lisbon, Portugal

    Isa Félix, Isabel Costa e Silva, Adriana Henriques & Mara Pereira Guerreiro

  3. Biosystems & Integrative Sciences Institute (BioISI), Faculdade de Ciências da Universidade de Lisboa, Lisbon, Portugal

    Ana Paula Cláudio & Maria Beatriz Carmo

  4. Escola Superior de Enfermagem de Lisboa, Lisbon, Portugal

    Ana Guerreiro & Maria Guedes

  5. Centro de Investigação Interdisciplinar Egas Moniz, Instituto Universitário Egas Moniz, Monte de Caparica, Almada, Portugal

    Mara Pereira Guerreiro

  6. Faculdade de Medicina da Universidade de Lisboa, Instituto de Saúde Ambiental (ISAMB), Lisbon, Portugal

    Adriana Henriques

Authors
  1. João Balsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Isa Félix
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ana Paula Cláudio
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maria Beatriz Carmo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Isabel Costa e Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ana Guerreiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Maria Guedes
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Adriana Henriques
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mara Pereira Guerreiro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to João Balsa.

Ethics declarations

Conflict of interests

All authors declare that they have no conflicts of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards. Besides, the study protocol, which encompasses iterative tests with older people with T2D and health professionals during the software development phase, was granted ethical approval from the Portuguese authoritiesFootnote 4 (6104/CES/2018 ARSLVT). This article does not contain any studies with animals performed by any of the authors.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Artificial Intelligence in Medicine (AIM)

Guest Editor: Filipe Portela

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Balsa, J., Félix, I., Cláudio, A.P. et al. Usability of an Intelligent Virtual Assistant for Promoting Behavior Change and Self-Care in Older People with Type 2 Diabetes. J Med Syst 44, 130 (2020). https://doi.org/10.1007/s10916-020-01583-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-020-01583-w

Keywords

Search

Navigation