Poor adherence to prescribed drug treatments is one of the leading causes of illness and treatment failure, which increases re-hospitalizations. In Mexico, the factors that most contribute to the non-adherence problem are age, polypharmacy, and education. For instance, elderly patients are prescribed with an average of seven medications after they are discharged from hospitals, and 25% of them face problems managing medications at home. A strategy that older adults use for medication adherence is to link their medication regimens to daily activities. We propose a system based in machine learning for audio-based activity recognition using Hidden Markov Models over Mel Frequency Cepstral Coefficients. The system triggers an assistive conversational agent that adapts its interaction model to the context detected. We report on two studies that provide evidence of the feasibility of our approach to assist older adults to develop consistent medication behaviors by associating them to daily routines. We first conducted an observational study with two older adults to understand the role of daily activities to develop consistent medication behaviors. Afterwards, we conducted an in situ assessment of the audio-based activity recognition system with the two study subjects. Our results showed that anchor activities with an audible manifestation were recognized with an accuracy of 79% for subject 1, and 97.6% for subject 2. Additionally, we validated how the integration of conversational agents into the system may support the mental association among activities and medication regimens that older adults fail to realize when, for instance, their intention plans involve multiple behaviors associated to an activity. The deployment of the proposed approach requires only a smart speaker, which increases its feasibility of adoption in Latin American and other developing countries.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
Tax calculation will be finalised during checkout.
Yach D (2003) Adherence to long-term therapies: evidence for action. Noncomunicable Diseases and Mental Health, World Health Organization.
Burnier M, Wuerzner G, Struijker-Boudier H, Urquhart J (2013) Measuring, analyzing, and managing drug adherence in resistant hypertension. Hypertension 62(2):218–225. https://doi.org/10.1161/HYPERTENSIONAHA.113.00687
Roux P, Pereira F, Santiago-Delefosse M, Verloo H (2018) Medication practices and experiences of older adults discharged home from hospital: a feasibility study protocol. Patient Prefer Adherence 12:1055–1063. https://doi.org/10.2147/PPA.S160990
Insel KC, Einstein GO, Morrow DG, Hepworth JT (2013) A multifaceted prospective memory intervention to improve medication adherence: design of a randomized control trial. Contemp Clin Trials 34(1):45–52. https://doi.org/10.1016/j.cct.2012.09.005
Moreno MG, Garza L, Interial MG (2013) Manejo de la medicación en el adulto mayor al alta hospitalaria. Ciencia y Enfermeria 19(3):11–20. https://doi.org/10.4067/S0717-95532013000300002
Wimmer BC, Bell JS, Fastbom J, Wiese MD, Johnell K (2016) Medication regimen complexity and number of medications as factors associated with unplanned hospitalizations in older people: a population-based cohort study. J Gerontol A Biol Sci Med Sci 71(6):831–837. https://doi.org/10.1093/gerona/glv219
Bogetti M, González C, Juárez T, Sánchez S, Rosas O (2016) Severe potential drug-drug interactions in older adults with dementia and associated factors. Clinics (Sao Paulo) 71(1):17–21. https://doi.org/10.6061/clinics/2016(01)04
Mahmood A, Elnour AA, Ali AAA, Hassan NAGM, Shehab A, Bhagavathula AS (2016) Evaluation of rational use of medicines (RUM) in four government hospitals in UAE. Saudi Pharm J 24(2):189–196. https://doi.org/10.1016/j.jsps.2015.03.003
Arriola MA (2015) The role of health regulations in the rational use of medicines. Gac Med Mex 151(5):690–698 Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/26526486. Accessed 20 Nov 2019
Poudel A, Nissen LM (2018) Rational and responsible medicines use. In social and administrative aspects of pharmacy in low-and middle-income countries: present challenges and future solutions, vol 16. Elsevier Inc, pp 263–277. https://doi.org/10.1016/B978-0-12-811228-1.00016-9
Graabæk T, Terkildsen BG, Lauritsen KE, Almarsdóttir AB (2019) Frequency of undocumented medication discrepancies in discharge letters after hospitalization of older patients: a clinical record review study. Ther Adv Drug Saf 10:204209861985804. https://doi.org/10.1177/2042098619858049
Boron JB, Rogers WA, Fisk AD (2006) Medication adherence strategies in older adults. Proc Hum Factors Ergon Soc Ann Meet 50(2):170–174. https://doi.org/10.1177/154193120605000201
Sanders MJ, Van T (2013) Using daily routines to promote medication adherence in older adults. Am J Occup Ther 67(1):91–99. https://doi.org/10.5014/ajot.2013.005033
Stawarz K, Rodríguez MD, Cox AL, Blandford A (2016) Understanding the use of contextual cues: design implications for medication adherence technologies that support remembering. Digit Health 2:205520761667870. https://doi.org/10.1177/2055207616678707
Conn VS, Ruppar TM (2017) Medication adherence outcomes of 771 intervention trials: systematic review and meta-analysis. Prev Med 99:269–276. https://doi.org/10.1016/j.ypmed.2017.03.008
Voicebot.ai (2019) Amazon echo & alexa stats. IOP Publishing Voicebotai. https://voicebot.ai/amazon-echo-alexa-stats/. Accessed 20 Nov 2019
Yen PY, Bakken S (2012) Review of health information technology usability study methodologies. J Am Med Inform Assoc 19(3):413–422. https://doi.org/10.1136/amiajnl-2010-000020
Agarawala A, Greenberg S, Ho G (2004) The context-aware pill bottle and medication monitor. Technical Report 2004-752-17. Department of Computer Science, University of Calgary, Calgary Retrieved from https://prism.ucalgary.ca/. Accessed 20 Nov 2019
de Oliveira R, Cherubini M, Oliver N (2010) MoviPill: improving medication compliance for elders using a mobile persuasive social game. In proceedings of the 12th ACM international conference on ubiquitous computing - Ubicomp ’10, pp 251–260. https://doi.org/10.1145/1864349.1864371
Lee ML, Dey AK (2014) Real-time feedback for improving medication taking. In: Conference on Human Factors in Computing Systems – Proceedings, pp 2259–2268. https://doi.org/10.1145/2556288.2557210
Dasgupta D, Johnson RA, Chaudhry B, Reeves KG, Willaert P, Chawla NV (2016) Design and evaluation of a medication adherence application with communication for seniors in independent living communities. AMIA, Annual Symposium proceedings. AMIA Symposium, 2016, 480-489
Furniss D, Barber N, Lyons I, Eliasson L, Blandford A (2014) Unintentional non-adherence: can a spoon full of resilience help the medicine go down? BMJ Qual Saf 23(2):95–98. https://doi.org/10.1136/bmjqs-2013-002276
Park LG, Howi J, Dracup K (2014) A quantitative systematic review of the efficacy of mobile phone interventions to improve medication adherence. J Adv Nurs 70(9):1932–1953. https://doi.org/10.1111/jan.12400
Stawarz K, Cox AL, Blandford A (2014) Don’t forget your pill!. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI ’14, 2269-2278). https://doi.org/10.1145/2556288.2557079
Lee ML, Dey AK (2015) Sensor-based observations of daily living for aging in place. Pers Ubiquit Comput 19(1):27–43. https://doi.org/10.1007/s00779-014-0810-3
Skubic M, Guevara RD, Rantz M (2015) Automated Health Alerts Using In-Home Sensor Data for Embedded Health Assessment. IEEE J Transl Eng Health Med 3:3–11. https://doi.org/10.1109/JTEHM.2015.2421499
Turner KJ, Gillespie A, McMichael LJ (2011) Rigorous development of prompting dialogues. J Biomed Inform 44(5):713–727. https://doi.org/10.1016/j.jbi.2011.03.010
Purington A, Taft JG, Sannon S, Bazarova NN, Taylor SH (2017) “Alexa is my new BFF”: Social roles, user satisfaction, and personification of the Amazon Echo. In: Conference on Human Factors in Computing Systems – Proceedings, vol Part F127655, pp 2853–2859. https://doi.org/10.1145/3027063.3053246
Pradhan A, Mehta K, Findlater L (2018) “Accessibility came by accident”: use of voice-controlled intelligent personal assistants by people with disabilities. In: Conference on Human Factors in Computing Systems – Proceedings, vol 2018-April, pp 1–13. https://doi.org/10.1145/3173574.3174033
Google Home - Smart Speaker & Home Assistant. Google Store (n.d.). https://store.google.com/mx/product/google_home Accessed 17 November 2019
Shalini S, Levins T, Robinson EL, Lane K, Park G, Skubic M (2019) Development and comparison of customized voice-assistant systems for independent living older adults. In: Zhou J, Salvendy G (eds) Human aspects of IT for the aged population. Social media, games and assistive environments. HCII 2019. Lecture Notes in Computer Science, vol 11593. Springer, Cham. https://doi.org/10.1007/978-3-030-22015-0_36
Lobo J, Ferreira L, Ferreira AJ (2017) CARMIE: A conversational medication assistant for heart failure. Int J E-Health Med Commun 8:21–37. https://doi.org/10.4018/IJEHMC.2017100102
Tschanz M, Dorner TL, Holm J, Denecke K (2018) Using eMMA to manage medication. Computer 51:18–25. https://doi.org/10.1109/MC.2018.3191254
Epicollect5 - Free and easy-to-use mobile data-gathering platform. (n.d.). https://five.epicollect.net/ Accessed June 4 2018
Claxton AJ, Cramer J, Pierce C (2001) A systematic review of the associations between dose regimens and medication compliance. Clin Ther 23:1296–1310. https://doi.org/10.1016/s0149-2918(01)80109-0
Maimone R, Guerini M, Dragoni M, Bailoni T, Eccher C (2018) PerKApp: a general purpose persuasion architecture for healthy lifestyles. J Biomed Inform 82:70–87. https://doi.org/10.1016/j.jbi.2018.04.010
Hermsen S, Frost J, Renes RJ, Kerkhof P (2016) Using feedback through digital technology to disrupt and change habitual behavior: a critical review of current literature. Comput Hum Behav 57:61–74. https://doi.org/10.1016/j.chb.2015.12.023
Lally P, Gardner B (2013) Promoting habit formation. Health Psychol Rev 7:S137–S158. https://doi.org/10.1080/17437199.2011.603640
Cruz-Sandoval D, Beltran-Marquez J, Garcia-Constantino M, Gonzalez-Jasso LA, Favela J, Lopez-Nava IH, Cleland I, Ennis A, Hernandez-Cruz N, Rafferty J, Synnott J, Nugent C (2019) Semi-automated data labeling for activity recognition in pervasive healthcare. Sensors 19:3035. https://doi.org/10.3390/s19143035
Richards M (2015) Software Architecture Patterns. O’Reilly Media, Inc
Beltrán J, Chávez E, Favela J (2015) Scalable identification of mixed environmental sounds, recorded from heterogeneous sources. Pattern Recogn Lett 68:153–160. https://doi.org/10.1016/j.patrec.2015.08.027
Ting KM (2017) Confusion Matrix. In: Sammut C, Webb GI (eds) Encyclopedia of machine learning and data mining. Springer, Boston. https://doi.org/10.1007/978-1-4899-7687-1_50
Rozanski N, Woods E (2005) Software systems architecture: working with stakeholders using viewpoints and perspectives. Addison-Wesley
Verplanken B, Orbell S (2003) Reflections on past behavior: a self-report index of habit strength1. J Appl Soc Psychol 33:1313–1330. https://doi.org/10.1111/j.1559-1816.2003.tb01951.x
Martin B, Hanington BM (2012) Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport Publishers
Lewis JR, Utesch BS, Maher DE (2015) Measuring perceived usability: the SUS, UMUX-LITE, and AltUsability. Int J Hum Comput Interact 31:496–505. https://doi.org/10.1080/10447318.2015.1064654
O’Brien HL, Cairns P, Hall M (2018) A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. Int J Hum Comput Stud 112:28–39. https://doi.org/10.1016/j.ijhcs.2018.01.004
This work was partially funded by the CONACYT through scholarships provided to Maribel Valenzuela and Dagoberto Cruz-Sandoval. Internal Gran Project 400/6/C/6/21 registered at UABC.
Conflict of interest
The authors declare that they have no conflict of interest.
The Ethics Review Board of the Nursing School of the UABC University approved the project protocol and provided the informed consent forms.
Consent to participate
Participants signed informed consent forms in which we indicate that all the data collected will be used for the project, their identity information will be preserved, and the audio data will be deleted once processed.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Rodríguez, M.D., Beltrán, J., Valenzuela-Beltrán, M. et al. Assisting older adults with medication reminders through an audio-based activity recognition system. Pers Ubiquit Comput 25, 337–351 (2021). https://doi.org/10.1007/s00779-020-01420-4
- Activity recognition
- Conversational agents
- Medication adherence
- Older adults