Skip to main content

Advertisement

SpringerLink
Log in

Getting a Technology-Based Diabetes Intervention Ready for Prime Time: a Review of Usability Testing Studies

  • Psychosocial Aspects (KK Hood, Section Editor)
  • Published:
Current Diabetes Reports Aims and scope Submit manuscript

Abstract

Consumer health technologies can educate patients about diabetes and support their self-management, yet usability evidence is rarely published even though it determines patient engagement, optimal benefit of any intervention, and an understanding of generalizability. Therefore, we conducted a narrative review of peer-reviewed articles published from 2009 to 2013 that tested the usability of a web- or mobile-delivered system/application designed to educate and support patients with diabetes. Overall, the 23 papers included in our review used mixed (n = 11), descriptive quantitative (n = 9), and qualitative methods (n = 3) to assess usability, such as documenting which features performed as intended and how patients rated their experiences. More sophisticated usability evaluations combined several complementary approaches to elucidate more aspects of functionality. Future work pertaining to the design and evaluation of technology-delivered diabetes education/support interventions should aim to standardize the usability testing processes and publish usability findings to inform interpretation of why an intervention succeeded or failed and for whom.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

  1. Bu D, Pan E, Walker J, Adler-Milstein J, Kendrick D, Hook JM, et al. Benefits of information technology-enabled diabetes management. Diabetes Care. 2007;30:1137–42.

    Article  PubMed  Google Scholar 

  2. Archer E, Groessl EJ, Sui X, McClain AC, Wilcox S, Hand GA, et al. An economic analysis of traditional and technology-based approaches to weight loss. Am J Prev Med. 2012;43:176–82.

    Article  PubMed  Google Scholar 

  3. Or CK, Tao D. Does the use of consumer health information technology improve outcomes in the patient self-management of diabetes? A meta-analysis and narrative review of randomized controlled trials. Int J Med Inform 2014.

  4. LeRouge C, Wickramasinghe N. A review of user-centered design for diabetes-related consumer health informatics technologies. J Diabetes Sci Technol. 2013;7:1039–56.

    PubMed  PubMed Central  Google Scholar 

  5. Corry MD, Frick TW, Hansen L. User-centered design and usability testing of a Web site: an illustrative case study. Etr & D-Educ Tech Res. 1997;45:65–76.

    Article  Google Scholar 

  6. Schleyer TK, Thyvalikakath TP, Hong J. What is user-centered design? J Am Dent Assoc. 2007;138:1081–2.

    Article  PubMed  Google Scholar 

  7. Jaspers MW. A comparison of usability methods for testing interactive health technologies: methodological aspects and empirical evidence. Int J Med Inform. 2009;78:340–53.

    Article  PubMed  Google Scholar 

  8. Napolitano MA, Hayes S, Russo G, Muresu D, Giordano A, Foster GD. Using avatars to model weight loss behaviors: participant attitudes and technology development. J Diabetes Sci Technol. 2013;7:1057–65.

    PubMed  PubMed Central  Google Scholar 

  9. Bus SA, Waaijman R, Nollet F. New monitoring technology to objectively assess adherence to prescribed footwear and assistive devices during ambulatory activity. Arch Phys Med Rehabil. 2012;93:2075–9.

    Article  PubMed  Google Scholar 

  10. Hazenberg CE, Bus SA, Kottink AI, Bouwmans CA, Schonbach-Spraul AM, van Baal SG. Telemedical home-monitoring of diabetic foot disease using photographic foot imaging—a feasibility study. J Telemed Telecare. 2012;18:32–6.

    Article  PubMed  Google Scholar 

  11. Morak J, Schwarz M, Hayn D, Schreier G. Feasibility of mHealth and Near Field Communication technology based medication adherence monitoring. Conf Proc IEEE Eng Med Biol Soc. 2012;2012:272–5.

    PubMed  Google Scholar 

  12. Keith-Hynes P, Guerlain S, Mize B, Hughes-Karvetski C, Khan M, McElwee-Malloy M, et al. DiAs user interface: a patient-centric interface for mobile artificial pancreas systems. J Diabetes Sci Technol. 2013;7:1416–26.

    PubMed  Google Scholar 

  13. Popay J, Roberts H, Sowden A, Petticrew M, Arai L, Rodgers M, Britten N, Roen K, Duffy S. Guidance on the conduct of narrative synthesis in systematic reviews. A Product from the ESRC Methods Programme 2006:1–92.

  14. Britto MT, Jimison HB, Munafo JK, Wissman J, Rogers ML, Hersh W. Usability testing finds problems for novice users of pediatric portals. J Am Med Inform Assoc. 2009;16:660–9.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Urowitz S, Wiljer D, Dupak K, Kuehner Z, Leonard K, Lovrics E, et al. Improving diabetes management with a patient portal: a qualitative study of diabetes self-management portal. J Med Internet Res. 2012;14:e158.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Dick JJ, Nundy S, Solomon MC, Bishop KN, Chin MH, Peek ME. Feasibility and usability of a text message-based program for diabetes self-management in an urban African-American population. J Diabetes Sci Technol. 2011;5:1246–54.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Boren MT, Ramey J. Thinking aloud: reconciling theory and practice. IEEE Trans Prof Commun. 2000;43:261–78.

    Article  Google Scholar 

  18. Simon AC, Holleman F, Gude WT, Hoekstra JB, Peute LW, Jaspers MW, et al. Safety and usability evaluation of a web-based insulin self-titration system for patients with type 2 diabetes mellitus. Artif Intell Med. 2013;59:23–31.

    Article  PubMed  Google Scholar 

  19. Wharton C, Rieman J, Lewis C, Polson P. The cognitive walkthrough method: a practitioner’s guide. In: Usability Inspection Methods. New York: Wiley; 1994. p. 105–41.

    Google Scholar 

  20. Lewis C, Rieman J. Chapter 4. Evaluating the design without users. In Task-centered user interface design: a practical introduction. Online via shareware; 1993, 1994.

  21. Nielsen J, Molich R. Heuristic evaluation of user interfaces. In Proc ACM CHI'90 Conf; April 1–5; Seattle, WA. 1990: 249–256.

  22. Demidowich AP, Lu K, Tamler R, Bloomgarden Z. An evaluation of diabetes self-management applications for Android smartphones. J Telemed Telecare. 2012;18:235–8.

    Article  PubMed  Google Scholar 

  23. Nijland N, van Gemert-Pijnen JE, Kelders SM, Brandenburg BJ, Seydel ER. Factors influencing the use of a Web-based application for supporting the self-care of patients with type 2 diabetes: a longitudinal study. J Med Internet Res. 2011;13:e71. This study used four usability approaches and collected data on reasons for discontinued use of a website. It only took 20 users to identify 166 unique usability problems, and users attributed a suboptimal user interface and limited interactivity as reasons for discontinued use.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Froisland DH, Arsand E, Skarderud F. Improving diabetes care for young people with type 1 diabetes through visual learning on mobile phones: mixed-methods study. J Med Internet Res. 2012;14:e111. This study illustrates how to conduct a comprehensive qualitative and quantitative usability testing approach to identify technical problems, obtain users’ ideas for improvements, and, in turn, inform further development and testing.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Bangor A, Kortum PT, Miller JT. An empirical evaluation of the System Usability Scale (SUS). Int J Hum Comput Interact. 2008;24:574–94.

    Article  Google Scholar 

  26. Brooke J. SUS: a “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland AL, editors. Usability evaluation in industry. London: Taylor and Francis; 1996.

    Google Scholar 

  27. Ossebaard HC, Seydel ER, van Gemert-Pijnen L. Online usability and patients with long-term conditions: a mixed-methods approach. Int J Med Inform. 2012;81:374–87. This study employed four usability testing approaches, including software that recorded screen action during think aloud sessions. Utilization of a variety of methods uncovered negative user experiences, negative think aloud expressions, incomplete user tasks by most participants, and usability issues pertaining to navigation and non-personalized content that could be improved.

    Article  PubMed  Google Scholar 

  28. Yen PY, Bakken S. Review of health information technology usability study methodologies. J Am Med Inform Assoc. 2012;19:413–22.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Lewis JR. IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. In Human Factors Group. Boca Raton, FL: IBM Corporation; 1993.

  30. Monkman H, Kushniruk A. A health literacy and usability heuristic evaluation of a mobile consumer health application. Stud Health Technol Inf. 2013;192:724–8.

    Google Scholar 

  31. Broderick J, Devine T, Langhans E, Lemerise AJ, Lier S, Harris L. Designing health literate mobile apps. Washington: Institute of Medicine; 2014.

    Google Scholar 

  32. Chakkalakal RJ, Kripalani S, Schlundt DG, Elasy TA, Osborn CY. Disparities in using technology to access health information: race versus health literacy. Diabetes Care. 2014;37:e53–4.

    Article  PubMed  Google Scholar 

  33. Sarkar U, Karter AJ, Liu JY, Adler NE, Nguyen R, Lopez A, et al. The literacy divide: health literacy and the use of an internet-based patient portal in an integrated health system—results from The Diabetes Study of Northern California (DISTANCE). J Health Commun. 2010;15 Suppl 2:183–96.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Smith A. Older adults and technology use. Washington: Pew Research Center; 2014.

    Google Scholar 

  35. Lyles CR, Harris LT, Jordan L, Grothaus L, Wehnes L, Reid RJ, et al. Patient race/ethnicity and shared medical record use among diabetes patients. Med Care. 2012;50:434–40.

    Article  PubMed  Google Scholar 

  36. Sarkar U, Karter AJ, Liu JY, Adler NE, Nguyen R, Lopez A, et al. Social disparities in internet patient portal use in diabetes: evidence that the digital divide extends beyond access. J Am Med Inform Assoc. 2011;18:318–21.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Gude WT, Simon AC, Peute LW, Holleman F, Hoekstra JB, Peek N, et al. Formative usability evaluation of a web-based insulin self-titration system: preliminary results. Stud Health Technol Inf. 2012;180:1209–11.

    Google Scholar 

  38. Jennings A, Powell J, Armstrong N, Sturt J, Dale J. A virtual clinic for diabetes self-management: pilot study. J Med Internet Res. 2009;11:e10.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Nielsen J, Mack RL, editors. Usability inspection methods. New York: Wiley; 1994.

    Google Scholar 

  40. Zayas-Caban T, Marquard JL, Radhakrishnan K, Duffey N, Evernden DL. Scenario-based user testing to guide consumer health informatics design. AMIA Annu Symp Proc. 2009;2009:719–23.

    PubMed  PubMed Central  Google Scholar 

  41. Arsand E, Tatara N, Ostengen G, Hartvigsen G. Mobile phone-based self-management tools for type 2 diabetes: the few touch application. J Diabetes Sci Technol. 2010;4:328–36.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res. 2012;14:e70.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Carroll AE, DiMeglio LA, Stein S, Marrero DG. Using a cell phone-based glucose monitoring system for adolescent diabetes management. Diabetes Educ. 2011;37:59–66.

    Article  PubMed  Google Scholar 

  44. Chomutare T, Tatara N, Arsand E, Hartvigsen G. Designing a diabetes mobile application with social network support. Stud Health Technol Inf. 2013;188:58–64.

    Google Scholar 

  45. DeShazo J, Harris L, Turner A, Pratt W. Designing and remotely testing mobile diabetes video games. J Telemed Telecare. 2010;16:378–82.

    Article  PubMed  Google Scholar 

  46. Garcia-Saez G, Hernando ME, Martinez-Sarriegui I, Rigla M, Torralba V, Brugues E, et al. Architecture of a wireless personal assistant for telemedical diabetes care. Int J Med Inform. 2009;78:391–403.

    Article  PubMed  Google Scholar 

  47. Mulvaney SA, Anders S, Smith AK, Pittel EJ, Johnson KB. A pilot test of a tailored mobile and web-based diabetes messaging system for adolescents. J Telemed Telecare. 2012;18:115–8.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Osborn CY, Mulvaney SA. Development and feasibility of a text messaging and interactive voice response intervention for low-income, diverse adults with type 2 diabetes mellitus. J Diabetes Sci Technol. 2013;7:612–22.

    Article  PubMed  PubMed Central  Google Scholar 

  49. Padman R, Jaladi S, Kim S, Kumar S, Orbeta P, Rudolph K, et al. An evaluation framework and a pilot study of a mobile platform for diabetes self-management: insights from pediatric users. Stud Health Technol Inf. 2013;192:333–7.

    Google Scholar 

  50. Tani S, Marukami T, Matsuda A, Shindo A, Takemoto K, Inada H. Development of a health management support system for patients with diabetes mellitus at home. J Med Syst. 2010;34:223–8.

    Article  PubMed  Google Scholar 

  51. Tatara N, Arsand E, Bratteteig T, Hartvigsen G. Usage and perceptions of a mobile self-management application for people with type 2 diabetes: qualitative study of a five-month trial. Stud Health Technol Inf. 2013;192:127–31.

    Google Scholar 

  52. Vuong AM, Huber Jr JC, Bolin JN, Ory MG, Moudouni DM, Helduser J, et al. Factors affecting acceptability and usability of technological approaches to diabetes self-management: a case study. Diabetes Technol Ther. 2012;14:1178–82.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

Dr. Lyles is supported by a Career Development Award from the Agency for Healthcare Research and Quality (K99 HS022408). Dr. Sarkar is also supported by the Agency for Healthcare Research and Quality (R21 HS021322). Dr. Osborn is supported by a Career Development Award from the National Institute of Diabetes and Digestive and Kidney Diseases (K01 DK087894). The contents of this manuscript are solely the responsibility of the authors and do not necessarily represent these funding entities. We thank Lina Tieu, MPH, for her assistance in reviewing and finalizing the manuscript.

Compliance with Ethics Guidelines

ᅟ

Conflict of Interest

Courtney R. Lyles, Urmimala Sarkar, and Chandra Y. Osborn declare that they have no conflict of interest.

Human and Animal Rights and Informed Consent

This article does not contain any studies with human or animal subjects performed by any of the authors.

Author information

Authors and Affiliations

  1. UCSF Center for Vulnerable Populations at San Francisco General Hospital, San Francisco, CA, USA

    Courtney R. Lyles & Urmimala Sarkar

  2. Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA

    Chandra Y. Osborn

  3. Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA

    Chandra Y. Osborn

  4. Diabetes Research & Training Center, Center for Diabetes Translational Research, Vanderbilt University Medical Center, Nashville, TN, USA

    Chandra Y. Osborn

  5. Division of General Internal Medicine at San Francisco General Hospital, University of California San Francisco, 1001 Potrero Ave., SFGH 10, San Francisco, CA, 94143, USA

    Courtney R. Lyles & Urmimala Sarkar

  6. Division of General Internal Medicine & Public Health, Vanderbilt University Medical Center, 1215 Twenty-First Ave South, Ste 6000, MCE - North Tower, Nashville, TN, 37232-8300, USA

    Chandra Y. Osborn

Authors
  1. Courtney R. Lyles
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Urmimala Sarkar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chandra Y. Osborn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Courtney R. Lyles.

Additional information

This article is part of the Topical Collection on Psychosocial Aspects

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lyles, C.R., Sarkar, U. & Osborn, C.Y. Getting a Technology-Based Diabetes Intervention Ready for Prime Time: a Review of Usability Testing Studies. Curr Diab Rep 14, 534 (2014). https://doi.org/10.1007/s11892-014-0534-9

Download citation

  • Published:

  • DOI: https://doi.org/10.1007/s11892-014-0534-9

Keywords

Search

Navigation