Skip to main content
SpringerLink
Log in

Iterative heuristic design of temporal graphic displays with clinical domain experts

  • Original Research
  • Published:
Journal of Clinical Monitoring and Computing Aims and scope Submit manuscript

Abstract

Conventional electronic health record information displays are not optimized for efficient information processing. Graphical displays that integrate patient information can improve information processing, especially in data-rich environments such as critical care. We propose an adaptable and reusable approach to patient information display with modular graphical components (widgets). We had two study objectives. First, reduce numerous widget prototype alternatives to preferred designs. Second, derive widget design feature recommendations. Using iterative human-centered design methods, we interviewed experts to hone design features of widgets displaying frequently measured data elements, e.g., heart rate, for acute care patient monitoring and real-time clinical decision-making. Participant responses to design queries were coded to calculate feature-set agreement, average prototype score, and prototype agreement. Two iterative interview cycles covering 64 design queries and 86 prototypes were needed to reach consensus on six feature sets. Interviewers agreed that line graphs with a smoothed or averaged trendline, 24-h timeframe, and gradient coloring for urgency were useful and informative features. Moreover, users agreed that widgets should include key functions: (1) adjustable reference ranges, (2) expandable timeframes, and (3) access to details on demand. Participants stated graphical widgets would be used to identify correlating patterns and compare abnormal measures across related data elements at a specific time. Combining theoretical principles and validated design methods was an effective and reproducible approach to designing widgets for healthcare displays. The findings suggest our widget design features and recommendations match critical care clinician expectations for graphical information display of continuous and frequently updated patient data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Roman LC, Ancker JS, Johnson SB, Senathirajah Y. Navigation in the electronic health record: a review of the safety and usability literature. J Biomed Inform. 2017;67:69–79. https://doi.org/10.1016/j.jbi.2017.01.005.

    Article  PubMed  Google Scholar 

  2. Zahabi M, Kaber DB, Swangnetr M. Usability and safety in electronic medical records interface design: a review of recent literature and guideline formulation. Hum Factors. 2015;57:805–34. https://doi.org/10.1177/0018720815576827.

    Article  PubMed  Google Scholar 

  3. Howe JL, Adams KT, Hettinger AZ, Ratwani RM. Electronic health record usability issues and potential contribution to patient harm. JAMA. 2018;319:1276–8. https://doi.org/10.1001/jama.2018.1171.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Tai DS, Hong J, Busuttil RW, Lipshutz GS. Information overload and missed test results in electronic health record-based settings. JAMA Intern Med. 2013;173(8):702–4.

    Article  Google Scholar 

  5. Shanafelt TD, Dyrbye LN, West CP. Addressing physician burnout: the way forward. JAMA. 2017;55902:4–5. https://doi.org/10.1001/JAMA.2017.0076.

    Article  Google Scholar 

  6. Reese T, Segall N, Nesbitt P, Del Fiol G, Waller R, Macpherson BC, Tonna JE, Wright MC. Patient information organization in the intensive care setting: expert knowledge elicitation with card sorting methods. J Am Med Inform Assoc. 2018;25:1026–35. https://doi.org/10.1093/jamia/ocy045.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Hegarty M. The cognitive science of visual-spatial displays: implications for design. Top Cogn Sci. 2011;3:446–74. https://doi.org/10.1111/j.1756-8765.2011.01150.x.

    Article  PubMed  Google Scholar 

  8. Varpula M, Tallgren M, Saukkonen K, Voipio-Pulkki LM, Pettilä V. Hemodynamic variables related to outcome in septic shock. Intensive Care Med. 2005;31:1066–71. https://doi.org/10.1007/s00134-005-2688-z.

    Article  PubMed  Google Scholar 

  9. Agutter J, Drews F, Syroid N, Westneskow D, Albert R, Strayer D, Bermudez J, Weinger MB. Evaluation of graphic cardiovascular display in a high-fidelity simulator. Anesth Analg. 2003;97:1403–13. https://doi.org/10.1213/01.ANE.0000085298.03143.CD.

    Article  PubMed  Google Scholar 

  10. Donnino M, Nguyen B, Jacobsen G, Tomlanovich M, Rivers E. Cryptic septic shock: a sub-analysis of early, goal-directed therapy. Chest. 2003. https://doi.org/10.1378/chest.124.4.

    Article  Google Scholar 

  11. Blow O, Magliore L, Claridge JA, Butler K, Young JS. The golden hour and the silver day: detection and correction of occult hypoperfusion within 24 hours improves outcome from major trauma. J Trauma Inj Infect Crit Care. 1999;47:964. https://doi.org/10.1097/00005373-199911000-00028.

    Article  CAS  Google Scholar 

  12. Ratwani RM. Electronic health records and improved patient care: opportunities for applied psychology. Curr Dir Psychol Sci. 2017;26:359–65. https://doi.org/10.1177/0963721417700691.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Waller RG, Wright MC, Segall N, Nesbitt P, Reese T, Borbolla D, Del Fiol G. Novel displays of patient information in critical care settings: a systematic review. J Am Med Inform Assoc. 2019;26:479–89. https://doi.org/10.1093/jamia/ocy193.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Wright MC, Borbolla D, Waller RG, Del Fiol G, Reese T, Nesbitt P, Segall N. Critical care information display approaches and design frameworks: a systematic review and meta-analysis. J Biomed Inform X. 2019. https://doi.org/10.1016/j.yjbinx.2019.100041.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Wachter BS, Johnson K, Albert R, Syroid N, Drews F, Westenskow D. The evaluation of a pulmonary display to detect adverse respiratory events using high resolution human simulator. J Am Med Inform Assoc. 2006;13:635–42. https://doi.org/10.1197/jamia.M2123.Introduction.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Bauer DT, Guerlain S, Brown PJ. The design and evaluation of a graphical display for laboratory data. J Am Med Inform Assoc. 2010;17:416–24. https://doi.org/10.1136/jamia.2009.000505.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Blike GT, Surgenor SD, Whalen K. A graphical object display improves anesthesiologists’ performance on a simulated diagnostic task. J Clin Monit Comput. 1999;15:37–44. https://doi.org/10.1023/A:1009914019889.

    Article  CAS  PubMed  Google Scholar 

  18. Tappan JM, Daniels J, Slavin B, Lim J, Brant R, Ansermino JM. Visual cueing with context relevant information for reducing change blindness. J Clin Monit Comput. 2009;23:223–32. https://doi.org/10.1007/s10877-009-9186-8.

    Article  PubMed  Google Scholar 

  19. Kennedy RR, Merry AF. The effect of a graphical interpretation of a statistic trend indicator (Trigg’s Tracking Variable) on the detection of simulated changes. Anaesth Intensive Care. 2011;39:881–6.

    Article  CAS  Google Scholar 

  20. Drews FA, Syroid N, Agutter J, Strayer DL, Westenskow DR. Drug delivery as control task: improving performance in a common anesthetic task. Hum Factors. 2006;48:85–94. https://doi.org/10.1518/001872006776412216.

    Article  PubMed  Google Scholar 

  21. Loeb RG, Weinger MB. Development and evaluation of a graphical anesthesia drug display. Anesthesiology. 2017;96:565–75.

    Google Scholar 

  22. Giuliano KK, Jahrsdoerfer M, Case J, Drew T, Raber G. The role of clinical decision support tools to reduce blood pressure variability in critically ill patients receiving vasopressor support. CIN Comput Inform Nurs. 2012;30:204–9. https://doi.org/10.1097/NCN.0b013e3182418c39.

    Article  PubMed  Google Scholar 

  23. Olchanski N, Dziadzko MA, Tiong IC, Daniels CE, Peters SG, O’Horo JC, Gong MN. Can a novel ICU data display positively affect patient outcomes and save lives? J Med Syst. 2017. https://doi.org/10.1007/s10916-017-0810-8.

    Article  PubMed  Google Scholar 

  24. Kirkness CJ, Burr RL, Cain KC, Newell DW, Mitchell PH. The impact of a highly visible display of cerebral perfusion pressure on outcome in individuals with cerebral aneurysms. Heart Lung. 2008;37:227–37. https://doi.org/10.1016/j.hrtlng.2007.05.015.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Kennedy RR, McKellow MA, French RA. The effect of predictive display on the control of step changes in effect site sevoflurane levels. Anaesthesia. 2010;65:826–30. https://doi.org/10.1111/j.1365-2044.2010.06410.x.

    Article  CAS  PubMed  Google Scholar 

  26. Segall N, Borbolla D, Del Fiol G, Waller R, Reese T, Nesbitt P, Wright MC (2017) Trend displays to support critical care: a systematic review. In: 2017 IEEE international conference on healthcare informatics (ICHI), Park City, UT, pp 305–313. https://doi.org/10.1109/ICHI.2017.85

  27. Tufte E. The visual display of quantitative information. Cheshire: Graphics Press; 2001.

    Google Scholar 

  28. Powsner S, Tufte E. Graphical summary of patient status. Lancet. 1994;344:386–9.

    Article  CAS  Google Scholar 

  29. Plaisant C, Mushlin R, Snyder A, Li J, Heller D, Shneiderman B (1998) LifeLines: using visualization to enhance navigation and analysis of patient records. In: Proceedings of the AMIA annual symposium. pp 76–80

  30. Aigner W, Miksch S, Müller W, Schumann H, Tominski C. Visualizing time-oriented data—a systematic view. Comput Graph. 2007;31:401–9. https://doi.org/10.1016/j.cag.2007.01.030.

    Article  Google Scholar 

  31. Friel SN, Curcio FR, Bright GW. Making sense of graphs: critical factors influencing comprehension and instructional implications. J Res Math Educ. 2001;32:124–58. https://doi.org/10.2307/749671.

    Article  Google Scholar 

  32. Shah P, Freedman EG. Bar and line graph comprehension: an interaction of top-down and bottom-up processes. Top Cogn Sci. 2009;3:560–78. https://doi.org/10.1111/j.1756-8765.2009.01066.x.

    Article  PubMed  Google Scholar 

  33. Shah P, Hoeffner J. Review of graph comprehension research: implications for instruction. Educ Psychol Rev. 2002;14:47–69. https://doi.org/10.1023/A:1013180410169.

    Article  Google Scholar 

  34. Pinker S (1990) A theory of graph comprehension. https://doi.org/10.1145/2046684.2046699

  35. Sanders EB-N. From user-centered to participatory design approaches. Des Soc Sci. 2002. https://doi.org/10.1201/9780203301302.ch1.

    Article  Google Scholar 

  36. Muller MJ (2003) Participatory design: the third space in HCI. In: Sears A, Jacko JA (eds) The human-computer interaction handbook. Lawrence Erlbaum, Mahwah, New Jersey

    Google Scholar 

  37. Spinuzzi C. The methodology of participatory design. Tech Commun. 2005;52:163–74. https://doi.org/10.1016/j.infsof.2008.09.005.

    Article  Google Scholar 

  38. Wright MC, Dunbar S, Macpherson BC, Moretti EW, Fiol GD, Bolte J, Taekman JM, Segall N. Toward designing information display to support critical care: a qualitative contextual evaluation and visioning effort. Appl Clin Inform. 2016;7:912–29. https://doi.org/10.4338/ACI-2016-03-RA-0033.

    Article  PubMed  PubMed Central  Google Scholar 

  39. ReeseT, Kawamoto K, Del Fiol G, Weir C, Tonna J, Segall N, Nesbitt P, Waller R, Borbolla D, Moretti EW, Wright MC (2017) Approaching the design of an information display to support critical care. In: 2017 IEEE international conference on healthcare informatics. pp 439–443

  40. Jones W, Grudin J, Czerwinski M, Spool J, Bellotti V. “Get real!” What’s wrong with HCI prototyping and how can we fix it? CHI. 2007;2007:1913–6.

    Google Scholar 

  41. Lim Y-K, Stolterman E, Tenenberg J. The anatomy of prototypes. ACM Trans Comput Interact. 2008;15:1–27. https://doi.org/10.1039/c5ra16144d.

    Article  Google Scholar 

  42. Wright MC, Waller R, Nesbitt P, Segall N, Borbolla D, Reese T, Del Fiol G. Display features that improve interpretation of critical care information: a systematic review. Crit Care Med, Honolulu. 2016;44(12S1):364.

    Article  Google Scholar 

  43. Tarrell A, Fruhling A, Borgo R, Forsell C, Grinstein G, Scholtz J (2014) Toward visualization-specific heuristic evaluation. In: BELIV ’14 proceedings of the fifth workshop on beyond time errors: novel evaluation methods for visualization. pp 110–117

  44. Pomerantz JR, Portillo MC. Grouping and emergent features in vision: toward a theory of basic Gestalts. J Exp Psychol Hum Percept Perform. 2011;37:1331–499. https://doi.org/10.1037/a0024330.

    Article  PubMed  Google Scholar 

  45. Wagemans J, Elder JH, Kubovy M, Palmer SE, Peterson MA, Singh M, von der Heydt R. A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychol Bull. 2012;138:1172–217. https://doi.org/10.1037/a0029333.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Wagemans J, Feldman J, Gepshtein S, Kimchi R, Pomerantz JR, Van der Helm PA, Van Leeuwen C. A century of Gestalt psychology in visual perception: II. Conceptual and theoretical foundations. Psychol Bull. 2012;138:1218–52. https://doi.org/10.1037/a0029334.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Aigner W, Miksch S, Schumann H, Tominski C. Visualization of time-oriented data. London: Springer; 2011.

    Book  Google Scholar 

  48. Shneiderman B (1996) The eyes have it: a task by data type taxonomy for information visualizations. In: Proceedings of the 1996 IEEE symposium on visual languages. pp 336–343. https://doi.org/10.1109/VL.1996.545307

  49. Araujo J, Born DG. Calculating percentage agreement correctly but writing its formula incorrectly. Behav Anal. 1985;8:207–8.

    Article  CAS  Google Scholar 

  50. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76:378–82. https://doi.org/10.1128/JCM.41.11.5325-5326.2003.

    Article  Google Scholar 

  51. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74.

    Article  CAS  Google Scholar 

  52. Sebat F, Musthafa AA, Johnson D, Kramer AA, Shoffner D, Eliason M, Henry K, Spurlock B. Effect of a rapid response system for patients in shock on time to treatment and mortality during 5 years. Crit Care Med. 2007;35:2568–75. https://doi.org/10.1097/01.CCM.0000287593.54658.89.

    Article  PubMed  Google Scholar 

  53. Rivers E, Nguyen B, Havstad S, Ressler J, Muzzin A, Knoblich B, Peterson E, Tomlanovich M. Early goal-directed therapy in the treatment of severe sepsis and septic shock. N Engl J Med. 2001;345:1368–77.

    Article  CAS  Google Scholar 

  54. Cleveland WS, McGill R. Graphical perception and graphical methods for analyzing scientific data. Science. 1985;229:828–33.

    Article  CAS  Google Scholar 

  55. Carswell CM. Choosing specifiers: an evaluation of the basic tasks model of graphical perception. Hum Factors. 1992;34:535–54. https://doi.org/10.1177/001872089203400503.

    Article  CAS  PubMed  Google Scholar 

  56. Few S. Information dashboard design. 2nd ed. Burlingame: Analytics Press; 2013.

    Google Scholar 

  57. Wickens CD, Carswell CM. The proximity compatibility principle: its psychological foundation and relevance to display design. Hum Factors. 1995;37:473–94. https://doi.org/10.1518/001872095779049408.

    Article  Google Scholar 

  58. Reese TJ, Del Fiol G, Tonna JE, et al. Impact of integrated graphical display on expert and novice diagnostic performance in critical care [published online ahead of print, 2020 Jun 17]. J Am Med Inform Assoc. 2020;ocaa086. https://doi.org/10.1093/jamia/ocaa086.

  59. Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc. 2016;23:899–908. https://doi.org/10.1093/jamia/ocv189.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Kawamoto K, Del Fiol G, Lobach DF, Jenders RA. Standards for scalable clinical decision support: need, current and emerging standards, gaps, and proposal for progress. Open Med Inform J. 2012;4:235–44. https://doi.org/10.2174/1874431101004010235.

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank and acknowledge the contributions of Brekk Macpherson, Atilio Barbeito, Jonathan Mark, and Eugene Moretti for feedback, participation, and interpretation of findings.

Funding

This work was supported by the National Library of Medicine of the National Institutes of Health Grant Numbers: R56LM011925 and T15LM007124.

Author information

Authors and Affiliations

  1. Department of Biomedical Informatics, University of Utah School of Medicine, 421 Wakara Way, Ste 140, Salt Lake City, UT, 84108-3514, USA

    Thomas J. Reese, Guilherme Del Fiol, Kensaku Kawamoto & Charlene Weir

  2. Department of Anesthesiology, Duke University School of Medicine, Durham, USA

    Noa Segall

  3. Divisions of Emergency Medicine and Cardiothoracic Surgery, University of Utah School of Medicine, Salt Lake City, USA

    Joseph E. Tonna

  4. College of Pharmacy, University of Idaho, Meridian, USA

    Melanie C. Wright

Authors
  1. Thomas J. Reese
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Noa Segall
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guilherme Del Fiol
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Joseph E. Tonna
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kensaku Kawamoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Charlene Weir
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Melanie C. Wright
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas J. Reese.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in the study involving human participants were in accordance with the ethical standards of the institutional committee (University of Utah Institutional Review Board IRB_00086050) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Verbal consent was obtained from all participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Reese, T.J., Segall, N., Del Fiol, G. et al. Iterative heuristic design of temporal graphic displays with clinical domain experts. J Clin Monit Comput 35, 1119–1131 (2021). https://doi.org/10.1007/s10877-020-00571-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10877-020-00571-2

Keywords

Search

Navigation