Skip to main content
SpringerLink
Log in
Book cover

Cognitive Informatics for Biomedicine pp 81–109Cite as

Evaluation of Health Information Technology: Methods, Frameworks and Challenges

  • Chapter

Part of the book series: Health Informatics ((HI))

Abstract

The widespread adoption and use of Health Information Technology (HIT) in clinical environments has broadly corresponded with several reports of workflow and usability concerns and, in some cases, resulting in sentinel and adverse events. Recent reports from the Institute of Medicine (IOM) and other federal agencies have highlighted the crucial role of rigorous evaluation methods for the design and development of safer HIT. In this chapter, we describe various methods for evaluating HIT: the first section focuses on the usability methods related to the intrinsic aspects of human-computer interaction (HCI), both from an analytic and user-centered perspective; the second section focuses on the usability methods related to the external and situated aspects of human-computer interaction in complex clinical settings, both from a descriptive and contextual perspective of HIT use. Each of the evaluation methods is described with examples from recent empirical studies published in the biomedical informatics research. Finally, we conclude the chapter by highlighting the challenges of conducting HIT evaluation studies, and potential mechanisms for mitigating some of these methodological challenges.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   84.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    While GOMS (See Sect. 5.2.1.3) is considered a task-analytic approach, we have categorized it as a model-based approach for predictions of task completion times. It is based on a task analytic decomposition of tasks.

Additional Readings

  • Johnson, C. M., Johnston, D., & Crowle, P. K. (2011). EHR usability toolkit: A background report on usability and electronic health records. Rockville: Agency for Healthcare Research and Quality.

    Google Scholar 

  • Kushniruk, A. W., & Patel, V. L. (2004). Cognitive and usability engineering methods for the evaluation of clinical information systems. Journal of Biomedical Informatics, 37(1), 56–76.

    Article  PubMed  Google Scholar 

  • Preece, J., et al. (1994). Human-computer interaction. Addison-Wesley Longman Ltd.

    Google Scholar 

  • Shortliffe, E. H., & Patel, V. L. (2011). Generation and formulation of knowledge: Human-intensive techniques. In R. A. Greenes (Ed.), Clinical decision support: The road ahead. Academic.

    Google Scholar 

References

  • Aarts, J., Ash, J., & Berg, M. (2007). Extending the understanding of computerized physician order entry: Implications for professional collaboration, workflow and quality of care. International Journal of Medical Informatics, 76(Suppl 1), S4–S13.

    Article  PubMed  Google Scholar 

  • Abraham, J. (2010). Meta-coordination activities: Exploring articulation work in hospitals, in information sciences and technology (IST). Doctoral dissertation, The Pennsylvania State University.

    Google Scholar 

  • Abraham, J., & Kannampallil, T. G. (2014). Quantifying physician activities in emergency care: An exploratory study. In Proceedings of the Human Factors and Ergonomics Society (HFES). Chicago: Sage.

    Google Scholar 

  • Abraham, J., & Reddy, M. C. (2008). Moving patients around: A field study of coordination between clinical and non-clinical staff in hospitals. In ACM conference on computer supported cooperative work (CSCW). ACM.

    Google Scholar 

  • Abraham, J., Kannampallil, T. G., & Reddy, M. (2009). Peripheral activities during EMR use in emergency care: A case study. In Proceedings of the American Medical Informatics Association (AMIA) annual symposium 2009. San Francisco.

    Google Scholar 

  • Abraham, J., Kannampallil, T., Patel, B., Almoosa, K., & Patel, V. L. (2012). Ensuring patient safety in care transitions: An empirical evaluation of a handoff intervention tool. In Proceedings of the AMIA annual symposium. Chicago.

    Google Scholar 

  • Allen, M., Currie, L. M., Bakken, S., Patel, V. L., & Cimino, J. J. (2006). Heuristic evaluation of paper-based web pages: A simplified inspection usability methodology. Journal of Biomedical Informatics, 39(4), 412–423.

    Article  PubMed  Google Scholar 

  • Annett, J., & Duncan, K. D. (1967). Task analysis and training design. Occupational Psychology, 41, 211–221.

    Google Scholar 

  • Ash, J. S., Stavri, P. Z., & Kuperman, G. J. (2003). A consensus statement on considerations for a successful CPOE implementation. Journal of the American Medical Informatics Association, 10(3), 229–234.

    Article  PubMed  PubMed Central  Google Scholar 

  • Ash, J. S., Berg, M., & Coiera, E. (2004). Some unintended consequences of information technology in health care: The nature of patient care information system-related errors. Journal of the American Medical Informatics Association, 11(2), 104–112.

    Article  PubMed  PubMed Central  Google Scholar 

  • Blumenthal, D. (2009). Stimulating the adoption of health information technology. New England Journal of Medicine, 360(15), 1477–1479.

    Article  CAS  PubMed  Google Scholar 

  • Brixey, J. J., Robinson, D. J., Tang, Z., Johnson, T. R., Zhang, J., & Turley, J. P. (2005). Interruptions in workflow for RNs in a level one trauma center. AMIA Annual Symposium Proceedings, 2005, 86.

    PubMed  PubMed Central  Google Scholar 

  • Brooke, J. (1996). SUS: A ‘quick and dirty’ usabiliy scale. In P. W. Jordan, B. Thomas, & I. L. McClell (Eds.), Usability evaluation in industry (pp. 189–195). London: Taylor & Francis.

    Google Scholar 

  • Burrows, D., & Kendall, S. (1997). Focus groups: What are they and how can they be used in nursing and health care research? Social Sciences in Health, 3, 244–253.

    Google Scholar 

  • Card, S. K., Moran, T. P., & Newell, A. L. (1980). The keystroke-level model for user performance time with interactive systems. Communications of ACM, 23(7), 396–410.

    Article  Google Scholar 

  • Card, S. K., Newell, A., & Moran, T. P. (1983). The psychology of human-computer interaction (p. 469). Hillsdale: Lawrence Erlbaum Associates.

    Google Scholar 

  • Caulton, D. A. (2001). Relaxing the homogeneity assumption in usability testing. Behaviour & Information Technology, 20, 1–7.

    Article  Google Scholar 

  • Chipman, S. F., Schraagen, J. M., & Shalin, V. L. (2000). Introduction to cognitive task analysis. In J. M. Schraagen, S. F. Chipman, & V. J. Shute (Eds.), Cognitive task analysis (pp. 3–23). Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Choi, J., & Bakken, S. (2010). Web-based education for low-literate parents in Neonatal Intensive Care Unit: Development of a website and heuristic evaluation and usability testing. International Journal of Medical Informatics, 79(8), 565–575.

    Article  PubMed  PubMed Central  Google Scholar 

  • Chung, P., Zhang, J., Johnson, T., et al. (2003). An extended hierarchical task analysis for error prediction in medical devices. In Proceedings of the annual symposium of the American Medical Informatics Association 2003.

    Google Scholar 

  • Cooke, N. J. (1994). Varieties of knowledge elicitation techniques. International Journal of Human-Computer Studies, 41, 801–849.

    Article  Google Scholar 

  • Dalai, V. V., Khalid, S., Gottipati, D., Kannampallil, T. G., John, V., Blatter, B., Patel, V. L., & Cohen, T. I. P. (in press). Evaluating the effects of cognitive support on psychiatric clinical comprehension. Aritificial Intelligence in Medicine, 62(2), 91–104.

    Google Scholar 

  • Donabedian, A. (1966). Evaluating the quality of medical care. Milbank Memorial Fund Quarterly, 44(2), 166–206.

    Article  Google Scholar 

  • Ericsson, K. A., & Simon, H. (1980). Verbal reports as data. Psychological Review, 87(3), 215–250.

    Article  Google Scholar 

  • Ericsson, K. A., & Simon, H. (1984). Protocol analysis: Verbal reports as data. Cambridge: MIT Press.

    Google Scholar 

  • Farr, R. M. (1982). Interviewing: The social psychology of the inter-view. In F. Fransella (Ed.), Psychology for occupational therapists (pp. 151–170). London: Macmillan.

    Chapter  Google Scholar 

  • Fetterman, D. M. (1998). Ethnography: Step by step (2nd ed.). Thousand Oaks: Sage.

    Google Scholar 

  • Fisher, A., & Fonteyn, M. E. (1995). An exploration of an innovative methodological approach for examining nurses’ heuristic use in clinical practice. Scholarly Inquiry for Nursing Practice, 9(3), 263–276.

    CAS  PubMed  Google Scholar 

  • Fitts, P. M. (1954). The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47, 381–391.

    Article  CAS  PubMed  Google Scholar 

  • Fonteyn, M. E., & Grobe, S. J. (1994). Expert system development in nursing: Implications for critical care nursing practice. Heart & Lung: The Journal of Critical Care, 23(1), 80–87.

    CAS  Google Scholar 

  • Fonteyn, M., Kuipers, B., & Grobe, S. (1993). A description of think aloud method and protocol analysis. Qualitative Health Research, 3, 430–441.

    Article  Google Scholar 

  • Forsythe, D. E. (1999). “It’s just a matter of common sense”: Ethnography as invisible work. Computer Supported Cooperative Work (CSCW), 8(1–2), 127–145.

    Article  Google Scholar 

  • Fowler, L. P. (1997). Clinical reasoning strategies used during care planning. Clinical Nursing Research, 6(4), 349–361.

    Article  CAS  PubMed  Google Scholar 

  • Friedman, C. P., & Wyatt, J. C. (2006). Evaluation methods in biomedical informatics. New York: Springer.

    Book  Google Scholar 

  • Funkesson, K. H., Anbäcken, E.-M., & Ek, A.-C. (2007). Nurses’ reasoning process during care planning taking pressure ulcer prevention as an example. A think-aloud study. International Journal of Nursing Studies, 44, 1109–1119.

    Article  PubMed  Google Scholar 

  • Gibbs, A. (1997). Focus groups. Social Research Update, 19, 1–7.

    Google Scholar 

  • Grobe, S. J., Drew, J. A., & Fonteyn, M. E. (1991). A descriptive analysis of experienced nurses’ clinical reasoning during a planning task. Research in Nursing & Health, 14(4), 305–314.

    Article  CAS  Google Scholar 

  • Hall, M., De Jong, M., & Steehouder, M. (2004). Cultural differences and usability evaluation: Individualistic and collectivistic participants compared. Technical Communication, 51(4), 489–503.

    Google Scholar 

  • Hart, S., & Staveland, L. (1988). Development of NASA TLX (Task Load Index): Results of empirical and theoretical research. In P. Hancock & N. Meshkati (Eds.), Human mental workload. Amsterdam: North Holland Press.

    Google Scholar 

  • Hermanns, H. (1991). Narratives interview. In U. Flick (Ed.), Handbuch qualitative socialforschung (pp. 182–185). Muenchen: Psychologie Verlags Union.

    Google Scholar 

  • Hewing, N. J., Kaufman, D. R., Chan, R. P., & Chiang, M. F. (2013). Plus disease in retinopathy of prematurity: Qualitative analysis of diagnostic process by experts. JAMA Ophthalmology, 131(8), 1026–1032.

    Article  PubMed  PubMed Central  Google Scholar 

  • Hick, W. E. (1951). A simple stimulus generator. Quarterly Journal of Experimental Psychology, 3, 94–95.

    Article  Google Scholar 

  • Holzinger, A., Kosec, P., Schwantzer, G., Debevc, M., Hofmann-Wellenhof, R., & Frühauf, J. (2011). Design and development of a mobile computer application to reengineer workflows in the hospital and the methodology to evaluate its effectiveness. Journal of Biomedical Informatics, 44(6), 968–977.

    Article  PubMed  Google Scholar 

  • Horsky, J., Gutnik, L., & Patel, V. L. (2006). Technology for emergency care: Cognitive and workflow considerations. In American medical informatics association symposium proceedings. Washington, DC.

    Google Scholar 

  • Hyman, R. (1953). Stimulus information as a determinant of reaction time. Journal of Experimental Psychology, 45, 188–196.

    Article  CAS  PubMed  Google Scholar 

  • Hysong, S. J., Sawhney, M. K., Wilson, L., Sittig, D. F., Espadas, D., Davis, T., & Singh, H. (2010). Provider management strategies of abnormal test result alerts: A cognitive task analysis. Journal of the American Medical Informatics Association, 17(1), 71–77.

    Article  PubMed  PubMed Central  Google Scholar 

  • IOM. (2011). Health IT and patient safety: Building safer systems for better care. Washington, DC: Institute of Medicine.

    Google Scholar 

  • Johnson, C. M., Johnson, T. R., & Zhang, J. (2005). A user-centered framework for redesigning health care interfaces. Journal of Biomedical Informatics, 38, 75–87.

    Article  PubMed  Google Scholar 

  • Kahn, M. J., & Prail, A. (1994). Formal usability inspections. In J. Nielson and R. L. Mack (Eds.) Usability inspection methods (pp. 141–171). New York: Wiley.

    Google Scholar 

  • Kannampallil, T. G., Schauer, G. F., Cohen, T., & Patel, V. L. (2011). Considering complexity in healthcare systems. Journal of Biomedical Informatics, 44(6), 943–947.

    Article  PubMed  Google Scholar 

  • Kannampallil, T. G., Franklin, A., Mishra, R., Cohen, T., Almoosa, K. F., & Patel, V. L. (2013). Understanding the nature of information seeking behavior in critical care: Implications for the design of health information technology. Artificial Intelligence in Medicine, 57(1), 21–29.

    Article  PubMed  Google Scholar 

  • Kannampallil, T. G., Jones, L. K., Patel, V. L., Buchman, T. G., & Franklin, A. (2014). Comparing the information seeking strategies of residents, nurse practitioners, and physician assistants in critical care settings. Journal of the American Medical Informatics Association, 21, e249–e256.

    Article  PubMed  PubMed Central  Google Scholar 

  • Karahoca, A., Bayraktar, E., Tatoglu, E., & Karahoca, D. (2010). Information system design for a hospital emergency department: A usability analysis of software prototypes. Journal of Biomedical Informatics, 43(2), 224–232.

    Article  PubMed  Google Scholar 

  • Kaufman, D. R., Patel, V. L., Hillman, C., Morin, P. C., Pevzner, J., Weinstock, R. S., Goland, R., Shea, S., & Starren, J. (2003). Usability in the real world: Assessing medical information technologies in patients’ homes. Journal of Biomedical Informatics, 36(1), 45–60.

    Article  PubMed  Google Scholar 

  • Kitzinger, J. (1995). Introducing focus groups. British Medical Journal, 311, 299–302.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Koppel, R., Metlay, J. P., Cohen, A., Abaluck, B., Localio, A. R., Kimmel, S. E., et al. (2005). Role of computerized physician order entry systems in facilitating medication errors. JAMA, 293(10), 1197–1203.

    Article  CAS  PubMed  Google Scholar 

  • Krueger, R. A. (2009). Focus groups: A practical guide for applied research. Thousand Oaks: Sage.

    Google Scholar 

  • Kurosu, M. (Ed.). (2014). Human-computer interaction. Theories, methods, and tools (pp. 469–480). Heidelberg: Springer.

    Google Scholar 

  • Kushniruk, A. W., Kaufman, D. R., Patel, V. L., Levesque, Y., & Lottin, P. (1996). Assessment of a computerized patient record system: A cognitive approach to evaluating medical technology. MD Computing, 13, 406–415.

    CAS  PubMed  Google Scholar 

  • Kushniruk, A. W., Borycki, E. M., Kuwata, S., & Watanabe, H. (2008). Using a low-cost simulation for assessing the impact of a medication administration system on workflow. In Studies in health technology and informatics (pp. 567–572).

    Google Scholar 

  • Lewis, J. R. (1991). Psychometric evaluation of an after-scenario questionnaire for computer usability studies: The ASQ. ACM SIGCHI Bulletin, 23(1), 78–81.

    Article  Google Scholar 

  • Lewis, A. (1992). Group child interviews as a research tool. British Educational Research Journal, 18, 413–421.

    Article  Google Scholar 

  • Lewis, J. R. (1994). Sample sizes for usability studies: Additional considerations. Human Factors, 36, 369–378.

    Article  Google Scholar 

  • Linder, J. A., Ma, J., Bates, D. W., Middleton, B., & Stafford, R. S. (2007). Electronic health record use and the quality of ambulatory care in the United States. Archives of Internal Medicine, 167(13), 1400–1405.

    Article  PubMed  Google Scholar 

  • Lundgrén-Laine, H., & Salanterä, S. (2010). Think-aloud technique and protocol analysis in clinical decision-making research. Qualitative Health Research, 20(4), 565–575.

    Article  PubMed  Google Scholar 

  • MacKenzie, I. S., Zhang, S. X., & Soukoreff, R. W. (1999). Text entry using soft keyboards. Behaviour and Information Technology, 18, 235–244.

    Article  Google Scholar 

  • Malhotra, S., Jordan, D., Shortliffe, E., & Patel, V. L. (2007). Workflow modeling in critical care: Piecing together your own puzzle. Journal of Biomedical Informatics, 40(2), 81–92.

    Article  PubMed  Google Scholar 

  • Mamykina, L., Vawdrey, D., Stetson, P., Zheng, K., & Hripcsak, G. (2012). Clinical documentation: Composition or synthesis? Journal of the American Medical Informatics Association, 19(6), 1025–1103.

    Article  PubMed  PubMed Central  Google Scholar 

  • Mason, J. (2002). Qualitative researching. London: Sage.

    Google Scholar 

  • McDonald, C. J., Callaghan, F. M., Weissman, A., Goodwin, R. M., Mundkur, M., & Kuhn, T. (2014). Use of internist’s free time by ambulatory care electronic medical record systems. JAMA Internal Medicine, 174(11), 1860–1863.

    Article  PubMed  Google Scholar 

  • Miller, R. H., & Sim, I. (2004). Physicians’ use of electronic medical records: Barriers and solutions. Health Affairs (Millwood), 23(2), 116–126.

    Article  Google Scholar 

  • Nielsen, J. (1994). Usability inspection methods. In Conference companion on human factors in computing systems (pp. 413–414). Boston: ACM.

    Chapter  Google Scholar 

  • Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 249–256). Seattle/Washington, DC: ACM.

    Google Scholar 

  • Olson, G. M., & Olson, J. S. (2003). Human-computer interaction: Psychological aspects of the human use of computing. Annual Review of Psychology, 54(1), 491–516.

    Article  PubMed  Google Scholar 

  • Overhage, J., Perkins, S., Tierney, W., & McDonald, C. (2001). Controlled trial of direct physician order entry: Effects on physicians’ time utilization in ambulatory primary care internal medicine practices. Journal of the American Medical Informatics Association, 8(4), 361–371.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Patel, V. L., & Groen, G. J. (1991a). The general and specific nature of medical expertise: A critical look. In K. A. E. J. Smith (Ed.), Toward a general theory of expertise: Prospects and limits (pp. 93–125). New York: Cambridge University Press.

    Google Scholar 

  • Patel, V. L., & Groen, G. J. (1991b). Developmental accounts of the transition from medical student to doctor: Some problems and suggestions. Medical Education, 25(6), 527–535.

    Article  CAS  PubMed  Google Scholar 

  • Patel, V. L., & Kannampallil, T. G. (2011). Cognitive approaches to clinical data management for decision support: Is it old wine in new bottle? In A. Holzinger & K.-M. Simonic (Eds.), Information quality in e-health (pp. 1–13). Berlin/Heidelberg: Springer.

    Google Scholar 

  • Patel, V. L., Arocha, J. F., & Kaufman, D. R. (1994). Diagnostic reasoning and medical expertise. In L. M. Douglas (Ed.), Psychology of learning and motivation (pp. 187–252). San Diego: Academic.

    Google Scholar 

  • Patel, V. L., Arocha, J. F., & Kaufman, D. R. (2001). A primer on aspects of cognition for medical informatics. Journal of the American Medical Informatics Association, 8(4), 324–343.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Patel, V. L., Kaufman, D. R., & Kannampallil, T. (2013). Diagnostic reasoning and decision making in the context of health information technology. Reviews of Human Factors and Ergonomics, 8, 149–190.

    Article  CAS  Google Scholar 

  • Patel, V. L., Kaufman, D. R., & Cohen, T. (2014). Cognitive informatics in health and biomedicine. London: Springer.

    Book  Google Scholar 

  • Patterson, E. S., Nguyen, A. D., Halloran, J. P., & Asch, S. M. (2004). Human factors barriers to the effective use of ten HIV clinical reminders. Journal of the American Medical Informatics Association, 11(1), 50–59.

    Article  PubMed  PubMed Central  Google Scholar 

  • Paul, S. A., Reddy, M., Abraham, J., & DeFlitch, C. (2008). The usefulness of information and communication technologies in crisis response. In AMIA annual symposium proceedings. American Medical Informatics Association.

    Google Scholar 

  • Pizziferri, L., Kittler, A., Volk, L., Honour, M., Gupta, S., & Wang, S. (2005). Primary care physician time utilization before and after implementation of an electronic health record: A time-motion study. Journal of Biomedical Informatics, 38(3), 176–188.

    Article  PubMed  Google Scholar 

  • Polson, P. G., Lewis, C., Rieman, J., & Wharton, C. (1992). Cognitive walkthroughs: A method for theory-based evaluation of user interfaces. International Journal of Man-Machine Studies, 36(5), 741–773.

    Article  Google Scholar 

  • Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., & Carey, T. (1994). Human-computer interaction. Essex: Addison-Wesley Longman Ltd.

    Google Scholar 

  • Reddy, M. C., Paul, S. A., Abraham, J., McNeese, M., DeFlitch, C., & Yen, J. (2009). Challenges to effective crisis management: Using information and communication technologies to coordinate emergency medical services and emergency department teams. International Journal of Medical Informatics, 78(4), 259–269.

    Article  PubMed  Google Scholar 

  • Rosson, M. B., & Carroll, J. M. (2009). Usability engineering: Scenario-based development of human-computer interaction. Morgan Kaufman: Redwood City, CA.

    Google Scholar 

  • Saitwal, H., Feng, X., Walji, M., et al. (2010). Assessing performance of an electronic health record (EHR) using cognitive task analysis. International Journal of Medical Informatics, 79(7), 501–506.

    Article  PubMed  Google Scholar 

  • Schatzberg, M. (2008). Seeing the invisible, hearing silence, thinking the unthinkable: The advantages of ethnographic immersion. In APSA 2008 annual meeting. Boston: Hynes Convention Center.

    Google Scholar 

  • Simmons, B., Lanuza, D., Fonteyn, M., Hicks, F., & Holm, K. (2003). Clinical reasoning in experienced nurses. Western Journal of Nursing Research, 25, 720–724.

    Article  Google Scholar 

  • Sittig, D. F., & Classen, D. C. (2010). Safe electronic health record use requires a comprehensive monitoring and evaluation framework. JAMA, 303(5), 450–451.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  • Stetler, C. B., Legro, M. W., Wallace, C. M., Bowman, C., Guihan, M., Hagedorn, H., …, Smith, J. L. (2006). The role of formative evaluation in implementation research and the QUERI experience. Journal of General Internal Medicine, 21(S2), S1–S8.

    Google Scholar 

  • Unertl, K. M., Weinger, M. B., Johnson, K. B., & Lorenzi, N. (2009). Describing and modeling workflow and information flow in chronic disease care. Journal of the American Medical Informatics Association, 16(6), 826–836.

    Article  PubMed  PubMed Central  Google Scholar 

  • Unertl, K., Johnson, K., Gadd, C., & Lorenzi, N. (2013). Bridging organizational divides in health care: An ecological view of health information exchange. JMIR Medical Informatics, 1(2), e3.

    Article  PubMed  PubMed Central  Google Scholar 

  • Vaughn, S., Schumm, J. S., & Sinagub, J. (1996). Focus group interviews in education and psychology. Thousand Oaks: Sage.

    Book  Google Scholar 

  • Vicente, K. J. (1999). Cognitive work analysis. Mahwah: Lawrence Erlbaum Associates.

    Book  Google Scholar 

  • Zhang, J., & Walji, M. F. (2011). TURF: Toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44(6), 1056–1067.

    Article  PubMed  Google Scholar 

  • Zhang, J., Johnson, T. R., Patel, V. L., Paige, D. L., & Kubose, T. (2003). Using usability heuristics to evaluate patient safety of medical devices. Journal of Biomedical Informatics, 36(1), 23–30.

    Article  PubMed  Google Scholar 

  • Zheng, K., Haftel, H. M., Hirschl, R. B., O’Reilly, M., & Hanauer, D. A. (2010). Quantifying the impact of health IT implementations on clinical workflow: A new methodological perspective. Journal of the American Medical Informatics Association, 17(4), 454–461.

    Article  PubMed  PubMed Central  Google Scholar 

  • Zheng, K., Guo, M. H., & Hanauer, D. A. (2011). Using the time and motion method to study clinical work processes and workflow: Methodological inconsistencies and a call for standardized research. Journal of the American Medical Informatics Association, 18(5), 704–710.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Illinois at Chicago, Chicago, IL, USA

    Thomas G. Kannampallil

  2. The New York Academy of Medicine, New York, NY, USA

    Thomas G. Kannampallil

  3. Department of Biomedical and Health Information Sciences, College of Applied Health Sciences, University of Illinois at Chicago, Chicago, IL, USA

    Joanna Abraham

Authors
  1. Thomas G. Kannampallil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Joanna Abraham
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas G. Kannampallil .

Editor information

Editors and Affiliations

  1. Center for Cognitive Studies in Medicine and Public Health, The New York Academy of Medicine, New York, NY, USA

    Vimla L. Patel

  2. Columbia University, Weill Cornell College of Medicine, New York, NY, USA

    Vimla L. Patel

  3. Arizona State University, Scottsdale, AZ, USA

    Vimla L. Patel  & David R. Kaufman  & 

  4. University of Illinois at Chicago, Chicago, IL, USA

    Thomas G. Kannampallil

  5. The New York Academy of Medicine, New York, NY, USA

    Thomas G. Kannampallil

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kannampallil, T.G., Abraham, J. (2015). Evaluation of Health Information Technology: Methods, Frameworks and Challenges. In: Patel, V.L., Kannampallil, T.G., Kaufman, D.R. (eds) Cognitive Informatics for Biomedicine. Health Informatics. Springer, Cham. https://doi.org/10.1007/978-3-319-17272-9_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-17272-9_5

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-17271-2

  • Online ISBN: 978-3-319-17272-9

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation