Skip to main content
SpringerLink
Log in

Evaluation for Public Health Informatics

  • Chapter
  • First Online:
Public Health Informatics and Information Systems

Part of the book series: Health Informatics ((HI))

Abstract

Evaluation is the application of specific criteria to determine the value or merit of the object of the study. Ensuring that public health information systems (ISs) and programs are managed wisely is essential. Evaluation answers the question of “why” a system is necessary, by collecting the data and performing the analysis needed to make determinations of efficiency and effectiveness and is a critical component to any public health informatics (PHI) project. Evaluation should occur at all stages of a PHI project. By using a combination of formative and summative evaluation, a well-designed plan provides key data to stakeholders that allow for informed decision-making about continuing, replacing, enhancing or retiring a public health IS. The design of the evaluation plan begins with identifying a mental model (e.g., information value cycle or data-information system-context-rings) from which to view the project and the evaluation objectives and determine what to evaluate. Conceptual frameworks, evaluation strategies, and methodology toolkits help define how the evaluation plan is developed and executed. A comprehensive program (e.g., the Centers for Disease Control and Prevention’s six-step evaluation framework) provides an example of an evaluation template.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Worthen BR, Sanders JR, Fitzpatrick JL. Program evaluation. 2nd ed. New York: Addison Wesley Longman; 1987.

    Google Scholar 

  2. Wagner JA. Encyclopedia of the Hundred Years War. Westport: Greenwood Publishing Group; 2006. p. 16–8.

    Google Scholar 

  3. Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR 1999;48(No. RR-11). Available at: http://www.cdc.gov/eval/framework/index.htm. Accessed June 19, 2013.

  4. Friedman CP. A “fundamental theorem” of biomedical informatics. J Am Med Inform Assoc. 2009;16(2):169–70. URL: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2649317/?tool=pubmed. Last accessed 12 Mar 2013.

  5. Scriven M. The methodology of evaluation. Perspectives of curriculum evaluation. In: AERA monograph series on curriculum evaluation, No. 1. Chicago: Rand McNally; 1967. p. 39–83.

    Google Scholar 

  6. Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H, Kimmel B, Sharp ND, Smith JL. The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med. 2006;21 Suppl 2:S1–8.

    Article  PubMed  Google Scholar 

  7. Benedict LG. Traditional research versus evaluation. U.S. Department of Education. 1975. ERIC. Web. ED100959. Available from: http://files.eric.ed.gov/fulltext/ED100959.pdf. Cited 8 Apr 2013.

  8. Unite for Sight. Module 10: Distinguishing evaluation from research. Available at: http://www.uniteforsight.org/evaluation-course/module10. Cited 1 Aug 2013.

  9. Guba E. Significant differences. Edu Res. 1969;20:4–5.

    Google Scholar 

  10. Bainbridge L. Mental models in cognitive skill: the example of industrial process operation. In: Rogers Y, Rutherford A, Bibby PA, editors. Models in the mind. San Diego: Academic; 1992. p. 119–43.

    Google Scholar 

  11. Ammenwerth E, et al. Evaluation of health information systems—problems and challenges. Int J Med Inform. 2003;71(2):125–35.

    Article  PubMed  Google Scholar 

  12. Taylor RS. Value-added processes in the information life cycle. J Am Soc Inf Sci. 1982;33:341–6.

    Article  Google Scholar 

  13. Thacker SB, Qualters JR, Lee LM. Public health surveillance in the United States: evolution and challenges. MMWR Surveill Summ. 2012;61(Suppl):3–9.

    PubMed  Google Scholar 

  14. Laventure M. State and local perspective on Public Health Data Standards: opportunities for strategic action. Presented at: Annual meeting on Public Health Data Standards Consortium, Bethesda, 16–18 March 2004. Powerpoint slides.

    Google Scholar 

  15. Avgerou C. The significance of context in information systems and organizational change. Inf Syst J. 2001;11(1):43–63.

    Article  Google Scholar 

  16. Lorenzi NM, et al. Antecedents of the people and organizational aspects of medical informatics review of the literature. J Am Med Inform Assoc. 1997;4(2):79–93.

    Article  PubMed  CAS  Google Scholar 

  17. Ramaprasad A, Rai A. Envisioning management of information. Omega. 1996;24(2):179–93.

    Article  Google Scholar 

  18. Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. J Am Med Inform Assoc. 2007;14(5):542–9.

    Article  PubMed  Google Scholar 

  19. Heeks R, Bathnagar S. Understanding success and failure in information age reform. In: Heeks R, editor. Reinventing government in the information age. London: Routledge; 1999. p. 49–74.

    Chapter  Google Scholar 

  20. One Health Initiative Web site. URL: http://www.onehealthinitiative.com/. Last accessed 1 Apr 2013.

  21. Kaufman D, et al. Applying an evaluation framework for health information system design, development, and implementation. Nurs Res. 2006;55(2):S37–42.

    Article  PubMed  Google Scholar 

  22. Stead WW, et al. Designing medical informatics research and library – resource projects to increase what is learned. J Am Med Inform Assoc. 1994;1(1):28–33.

    Article  PubMed  CAS  Google Scholar 

  23. Currie LM. Evaluation frameworks for nursing informatics. Int J Med Inform 2005;74(11–12):908–16.

    Google Scholar 

  24. Yusof MM, Parazafeiropoulou A, Paul RJ, Stergioulas LK. Investigating evaluation frameworks for health information systems. Int J Med Inform 2008;77(6):377–85.

    Google Scholar 

  25. Cronholm S, Goldkuhl G. Strategies for information systems evaluation-six generic types. Electronic Journal of Information Systems Evaluation 2003;6(2):65–74.

    Google Scholar 

  26. Friedman CP, Wyatt J. Evaluation methods in biomedical informatics. New York, NY: Springer Science+Business Media, 2006.

    Google Scholar 

  27. Westbrook JI, et al. Evaluating the impact of information communication technologies on complex organizational systems: a multi-disciplinary, multi-method framework. Stud Health Technol Inform. 2004;107(Pt 2):1323–7.

    PubMed  Google Scholar 

  28. Anderson JG. Evaluation in health informatics: social network analysis. Comput Biol Med. 2002;32(3):179.

    Google Scholar 

  29. Merrill J, et al. Description of a method to support public health information management: organizational network analysis. J Biomed Inform. 2007;40(4):422.

    Article  PubMed  Google Scholar 

  30. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3):166–206.

    Article  PubMed  Google Scholar 

  31. DeLone WH, McLean ER. Information systems success: the quest for the dependent variable. Inf Syst Res. 1992;3(1):60–95.

    Article  Google Scholar 

  32. Delone WH. The DeLone and McLean model of information systems success: a ten-year update. J Manage Inf Syst. 2003;19(4):9–30.

    Google Scholar 

  33. Fehrenbach N, Ross D, Hastings T, Renahan-White A. Towards measuring value: an evaluation framework for public health information systems. Public Health Informatics Institute. 2005. Web. Last accessed 1 Apr 2013.

    Google Scholar 

  34. Hebert M. Telehealth success: evaluation framework development. Stud Health Technol Inform. 2001;2:1145–9.

    Google Scholar 

  35. Cronholm S. Information Systems Evaluation-adding process descriptions to six evaluation types. European Conference on Information Technology Evaluation ECITE, Amsterdam, The Netherlands: Academic Conferences Limited; 2004.

    Google Scholar 

  36. “tool, n.”. OED Online. March 2013. Oxford University Press. Accessible from: http://www.oed.com/view/Entry/203258?rskey=WzPh27&result=1. Cited 8 Apr 2013.

  37. Health Resource and Services Administration. Health IT Adoption Toolbox. Accessed from: http://www.hrsa.gov/healthit/toolbox/. Cited 8 Apr 2013.

  38. Cusack CM, Byrne C, Hook JM, McGowan J, Poon EG, Zafar A. Health Information Technology Evaluation Toolkit: 2009 Update (Prepared for the AHRQ National Resource Center for Health Information Technology under Contract No. 290-04-0016.) AHRQ Publication No. 09-0083-EF. Rockville: Agency for Healthcare Research and Quality. 2009.

    Google Scholar 

  39. Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR. 1999;48 (No. RR-11). URL: http://www.cdc.gov/eval/framework/index.htm. Last accessed 12 Mar 2013.

  40. Friedman CP. “Smallball” evaluation: a prescription for studying community-based information interventions. J Med Libr Assoc. 2005;93(4 Suppl):S43.

    PubMed  Google Scholar 

  41. Brender J. Handbook of evaluation methods for health informatics. Burlington, MA: Academic Press, 2006.

    Google Scholar 

  42. Creswell JW. Research design: qualitative, quantitative, and mixed methods approaches. Thousand Oaks: SAGE Publications, Incorporated; 2008.

    Google Scholar 

  43. Ericsson KA, Ralf TK, Clemens TR. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100(3):363.

    Article  Google Scholar 

  44. Butler MA, Bender AD. Intensive care unit bedside documentation systems. Realizing cost savings and quality improvements. Comput Nurs. 1999;17(1):32.

    PubMed  CAS  Google Scholar 

  45. Heathfield H, Pitty D, Hanka R. Evaluating information technology in health care: barriers and challenges. BMJ. 1998;316:1959–61.

    Article  PubMed  CAS  Google Scholar 

  46. Ammenwerth E, Mansmann U, Mahler C, Kandert M, Eichstadter R. Are quantitative methods sufficient to show why wards react differently to computer-based nursing document? In: Surjan G, Engelbrecht R, McNair P, editors. Proceedings of the 17th international congress of the European Federation for Medical Informatics (Medical Informatics Europe 2002 – health Data in the Information Society), 25–29 Aug 2002, Budapest, Studies in Health Technology and Informatics. Vol. 90. Amsterdam: IOS Press; 2002, p. 377–81.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. David Geffen School of Medicine at UCLA and UCLA Fielding School of Public Health, Los Angeles County Harbor-UCLA Medical Center, 1000 W. Carson Street, Building E-4, Torrance, CA, 90509, USA

    Paul C. Fu Jr. MD, MPH

  2. Scientific Education and Professional Development Program Office, Centers for Disease Control and Prevention (CDC), 1600 Clifton Rd NE, Mailstop E-92, Atlanta, GA, 30333, USA

    Herman Tolentino MD & Laura H. Franzke PhD, MPH

Authors
  1. Paul C. Fu Jr. MD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Herman Tolentino MD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laura H. Franzke PhD, MPH
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul C. Fu Jr. MD, MPH .

Editor information

Editors and Affiliations

  1. Department of Medical Informatics, Oregon Health & Science University, Portland, Oregon, USA

    J.A. Magnuson

  2. Harbor-UCLA Medical Center, Torrance, California, USA

    Paul C. Fu, Jr.

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag London

About this chapter

Cite this chapter

Fu, P.C., Tolentino, H., Franzke, L.H. (2014). Evaluation for Public Health Informatics. In: Magnuson, J., Fu, Jr., P. (eds) Public Health Informatics and Information Systems. Health Informatics. Springer, London. https://doi.org/10.1007/978-1-4471-4237-9_13

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4237-9_13

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4236-2

  • Online ISBN: 978-1-4471-4237-9

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation