Tracking Success: Outputs Versus Outcomes—A Comparison of Accredited and Non-Accredited Public Health Agencies’ Community Health Improvement Plan objectives

Abstract

With funding for public health initiatives declining, creating measurable objectives that are focused on tracking and changing population outcomes (i.e., knowledge, attitudes, or behaviors), instead of those that are focused on health agencies’ own outputs (e.g., promoting services, developing communication messages) have seen a renewed focus. This study analyzed 4094 objectives from the Community Health Improvement Plans (CHIPs) of 280 local PHAB-accredited and non-accredited public health agencies across the United States. Results revealed that accredited agencies were no more successful at creating outcomes-focused objectives (35% of those coded) compared to non-accredited agencies (33% of those coded; Z = 1.35, p = .18). The majority of objectives were focused on outputs (accredited: 61.2%; non-accredited: 63.3%; Z = 0.72, p = .47). Outcomes-focused objectives primarily sought to change behaviors (accredited: 85.43%; non-accredited: 80.6%), followed by changes in knowledge (accredited: 9.75%; non-accredited: 10.8%) and attitudes (accredited: 1.6%; non-accredited: 5.1%). Non-accredited agencies had more double-barreled objectives (49.9%) compared to accredited agencies (32%; Z = 11.43, p < .001). The authors recommend that accreditation procedures place a renewed focus on ensuring that public health agencies strive to achieve outcomes. It is also advocated that public health agencies work with interdisciplinary teams of Health Communicators who can help them develop procedures to effectively and efficiently measure outcomes of knowledge and attitudes that are influential drivers of behavioral changes.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    When an email address to contact the state-level health agency was clearly present on an agency’s website (n = 47) an email request was sent. When an email address was unable to be found (n = 3) a member of the research team called the agency via the phone to ask whether or not CHIPs were required of their local agencies.

References

  1. 1.

    Abbott, E., Cantalupo, J., & Dixon, L. (1998). Performance measures: Linking outputs and outcomes to achieve goals. Transportation Research Record: Journal of the Transportation Research Board, 1617, 90–95.

    Article  Google Scholar 

  2. 2.

    Ajzen, I. (1985). From intentions to actions: A theory of planned behavior. In J. Kuhl (Ed.) & J. Beckmann (Eds.), Action control: From cognition to behavior (pp. 11–39). Berlin: Springer

    Google Scholar 

  3. 3.

    Bandura, A. (1998). Health promotion from the perspective of social cognitive theory. Psychology & Health, 13(4), 623–649. https://doi.org/10.1080/08870449808407422.

    Article  Google Scholar 

  4. 4.

    Bishai, D., Leider, J. P., & Resnick, B. (2016). Public health’s share of US health spending. American Journal of Public Health, 106(7), e11.

    Article  PubMed  Google Scholar 

  5. 5.

    Carrilio, T. E., Packard, T., & Clapp, J. D. (2003). Nothing in—nothing out: Barriers to the use of performance data in social service programs. Administration in Social Work, 27(4), 61–75.

    Article  Google Scholar 

  6. 6.

    Chudgar, R. B., Shirey, L. A., Sznycer-Taub, M., Read, R., Pearson, R. L., & Erwin, P. C. (2014). Local health department and academic institution linkages for community health assessment and improvement processes: A national overview and local case study. Journal of Public Health Management and Practice, 20(3), 349–355.

    Article  PubMed  Google Scholar 

  7. 7.

    Dugan, R. E., & Hernon, P. (2002). Outcomes assessment: Not synonymous with inputs and outputs. The Journal of Academic Librarianship, 28(6), 376–380.

    Article  Google Scholar 

  8. 8.

    Fisher, E. A. (2005). Facing the challenges of outcomes measurement: The role of transformational leadership. Administration in Social Work, 29(4), 35–49.

    Article  Google Scholar 

  9. 9.

    Glasgow, R. E., Vogt, T. M., & Boles, S. M. (1999). Evaluating the public health impact of health promotion interventions: The RE-AIM framework. American Journal of Public Health, 89(9), 1322–1327. https://doi.org/10.2105/AJPH.89.9.1322.

    CAS  Article  PubMed  PubMed Central  Google Scholar 

  10. 10.

    Himmelstein, D. U., & Woolhandler, S. (2016). Public health’s falling share of US health spending. American Journal of Public Health, 106(1), 56–57.

    Article  PubMed  PubMed Central  Google Scholar 

  11. 11.

    Hussey, P. S., De Vries, H., Romley, J., Wang, M. C., Chen, S. S., Shekelle, P. G., & McGlynn, E. A. (2009). A systematic review of health care efficiency measures. Health Services Research, 44(3), 784–805.

    Article  PubMed  PubMed Central  Google Scholar 

  12. 12.

    Issel, L. M., & Rosenberg, D. (2014). Program objectives and setting targets. In L. M. Issel (Ed.), Health program planning and evaluation: A practical, systematic approach for community health (pp. 215–248). Burlington, MA: Jones & Bartlett Learning.

    Google Scholar 

  13. 13.

    Kotler, P., & Lee, N. R. (2008). Social marketing: Influencing behaviors for good. Thousand Oaks, CA: Sage.

    Google Scholar 

  14. 14.

    Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.

    CAS  Article  PubMed  Google Scholar 

  15. 15.

    Lynch-Cerullo, K., & Cooney, K. (2011). Moving from outputs to outcomes: A review of the evolution of performance measurement in the human service nonprofit sector. Administration in Social Work, 35(4), 364–388.

    Article  Google Scholar 

  16. 16.

    McGuire, W. J. (2001). Input and output variables currently promising for constructing persuasive communications. In R. Rice & C. K. Atkin (Eds.), Public communication campaigns (3rd Edition, pp. 22–48). Thousand Oaks, CA: Sage.

    Google Scholar 

  17. 17.

    Mecca, W., Rivera, A., & Esposito, A. (2000). Instituting an outcomes assessment effort: Lessons from the field. Families in Society: The Journal of Contemporary Social Services, 81(1), 85–89.

    Article  Google Scholar 

  18. 18.

    National Cancer Institute. (2005). Theory at a glance: A guide for health promotion practice. Bethesda, MD: U.S. Dept. of Health and Human Services, National Institutes of Health, National Cancer Institute. NIH Publication No. pp. 05–3896.

    Google Scholar 

  19. 19.

    Norman, R. (2007). Managing outcomes while accounting for outputs: redefining “public value” in New Zealand’s performance management system. Public Performance & Management Review, 30(4), 536–549.

    Article  Google Scholar 

  20. 20.

    Pitman, B. (1994). Stop wasting training dollars: Train for outcomes, not outputs. Journal of Systems Management, 45(6), 25.

    Google Scholar 

  21. 21.

    Public Health Accreditation Board. (2013). Standards & Measures: Version 1.5. Retrieved from: http://www.phaboard.org/wp-content/uploads/SM-Version-1.5-Board-adopted-FINAL-01-24-2014.docx.pdf.

  22. 22.

    Public Health Accreditation Board. (2016). What does it cost? Retrieved from: http://www.phaboard.org/accreditation-overview/what-does-it-cost/.

  23. 23.

    Rosenstock, I. M. (1974). Historical origins of the health belief model. Health Education Monographs, 2(4), 328–335.

    Article  Google Scholar 

  24. 24.

    Rosner, B. (2006). Fundamentals of Biostatistics, 6th Edition. Belmont, CA: Thomson.

    Google Scholar 

  25. 25.

    Sanson-Fisher, R. W., Campbell, E. M., Htun, A. T., Bailey, L. J., & Millar, C. J. (2008). We are what we do: Research outputs of public health. American Journal of Preventive Medicine, 35(4), 380–385.

    Article  PubMed  Google Scholar 

  26. 26.

    United Way. (1996). Measuring program outcomes: A practical approach. Alexandria, VA: United Way of America.

    Google Scholar 

  27. 27.

    Weiss, A. P. (2007). Measuring the impact of medical research: Moving from outputs to outcomes. American Journal of Psychiatry, 164(2), 206–214.

    Article  PubMed  Google Scholar 

  28. 28.

    Wilson, L. J., & Ogden, J. D. (2008). Strategic communications planning for effective public relations and marketing. Dubuque, IA: Kendall.

    Google Scholar 

  29. 29.

    Woolf, S. H., Johnson, R. E., Fryer, G. E., Rust, G., & Satcher, D. (2004). The health impact of resolving racial disparities: An analysis of US mortality data. American Journal of Public Health, 94(12), 2078–2081. https://doi.org/10.2105/AJPH.94.12.2078.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Evan K. Perrault.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Perrault, E.K., Inderstrodt-Stephens, J. & Hintz, E.A. Tracking Success: Outputs Versus Outcomes—A Comparison of Accredited and Non-Accredited Public Health Agencies’ Community Health Improvement Plan objectives. J Community Health 43, 570–577 (2018). https://doi.org/10.1007/s10900-017-0454-0

Download citation

Keywords

  • Objectives
  • Outcomes
  • Public health departments
  • CHIP