The ecology of outcomes: System accountability in children's mental health

  • Mario Hernandez
  • Sharon Hodges
  • Michelle Cascardi


This article provides a conceptual and practical framework called the Ecology of Outcomes. Based on this framework, agencies that serve children and families build and use outcome-oriented information systems to respond to their clients in a more flexible manner. The goal is to improve promising programs by involving stakeholders in outcome identification and in utilization of results. Problems addressed include the emphasis human services place on rules compliance, lack of feedback to program staff to allow for midcourse correction, and lack of input by key stakeholders in the identification of outcomes to be measured. The following components of the framework are described: principles of outcome accountability, prerequisites and building blocks, implementing an outcome information system, and utilizing the results. Key elements of the framework are the integration of outcome information into a service system's decision-making process and the inclusion of client, stakeholder, and provider satisfaction information.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Nelson DW: Introduction. In: Annie E. Casey Foundation and Center for the Study of Social Policy:Kids Count Data Book. Washington, DC: Center for the Study of Social Policy, 1993, pp. 4–7.Google Scholar
  2. 2.
    Burchard JD, Schaefer M: Improving accountability in a service delivery system in children's mental health.Clinical Psychology Review 1992; 12:867–882.Google Scholar
  3. 3.
    Gore A, Brown R:The National Information Infrastructure: Agenda for Action. Washington, DC: Executive Office of the President, 1993.Google Scholar
  4. 4.
    Stroul BA, Friedman R:A System of Care for Severely Emotionally Disturbed Children and Youth. Washington, DC: Georgetown University Child Development Center, CASSP Technical Assistance Center, 1986.Google Scholar
  5. 5.
    Schorr LB, Farrow F, Hornbeck D, et al.: The case for shifting to results-based accountability. In: Young N, Gardner S, Coley S, et al. (Eds.):Making a Difference: Moving to Outcome-Based Accountability for Comprehensive Service Reforms. National Center for Service Integration Resource Brief 7. Falls Church, VA: National Center for Service Integration, 1994, pp. 13–28.Google Scholar
  6. 6.
    Feltman R: Testimony before the select committee on children, youth and families. In: Feltman R (Ed.):California System of Care Planning Model for Seriously Emotionally Disturbed Children and Seriously Mentally Disordered Adults and Older Adults. Ventura, CA: Ventura County Mental Health Services, 1991.Google Scholar
  7. 7.
    Cohen E, Ooms T:Data Integration and Evaluation: Essential Components of Family-Centered Systems Reform. Washington, DC: The American Association for Marriage and Family Therapy Research and Education Foundation, 1993.Google Scholar
  8. 8.
    Usher L: Balancing Stakeholder Interests in Evaluations of Innovative Programs to Serve Families and Children. Paper presented at the annual meeting of the Association for Policy Analysis and Management, Washington, DC, October 1993.Google Scholar
  9. 9.
    Usher L: Building Capacity for Self-Evaluation in Family and Children's Services Reform Efforts. Paper presented at the annual meeting of the American Evaluation Association, Dallas, TX, November 1993.Google Scholar
  10. 10.
    Weiss HB, Jacobs F:Evaluating Family Programs. New York: Aldine de Gruyter, 1988.Google Scholar
  11. 11.
    Flaherty EW, Windle C: Mandated evaluation in community mental health centers: Framework for a new policy.Evaluation Review 1981; 5:620–638.Google Scholar
  12. 12.
    Elmore RF: Organizational model of social program implementation.Public Policy 1978; 26:185–228.Google Scholar
  13. 13.
    Evaluation and Analysis to Support Decision-Making. PAD-76-9. U.S. General Accounting Office, Washington, DC, September 1976.Google Scholar
  14. 14.
    Wortmann PM: Evaluation research: A psychological perspective.American Psychologist 1975; 30:562–575.Google Scholar
  15. 15.
    Bickman L, Heflinger CA, Pion G, et al.: Evaluation planning for an innovative children's mental health system. Special issue on Child and Adolescent Mental Health.Clinical Psychology Review 1992; 12:835–865.Google Scholar
  16. 16.
    Stevenson JF, Longabaugh RH: The role of evaluation in mental health.Evaluation Review 1980; 4:461–480.Google Scholar
  17. 17.
    Weiss HB, Greene JC: An empowerment partnership for family support and education programs and evaluations.Family Science Review 1992; 5:131–148.Google Scholar
  18. 18.
    Casas JM: A culturally sensitive model for evaluating alcohol and other drug abuse preventions programs: A Hispanic perspective. In: Orlandi MA, Weston R, Epstein LG (Eds.):Cultural Competence for Evaluators: A Guide for Alcohol and Other Drug Abuse Prevention Practitioners Working With Ethnic/Racial Communities. Rockville, MD: Office for Substance Abuse Prevention, 1992, pp. 75–116.Google Scholar
  19. 19.
    Bartlett J, Cohen J: Building an accountable, improvable delivery system.Administration and Policy in Mental Health 1993; 21:51–58.Google Scholar
  20. 20.
    Greene RS, Newman F: Utilizing client outcomes to improve the delivery of mental health services: Coping with organizational realities, concerns about relevance, and the believability of the data.Evaluation and Program Planning 1993; 16:325–327.Google Scholar
  21. 21.
    Berwick D: Continuous improvement as an ideal in health care.New England Journal of Medicine 1989; 320:53–56.Google Scholar
  22. 22.
    Evans ON, Faulkner LR, Hodo GL, et al.: A quality improvement process for state mental health systems.Hospital and Community Psychiatry 1992; 5:465–469.Google Scholar
  23. 23.
    Fountain DL: Avoiding the quality assurance boondoggle in drug treatment programs through total quality management.Journal of Substance Abuse Treatment 1992; 9:355–364.Google Scholar
  24. 24.
    Newman FL, Sorensen JL:Integrated Clinical and Fiscal Management in Mental Health. Norwood, NJ: Ablex, 1985.Google Scholar
  25. 25.
    Eckert PA: Cost control through quality improvement: The new challenge for psychology.Professional Psychology: Research and Practice 1994; 25:3–8.Google Scholar
  26. 26.
    Lonborg R, Fenster LF: Value assessment and enhancement: The link between outcomes management and CQI.Quality Source for Group Practice 1992; 1:1–6.Google Scholar
  27. 27.
    O'Leary D: Improved Performance and Continuous Quality Improvement: The Agenda for Change in 1994 and Beyond. Plenary presentation at the Fifth Annual National Forum on Health Care Quality Improvement (sponsored by the Joint Commission on Accreditation of Health Care Organization), Chicago, November 1992.Google Scholar
  28. 28.
    Hodges S, Hernandez M: Building Outcome Accountability in Children's Mental Health Systems. Poster presentation at the 9th Annual Research Conference for Children's Mental Health, Tampa, FL, February 1996.Google Scholar
  29. 29.
    Hernandez M, Hodges S: The Ecology of Outcomes: Successful Approaches to Building Outcome Information Systems. Paper presented at the 9th Annual Research Conference for Children's Mental Health, Tampa, FL, February 1996.Google Scholar
  30. 30.
    Burns BJ: What drives outcomes for emotional and behavioral disorders in children and adolescents? In:Using Outcomes to Improve Care: New Directions in Mental Health Services. San Francisco: Jossey-Bass, 1996.Google Scholar
  31. 31.
    Hodges S, Hernandez M:Local Processes of Outcome Evaluation: A Survey of CMHS Grantees. Tampa: Florida Mental Health Institute, University of South Florida, 1996.Google Scholar
  32. 32.
    Meadowcroft P, Pierce J, Beck S: Developing a Multi-Agency Outcome Evaluation System for Children's Services: Lessons Learned. Paper presented at the 7th Annual Research Conference, Tampa, FL, February 1994.Google Scholar
  33. 33.
    Lourie I:Development of Local Systems of Care: Core Elements, Strategies, and Urban Issues. Washington, DC: Human Service Collaborative, 1993.Google Scholar
  34. 34.
    Rouse LW, MacCabe N, Toprac MG: The Development of a State-Wide Continuous Evaluation System for the Texas Children's Mental Health Plan: A Total Quality Management Approach. Paper presented at the 8th Annual Research Conference for Children's Mental Health, Tampa, FL, February 1995.Google Scholar
  35. 35.
    Office of Research and Evaluation:Consumer Outcomes Assessment: Proposed Instruments for Child and Adolescent Mental Health Services. Richmond, VA: Department of Mental Health, Mental Retardation and Substance Abuse Services, July 1995.Google Scholar
  36. 36.
    Office of Research and Evaluation:Outcome Assessment Pilot Project. Final Report: Instruments for Child/Adolescent Mental Health Services. Richmond, VA: Department of Mental Health, Mental Retardation and Substance Abuse Services, January 1996.Google Scholar
  37. 37.
    What Pennsylvanians Want From Children's Services. The Pennsylvania Outcome Project for Children's Services, Center for Research and Public Policy. Pittsburgh, PA: The Pressley Ridge Schools. September 1992.Google Scholar
  38. 38.
    Hernandez M, Hodges S, MacBeth G, et al.:Preliminary Report of the Michigan Outcome Identification Project. Tampa: Florida Mental Health Institute, University of South Florida, May 1996.Google Scholar
  39. 39.
    From Outcomes to Budgets: An Approach to Outcome-Based Budgeting for Families and Children's Services (Draft). Washington, DC: Center for the Study of Social Policy, June 1995.Google Scholar
  40. 40.
    Hedrick TE, Bickman L, Rog DJ:Applied Research Design: A Practical Guide. Applied Social Research Methods Series 32. Newbury Park, CA: Sage, 1993.Google Scholar
  41. 41.
    Jordan D, Hernandez M: The Ventura planning model: A proposal for mental health reform.The Journal of Mental Health Administration 1990; 1:26–47.Google Scholar
  42. 42.
    Dym K: Self-Evaluation Strategies Applied to Multiple Child-Serving Systems. Presentation at the 9th Annual Research Conference for Children's Mental Health, Tampa, FL, February 1996.Google Scholar
  43. 43.
    Weiss CH: Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In: Conell JP, Kubisch AC, Schorr LB, et al. (Eds.):New Approaches to Evaluating Community Initiatives: Concept, Methods, and Contexts. Washington, DC: The Aspen Institute, 1995, pp. 65–92.Google Scholar
  44. 44.
    Hernandez M, Goldman SA: Local approach to system development: Ventura County, California. In: Stroul BA (Ed.):Children's Mental Health: Creating Systems of Care in a Changing Society. Baltimore, MD: Paul H. Brookes, 1996, pp. 177–196.Google Scholar
  45. 45.
    Stroul BA: Profiles of local systems of care. In: Stroul BA (Ed.):Children's Mental Health: Creating Systems of Care in a Changing Society. Baltimore, MD: Paul H. Brookes, 1996, pp. 149–176.Google Scholar
  46. 46.
    Rosenblatt A: Bows and ribbons, tape and twine: Wrapping the wraparound process for children with multi-system needs.Journal of Child Family Studies 1996; 5:101–116.Google Scholar
  47. 47.
    United Way of America:Measuring Program Outcomes: A Practical Approach. Alexandria, VA: United Way, 1996.Google Scholar
  48. 48.
    Rosenblatt A, Attkisson C: Assessing outcomes for sufferers of severe mental disorder: A conceptual framework and review.Evaluation and Program Planning 1993; 16:347–362.Google Scholar
  49. 49.
    Performance Measures for Managed Behavioral Healthcare Programs. Washington, DC: American Managed Behavioral Healthcare Association, 1995.Google Scholar
  50. 50.
    Freeman MA, Trabin T:Managed Behavioral Healthcare: History, Models, Key Issues, and Future Course. Rockville, MD: Department of Health and Human Services, 1994.Google Scholar

Copyright information

© Association of Behavioral Healthcare Management 1998

Authors and Affiliations

  • Mario Hernandez
    • 1
  • Sharon Hodges
    • 2
  • Michelle Cascardi
    • 3
  1. 1.Division of Service Demonstration Evaluation & Professional Training, Department of Child and Family Studies, Louis de la Parte Florida Mental Health InstituteUniversity of South FloridaTampa
  2. 2.System Accountability Project for Children's Mental Health, Department of Child and Family Studies, Louis de la Parte Florida Mental Health InstituteUniversity of South FloridTampa
  3. 3.Dating Violence Prevention ProgramBala Cynwyd

Personalised recommendations