A multistakeholder-driven model for developing an outcome management system

  • J. Randy Koch
  • Allen Lewis
  • Dwight McCall


Increasing demands for accountability in the delivery of public mental health services are engendering organized systems of treatment outcome monitoring. As such systems are designed and implemented, it is critical that key stakeholders be involved to the greatest extent possible in developing assessment instruments and methodologies so as to ensure relevance and acceptability of the outcome management system. A multistakeholder-based initiative for developing standardized outcome assessment for public mental health services in Virginia is described. Key components of the initiative include the process by which stakeholders were enlisted, the identification of recommended assessment instruments, and a pilot project that began the evaluation of the feasibility, utility, and cost-benefit of using the instruments. To illustrate features of this initiative, the child/adolescent mental health pilot project is described in detail. Implications for behavioral health administrators and next steps for Virginia's outcome management system are discussed, highlighting the role of key stakeholders.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Rounsalville BJ, Tims FM, Horton AH, et al. (Eds.):Diagnostic Source Book on Drug Abuse Research and Treatment. NIDA Publication No. 93-3508. Rockville, MD: National Institute on Drug Abuse, 1993.Google Scholar
  2. 2.
    Slayton LA (Ed.):Behavioral Healthcare Outcomes: A Reference Guide to Measurement Tools. Rockville, MD: National Community Healthcare Council, 1995.Google Scholar
  3. 3.
    Young SC, Nicholson J, Davis M:Guide to Instruments Assessing Consumer Satisfaction With Child and Adolescent Mental Health Services. Working Paper No. 95-1. Worcester, MA: Center for Psychosocial and Forensic Services Research, University of Massachusetts Medical Center, 1995.Google Scholar
  4. 4.
    Sederer LI, Dickey B (Eds.):Outcomes Assessment in Clinical Practice. Baltimore, MD: Williams and Wilkins, 1996.Google Scholar
  5. 5.
    Relman AS: Assessment and accountability: The third revolution in medical care.New England Journal of Medicine 1988; 319:1220–1222.Google Scholar
  6. 6.
    Cross TP, McDonald E:Evaluating the Outcome of Children's Mental Health Services: A Guide for the Use of Available Child and Family Outcome Measures. Boston: Judge Baker Children's Center, 1995.Google Scholar
  7. 7.
    Graham K: Guidelines for using standardized outcome measures following addictions treatment.Evaluation & the Health Professions 1994; 17(1):43–59.Google Scholar
  8. 8.
    Rosenberg G, Holden G: Social work effectiveness: A response to Cheetham. Special issue on Research and Practice: Bridging the Gap.Research on Social Work Practice 1992; 2(3):288–296.Google Scholar
  9. 9.
    Schalock RL, Kiernan WE, McGaughey MJ, et al.: State MR/DD agency information systems and available data related to day and employment programs.Mental Retardation 1993; 31(1):29–34.Google Scholar
  10. 10.
    Dialogue on Outcomes for Mental and Addictive Disorders. Arlington, VA: National Alliance for the Mentally Ill, 1996.Google Scholar
  11. 11.
    The MHSIP Consumer-Oriented Mental Health Report Card. Washington, DC: Center for Mental Health Services, 1996.Google Scholar
  12. 12.
    Bickman L: Evaluation of demonstration programs: Measuring client outcomes. In: Stroul BA (Ed.):Proceedings of the Children and Adolescents Service System Programs Technical Assistance Research Meeting. Washington, DC: Children and Adolescents Service System Programs Technical Assistance Center, 1990.Google Scholar
  13. 13.
    VanDenBerg J, Beci S, Pierce J: The Pennsylvania Outcome Project for Children's Services. In: Kutash K, Liberton CJ, Algarin A, et al. (Eds.):Children's Mental Health Services and Policy: Building a Research Base. 5th Annual Research Conference Proceedings. Tampa: University of South Florida, Florida Mental Health Institute, Research and Training Center for Children's Mental Health, 1992, pp. 233–238.Google Scholar
  14. 14.
    Schorr LB, Farrow F, Hornbeck D, et al.:The Case for Shifting to Results-Based Accountability. Boston: Judge Baker Children's Center, 1994.Google Scholar
  15. 15.
    Goerge RW: The promise of information systems.Research and Evaluation in Group Care 1993; 3:3–4.Google Scholar
  16. 16.
    Stroul BA, Friedman FM:A System of Care for Severely Emotionally Disturbed Children & Youth. Washington, DC: Children and Adolescent Service System Programs Technical Advisory Committee, 1986.Google Scholar
  17. 17.
    Horsch K: Results-based accountability systems: Opportunities and challenges. In:The Evaluation Exchange: Emerging Strategies in Evaluating Child and Family Services. Vol. 1. Boston: Harvard Family Research Project, 1996, issue 1.Google Scholar
  18. 18.
    Dunton N: Challenges to data capacity for outcome-based accountability. In:The Evaluation Exchange: Emerging Strategies in Evaluating Child and Family Services. Vol. 1. Boston: Harvard Family Research Project, 1996, issue 1.Google Scholar
  19. 19.
    Law CE: Evaluating children's managed mental health care.Technical Assistance Brief 1996; 1(2):2–4.Google Scholar
  20. 20.
    Burchard JD, Schaefer M: Improving accountability in a service delivery system in children's mental health.Clinical Psychology Review 1992; 12:867–882.Google Scholar
  21. 21.
    Hodges K, Gust BS: Measures of impairment for children and adolescents.The Journal of Mental Health Administration 1995; 22(4):403–413.Google Scholar
  22. 22.
    Robin G: Creating policy alternatives using stakeholder values.Management Science 1994; 40:1035–1048.Google Scholar
  23. 23.
    Wallace GW: Balancing conflicting stakeholder requirements.Journal for Quality and Participation 1995; 18(2):84–89.Google Scholar
  24. 24.
    Wallace D, White JB: Building integrity in organizations.New Management 1988; 6(1):30–35.Google Scholar
  25. 25.
    Achenbach TM:Manual for the Child Behavior Checklist/4–18 and 1991 Profile. Burlington: University of Vermont Department of Psychiatry, 1991.Google Scholar
  26. 26.
    Hodges K, Bickman L, Kurtz S, et al.: A multi-dimensional measure of level of functioning for children and adolescents. In: Algarin A, Friedman MR (Eds.):A System of Care for Children's Mental Health. Tampa: Research and Training Center for Children's Mental Health, Florida Mental Health Institute, University of South Florida, 1988.Google Scholar
  27. 27.
    Epstein NB, Baldwin LM, Bishop DS: The McMaster Family Assessment Device.Journal of Marital and Family Therapy 1983; 9:171–180.Google Scholar
  28. 28.
    Attkisson CC, Greenfield TK: The Client Satisfaction Questionnaire (CSQ) scales. In: Sederer LL, Dickey B (Eds.):Outcome Assessment in Clinical Practice. Baltimore, MD: Williams and Wilkins, 1995.Google Scholar
  29. 29.
    Evaluation of the Comprehensive Mental Health Services Program for Children With Severe Emotional Disturbances. Atlanta, GA: Macro International Inc. and University of South Florida, 1995.Google Scholar

Copyright information

© Association of Behavioral Healthcare Management 1998

Authors and Affiliations

  • J. Randy Koch
    • 1
  • Allen Lewis
    • 2
  • Dwight McCall
    • 1
  1. 1.Virginia Department of Mental Health, Mental Retardation, and Substance Abuse ServicesRichmond
  2. 2.the Virginia Cancer Registry, Department of HealthRichmond

Personalised recommendations