Long-term effects of a system of care on children and adolescents

  • Leonard BickmanEmail author
  • Kelly Noser
  • Wm. Thomas Summerfelt
Distinguished Research Paper


This study evaluates an exemplary system of care designed to provide comprehensive mental health services to children and adolescents. It was believed that the system would lead to more improvement in the functioning and symptoms of clients compared to those receiving care as usual. The project employed a randomized experimental five-wave longitudinal design with 350 families. While access to care, type of care, and the amount of care were better in the system of care, there were no differences in clinical outcomes compared to care received outside the system. In addition, children who did not receive any services, regardless of experimental condition, improved at the same rate as treated children. Similar to the Fort Bragg results, the effects of systems of care are primarily limited to system-level outcomes but do not appear to affect individual outcomes such as functioning and symptomatology.


Public Health Mental Health Clinical Outcome Health Service Health Promotion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Behar L: An integrated state system of services for seriously disturbed children. In: Looney JG (Ed.):In Chronic Mental Illness in Children and Adolescents. Washington, DC: American Psychiatric Press, 1988, pp. 131–158.Google Scholar
  2. 2.
    Stroul BA, Friedman RM: A System of Care for Severely Emotionally Disturbed Youth. Washington, DC: CASSP Technical Assistance Center, 1986.Google Scholar
  3. 3.
    Saxe L, Cross T, Silverman N, et al.:Children's Mental Health: Problems and Treatment. Durham, NC: Duke University Press, 1987.Google Scholar
  4. 4.
    Friedman RM, Duchnowski AJ: Children's mental health services [Special issue].Journal of Mental Health Administration 1990; 17(1).Google Scholar
  5. 5.
    Behar L: The Fort Bragg Evaluation: A snapshot in time.American Psychologist 1997; 52:557–559.Google Scholar
  6. 6.
    Stroul BA:Children's mental health: Creating Systems of Care in a Changing Society. Baltimore, MD: Paul H. Brookes, 1996.Google Scholar
  7. 7.
    Bickman L, Summerfelt WT, Noser K: Comparative outcomes of emotionally disturbed children and adolescents in a system of services and usual care.Psychiatric Services 1997; 48:1543–1548.Google Scholar
  8. 8.
    Bickman L: A continuum of care: More is not always better. American Psychologist 1996; 51:689–701.Google Scholar
  9. 9.
    Bickman L, Guthrie P, Foster EM, et al.:Evaluating Managed Mental Health Services: The Fort Bragg Experiment. New York: Plenum, 1995.Google Scholar
  10. 10.
    Bickman L: Resolving issues raised by the Fort Bragg evaluation: New directions for mental health services.American Psychologist 1997; 52:562–565.Google Scholar
  11. 11.
    Bickman L, Heflinger CA, Lambert EW, et al.: The Fort Bragg managed care experiment: Short term impact on psychopathology.Journal of Child and Family Studies 1996; 5:137–160.Google Scholar
  12. 12.
    Bickman L, Summerfelt WT, Foster EM: Research on systems of care: Implications of the Fort Bragg Evaluation, in children's mental health. In: Stroul BA (Ed.):Creating Systems of Care in a Changing Society. Baltimore: Paul H. Brookes, 1996, pp. 337–355.Google Scholar
  13. 13.
    Friedman RM, Burns BJ: The evaluation of the Fort Bragg Demonstration Project: An alternative interpretation of the findings.Journal of Mental Health Administration 1996; 23:128–136.Google Scholar
  14. 14.
    Burchard JD: Evaluation of the Fort Bragg managed care experiment.Journal of Child and Family Studies 1996; 5:173–176.Google Scholar
  15. 15.
    Sechrest L, Walsh M: Dogma or data: Bragging rights.American Psychologist 1997; 52:536–540.Google Scholar
  16. 16.
    Weisz JR, Han SS, Valeri, SM: More of what? Issues raised by the Fort Bragg study.American Psychologist 1997; 52:541–545.Google Scholar
  17. 17.
    Hoagwood K: Interpreting nullity: The Fort Bragg experiment—A comparative success or failure?American Psychologist 1997; 52:546–550.Google Scholar
  18. 18.
    DeLeon PH, Williams JG: Evaluation research and public policy formation: Are psychologists collectively willing to accept unpopular findings?American Psychologist 1997; 52:551–552.Google Scholar
  19. 19.
    Henggeler SW, Schoenwald SK, Munger, RL: Families and therapists achieve clinical outcomes, systems of care mediate the process.Journal of Child and Family Studies 1996; 5:177–183.Google Scholar
  20. 20.
    Weisz JR, Han SS, Valeri SM: What can we learn from Fort Bragg?Journal of Child and Family Studies 1996; 5:185–190.Google Scholar
  21. 21.
    The National Comprehensive Community Mental Health for Child and Families Program Evaluation: Report from Year One, Stark County, OH. Atlanta, GA: Macro International, 1996.Google Scholar
  22. 22.
    Community-Based Mental Health Services for Children in the Child Welfare System. Final Report Contract No. HHS-100-91-0016-01. Atlanta, GA: Macro International, 1992.Google Scholar
  23. 23.
    Stroul BA, Goldman SK, Lourie IS, et al.:Profiles of Local Systems of Care for Children and Adolescents with Severe Emotional Disturbances. Rev. ed. Washington, DC: CASSP Technical Assistance Center, 1992.Google Scholar
  24. 24.
    Bickman L: Reinterpreting the Fort Bragg Evaluation findings: The message does not change.Journal of Mental Health Administration 1996; 23:137–145.Google Scholar
  25. 25.
    Bickman L, Summerfelt WT, Firth JM, et al.: The Stark County Evaluation Project: Baseline results of a randomized experiment. In Nixon CT, Northrup DA (Eds.):Evaluating Mental Health Services: How Do Programs “Work” in the Real World? Thousand Oaks, CA: Sage, 1997, pp. 231–258.Google Scholar
  26. 26.
    Hodges K, Kline J, Stern L, et al.: The development of a child assessment interview for research and clinical use.Journal of Abnormal Child Psychology 1982; 10:173–189.Google Scholar
  27. 27.
    Summerfelt WT, Hodges, VK: Test-Retest Reliability of an Interactive Computerized Interview with Adolescents: The Computerized Child Assessment Schedule. Poster presented at the 5th annual research conference on a System of Care for Children's Mental Health Expanding the Research Base, Tampa, FL, March 2–4, 1992.Google Scholar
  28. 28.
    Hodges, K: Structured interviews for assessing children.Journal of Child Psychology and Psychiatry and Allied Disciplines 1993; 34:49–68.Google Scholar
  29. 29.
    Achenbach TM:Manual for the Child and Behavior Checklist and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry, 1991.Google Scholar
  30. 30.
    Achenbach TM:Manual for the Teacher Report Form and 1991 profile. Burlington, VT: University of Vermont, Department of Psychiatry, 1991.Google Scholar
  31. 31.
    Bickman L, Lambert EW, Karver M, et al.: Two low-cost measures of child and adolescent functioning for services research.Evaluation & Program Planning 1998; 21:263–275.Google Scholar
  32. 32.
    Epstein NB, Baldwin LM, Bishop DS: The McMaster Family Assessment Device.Journal of Marital and Family Therapy 1983; 9:171–180.Google Scholar
  33. 33.
    Miller IW, Epstein NB, Bishop DS, et al.: The McMaster Family Assessment Device: Reliability and validity.Journal of Marital and Family Therapy 1985; 11:345–356.Google Scholar
  34. 34.
    Harter S:Manual for the Self-Perception Profile for Children. Denver, CO: University of Denver Press, 1985.Google Scholar
  35. 35.
    Bickman L, Earl E, Klindworth L: Vanderbilt Mental Health Service Efficacy Questionnaire. Unpublished manuscript, Vanderbilt University, Nashville, TN, 1991.Google Scholar
  36. 36.
    Kelker KA: Working together: The parent-professional partnership. Families as Allies Project Research and Training Center to Improve Services for Seriously Emotionally Handicapped Children and Their Families, Portland, OR, April 1987.Google Scholar
  37. 37.
    Dunst CJ, Leet HE: Measuring the adequacy of resources in households with young children.Child: Care Health and Development 1987; 13:111–125.Google Scholar
  38. 38.
    Achenbach TM:Manual for the Youth Self Report and 1991 Profile. Burlington, VT: University of Vermont, Department of Psychiatry, 1991.Google Scholar
  39. 39.
    Brannan AM, Heflinger CA, Bickman L: The Caregiver Strain Questionnaire: Measuring the impact of living with a child with serious emotional disturbance.Journal of Emotional and Behavioral Disorders 1997; 5:212–222.Google Scholar
  40. 40.
    Gibbons, RD, Hedeker DR, Davis JM: Estimation of effect size from a series of experiments involving paired comparisons.Journal of Educational Statistics 1993; 18:271–279.Google Scholar
  41. 41.
    Bryk AS, Raudenbush SW: Application of hierarchical linear models to assessing change.Psychological Bulletin 1987; 101:147–158.Google Scholar
  42. 42.
    Hedeker D, Gibbons RD, Flay BR: Random-effects regression models for clustered data with an example from smoking prevention research.Journal of Consulting & Clinical Psychology 1994; 62:757–765.Google Scholar
  43. 43.
    Rogosa D: Myths about longitudinal research. In: Schaie W (Ed.):Methodological Issues in Aging Research. New York: Springer, 1988, pp. 171–209.Google Scholar
  44. 44.
    Rogosa D, Brandt D, Zimowski M: A growth curve approach to the measurement of change.Psychological Bulletin, 1982; 92:726–748.Google Scholar
  45. 45.
    Bryk AS, Raudenbush SW:Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: Sage, 1992.Google Scholar
  46. 46.
    Bryk A, Raudenbush S, Congdon R:HLM Hierarchical Linear and Nonlinear Modeling with the HLM/2L and HLM/3L Programs. Chicago: Scientific Software International, 1996.Google Scholar
  47. 47.
    Raudenbush SW: Random effects models. In Cooper H (Ed.):The Handbook of Research Synthesis. New York: Russell Sage, 1994, pp. 301–321.Google Scholar
  48. 48.
    Little RC, Milliken GA, Stroup WW, et al.: SAS System for Mixed Models. Cary, NC: SAS Institute, 1996.Google Scholar
  49. 49.
    Diggle PJ, Liang, KY, Zeger SC:Analysis of Longitudinal Data. Oxford, UK: Clarendon, 1994.Google Scholar
  50. 50.
    Cohen J:Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum, 1988.Google Scholar
  51. 51.
    Foster EM, Bickman L: An evaluator's guide to detecting attrition problems.Evaluation Review 1996; 20:695–723.Google Scholar
  52. 52.
    Verbeek M, Nijman T: Testing for selectivity bias in panel data models.International Economic Review 1992; 33:681–703.Google Scholar
  53. 53.
    Service Activity Log and Manual. Canton, OH: Child and Adolescent Center, March 1997.Google Scholar
  54. 54.
    Friedman RM, Street S: Admission and discharge criteria for children's mental health services: A review of the issues and options.Journal of Clinical Child Psychology 1985; 14:229–235.Google Scholar
  55. 55.
    Bickman L, Karver MS, Schut LJA: Clinician reliability and accuracy in judging appropriate level of care.Journal of Clinical and Consulting Psychology 1997; 65:515–520.Google Scholar
  56. 56.
    Weisz JR, Doneberg GR, Han SS, et al.: Bridging the gap between lab and clinic in child and adolescent psychotherapy.Journal of Consulting and Clinical Psychology 1995; 63:688–701.Google Scholar
  57. 57.
    Rivera VR, Kutash K:What Works in Children's Mental Health Services?: Uncovering Answers to Critical Questions. Baltimore, MD: Paul H. Brookes, 1996.Google Scholar
  58. 58.
    Bickman L, Noser K: Meeting the challenges in the delivery of child and adolescent mental health services in the next millennium: The continuous quality improvement approach.Applied and Preventive Psychology. In press.Google Scholar

Copyright information

© Association of Behavioral Healthcare Management 1999

Authors and Affiliations

  • Leonard Bickman
    • 1
    Email author
  • Kelly Noser
    • 2
  • Wm. Thomas Summerfelt
    • 2
  1. 1.Psychiatry, and Public Policy, Center for Mental Health Policy, Vanderbilt Institute of Public Policy StudiesVanderbilt UniversityNashville
  2. 2.the Center for Mental Health Policy, Vanderbilt Institute of Public Policy StudiesVanderbilt UniversityUSA

Personalised recommendations