Advertisement

Evaluating Mental Health Services for Children and Adolescents

  • Susan Douglas Kelley
  • Carol T. Nixon
  • Leonard Bickman
Part of the Issues in Clinical Child Psychology book series (ICCP)

Abstract

Aside from a different emphasis on intervention, such as job training or housing for adults, many have characterized a split between child and adult mental health services research (e.g., Mechanic, 1996). The research, theory, knowledge, and legislation in children’s mental health services has lagged far behind that in the adult field (Institute of Medicine, 1989). This chapter provides an overview of key issues in children’s mental health services research. First, the definition and functions of mental health services research are discussed. Although the field is still developing, mental health services research serves several functions, primarily to provide an objective means of informing public policy and program decision making, and ultimately contributing to the improvement of clinical practice. A distinction is then made between efficacy research, or treatment research, and effectiveness research, which also can be called services research. The discussion focuses not only on the differences between the two, but also suggests the importance of the complementary use of both approaches in order to increase our knowledge of “what works” in children’s mental health services in applied settings.

Keywords

Mental Health Mental Health Service Service Research Multiple Stakeholder Program Staff 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ackoff, R. L. (1994), The democratic corporation: A radical prescription for recreating corporate America and rediscovering success. New York: Oxford University Press.Google Scholar
  2. Agency for Health Care Policy and Research. (1995). Conference summary: Consumer survey information in a reforming health care system (AHCPR Publication No. 95-0083). Rockville, MD: US Department of Health and Human Services.Google Scholar
  3. Arnold, L. E., Hoagwood, K., Jensen, P. S., & Vitiello, B. (1997). Towards clinically relevant trials. Psychopharmacology Bulletin, 33(1), 135–142.PubMedGoogle Scholar
  4. Behar, L. B. (1997). The Fort Bragg Evaluation: A snapshot in time. American Psychologist, 52(5), 557–559.CrossRefGoogle Scholar
  5. Bickman, L. (Ed.). (1987). Using program theory in evaluation: Vol. 33. New directions for program evaluation. San Francisco: Jossey-Bass.Google Scholar
  6. Bickman, L. (1990). Study design. In Y. T. Yuan & M. Rivest (Eds.), Preserving families: Evaluation resources for practitioners and policymakers (pp. 132–166). Newbury Park, CA: Sage.Google Scholar
  7. Bickman, L. (1992). Designing outcome evaluations for children’s mental health services: Improving internal validity. In L. Bickman & D. J. Rog (Eds.), Evaluating mental health services for children. Vol. 54. New directions for program evaluation (pp. 57–68). San Francisco: Jossey-Bass.Google Scholar
  8. Bickman, L. (1995, March). The Fort Bragg Managed Care Experiment: Findings and implications. Paper presented at the Eighth Annual Research Conference, A System of Care for Children’s Mental Health. Expanding the Research Base, Florida Mental Health Institute, Tampa, FL.Google Scholar
  9. Bickman, L. (1996a). A continuum of care: More is not always better. American Psychologist, 51, 689–701.PubMedCrossRefGoogle Scholar
  10. Bickman, L. (1996b). The application of program theory to the evaluation of a managed mental health care system. Evaluation and Program Planning, 19(2), 111–119.CrossRefGoogle Scholar
  11. Bickman, L. (Ed.). (1996c). The Fort Bragg Experiment [Special Issue]. Journal of Mental Health Administration, 23(1).Google Scholar
  12. Bickman, L. (1997). Resolving issues raised by the Fort Bragg evaluation: New directions for mental health services research. American Psychologist, 52(5), 562–565.PubMedCrossRefGoogle Scholar
  13. Bickman, L., Guthrie, P. R., Foster, E. M., Lambert, E. W., Summerfelt, W. T., Breda, C. S., & Heflinger, C. A. (1995a). Evaluating managed mental health services: The Fort Bragg experiment. New York: Plenum Press.Google Scholar
  14. Bickman, L., Lambert, W., Northrup, D., Salzer, M., & Summerfelt, W. T. (1997a, May). A conceptually-based adolescent outcome monitoring system with empirical foundations (Report to Compass Information Systems, Inc.). Nashville, TN: Vanderbilt University.Google Scholar
  15. Bickman, L., & Rog, D. (Eds.). (1997). Handbook of applied social research methods. Thousand Oaks, CA: Sage.Google Scholar
  16. Bickman, L., Rog, D., & Hedrick, T. E. (1997b). Applied research design: A practical approach. In L. Bickman & D. Rog (Eds.), Handbook of applied social research methods (pp. 5–38). Newbury Park, CA: Sage.Google Scholar
  17. Bickman, L., & Salzer, M. S. (1997). Measuring quality in mental health services. Evaluation Review, 21(3), 285–291.PubMedCrossRefGoogle Scholar
  18. Bickman, L., Summerfelt, W. T., & Bryant, D. (1995b). The quality of services at the Fort Bragg Demonstration. The Journal of Mental Health Administration, 23(1), 30–39.CrossRefGoogle Scholar
  19. Bickman, L., Summerfelt, W. T., Firth, J. M., & Douglas, S. M. (1997c). The Stark County Evaluation Project: Baseline results of a randomized experiment. In L. Bickman (Series Ed.), C. Nixon, & D. A. Northrup (Vol. Eds.), Children’s mental health services: Vol. 3. Evaluating mental health services: How do programs for children “work” in the real world? (pp. 209–230). Thousand Oaks, CA: Sage.Google Scholar
  20. Bickman, L., Summerfelt, W. T., & Foster, M. (1996). Research on systems of care: Implications of the Fort Bragg Evaluation. In B. A. Stroul (Ed.), Children’s mental health: Creating systems of care in a changing society (pp. 337–355). Baltimore: Paul H. Brookes.Google Scholar
  21. Bickman, L., Summerfelt, W. T., & Noser, K. (1997d). Comparative outcomes of emotionally disturbed children and adolescents in a system of services and usual care. Psychiatric Services, 48(12), 1543–1548.PubMedGoogle Scholar
  22. Boruch, R. F. (1997). Randomized experiments for planning and evalulation: A practical guide. In L. Bickman & D. J. Rog (Series Eds.), Applied social research methods (Vol. 44). Thousand Oaks, CA: Sage.Google Scholar
  23. Boruch, R. F., & Gomez, H. (1977). Sensitivity, bias, and theory in impact evaluation. Professional Psychology, 8, 411–434.CrossRefGoogle Scholar
  24. Boruch, R. F., & Wothke, W. (Eds.) (1985). Randomization and field experimentation: Vol. 28. New directions for program evaluation. San Francisco: Jossey-Bass.Google Scholar
  25. Brandenburg, N., Friedman, R., & Silver, S. (1990). The epidemiology of childhood psychiatric disorders: Prevalence findings from recent studies. Journal of the American Academy of Child and Adolescent Psychiatry, 29, 76–83.PubMedCrossRefGoogle Scholar
  26. Brown, S. W., & Swartz, T. A. (1989). A gap analysis of professional service quality. Journal of Marketing, 53, 92–98.CrossRefGoogle Scholar
  27. Burns, B. J. (1990). Mental health service use by adolescents in the 1970s and 1980s. Journal of the American Academy of Child and Adolescent Psychiatry, 30, 144–149.CrossRefGoogle Scholar
  28. Burns, B. J. (1994). The challenges of child mental health services research. Journal of Emotional and Behavioral Disorders, 2, 254–259.CrossRefGoogle Scholar
  29. Burns, B. I., Farmer, E. M. Z., Angold, A., Costello, E. J., & Behar, L. B. (1996). A randomized trial of case management for youths with serious emotional disturbance. Journal of Clinical Child Psychology, 25(4), 476–486.CrossRefGoogle Scholar
  30. Burns, B. J., & Friedman, R. M. (1990). Examining the research base for child mental health services and policy. Journal of Mental Health Administration, 17, 87–98.PubMedCrossRefGoogle Scholar
  31. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasi-experimental designs for research. Chicago: Rand McNally.Google Scholar
  32. Chambless, D. L., & Hollon, S. D. (1998). Defining empirically supported therapies. Journal of Consulting and Clinical Psychology, 66(1), 7–18.PubMedCrossRefGoogle Scholar
  33. Chelimsky, E. (1995). Preamble: New directions in evaluation. In R. Picciotto & R. C. Rist (Eds.), Evaluating country development policies and programs: New approaches for a newagenda:Vol. 67. New directions for evaluations (pp. 3–12). San Francisco: Jossey-Bass.Google Scholar
  34. Chen, H. (1990). Theory-driven evaluations. Newbury Park, CA: Sage.Google Scholar
  35. Chen, H., & Rossi, P. H. (1983). Evaluating with sense: The theory-driven approach. Evaluation Review, 7(3), 283–302.CrossRefGoogle Scholar
  36. Cohen, L., Sargent, M., & Sechrest, L. (1986). Use of psychotherapy research by professional psychologists. American Psychologist, 41, 198–206.PubMedCrossRefGoogle Scholar
  37. Connolly, T., Conlon, E. J., & Deutsch, S. J. (1980). Organisational effectiveness: A multiple-constituency approach. Academy of Management Review, 5, 211–217.Google Scholar
  38. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally.Google Scholar
  39. Coser, R. L. (1956). A home away from home. Social Problems, 4, 3–17.CrossRefGoogle Scholar
  40. Costello, E. J. (1989). Developments in child psychiatric epidemiology. Journal of the American Academy of Child and Adolescent Psychiatry, 28, 836–841.PubMedCrossRefGoogle Scholar
  41. Coulton, C. J. (1995). Using community-level indicators of children’s well-being in comprehensive community initiatives. In J. P. Connell, A. C. Kubisch, L. B. Schorr, & C. H. Weiss (Eds.), New approaches to evaluating community initiatives: Concepts, methods, and contexts (pp. 173–199). Washington, DC: The Aspen Institute.Google Scholar
  42. Cronbach, L. J., Ambron, S. R., Dornbusch, S. M., Hess, R. D., Hornik, R. C., Phillips, D. C., Walker. D. F., & Weiner, S. S. (1980). Toward reform of program evaluation. San Francisco: Jossey-Bass.Google Scholar
  43. Cross, T. P., & Saxe, L. (1997). Many hands make mental health systems of care a reality: Lessons from the mental health services program for youth. In L. Bickman (Series Ed.), C. Nixon, & D. A. Northrup (Vol. Eds.), Children’s mental health services: Vol. 3. Evaluating mental health services: How do programs for children “work” in the real world? (pp. 45–72). Thousand Oaks, CA: Sage.Google Scholar
  44. Evans, M. E., & Armstrong, M. I. (1997). The development of a state’s perspective on keeping children in the community: Strategies for change. In L. Bickman (Series Ed.), C. Nixon, & D. A. Northrup (Vol. Eds.), Children’s mental health services: Vol. 3. Evaluating mental health services: How do programs for children “work” in the real world? (pp. 73–94). Thousand Oaks, CA: Sage.Google Scholar
  45. Farmer, E. M. Z., Burns, B. T., Guiles, H. B., Behar, L., & Gerber, D. I. (1997). Conducting randomized clinical trials in children’s mental health. In L. Bickman (Series Ed.), C. Nixon, & D. A. Northrup (Vol. Eds.), Children’s mental health services: Vol. 3. Evaluating mental health services: How do programs for children “work” in the real world? (pp. 209–230). Thousand Oaks, CA: Sage.Google Scholar
  46. Friedman, R. M. (1997). Services and service delivery systems for children with serious emotional disorders: Issues in assessing effectiveness. In L. Bickman (Series Ed.), C. Nixon, & D. A. Northrup (Vol. Eds.), Children’s mental health services: Vol. 3. Evaluating mental health services: How do programs for children “work” in the real world? (pp. 209–230). Thousand Oaks, CA: Sage.Google Scholar
  47. Friedman, R. M., & Burns, B. J. (1996). The evaluation of the Fort Bragg Demonstration Project: An alternative interpretation of the findings. Journal of Mental Health Administration, 21(1), 128–136.CrossRefGoogle Scholar
  48. Friedman, R. M., & Hernandez, M. (1993, October). Special challenges in evaluating multisite system reform efforts: A focus on mental health services. Paper presented at the Fifteenth Annual Research Conference of the Association for Public Policy Analysis and Management, Washington, DC.Google Scholar
  49. Hall, J. A., & Dornan, M. C. (1988). What parents like about their medical care and how often they are asked: A meta-analysis of the satisfaction literature. Social Science and Medicine, 27, 935–939.PubMedCrossRefGoogle Scholar
  50. Heflinger, C. A. (1987). Psychotherapy research ethics: Continuing the debate on controlled clinical trials. American Psychologist, 42(10), 956–957.CrossRefGoogle Scholar
  51. Heflinger, C. A. (1993, October). Final report of the implementation study of the Fort Bragg evaluation project. Nashville, TN: Center for Mental Health Policy, Vanderbilt University.Google Scholar
  52. Heflinger, C. A. (1996a). Implementing a system of care: Findings from the Fort Bragg evaluation project. Journal of Mental Health Administration, 23(1), 16–29.PubMedCrossRefGoogle Scholar
  53. Heflinger, C. A. (1996b). Measuring service system coordination in managed mental health care for children and youth. Evaluation and Program Planning, 19(2), 155–163.CrossRefGoogle Scholar
  54. Henggeler, S. W., & Borduin, C. M. (1990). Family therapy and beyond: A multisystemic approach to treating the behavior problems of children and adolescents. Pacific Grove, CA: Brooks/Cole.Google Scholar
  55. Henggeler, S. W., & Borduin, C. M. (1995). Multisystemic treatment of serious juvenile offenders and their families. In I. M. Schwartz & P. AuClaire (Eds.), Home-based services for troubled children (pp. 113–130). Lincoln: University of Nebraska Press.Google Scholar
  56. Henggeler, S. W., Melton, G. B., Smith, L. A., Schoenwald, S. K., & Hanley, J. H. (1993). Family preservation using multisystemic treatment: Long-term follow-up to a clinical trial with serious juvenile offenders. Journal of Child and Family Studies, 2, 283–293.CrossRefGoogle Scholar
  57. Henggeler, S. W., Schoenwald, S. K., & Pickrel, S. G. (1995). Multisystemic therapy: Bridging the gap between university-and community-based treatment. Journal of Consulting and Clinical Psychology, 63, 709–717.PubMedCrossRefGoogle Scholar
  58. Henggeler, S. W., Schoenwald, S. K., Pickrel, S. G., Rowland, M. D., & Santos, A. B. (1994). The contribution of treatment outcome research to the reform of children’s mental health services: Multisystemic therapy as an example. Journal of Mental Health Administration, 21(3), 229–239.PubMedCrossRefGoogle Scholar
  59. Henggeler, S. W., Schoenwald, S. K. Borduin, C. M., Rowland, M. D., & Cunningham, P. B. (1998). Multisystemic treatment of antisocial behavior in children and adolescents. New York: Guilford Press.Google Scholar
  60. Hernandez, M., Hodges, S., & Cascardi, M. (1998). The ecology of outcomes: System accountability in children’s mental health. Journal of Behavioral Health Services and Research, 25(2), 136–150.PubMedCrossRefGoogle Scholar
  61. Hibbard, J. H., & Jewett, J. J. (1995, June). Using report cards to inform and empower: Consumer understanding of quality-of-care information. Paper presented at the Annual Meeting of the Association for Health Services Research.Google Scholar
  62. Hibbs, E. D., & Jensen, P. S. (Eds.) (1996). Psychosocial treatments for child and adolescent disorders: Empirically based strategies for clinical practice. Washington, DC: American Psychological Association.Google Scholar
  63. Hoagwood, K. (1994). Introduction to the special section: Issues in designing and implementing studies in non-mental health care sectors. Journal of Clinical Child Psychology, 23, 114–120.CrossRefGoogle Scholar
  64. Hoagwood, K., Hibbs, E., Brent, D., & Jensen, P. (1995). Introduction to the special edition: Efficacy and effectiveness studies of child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology, 63, 683–687.PubMedCrossRefGoogle Scholar
  65. Hobbs, N., Perrin, J., & Treys, H. (1985). Chronically ill children and their families. San Francisco: Jossey-Bass.Google Scholar
  66. Institute of Medicine. (1989). Research on children and adolescents with mental, behavioral, and developmental disorders. Washington, DC: National Academy Press.Google Scholar
  67. Kazdin, A. E., Bass, D., Ayers, W. A., & Rodgers, A. (1990). Empirical and clinical focus of child and adolescent psychotherapy research. Journal of Consulting and Clinical Psychology, 58, 729–740.PubMedCrossRefGoogle Scholar
  68. Kazdin, A. E., & Kendall, P. C. (1998). Current progress and future plans for developing effective treatments: Comments and perspectives. Journal of Clinical Child Psychology, 27(2), 217–226.PubMedCrossRefGoogle Scholar
  69. Kelleher, K., & Long, N. (1994). Barriers and new directions in mental health services research in the primary care setting. Journal of Clinical Child Psychology, 23, 133–142.CrossRefGoogle Scholar
  70. Kendall, P. C, & Southam-Gerow, M. A. (1995). Issues in the transportability of treatment: The case of anxiety disorders in youth. Journal of Consulting and Clinical Psychology, 63, 702–708.PubMedCrossRefGoogle Scholar
  71. Kraemer, H. C., & Thiemann, S. (1987). How many subjects? Statistical power analysis in research. Newbury Park, CA: Sage.Google Scholar
  72. Kurata, J. H., Nogawa, A. K., Phillips, D. M., Hoffman, S., & Werblun, M. N. (1992). Patient and provider satisfaction with medical care. Journal of Family Practice, 35, 176–179.PubMedGoogle Scholar
  73. Lambert, E. W., Nixon, C. T., Simpkins, G., & Bickman, L. (1996, December). Mental health outcomes for treated children: Definitions of improvement, effects of severity, and predicting clinical results. Final Report to the US Army MEDCOM contract DA-DA10-96-C-030. Nashville, TN: Vanderbilt University.Google Scholar
  74. Lipsey, M. W. (1989). Design sensitivity: Statistical power for experimental research. Newbury Park, CA: Sage.Google Scholar
  75. Lipsey, M. W. (1990). Design sensitivity. Newbury Park: Sage.Google Scholar
  76. Lipsey, M. W, & Wilson, D. B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48(12), 1181–1209.PubMedCrossRefGoogle Scholar
  77. Lohr, K. N., Donaldson, M., & Walker, A. (1991). Medicare: A strategy for quality assurance, III: Beneficiary and physician focus groups. Quality Review Bulletin, 17, 242–253.PubMedGoogle Scholar
  78. Lonigan, C. J., Elbert, J. C., & Johnson, S. B. (1998). Empirically supported psychosocial interventions for children: An overview. Journal of Clinical Child Psychology, 27(2), 138–145.PubMedCrossRefGoogle Scholar
  79. MACRO International and University of South Florida (1995). Evaluation of the comprehensive mental health services program for children with severe emotional disturbances: Overview of the evaluation. Atlanta, GA: MACRO International.Google Scholar
  80. Mechanic, D. (1996). Emerging issues in international mental health services research. Psychiatric Services, 47(4), 371–375.PubMedGoogle Scholar
  81. Mechanic, D., Bevilacqua, J. J., Goldman, H., Hargreaves, W., Howe, J., Knisley, M., Scherl, D. J., Stuart, G., Unhjem, M. B., & Lalley, T. L. (1992). Research resources. Schizophrenia Bulletin, 15, 669–696.CrossRefGoogle Scholar
  82. Morrissey, J. P., Calloway, M., Bartko, W. T., Ridgely, M. S., Goldman, H., & Paulson, R. I. (1994). Local mental health authorities and service system change: Evidence from the Robert Wood Johnson Foundation Program on Chronic Mental Illness. Milbank Quarterly, 72, 1–32.CrossRefGoogle Scholar
  83. Morrissey, J. P., Johnsen, M. C., & Calloway, M. O. (1997). Evaluating performance and change in mental health systems serving children and youth: An interorganizational network approach. Journal of Mental Health Administration, 24(1), 4–22.PubMedGoogle Scholar
  84. Morrow-Bradley, C., & Elliott, R. (1986). Utilization of psychotherapy research by practicing psychologists. American Psychologist, 41, 188–197.PubMedCrossRefGoogle Scholar
  85. National Committee on Quality Assurance. (1995). NCQA consumer information project focus group report: Executive summary. Washington, DC: Author.Google Scholar
  86. National Institute of Mental Health (1990). National plan for research on child and adolescent mental disorders. Rockville, MD: National Institutes of Health.Google Scholar
  87. National Institute of Mental Health (1999). Child and adolescent services research strategic planning meeting (Belmont Conference Center Draft Report). Rockland, MA: National Institute of Mental Health, Division of Services and Intervention Research.Google Scholar
  88. Nixon, C. T. (1997). How does evaluation of mental health services for children work in the real world? In L. Bickman (Series Ed.), C. Nixon, & D. A. Northrup (Vol. Eds.), Children’s mental health services: Vol. 3. Evaluating mental health services: How do programs for children “work” in the real world? (pp. 1–15). Thousand Oaks, CA: Sage.Google Scholar
  89. Ogles, B. M., Trout, S. C, Gillespie, D. K., & Penkert, K. S. (1998). Managed care as a platform for cross-system integration. Journal of Behavioral Health Services and Research, 25(3), 252–268.PubMedCrossRefGoogle Scholar
  90. Palumbo, D. T., & Oliverio, A. (1989). Implementation theory and the theory-driven approach to validity. Evaluation and Program Planning, 12, 337–344.CrossRefGoogle Scholar
  91. Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1985). A conceptual model of service quality and its implications for future research. Journal of Marketing, 49, 41–50.CrossRefGoogle Scholar
  92. Persons, J. B. (1991). Psychotherapy outcome studies do not accurately represent current models of psychotherapy: A proposed remedy. American Psychologist, 46(2), 99–106.PubMedCrossRefGoogle Scholar
  93. Pires, S. A., Stroul, B. A., & Armstrong, M. I. (1998). Impact of managed care in the public sector: Children and adolescents with behavioral health problems and their families. Behavioral Healthcare Tomorrow, 7(3), 41–46.PubMedGoogle Scholar
  94. Raw, S. D. (1993, March). Does psychotherapy research teach us anything about psychotherapy? Behavior Therapist, 75–76.Google Scholar
  95. Rezmovic, E. L. (1984). Assessing treatment implementation amid the slings and arrows of reality. Evaluation Review, 8,187–204.CrossRefGoogle Scholar
  96. Santos, A. B., Henggeler, S. W., Burns, B. J., Arana, G. W., & Meisler, N. (1995). Research on field-based services: Models for reform in the delivery of mental health care to populations with complex clinical problems. American Journal of Psychiatry, 152, 1111–1123.PubMedGoogle Scholar
  97. Saxe, L., Cross, T. P., Silverman, N., Batchelor, W. F., & Dougherty, D. (1987). Children’s mental health: Problems and treatment. Durham, NC: Duke University Press.Google Scholar
  98. Scheirer, M. A. (1987). Program theory and implementation theory: Implications for evaluators. In L. Bickman (Ed.), Using program theory in evaluation: Vol. 33. New directions for program evaluation (pp. 19–42). San Francisco: Jossey-Bass.Google Scholar
  99. Scherer, D. G., Brondino, M. J., Henggeler, S. W., Melton, G. B., & Hanley, J. H. (1994). Multisystemic family preservation therapy: Preliminary findings from a study of rural and minority serious adolescent offenders. Journal of Emotional and Behavioral Disorders, 2, 198–206.CrossRefGoogle Scholar
  100. Sechrest, L., & Walsh, M. (1997). Dogma or data: Bragging rights. American Psychologist, 52(5), 536–540.CrossRefGoogle Scholar
  101. Sechrest, L. B., West, S. G., Phillips, M. A., Redner, R., & Yeaton, W. (1979). Some neglected problems in evaluation research: Strength and integrity of treatments. In L. Sechrest, S. G. West, M. A. Phillips, R. Redner, & W. Yeaton (Eds.), Evaluation studies review annual (Vol. 4, pp. 15–35). Beverly Hills, CA: Sage.Google Scholar
  102. Smith, M. L., & Glass, G. V. (1977). Meta-analysis of psychotherapy outcome studies. American Psychologist, 32, 752–760.PubMedCrossRefGoogle Scholar
  103. Speer, D. C. (1994). Can treatment research inform decision makers? Nonexperimental methods issues and examples among older outpatients. Journal of Consulting and Clinical Psychology, 62, 560–568.PubMedCrossRefGoogle Scholar
  104. Speer, D. C., & Newman, F. L. (1996). Mental health services outcome evaluation. Clinical Psychology: Science and Practice, 3(2), 105–129.CrossRefGoogle Scholar
  105. Starfield, B. (1973). Health services research: A working model. New England Journal of Medicine, 289(3), 132–136.PubMedCrossRefGoogle Scholar
  106. Stroul, B. A. (1996). Introduction: Progress in children’s mental health. In B. A. Stroul (Ed.), Children’s mental health: Creating systems of care in a changing society (pp. xxi–xxxiii). Baltimore: Paul H. Brookes.Google Scholar
  107. Stroul, B. A., & Friedman, R. M. (1986). A system of care for severely emotionally disturbed children and youth. Washington, DC: Georgetown University Child Development Center, CASSP Technical Assistance Center.Google Scholar
  108. Stroul, B. A., & Friedman, R. M. (1996). The system of care concept and philosophy. In B. A. Stroul (Ed.), Children’s mental health: Creating systems of care in a changing society (pp. 3–21). Baltimore: Paul H. Brookes.Google Scholar
  109. Stroul, B. A., Pires, S. A., Armstrong, M. I., & Meyers, J. C. (1998). The impact of managed care on mental health services for children and their families. Future of Children, 8(2), 119–133.PubMedCrossRefGoogle Scholar
  110. Summerfelt, W. T. (1992). The development of a computerized version of the Child Assessment Schedule: An initial study. Unpublished master’s thesis.Google Scholar
  111. Summerfelt, W. T., & Hodges, V. K. (1992, March). Test-retest reliability of an interactive computerized interview with adolescents: The computerized Child Assessment Schedule. Poster presented at the 5th Annual Research Conference, A System of Care for Children’s Mental Health, Research and Training Center for Children’s Mental Health, University of South Florida, Tampa, FL.Google Scholar
  112. Trochim, W. M. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning, 12, 1–16.CrossRefGoogle Scholar
  113. Ware, J. E., & Davies, A. R. (1983). Behavioral consequences of consumer dissatisfaction with medical care. Evaluation and Program Planning, 6, 291–297.PubMedCrossRefGoogle Scholar
  114. Weiss, B. Catron, T., Harris, V., & Phung, T. M. (1999). The effectiveness of traditional child psychotherapy. Journal of Consulting and Clinical Psychology, 67(1), 82–94.PubMedCrossRefGoogle Scholar
  115. Weiss, C. H. (1983a). The stakeholder approach to evaluation: Origins and promise. In A. S. Byrk (Ed.), Stakeholder-based evaluation: Vol. 17. New directions for program evaluation (pp. 3–14). San Francisco: Jossey-Bass.Google Scholar
  116. Weiss, C. H. (1983b). Toward the future of stakeholder approaches in evaluation. In A. S. Byrk (Ed.), Stakeholder-based evaluation: Vol. 17. New directions for program evaluation (pp. 83–96). San Francisco: Jossey-Bass.Google Scholar
  117. Weiss, C. H. (1995). Nothing as practical as good theory: Exploring theory-based evaluation for comprehensive community initiatives for children and families. In J. P. Connell, A. C. Kubisch, L. B. Schorr, & C. H. Weiss (Eds.), New approaches to evaluating community initiatives: Concepts, methods, and contexts (pp. 65–92). Washington, DC: The Aspen Institute.Google Scholar
  118. Weiss, H. B., & Greene, J. C. (1995). An empowerment partnership for family support and education programs and evaluations. Family Science Review, 5, 131–148.Google Scholar
  119. Weisz, J. R., & Jensen, P. S. (1999). Efficacy and effectiveness of psychotherapy and psychopharmacotherapy with children and adolescent. Mental Health Services Research, 1(3), 125–157.PubMedCrossRefGoogle Scholar
  120. Weisz, J. R., Donenberg, G. R., Han, S. S., & Weiss, B. (1995). Bridging the gap between laboratory and clinic in child and adolescent psychotherapy. Journal of Consulting and Clinical Psychology, 63, 688–701.PubMedCrossRefGoogle Scholar
  121. Weisz, J. R., Han, S. S., & Valeri, S. M. (1997). More of what? Issues raised by the Fort Bragg study. American Psychologist, 52(5), 541–545.PubMedCrossRefGoogle Scholar
  122. Weisz, J. R., Weiss, B., Alicke, M. D., & Klotz, M. L. (1987). Effectiveness of psychotherapy with children and adolescents: A meta-analysis for clinicians. Journal of Consulting and Clinical Psychology, 55, 542–549.PubMedCrossRefGoogle Scholar
  123. Weisz, J. R., Weiss, B., & Donenberg, G. R. (1992). The lab versus the clinics: Effects of child and adolescent psychotherapy. American Psychologist, 47, 1578–1585.PubMedCrossRefGoogle Scholar
  124. West, S. G., & Sagarin, B. J. (in press). Participant selection and loss in randomized experiments. In L. Bickman (Ed.), Volume II: Contributions to research design: Donald Campbell’s legacy. Newbury Park: Sage.Google Scholar
  125. Wilde, B., Starrin, B., Larsson, G., & Larsson, M. (1993). Quality of care from a patient perspective: A grounded theory study. Scandinavian Journal of the Caring Sciences, 7, 113–120.Google Scholar
  126. Windle, C. (1994). Mental health services research: The shift to research synthesis. Administration and Policy in Mental Health, 21(3), 263–267.CrossRefGoogle Scholar
  127. Yin, R. K. (1994). Discovering the future of the case study method in evaluation research. Evaluation Practice, 15, 283–290.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2000

Authors and Affiliations

  • Susan Douglas Kelley
    • 1
  • Carol T. Nixon
    • 1
  • Leonard Bickman
    • 1
  1. 1.Center for Mental Health PolicyVanderbilt UniversityNashvilleUSA

Personalised recommendations