Mental Health Services Research

, Volume 2, Issue 2, pp 75–87 | Cite as

Measurement of Fidelity in Psychiatric Rehabilitation

  • Gary R. Bond
  • Lisa Evans
  • Michelle P. Salyers
  • Jane Williams
  • Hea-Won Kim
Article

Abstract

Until recently, most psychiatric rehabilitation models have been poorly defined and few have had systematic methods for measuring their implementation. We review the historical roots for the development of fidelity measures and describe recent applications in both research and practice.

fidelity program implementation program standards program models psychiatric rehabilitation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

REFERENCES

  1. Allness, D. J., & Knoedler, W. H. (1998). The PACT model of community-based treatment for persons with severe and persistent mental illness: A manual for PACT start-up. Arlington, VA: NAMI.Google Scholar
  2. Anthony, W. A. (1994). Whither the ''Boston University model''? Psychosocial Rehabilitation Journal, 17(4), 169-170.Google Scholar
  3. Anthony, W. A., Cohen, M. R., & Farkas, M. D. (1982). Apsychiatric rehabilitation program: Can I recognize one when I see one? Community Mental Health Journal, 18, 83-96.Google Scholar
  4. Anthony, W. A., Cohen, M. R., & Pierce, R. M. (1980). Instructor's guide to the psychiatric rehabilitation practice series. Baltimore, MD: University Park Press.Google Scholar
  5. Bachrach, L. L. (1988). The chronic patient: On exporting and importing model programs. Hospital and Community Psychiatry, 39, 1257-1258.Google Scholar
  6. Barton, R. (1997). Psychosocial rehabilitation services and community support systems: Outcomes research and policy implications. Springfield, IL: Office of Mental Health of the Illinois Department of Human Services.Google Scholar
  7. Beard, J. H., Propst, R. N., & Malamud, T. J. (1982). The Fountain House model of rehabilitation. Psychosocial Rehabilitation Journal, 5(1), 47-53.Google Scholar
  8. Becker, D. R., & Drake, R. E. (1993).A working life: The Individual Placement and Support (IPS) Program. Concord, NH: New Hampshire-Dartmouth Psychiatric Research Center.Google Scholar
  9. Becker, T., Holloway, F., McCrone, P., & Thornicroft, G. (1998). Evolving service interventions in Nunhead and Norwood: PRiSM Psychosis Study 2. British Journal of Psychiatry, 173, 371-375.Google Scholar
  10. Bond, G. R. (1991). Variations in an assertive outreach model. New Directions for Mental Health Services, 52, 65-80.Google Scholar
  11. Bond, G. R., Becker, D. R., Drake, R. E., & Vogler, K. M. (1997). A fidelity scale for the Individual Placement and Support model of supported employment. Rehabilitation Counseling Bulletin, 40, 265-284.Google Scholar
  12. Bond, G. R., Drake, R. E., Becker, D. R., & Mueser, K. T. (1999a). Effectiveness of psychiatric rehabilitation approaches for employment of people with severe mental illness. Journal of Disability Policy Studies, 10(1), 18-52.Google Scholar
  13. Bond, G. R., Evans, L., Kim, H., & Goodman, C. (1999b). Evaluation of Connect98. Phase II: Site visits by network staff.Final report to Illinois Office of Mental Health. Indianapolis, IN: Indiana University-Purdue University Indianapolis.Google Scholar
  14. Bond, G. R., Miller, L. D., Krumwied, R. D., & Ward, R. S. (1988). Assertive case management in three CMHCs: A controlled study. Hospital and Community Psychiatry, 39, 411-418.Google Scholar
  15. Bond, G. R., Picone, J., Mauer, B., Fishbein, S., & Stout, R. (1999). The Quality of Supported Employment Implementation Scale. In G. Revell, K. J. Inge, D. Mank, & P. Wehman (Eds.), The impact of supported employment for people with significant disabilities: Preliminary findings from the National Supported Employment Consortium (pp. 73-88). Richmond, VA: Virginia Commonwealth University.Google Scholar
  16. Brekke, J. S. (1987). The model-guided method for monitoring program implementation. Evaluation Review, 11(3), 281-299.Google Scholar
  17. Brekke, J. S. (1988). What do we really know about community support programs? Strategies for better monitoring. Hospital and Community Psychiatry, 39, 946-952.Google Scholar
  18. Brekke, J. S., & Aisley, R. A. (1990). The client interaction scale. Evaluation and the Health Professions, 13, 215-226.Google Scholar
  19. Brekke, J. S., & Test, M. A. (1987). An empirical analysis of services delivered in a model community support program. Psychosocial Rehabilitation Journal, 10(4), 51-61.Google Scholar
  20. Brekke, J. S., & Test, M. A. (1992). A model for measuring the implementation of community support programs: Results from three sites. Community Mental Health Journal, 28, 227-247.Google Scholar
  21. Brekke, J. S., & Wolkon, G. H. (1988). Monitoring program implementation in community mental health settings. Evaluation and the Health Professions, 11, 425-440.Google Scholar
  22. Burns, T., Creed, F., Fahy, T., Thompson, S., Tyrer, P., & White, I. (1999). Intensive versus standard case management for severe psychotic illness: A randomised trial. UK 700 Group. Lancet, 353, 2185-2189.Google Scholar
  23. Burt, M., Duke, A. E., & Hargreaves, W. A. (1998). The Program Environment Scale: Assessing client perceptions of community-based programs for the severely mentally ill. American Journal of Community Psychology, 26, 853-879.Google Scholar
  24. Cochrane, J., Durbin, J., & Goering, P. (1997). Best practices in mental health reform: Discussion paper. Toronto, ON: Clarke Institute of Psychiatry.Google Scholar
  25. Deci, P. A., Santos, A. B., Hiott, D. W., Schoenwald, S., & Dias, J. K. (1995). Dissemination of assertive community treatment teams. Psychiatric Services, 46, 676-678.Google Scholar
  26. Dewan, N. A., & Carpenter, D. (1997). Performance measurement in healthcare delivery systems. In L. J. Dickstein, M. B. Riba, & J. M. Oldham (Eds.), American Psychiatric Press Review of Psychiatry (Vol. 16, pp. 81-102). Washington, DC: American Psychiatric Press.Google Scholar
  27. Dincin, J. (1995). A pragmatic approach to psychiatric rehabilitation: Lessons from Chicago's Thresholds program. New Directions for Mental Health Services, 69 (Whole Issue).Google Scholar
  28. Drake, R. E., McHugo, G. J., Becker, D. R., Anthony, W. A., & Clark, R. E. (1996). The New Hampshire study of supported employment for people with severe mental illness: Vocational outcomes. Journal of Consulting and Clinical Psychology, 64, 391-399.Google Scholar
  29. Eysenck, H. (1952). The effects of psychotherapy, an evaluation. Journal of Consulting Psychology, 16, 319-324.Google Scholar
  30. Fishbein, S. M. (1988). Partial care as a vehicle for rehabilitation of individuals with severe psychiatric disability. Rehabilitation Psychology, 33, 57-64.Google Scholar
  31. Giesler, L. J., & Hodge, M. (1998). Case management in behavioral health care. International Journal of Mental Health, 27, 26-40.Google Scholar
  32. Gold Award. (1999). The wellspring of the clubhouse model for social and vocational adjustment of persons with serious mental illness: Fountain House, New York City. Psychiatric Services, 50, 1473-1476.Google Scholar
  33. Gorey, K. M., Leslie, D. R., Morris, T., Carruthers, W. V., John, L., & Chacko, J. (1998). Effectiveness of case management with severely and persistently mentally ill people. Community Mental Health Journal, 34, 241-250.Google Scholar
  34. Hall, L. L., Edgar, E. R., & Flynn, L. M. (1997). Stand and deliver: Action call to a failing industry. Arlington, VA: National Alliance for the Mentally Ill.Google Scholar
  35. Hargreaves, W. A., Shumway, M., Hu, T. W., & Cuffel, B. (1998). Cost-outcomemethods for mental health. San Diego, CA: Academic Press.Google Scholar
  36. Heflinger, C. A. (1996). Implementing a system of care: Finding from the Fort Bragg Evaluation Project. Journal of Mental Health Administration, 23, 16-29.Google Scholar
  37. Henggeler, S. W., Pickrel, S. G., & Brondino, M. J. (1999). Multisystemic treatment of substance-abusing and-dependent delinquents: Outcomes, treatment fidelity, and transportability. Mental Health Services Research, 1, 171-184.Google Scholar
  38. Herz, M. I., Liberman, R. P., Lieberman, J. A., Marder, S. R., McGlashan, T. H., Wyatt, R. J., & Wang, P. (1997). Practice guideline for the treatment of patients with schizophrenia. American Journal of Psychiatry, 154(Supplement), 1-63.Google Scholar
  39. Hogan, M. F. (1999). Public-sector mental health care: New challenges. Health Affairs, 18(5), 106-111.Google Scholar
  40. Hogarty, G. E. (1995). Schizophrenia and modern mental health services. Decade of the Brain, 6(1), 3-6.Google Scholar
  41. Hughes, R. (1999). The meaning of ''evidence based'' services in PSR. IAPSRS Connection, 2(1) 10-12.Google Scholar
  42. Hughes, R., & Clement, J. (1999, March 1). Time to end the model wars. IAPSRS Connection, 1, 1.Google Scholar
  43. IAPSRS. (1997a). IAPSRS position paper: The single model trap. Columbia, MD: International Association of Psychosocial Rehabilitation Services.Google Scholar
  44. IAPSRS. (1997b). Practice guidelines for the psychiatric rehabilitation of persons with severe and persistent mental illness in a managed care environment. Columbia, MD: International Association of Psychosocial Rehabilitation Services.Google Scholar
  45. Jerrell, J. M., & Hargreaves, W. A. (1991). The operating philosophy of community programs (Working Paper 18). Berkeley, CA: Institute for Mental Health Services Research.Google Scholar
  46. Jerrell, J. M., & Ridgely, M. S. (1999). Impact of robustness of program implementation on outcomes of clients in dual diagnosis programs. Psychiatric Services, 50, 109-112.Google Scholar
  47. Johnsen, M., Samberg, L., Calsyn, R., Blasinsky, M., Landow, W., & Goldman, H. (1999). Case management models for persons who are homeless and mentally ill: The ACCESS Demonstration Project. Community Mental Health Journal, 35, 325-346.Google Scholar
  48. Kerlinger, F. N. (1986). Foundations of behavioral research. (3rd ed.). New York: Holt, Rinehart, & Winston.Google Scholar
  49. Latimer, E. (1999a). Conseil d'e´valuation des technologies de la sante´ du Que´bec. Suivi intensif en e´quipe dans la communaute´ pour personnes atteintes de troubles mentaux graves (CETS 99-1 RF). Montreal: CETS.Google Scholar
  50. Latimer, E. (1999b). Economic impacts of assertive community treatment: A review of the literature. Canadian Journal of Psychiatry, 44, 443-454.Google Scholar
  51. Lehman, A. F., Steinwachs, D. M., & PORT Co-Investigators. (1998). At issue: Translating research into practice: The Schizophrenia Patient Outcomes Research Team (PORT) treatment recommendations. Schizophrenia Bulletin, 24, 1-10.Google Scholar
  52. Leithwood, K. A., & Montgomery, D. J. (1980). Evaluating program implementation. Evaluation Review, 4(2), 193-214.Google Scholar
  53. Lieberman, M. A., Yalom, I. D., & Miles, M. B. (1973). Encounter groups: First facts. New York: Basic Books.Google Scholar
  54. Lipsey, M. W. (1990). Design sensitivity. Newbury Park, CA: Sage.Google Scholar
  55. Marshall, M., & Creed, F. (in press). Assertive community treatment: Is it the future of community care in the UK? Social Psychiatry and Psychiatric Epidemiology. Google Scholar
  56. Marshall, M., Lockwood, A., Green, R., & Gray, A. (1998). Case management for people with severe mental disorders (Cochrane Review), Cochrane Library (Updated quarterly ed.). Oxford, England: Update Software.Google Scholar
  57. McCarthy, D., Thompson, D., & Olson, S. (1998). Planning a statewide project to convert day treatment to supported employment. Psychiatric Rehabilitation Journal, 22(1), 30-33.Google Scholar
  58. McEvoy, J. P., Scheifler, P. L., & Frances, A. (1999). The expert consensus guideline series: Treatment of schizophrenia 1999. Journal of Clinical Psychiatry, 11(Supplement), 1-80.Google Scholar
  59. McGrew, J. H., & Bond, G. R. (1995). Critical ingredients of assertive community treatment: Judgments of the experts. Journal of Mental Health Administration, 22, 113-125.Google Scholar
  60. McGrew, J. H., Bond, G. R., Dietzen, L. L., & Salyers, M. P. (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62, 670-678.Google Scholar
  61. McHugo, G. J., Drake, R. E., Teague, G. B., & Xie, H. (1999). The relationship between model fidelity and client outcomes in the New Hampshire Dual Disorders Study. Psychiatric Services, 50, 818-824.Google Scholar
  62. McHugo, G. J., Hargreaves, W. A., Drake, R. E., Clark, R. E., Xie, H., Bond, G. R., & Burns, B. J. (1998). Methodological issues in assertive community treatment studies. American Journal of Orthopsychiatry, 68, 246-260.Google Scholar
  63. Mitchell, K. M., Bozarth, J. D., & Krauft, C. C. (1977). A reappraisal of the therapeutic effectiveness of accurate empathy, nonpossessive warmth, and genuineness. In A. S. Gurman & A. M. Razin (Eds.), Effective psychotherapy: A handbook of research (pp. 482-502). Oxford: Pergamon.Google Scholar
  64. Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247-266.Google Scholar
  65. Moos, R. H. (1974). Community Oriented Program Environment Scale. Palo Alto, CA: Consulting Psychologist Press.Google Scholar
  66. Mowbray, C. T., Plum, T. B., & Masterton, T. (1998). Harbinger II: Deployment and evolution of assertive community treatment in Michigan. Administration and Policy in Mental Health, 25, 125-139.Google Scholar
  67. Moxley, D. P. (1993). Clubhouse standards as a program development tool. Psychsocial Rehabilitation Journal, 17(2), 177-183.Google Scholar
  68. Mueser, K. T., Bond, G. R., Drake, R. E., & Resnick, S. G. (1998). Models of community care for severe mental illness: A review of research on case management. Schizophrenia Bulletin, 24, 37-74.Google Scholar
  69. Mueser, K. T., & Glynn, S. M. (1999). Behavioral family therapy for psychiatric disorders. (2nd ed.). Oakland, CA: NewHarbinger.Google Scholar
  70. Noble, J. H. (1991). The benefits and costs of supported employment for people with mental illness and with traumatic brain injury in New York State (C-0023180). Buffalo, NY: Research Foundation of the State University of New York.Google Scholar
  71. Orlinsky, D. E., Grawe, K., & Parks, B. K. (1994). Process and outcome in psychotherapy-Noch einmal. In A. E. Bergin & S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change (4th ed., pp. 270-376). New York: Wiley.Google Scholar
  72. Paul, G. L., & Lentz, R. J. (1977). Psychosocial treatment of chronic mental patients: Milieu versus social learning programs. Cambridge, MA: Harvard University Press.Google Scholar
  73. Propst, R. (1992). Standards for clubhouse programs: Why and how they were developed. Psychosocial Rehabilitation Journal, 16(2), 25-30.Google Scholar
  74. Rapp, C. A. (1999). Best Practice Fidelity Tools. Lawrence, KS: University of Kansas School of Social Welfare.Google Scholar
  75. Rezmovic, E. L. (1984). Assessing treatment implementation amid the slings and arrows of reality. Evaluation Review, 8, 187-204.Google Scholar
  76. Rogers, C. R. (1951). Client-centered therapy. Boston: Houghton Mifflin.Google Scholar
  77. Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology, 21, 95-103.Google Scholar
  78. Rosenheck, R., Neale, M., Leaf, P., Milstein, R., & Frisman, L. (1995). Multisite experimental cost study of intensive psychiatric community care. Schizophrenia Bulletin, 21, 129-140.Google Scholar
  79. Salyers, M. P., Masterton, T. W., Fekete, D. M., Picone, J. J., & Bond, G. R. (1998). Transferring clients from intensive case management: Impact on client functioning. American Journal of Orthopsychiatry, 68, 233-245.Google Scholar
  80. Schaedle, R., & Epstein, I. (2000). Specifying intensive case management: A multiple perspective approach. Mental Health Services Research, this issue.Google Scholar
  81. Schreier, M. A., & Rezmovic, E. L. (1983). Measuring the degree of program implementation:Amethodological review. Evaluation Review, 7, 599-633.Google Scholar
  82. Sloane, R. B., Staples, F. R., Cristol, A. H., Yorkston, N. J., & Whipple, K. (1975). Psychotherapy versus behavior therapy. Cambridge, MA: Harvard University Press.Google Scholar
  83. Solomon, P. (1992). The efficacy of case management services for severely mentally disabled clients. Community Mental Health Journal, 28, 163-180.Google Scholar
  84. Stein, L. I., & Santos, A. B. (1998). Assertive community treatment of persons with severe mental illness. New York: Norton.Google Scholar
  85. Stein, L. I., & Test, M. A. (1980). An alternative to mental health treatment. I: Conceptual model, treatment program, and clinical evaluation. Archives of General Psychiatry, 37, 392-397.Google Scholar
  86. Suh, C. S., O'Malley, S. S., & Strupp, H. H. (1986). The Vanderbilt process measures: The Psychotherapy Process Scale (VPPS) and the Negative Indicators Scale (VNIS). In L. S. Greenberg & W. M. Pinsof (Eds.), The psychotherapeutic process: Aresearch handbook (pp. 285-323). NewYork: Guilford Press.Google Scholar
  87. Teague, G. B., Bond, G. R., & Drake, R. E. (1998). Program fidelity in assertive community treatment: Development and use of a measure. American Journal of Orthopsychiatry, 68, 216-232.Google Scholar
  88. Teague, G. B., Drake, R. E., & Ackerson, T. H. (1995). Evaluating use of continuous treatment teams for persons with mental illness and substance abuse. Psychiatric Services, 46, 689-695.Google Scholar
  89. Torrey, E. F., Erdman, K., Wolfe, S. M., & Flynn, L. M. (1990). Care of the seriously mentally ill: A rating of state programs (3rd ed.). Arlington, VA: National Alliance for the Mentally Ill.Google Scholar
  90. Torrey, W. C., & Wyzik, P. F. (1997). New Hampshire clinical practice guidelines for adults in community support programs. Lebanon, NH: West Central Services.Google Scholar
  91. Turner, J. C., & TenHoor, W. J. (1978). The NIMH community support program: Pilot approach to a needed social reform. Schizophrenia Bulletin, 4, 319-348.Google Scholar
  92. Wallace, C. J., Liberman, R. P., MacKain, S. J., Blackwell, G., & Eckman, T. A. (1992). Effectiveness and replicability of modules for teaching social and instrumental skills to the severely mentally ill. American Journal of Psychiatry, 149, 654-658.Google Scholar
  93. Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620-630.Google Scholar
  94. Wang, Q., Macias, C., & Jackson, R. (1999). First step in the development of a clubhosue fidelity instrument: Content analysis of clubhouse certification reports. Psychiatric Rehabilitation Journal, 22(3), 294-301.Google Scholar
  95. Zahrt, D. M., Bond, G. R., Salyers, M. P., & Teague, G. B. (1999). Dartmouth ACT Fidelity Scale: A practical application in a statewide project. Unpublished paper. Indiana University-Purdue University Indianapolis, Indianapolis, Indiana.Google Scholar

Copyright information

© Plenum Publishing Corporation 2000

Authors and Affiliations

  • Gary R. Bond
    • 1
  • Lisa Evans
    • 1
  • Michelle P. Salyers
    • 1
  • Jane Williams
    • 1
  • Hea-Won Kim
    • 1
  1. 1.Department of PsychologyIndiana University-Purdue University IndianapolisIndianapolis

Personalised recommendations