Current Status of Training Evaluation in Clinical Psychology

  • John F. Stevenson
  • John C. Norcross


Clinical psychology has matured to a stage at which it is both possible and necessary to evaluate its training activities. The need to evaluate training process and outcome has become particularly intense at this time of declining federal support, shrinking job market, and increasing demands for demonstration of professional competence as well as documentation of special skills imparted by training programs More locally, training can be directly improved by formative feedback, and students can be socialized to understand and appreciate quantitative evaluation efforts.


Training Process Clinical Psychologist Clinical Training Importance Rating Training Evaluation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Albee, G. W., & Kessler, M. (1977). Evaluating individual deliverers: Private practice and professional standards review organizations. Professional Psychology, 8, 502–515.CrossRefGoogle Scholar
  2. American Psychological Association (APA). (1983). Criteria for accreditation of doctoral training programs and internships in professional psychology. Washington, DC: Author.Google Scholar
  3. American Psychological Association Task Force on Education, Training, and Service in Psychology. (1982). Summary report. Washington, DC: Author.Google Scholar
  4. Autor, S. B., & Zide, E. D. (1974). Masters’ level professional training in clinical psychology and community mental health. Professional Psychology, 5, 115–121.CrossRefGoogle Scholar
  5. Ball, S., & Anderson, S. B. (1975). Practices in program evaluation: A survey and some case studies. Princeton, NJ: Educational and Testing Services.Google Scholar
  6. Barrett-Lennard, G. T. (1962). Dimensions of therapist response as causal factors in therapeutic change. Psychological Monographs, 76 (Whole No. 43).Google Scholar
  7. Bent, R. (1982). The quality assurance process as a management method for psychology training programs. Professional Psychology, 13, 98–104.CrossRefGoogle Scholar
  8. Bernstein, G. S. (1982). Training behavior change agents: A conceptual review. Behavior Therapy, 13, 1–23.CrossRefGoogle Scholar
  9. Blanton, J. S., Swope, A., Armstrong, P. M., Tori, C., & Dienst, E. R. (1984, October). Assessing competency of clinical skills: Methodological and process issues. Panel presented at Evaluation 84 Conference, San Francisco, CA.Google Scholar
  10. Buckley, P., Conte, H. R., Plutchik, R., Karasu, T. B., & Wild, K. V. (1982). Learning dynamic psychotherapy: A longitudinal study. American Journal of Psychiatry, 139, 1607–1610.PubMedGoogle Scholar
  11. Carkhuff, R. R. (1969). Helping and human relations. New York: Holt, Rinehart & Winston.Google Scholar
  12. Chen, H., & Rossi, P. (1983). Evaluating with sense: The theory-driven approach. Evaluation Review, 7, 283–302.CrossRefGoogle Scholar
  13. Ciarlo, J. A. (1981). Final report: The assessment of client/patient outcome techniques for use in mental health programs. NIMH Contract. Denver: University of Denver.Google Scholar
  14. Cohen, L. H. (1977). Factors affecting the utilization of mental health evaluation research findings. Professional Psychology, 8, 526–534.CrossRefGoogle Scholar
  15. Cohen, L. H. (1983). Document-based peer review in a psychology training clinic. Professional Psychology: Research and Practice, 14, 362–367.CrossRefGoogle Scholar
  16. Cook, T., & Reichardt, C. (1979). Qualitative and quantitative methods in evaluation research. Beverly Hills, CA: Sage.Google Scholar
  17. Dana, R. H., Gilliam, M., & Dana, J. M. (1976). Adequacy of academic-clinical preparation for internship. Professional Psychology, 7, 112–116.CrossRefGoogle Scholar
  18. Dickey, B., Vasile, R. G., & Gudeman, J. E. (1982). Increasing evaluation utilization in postgraduate training progams: Developing an alliance between evaluator, faculty, and trainees. Unpublished manuscript, Harvard Medical School and Massachusetts Mental Health Center.Google Scholar
  19. Edelstein, B. A. (1982, April). Empirical evaluation of clinical training. Paper presented at the Working Conference on Behavioral Clinical Training, Morgantown, WV.Google Scholar
  20. Edelstein, B. A., & Brasted, W. S. (1983). Clinical training. In M. Hersen, A. E. Kazdin, & A. S. Bellack (Eds.), The clinical psychology handbook. New York: Pergamon.Google Scholar
  21. Fishman, D. B., & Neigher, W. D. (1982). American psychology in the eighties: Who will buy? American Psychologist, 37, 533–546.CrossRefGoogle Scholar
  22. Flaherty, E. W., & Morell, J. A. (1978). Evaluation: Manifestations of a new field. Evaluation and Program Planning, 1, 1–10.CrossRefGoogle Scholar
  23. Ford, J. D. (1979). Research on training counselors and clinicians. Review of Educational Research, 49, 87–130.Google Scholar
  24. Garfield, S. L. (1977). Research on the training of professional psychotherapists. In A. S. Gurman & A. M. Razin (Eds.), Effective psychotherapy: A handbook of research (pp. 63–83 ). New York: Pergamon.Google Scholar
  25. Garfield, S. L., & Kurtz, R. (1976). Clinical psychologists in the 1970s. American Psychologist, 31, 1–9.CrossRefGoogle Scholar
  26. Gottfredson, G. D. (1984). A theory-ridden approach to program evaluation: A method for stimulating researcher-implementer collaboration. American Psychologist, 39(10), 1101–1112.CrossRefGoogle Scholar
  27. Grant, D. L., & Anderson, S. B. (1977). Issues in the evaluation of training. Professional Psychology, 8, 659–677.CrossRefGoogle Scholar
  28. Greenspan, S. I., & Sharfstein, S. S. (1981). Efficacy of psychotherapy: Asking the right questions. Archives of General Psychiatry, 38, 1213–1219.PubMedCrossRefGoogle Scholar
  29. Haley, J. (1976). Problem-solving therapy. San Francisco: Jossey-Bass.Google Scholar
  30. Harway, N. I., & Serafica, F. C. (1977). Issues in the evaluation of the psychology department clinic. Professional Psychology, 8, 650–658.CrossRefGoogle Scholar
  31. Henry, B. (1984). The future of clinical training: Forward into the past. The Clinical Psychologist, Winter, 1, 25–26.Google Scholar
  32. Hess, A. K., & Hess, K. A. (1983). Psychotherapy supervision: A survey of internship training practices. Professional Psychology: Research and Practice, 4, 504–513.CrossRefGoogle Scholar
  33. Hoffman, P. J., & Aderet, A. (1984). Empirical validity of the EPPP. A report by COGITAN to the American Association of State Psychology Boards, Palo Alto, CA.Google Scholar
  34. Howard, A. (1983). Work samples and simulations in competency evaluation. Professional Psychology: Research and Practice, 14, 780–796.CrossRefGoogle Scholar
  35. Howe, H. E., & Neimeyer, R. A. (1979). Empirical description of clinical training: A reevaluation and validation. Professional Psychology: Research and Practice, 10, 168–174.Google Scholar
  36. Irby, D. M., Evans, J., & Larson, L. (1978). Trends in clinical evaluation. In M. K. Morgan D. M. Irby (Eds.), Evaluating clinical competence in the health professions. St. Louis: C. V. Mosby.Google Scholar
  37. Kalafat, J., & Neigher, W. D. (1983). Can quality survive in public health programs? The challenge for training. Professional Psychology, 14, 90–104.CrossRefGoogle Scholar
  38. Khol, T., Matefy, R., & Turner, J. (1972). Evaluation of APA internship programs: A survey of clinical psychology interns. Journal of Clinical Psychology, 28, 562–569.PubMedCrossRefGoogle Scholar
  39. Kirchner, E. P. (1969). Graduate education in psychology: Retrospective views of advanced degrees recipients. Journal of Clinical Psychology, 25, 207–213.CrossRefGoogle Scholar
  40. Koocher, G. P. (1979). Credentialing in psychology: Close encounters with competence. American Psychologist, 34, 696–702.CrossRefGoogle Scholar
  41. Korman, M. (1973). Resolutionsof the National Conference on Levels and Patterns of Training in Professional Psychology. Vail, CO.Google Scholar
  42. Korman, M. (1974). National conference on levels and patterns of professional training in psychology: The major themes. American Psychologist, 29, 441–449.CrossRefGoogle Scholar
  43. Korn, J. H., Keiser, K. W., & Stevenson, J. F. (1982). Practicum and internship training in program evaluation. Professional Psychology, 13, 462–469.CrossRefGoogle Scholar
  44. Krasner, L. (1969). Behavior modification: Values and training. In C. M. Franks (Ed.), Behavior therapy: Appraisal and status. New York: McGraw-Hill.Google Scholar
  45. Katochwill, T. R., & Van Someren, K. R. (1984). Training behavioral consultants: Issues and directions. Behavior Therapist, 7, 19–22.Google Scholar
  46. Kurz, R. B., Fuchs, M., Dabeks, R. F., Kurtz, S. M. S., & Helfrich, W. T. (1982). Characteristics of predoctoral internships in professional psychology. American Psychologist, 37, 1213–1220.CrossRefGoogle Scholar
  47. Lambert, M. J., & Bergin, N. E. (1983). Therapist characteristics and their contribution to psychotherapy outcome. In C. E. Walker (Ed.), The handbook of clinical psychology (pp. 205–241 ). Homewood, IL: Dow Jones-Irwin.Google Scholar
  48. Levy, L. H. (1983). Evaluation of students in clinical psychology programs: A program evaluation perspective. Professional Psychology: Research and Practice, 15, 497–508.CrossRefGoogle Scholar
  49. Longabaugh, R., Stout, R., Kriebel, G. W., McCullough, L., & Bishop, D. (1986). DSM-III and clinically identified problems as a guide to treatment. Archives of General Psychiatry, 43, 1097–1103.PubMedCrossRefGoogle Scholar
  50. Malouf, J. L., Haas, L. J., & Forah, J. (1983). Issues in the preparation of interns: Views of trainers and trainees. Professional Psychology: Research and Practice, 14, 624–631.CrossRefGoogle Scholar
  51. Matarazzo, R. G. (1978). Research on the teaching and learning of psychotherapeutic skills. In S. L. Garfield & A. E. Bergin (Eds.), Handbook of psychotherapy and behavior change ( 2nd ed., pp. 941–966 ). New York: Wiley.Google Scholar
  52. Norcross, J. C., & Prochaska, J. O. (1982a). A national survey of clinical psychologists: Affiliation and orientations. The Clinical Psychologist, 35(1), 4–6.Google Scholar
  53. Norcross, J. C., & Prochaska, J. O. (1982b). A national survey of clinical psychologists: Views on training, APA, and career choice. The Clinical Psychologist, 35(2), 1–8.Google Scholar
  54. Norcross, J. C., & Stevenson, J. F. (1984). How shall we judge ourselves? Training evaluation in clinical psychology. Professional Psychology: Research and Practice, 15, 497–508.CrossRefGoogle Scholar
  55. Norcross, J. C., Stevenson, J. F., & Nash, J. M. (1986). Evaluation of internship training: Practices, problems and prospects. Professional Psychology, 17, 280–282.CrossRefGoogle Scholar
  56. Nurcombe, B., & Gallagher, R. M., III (1986). Clinical reasoning in psychiatry: Research and education. In B. Nurcombe & R. M. Gallagher, III (Eds.), The clinicalprocess in psychiatry (pp. 605–634 ). New York: Cambridge University Press.Google Scholar
  57. Parloff, M. B. (1980). Psychotherapy and research: An anaclitic depression. Psychiatry, 43, 279–293.PubMedGoogle Scholar
  58. Parloff, M. B., Waskow, I. E., & Wolfe, B. E. (1978). Research on therapist variables in relation to process and outcome. In S. L. Garfield & A. E. Bergin (Eds.), Handbook of psychotherapyand behavior change: An empirical analysis (2nd ed., pp. 233282 ). New York: Wiley.Google Scholar
  59. Patterson, C. H. (1984). Empathy, warmth, and genuineness in psychotherapy: A review of reviews. Psychotherapy, 21, 431–438.CrossRefGoogle Scholar
  60. Patton, M. Q. (1978). Utilization-focused evaluation. Beverly Hills, CA: Sage.Google Scholar
  61. Patton, M. Q. (1980). Qualitative evaluation methods. Beverly Hills, CA: Sage.Google Scholar
  62. Peterson, D. R. (1985). Twenty years of practitioner training in psychology. American Psychologist, 40(4), 441–451.PubMedCrossRefGoogle Scholar
  63. Peterson, D. R., & Bry, B. H. (1980). Dimensions of perceived competence in professional psychology. Professional Psychology, 11, 965–971.CrossRefGoogle Scholar
  64. Rice, D. G., & Gurman, A. S. (1973). Unresolved issues in the clinical psychology internship. Professional Psychology, 4, 403–407.CrossRefGoogle Scholar
  65. Robertson, M. (1984). Teaching psychotherapy in an academic setting. Psychotherapy, 21, 209–212.CrossRefGoogle Scholar
  66. Robertson, M. (1985). Training eclectic psychotherapists. In J. C. Norcross (Ed.), Handbookof eclectic psychotherapy(pp. 416–435 ). New York: Brunner/Mazel.Google Scholar
  67. Rogers, C. R. (1957). The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology, 21, 95–103.PubMedCrossRefGoogle Scholar
  68. Rosenfeld, M., Shimberg, B., & Thornton, R. F. (1984). Job analysis of licensed psychologists in the UnitedStates and Canada. Princeton, NJ: The Educational Testing Service.Google Scholar
  69. Rosenkrantz, A. L., & Holmes, G. R. (1974). A pilot study of clinical internship at the William S. Hall Psychiatric Institute. Journal of Clinical Psychology, 25, 417–419.CrossRefGoogle Scholar
  70. Schacht, T. E. (1984). Toward a national clinical training system. The Clinical Psychologist, Winter, 1, 26–27.Google Scholar
  71. Scofield, M. E., & Yoxtheimer, L. L. (1983). Psychometric issues in the assessment of clinical competencies. Journal of Counseling Psychology, 30, 413–420.CrossRefGoogle Scholar
  72. Sechrest, L., & Hoffman, P. E. (1982). The philosophical underpinnings of peer review. Professional Psychology, 13, 14–18.CrossRefGoogle Scholar
  73. Serafica, F. C., & Harway, N. I. (1980). The psychology department clinic: Its organization and development. Professional Psychology, 11, 741–748.CrossRefGoogle Scholar
  74. Shadish, W. R. (1984). Policy research: Lessons from the implementation of deinstitutionalization. American Psychologist, 39, 725–738.PubMedCrossRefGoogle Scholar
  75. Shemberg, K. M., & Keeley, S. M. (1974). Training practices and satisfactions with preinternship preparation. Professional Psychology, 5, 98–105.CrossRefGoogle Scholar
  76. Shemberg, K. M., & Leventhal, D. B. (1981). Attitudes of internship directors toward preinternship training and clinical training models. Professional Psychology, 12, 639–646.CrossRefGoogle Scholar
  77. Steinhelber, J., & Gaynor, J. (1981). Attitudes, satisfaction, and training recommendations of former clinical psychology interns: 1968 and 1977. Professional Psychology, 12, 253–260.CrossRefGoogle Scholar
  78. Stern, S. (1984). Professional training and professional competence: A critique of current thinking. Professional Psychology: Research and Practice, 15, 230–243.CrossRefGoogle Scholar
  79. Stevenson, J. F., & Norcross, J. C. (1984, August). Local training evaluation: Who’s doing it andwhat works? Paper presented at the annual convention of the American Psychological Association, Toronto, Canada.Google Scholar
  80. Stevenson, J. F., & Norcross, J. C. (1985). Evaluation activity in psychology training clinics: National survey findings. Professional Psychology: Research and Practice, 16, 2941.CrossRefGoogle Scholar
  81. Stevenson, J. F., Norcross, J. C., King, J. T., & Tobin, K. G. (1984). Evaluating clinical training programs: A formative effort. Professional Psychology: Research and Practice, 15, 218–229.CrossRefGoogle Scholar
  82. Stevenson, J. F., Norcross, J. C., Nash, J., & Grebstein, L. (1985). University of Rhode Island Clinical Psychology Program Self-Study. Kingston, RI: University of Rhode Island.Google Scholar
  83. Stout, A. L., Holmes, G. R., & Rothstein, W. (1977). Responses by graduates to the memory of their internship in clinical psychology. Perceptual and Motor Skills, 45, 863–870.PubMedCrossRefGoogle Scholar
  84. Strupp, H. H. (1975). Training the complete clinician. The Clinical Psychologist, 28(4), 1–2.Google Scholar
  85. Strupp, H. H. (1981, August). Some implications of psychotherapy research for training. Paper presented at American Psychological Association Annual Meeting, Los Angeles, CA.Google Scholar
  86. Strupp, H. H. (1984). The Vanderbilt psychotherapy research project: Past, present, and future. In J. B. Williams & R. L. Spitzer (Eds.), Psychotherapy research: Where are we and where should we go? (pp. 235–246 ). New York: Guilford.Google Scholar
  87. Strupp, H., & Schacht, T. (Eds.). (1984). The training of clinical psychologists. The Clinical Psychologist, Winter, 1, 19–35.Google Scholar
  88. Taylor, C. W., Albo, D., Holland, J., & Brandt, G. (1985). Attributes of excellence in various professions: Their relevance to the selection of gifted/talented persons. Gifted Child Quarterly, 29, 29–34.CrossRefGoogle Scholar
  89. Truax, C. B., & Carkhuff, R. R. (1967). Toward effective counseling and psychotherapy: Training and practice. Chicago: Aldine.Google Scholar
  90. Truax, C. B., & Mitchell, K. M. (1971). Research on certain therapist interpersonal skills in relation to process and outcome. In A. E. Bergin S. L. Garfield (Eds.), Handbook of psychotherapy and behavior change. New York: Wiley.Google Scholar
  91. Tucker, R. (1970). Strangers in paradise. Journal of Consulting and Clinical Psychology, 34, 140–143.PubMedCrossRefGoogle Scholar
  92. Tyler, J. D., & Weaver, S. H. (1981). Evaluating the clinical supervisee: A survey of practices in graduate training programs. Professional Psychology: Research and Practice, 12, 434–437.Google Scholar
  93. Walfish, S., Kaufman, K., & Kinder, B. N. (1980). Graduate training in clinical psychology: A view from the consumer. Journal of Clinical Psychology, 36, 1040–1045.PubMedCrossRefGoogle Scholar
  94. Waskow, I. E., Parloff, M. B. (1975). Psychotherapy change measures. DHEW Publi- cation ADM 74–120. Washington, DC: National Institutes of Mental Health.Google Scholar
  95. Weiss, C. H. (1972). Evaluation research. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
  96. Weiss, S. L. (1975). The clinical psychology intern evaluates the training experience. Professional Psychology, 6, 435–441.CrossRefGoogle Scholar

Copyright information

© Plenum Press, New York 1987

Authors and Affiliations

  • John F. Stevenson
    • 1
  • John C. Norcross
    • 2
  1. 1.Department of PsychologyUniversity of Rhode IslandKingstonCanada
  2. 2.Department of PsychologyUniversity of ScrantonScrantonUSA

Personalised recommendations