Field Reliability of Competence to Stand Trial Opinions: How Often Do Evaluators Agree, and What Do Judges Decide When Evaluators Disagree?

  • W. Neil Gowensmith
  • Daniel C. Murrie
  • Marcus T. Boccaccini
Original Article


Despite many studies that examine the reliability of competence to stand trial (CST) evaluations, few shed light on “field reliability,” or agreement among forensic evaluators in routine practice. We reviewed 216 cases from Hawaii, which requires three separate evaluations from independent clinicians for each felony defendant referred for CST evaluation. Results revealed moderate agreement. In 71% of initial CST evaluations, all evaluators agreed about a defendant’s competence or incompetence (kappa = .65). Agreement was somewhat lower (61%, kappa = .57) in re-evaluations of defendants who were originally found incompetent and sent for restoration services. We also examined the decisions judges made about a defendant’s CST. When evaluators disagreed, judges tended to make decisions consistent with the majority opinion. But when judges disagreed with the majority opinion, they more often did so to find a defendant incompetent than competent, suggesting a generally conservative approach. Overall, results reveal moderate agreement among independent evaluators in routine practice. But we discuss the potential for standardized training and methodology to further improve the field reliability of CST evaluations.


  1. Bernstein, D. E. (2008). Expert witnesses, adversarial bias, and the (partial) failure of the Daubert revolution. Iowa Law Review, 92, 102–137.Google Scholar
  2. Boccaccini, M. T., Turner, D., & Murrie, D. C. (2008). Do some evaluators report consistently higher or lower psychopathy scores than others? Findings from a statewide sample of sexually violent predator evaluations. Psychology, Public Policy, and Law, 14, 262–283. doi:10.1037/a0014523.CrossRefGoogle Scholar
  3. Bonnie, R., & Grisso, T. (2000). Adjudicative competence and youthful offenders. In T. Grisso & R. Schwartz (Eds.), Youth on trial (pp. 73–103). Chicago, IL: University of Chicago Press.Google Scholar
  4. Brennan, R. L., & Prediger, D. J. (1981). Coefficient Kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement, 41, 687–699. doi:10.1177/001316448104100307.CrossRefGoogle Scholar
  5. Christy, A., Douglas, K., Otto, R. K., & Petrila, J. (2004). Juveniles evaluated incompetent to proceed: Characteristics and quality of mental health professionals’ evaluations. Professional Psychology: Research and Practice, 35, 380–388. doi:10.1037/0735-7028.35.4.380.CrossRefGoogle Scholar
  6. Cicchetti, D. V., & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86, 127–137.PubMedGoogle Scholar
  7. Cruise, K. R., & Rogers, R. (1998). An analysis of competency to stand to stand trial: An integration of case law and clinical knowledge. Behavioral Sciences and the Law, 16, 35–50.CrossRefPubMedGoogle Scholar
  8. Fleiss, J. L. (1980). Statistical methods for rates and proportions (2nd ed.). New York: Wiley.Google Scholar
  9. Golding, S. L. (1992). Studies of incompetent defendants: Research and social policy implications. Forensic Reports, 5, 77–83.Google Scholar
  10. Golding, S. L. (2008). Evaluations of adult adjudicative competency. In R. Jackson (Ed.), Learning forensic assessment (pp. 75–108). Mahwah, NJ: Erlbaum.Google Scholar
  11. Golding, S. L., Roesch, R., & Schreiber, J. (1984). Assessment and conceptualization of competency to stand trial: Preliminary data on the interdisciplinary fitness interview. Law and Human Behavior, 8, 321–334. doi:10.1007/BF01044699.CrossRefGoogle Scholar
  12. Goldstein, R. L., & Stone, M. (1977). When doctors disagree: Differing views on competency. Bulletin of the American Academy of Psychiatry and the Law, 8, 90–97.Google Scholar
  13. Gowensmith, W. N. (2010). Comparison of state forensic evaluation systems in the United States. unpublished data.Google Scholar
  14. Hart, S. D., & Hare, R. D. (1992). Predicting fitness to stand trial: The relative power of demographic, criminal, and clinical variables. Forensic Reports, 5, 53–65.Google Scholar
  15. Hawaii Revised Statutes, Vol. 14, 704-404 (2003). doi:10.1007/BF01044699.
  16. Hoge, S., Bonnie, R., Poythress, N., & Monahan, J. (1992). Attorney-client decision making in criminal cases: Client competence and participation as perceived by their attorneys. Behavioral Sciences and the Law, 10, 385–406. doi:10.1002/bsl.2370100308.CrossRefGoogle Scholar
  17. Homant, R. J., & Kennedy, D. B. (1987). Subjective factors in the judgment of insanity. Criminal Justice and Behavior, 14, 38–61. Retrieved from
  18. Krafka, C., Meghan, A., Dunn, M. A., Johnson, M. T., Cecil, J. S., & Miletich, D. (2002). Judge and attorney experiences, practices, and concerns regarding expert testimony in federal civil trials. Psychology, Public Policy, and Law, 8, 309–332. doi:10.1037/1076-8971.8.3.309.CrossRefGoogle Scholar
  19. Laboratory of Community Psychiatry, Harvard Medical School. (1973). Competency to stand trial and mental illness (DHEW Publication No. ADM77-103). Rockville, MD: Department of Health, Education, and Welfare.Google Scholar
  20. Melton, G. B., Petrila, J., Poythress, N. G., & Slobogin, C. (2007). Psychological evaluations for the courts: A handbook for mental health professionals and lawyers (3rd ed.). New York, NY: The Guilford Press.Google Scholar
  21. Morris, G. H., Haroun, A. M., & Naimark, D. (2004). Health law in the criminal justice system symposium: Competency to stand trial on trial. Houston Journal of Health Law and Policy, 4, 193–238.Google Scholar
  22. Mossman, D., Bowen, M. D., Vanness, D. J., Bienenfeld, D., Correll, T., Kay, J., …Lehrer, D. S. (2009). Quantifying the accuracy of forensic examiners in the absence of a “gold standard”. Law and Human Behavior, 34(5), 402. doi:10.1007/s10979-009-9197-5.CrossRefPubMedGoogle Scholar
  23. Murrie, D. C., Boccaccini, M. T., Turner, D., Meeks, M., Woods, C., & Tussey, C. (2009). Rater (dis)agreement on risk assessment measures in sexually violent predator proceedings: Evidence of adversarial allegiance in forensic evaluation? Psychology, Public Policy and Law, 15, 19–53. doi:10.1037/a0014897.CrossRefGoogle Scholar
  24. Murrie, D. C., Boccaccini, M., Zapf, P. A., Warren, J. I., & Henderson, C. E. (2008). Clinician variation in findings of competence to stand trial. Psychology, Public Policy, and Law, 14, 177–193. doi:10.1037/a0013578.CrossRefGoogle Scholar
  25. Murrie, D. C., & Warren, J. I. (2005). Clinician variation in rates of legal sanity opinions: Implications for self-monitoring. Professional Psychology: Research and Practice, 36, 519–524. doi:10.1037/0735-7028.36.5.519.CrossRefGoogle Scholar
  26. Perlin, M. L. (2004). Everything’s a little upside down, as a matter of fact the wheels have stopped: The fraudulence of the incompetency evaluation process. Houston Journal of Health Law and Policy, 4, 239–253.Google Scholar
  27. Pirelli, G., Gottdiener, W. H., & Zapf, P. A. (in press). A meta-analytic review of competency to stand trial research. Psychology, Public Policy, and Law.Google Scholar
  28. Poythress, N. G., & Stock, H. V. (1980). Competency to stand trial: A historical review and some new data. Psychiatry and Law, 8, 131–146.Google Scholar
  29. Randolph, J. J. (2008). Online kappa calculator [online application]. Retrieved March 2, 2010, from
  30. Robinson, R., & Acklin, M. W. (2010). Fitness in paradise: Quality of forensic reports submitted to the Hawaii judiciary. International Journal of Law and Psychiatry, 33, 131–137. doi:10.1016/j.ijlp.2010.03.001.CrossRefPubMedGoogle Scholar
  31. Roesch, R., & Golding, S. L. (1980). Competency to stand trial. Urbana, IL: University of Illinois Press.Google Scholar
  32. Roesch, R., Zapf, P. A., & Eaves, D. (2006). Fitness interview test-revised: A structured interview for assessing competency to stand trial. Sarasota, FL: Professional Resource Press.Google Scholar
  33. Rogers, R., & Johannsson-Love, J. (2009). Evaluating competency to stand trial with evidence-based practice. Journal of the Academy of Psychiatry and the Law, 37, 450–460.Google Scholar
  34. Rogers, R., Tillbrook, C. E., & Sewell, K. W. (2004). Evaluation of competency to stand trial—revised professional manual. Lutz, FL: Psychological Assessment Resources.Google Scholar
  35. Shuman, D., Whitaker, E., & Champagne, A. (1994). An empirical examination of the use of expert witnesses in the courts—Part II: A three part study. Jurimetrics, 34, 193–208. Retrieved from
  36. Skeem, J., & Golding, S. (1998). Community examiners’ evaluations of competence to stand trial: Common problems and suggestions for improvement. Professional Psychology: Research and Practice, 29, 357–367.CrossRefGoogle Scholar
  37. Skeem, J., Golding, S., Cohn, N., & Berge, G. (1998). The logic and reliability of expert opinion on competence to stand trial. Law and Human Behavior, 22, 519–547. doi:10.1023/A:1025787429972.CrossRefPubMedGoogle Scholar
  38. Zapf, P. A., Hubbard, K. L., Galloway, V. A., Cox, M., & Ronan, K. A. (2004). Have the courts abdicated their responsibility for determinations of competency to stand trial to clinicians? Journal of Forensic Psychology Practice, 4, 27–44. doi:10.1300/J158v04n01_02.CrossRefGoogle Scholar
  39. Zapf, P. A., & Roesch, R. (2006). Competency to stand trial: A guide for evaluators. In A. K. Hess & I. B. Weiner (Eds.), Handbook of forensic psychology (3rd ed., pp. 305–331). New York: Wiley.Google Scholar
  40. Zapf, P. A., & Roesch, R. (2009). Evaluation of competence to stand trial. New York, NY: Oxford University Press.Google Scholar

Copyright information

© American Psychology-Law Society/Division 41 of the American Psychological Association 2010

Authors and Affiliations

  • W. Neil Gowensmith
    • 1
  • Daniel C. Murrie
    • 2
  • Marcus T. Boccaccini
    • 3
  1. 1.Forensic Services, Adult Mental Health DivisionState of Hawaii, Courts and Corrections BranchHonoluluUSA
  2. 2.Institute of Law, Psychiatry, and Public PolicyUniversity of VirginiaCharlottesvilleUSA
  3. 3.Sam Houston State UniversityHuntsvilleUSA

Personalised recommendations