Skip to main content
SpringerLink
Log in

Field Reliability of Competence to Stand Trial Opinions: How Often Do Evaluators Agree, and What Do Judges Decide When Evaluators Disagree?

  • Original Article
  • Published:
Law and Human Behavior

Abstract

Despite many studies that examine the reliability of competence to stand trial (CST) evaluations, few shed light on “field reliability,” or agreement among forensic evaluators in routine practice. We reviewed 216 cases from Hawaii, which requires three separate evaluations from independent clinicians for each felony defendant referred for CST evaluation. Results revealed moderate agreement. In 71% of initial CST evaluations, all evaluators agreed about a defendant’s competence or incompetence (kappa = .65). Agreement was somewhat lower (61%, kappa = .57) in re-evaluations of defendants who were originally found incompetent and sent for restoration services. We also examined the decisions judges made about a defendant’s CST. When evaluators disagreed, judges tended to make decisions consistent with the majority opinion. But when judges disagreed with the majority opinion, they more often did so to find a defendant incompetent than competent, suggesting a generally conservative approach. Overall, results reveal moderate agreement among independent evaluators in routine practice. But we discuss the potential for standardized training and methodology to further improve the field reliability of CST evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Evaluators become “certified” if the state forensic administrator approves their background credentials. Evaluators may also attend DOH-sponsored annual trainings on forensic mental health assessment.

  2. Cases in which one or more evaluator opinions were coded as “unknown” contributed to the “other disagreement” categories listed in Tables 1 and 2.

References

  • Bernstein, D. E. (2008). Expert witnesses, adversarial bias, and the (partial) failure of the Daubert revolution. Iowa Law Review, 92, 102–137.

    Google Scholar 

  • Boccaccini, M. T., Turner, D., & Murrie, D. C. (2008). Do some evaluators report consistently higher or lower psychopathy scores than others? Findings from a statewide sample of sexually violent predator evaluations. Psychology, Public Policy, and Law, 14, 262–283. doi:10.1037/a0014523.

    Article  Google Scholar 

  • Bonnie, R., & Grisso, T. (2000). Adjudicative competence and youthful offenders. In T. Grisso & R. Schwartz (Eds.), Youth on trial (pp. 73–103). Chicago, IL: University of Chicago Press.

    Google Scholar 

  • Brennan, R. L., & Prediger, D. J. (1981). Coefficient Kappa: Some uses, misuses, and alternatives. Educational and Psychological Measurement, 41, 687–699. doi:10.1177/001316448104100307.

    Article  Google Scholar 

  • Christy, A., Douglas, K., Otto, R. K., & Petrila, J. (2004). Juveniles evaluated incompetent to proceed: Characteristics and quality of mental health professionals’ evaluations. Professional Psychology: Research and Practice, 35, 380–388. doi:10.1037/0735-7028.35.4.380.

    Article  Google Scholar 

  • Cicchetti, D. V., & Sparrow, S. A. (1981). Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86, 127–137.

    PubMed  Google Scholar 

  • Cruise, K. R., & Rogers, R. (1998). An analysis of competency to stand to stand trial: An integration of case law and clinical knowledge. Behavioral Sciences and the Law, 16, 35–50.

    Article  PubMed  Google Scholar 

  • Fleiss, J. L. (1980). Statistical methods for rates and proportions (2nd ed.). New York: Wiley.

    Google Scholar 

  • Golding, S. L. (1992). Studies of incompetent defendants: Research and social policy implications. Forensic Reports, 5, 77–83.

    Google Scholar 

  • Golding, S. L. (2008). Evaluations of adult adjudicative competency. In R. Jackson (Ed.), Learning forensic assessment (pp. 75–108). Mahwah, NJ: Erlbaum.

    Google Scholar 

  • Golding, S. L., Roesch, R., & Schreiber, J. (1984). Assessment and conceptualization of competency to stand trial: Preliminary data on the interdisciplinary fitness interview. Law and Human Behavior, 8, 321–334. doi:10.1007/BF01044699.

    Article  Google Scholar 

  • Goldstein, R. L., & Stone, M. (1977). When doctors disagree: Differing views on competency. Bulletin of the American Academy of Psychiatry and the Law, 8, 90–97.

    Google Scholar 

  • Gowensmith, W. N. (2010). Comparison of state forensic evaluation systems in the United States. unpublished data.

  • Hart, S. D., & Hare, R. D. (1992). Predicting fitness to stand trial: The relative power of demographic, criminal, and clinical variables. Forensic Reports, 5, 53–65.

    Google Scholar 

  • Hawaii Revised Statutes, Vol. 14, 704-404 (2003). doi:10.1007/BF01044699.

  • Hoge, S., Bonnie, R., Poythress, N., & Monahan, J. (1992). Attorney-client decision making in criminal cases: Client competence and participation as perceived by their attorneys. Behavioral Sciences and the Law, 10, 385–406. doi:10.1002/bsl.2370100308.

    Article  Google Scholar 

  • Homant, R. J., & Kennedy, D. B. (1987). Subjective factors in the judgment of insanity. Criminal Justice and Behavior, 14, 38–61. Retrieved from http://proquest.umi.com/pqdweb?did=41435115&Fmt=1&clientId=82498&RQT=309&VName=PQD.

  • Krafka, C., Meghan, A., Dunn, M. A., Johnson, M. T., Cecil, J. S., & Miletich, D. (2002). Judge and attorney experiences, practices, and concerns regarding expert testimony in federal civil trials. Psychology, Public Policy, and Law, 8, 309–332. doi:10.1037/1076-8971.8.3.309.

    Article  Google Scholar 

  • Laboratory of Community Psychiatry, Harvard Medical School. (1973). Competency to stand trial and mental illness (DHEW Publication No. ADM77-103). Rockville, MD: Department of Health, Education, and Welfare.

  • Melton, G. B., Petrila, J., Poythress, N. G., & Slobogin, C. (2007). Psychological evaluations for the courts: A handbook for mental health professionals and lawyers (3rd ed.). New York, NY: The Guilford Press.

    Google Scholar 

  • Morris, G. H., Haroun, A. M., & Naimark, D. (2004). Health law in the criminal justice system symposium: Competency to stand trial on trial. Houston Journal of Health Law and Policy, 4, 193–238.

    Google Scholar 

  • Mossman, D., Bowen, M. D., Vanness, D. J., Bienenfeld, D., Correll, T., Kay, J., …Lehrer, D. S. (2009). Quantifying the accuracy of forensic examiners in the absence of a “gold standard”. Law and Human Behavior, 34(5), 402. doi:10.1007/s10979-009-9197-5.

    Article  PubMed  Google Scholar 

  • Murrie, D. C., Boccaccini, M. T., Turner, D., Meeks, M., Woods, C., & Tussey, C. (2009). Rater (dis)agreement on risk assessment measures in sexually violent predator proceedings: Evidence of adversarial allegiance in forensic evaluation? Psychology, Public Policy and Law, 15, 19–53. doi:10.1037/a0014897.

    Article  Google Scholar 

  • Murrie, D. C., Boccaccini, M., Zapf, P. A., Warren, J. I., & Henderson, C. E. (2008). Clinician variation in findings of competence to stand trial. Psychology, Public Policy, and Law, 14, 177–193. doi:10.1037/a0013578.

    Article  Google Scholar 

  • Murrie, D. C., & Warren, J. I. (2005). Clinician variation in rates of legal sanity opinions: Implications for self-monitoring. Professional Psychology: Research and Practice, 36, 519–524. doi:10.1037/0735-7028.36.5.519.

    Article  Google Scholar 

  • Perlin, M. L. (2004). Everything’s a little upside down, as a matter of fact the wheels have stopped: The fraudulence of the incompetency evaluation process. Houston Journal of Health Law and Policy, 4, 239–253.

    Google Scholar 

  • Pirelli, G., Gottdiener, W. H., & Zapf, P. A. (in press). A meta-analytic review of competency to stand trial research. Psychology, Public Policy, and Law.

  • Poythress, N. G., & Stock, H. V. (1980). Competency to stand trial: A historical review and some new data. Psychiatry and Law, 8, 131–146.

    Google Scholar 

  • Randolph, J. J. (2008). Online kappa calculator [online application]. Retrieved March 2, 2010, from http://justus.randolph.name/kappa.

  • Robinson, R., & Acklin, M. W. (2010). Fitness in paradise: Quality of forensic reports submitted to the Hawaii judiciary. International Journal of Law and Psychiatry, 33, 131–137. doi:10.1016/j.ijlp.2010.03.001.

    Article  PubMed  Google Scholar 

  • Roesch, R., & Golding, S. L. (1980). Competency to stand trial. Urbana, IL: University of Illinois Press.

    Google Scholar 

  • Roesch, R., Zapf, P. A., & Eaves, D. (2006). Fitness interview test-revised: A structured interview for assessing competency to stand trial. Sarasota, FL: Professional Resource Press.

    Google Scholar 

  • Rogers, R., & Johannsson-Love, J. (2009). Evaluating competency to stand trial with evidence-based practice. Journal of the Academy of Psychiatry and the Law, 37, 450–460.

    Google Scholar 

  • Rogers, R., Tillbrook, C. E., & Sewell, K. W. (2004). Evaluation of competency to stand trial—revised professional manual. Lutz, FL: Psychological Assessment Resources.

    Google Scholar 

  • Shuman, D., Whitaker, E., & Champagne, A. (1994). An empirical examination of the use of expert witnesses in the courts—Part II: A three part study. Jurimetrics, 34, 193–208. Retrieved from http://proquest.umi.com/pqdweb?did=41486506&Fmt=1&clientId=82498&RQT=309&VName=PQD.

  • Skeem, J., & Golding, S. (1998). Community examiners’ evaluations of competence to stand trial: Common problems and suggestions for improvement. Professional Psychology: Research and Practice, 29, 357–367.

    Article  Google Scholar 

  • Skeem, J., Golding, S., Cohn, N., & Berge, G. (1998). The logic and reliability of expert opinion on competence to stand trial. Law and Human Behavior, 22, 519–547. doi:10.1023/A:1025787429972.

    Article  PubMed  Google Scholar 

  • Zapf, P. A., Hubbard, K. L., Galloway, V. A., Cox, M., & Ronan, K. A. (2004). Have the courts abdicated their responsibility for determinations of competency to stand trial to clinicians? Journal of Forensic Psychology Practice, 4, 27–44. doi:10.1300/J158v04n01_02.

    Article  Google Scholar 

  • Zapf, P. A., & Roesch, R. (2006). Competency to stand trial: A guide for evaluators. In A. K. Hess & I. B. Weiner (Eds.), Handbook of forensic psychology (3rd ed., pp. 305–331). New York: Wiley.

    Google Scholar 

  • Zapf, P. A., & Roesch, R. (2009). Evaluation of competence to stand trial. New York, NY: Oxford University Press.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Forensic Services, Adult Mental Health Division, State of Hawaii, Courts and Corrections Branch, 1700 Lanikila Ave, Honolulu, HI, 96817, USA

    W. Neil Gowensmith

  2. Institute of Law, Psychiatry, and Public Policy, University of Virginia, Charlottesville, VA, USA

    Daniel C. Murrie

  3. Sam Houston State University, Huntsville, TX, USA

    Marcus T. Boccaccini

Authors
  1. W. Neil Gowensmith
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Daniel C. Murrie
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marcus T. Boccaccini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to W. Neil Gowensmith.

About this article

Cite this article

Gowensmith, W.N., Murrie, D.C. & Boccaccini, M.T. Field Reliability of Competence to Stand Trial Opinions: How Often Do Evaluators Agree, and What Do Judges Decide When Evaluators Disagree?. Law Hum Behav (2010). https://doi.org/10.1007/s10979-010-9259-8

Download citation

  • Published:

  • DOI: https://doi.org/10.1007/s10979-010-9259-8

Keywords

Search

Navigation