Skip to main content

Does Interrater (Dis)agreement on Psychopathy Checklist Scores in Sexually Violent Predator Trials Suggest Partisan Allegiance in Forensic Evaluations?

Law and Human Behavior volume 32, pages 352–362 (2008)Cite this article

Abstract

Many studies reveal strong interrater agreement for Hare’s Psychopathy Checklist-Revised (PCL-R) when used by trained raters in research contexts. However, no systematic research has examined agreement between PCL-R scores from independent clinicians who are retained by opposing sides in adversarial legal proceedings. We reviewed all 43 sexual-offender civil-commitment trials in one state and identified 23 cases in which opposing evaluators reported PCL-R total scores for the same individual. Differences between scores from opposing evaluators were usually in a direction that supported the party who retained their services. These score differences were greater in size than would be expected based on the instrument’s standard error of measurement or the rater agreement values reported in previous PCL-R research. The intraclass correlation for absolute agreement for the PCL-R Total score from a single rater (ICC1,A = .39) was well below levels of agreement observed for the PCL-R in research contexts, and below published test-retest values for the PCL-R. Results raise concerns about the potential for a forensic evaluator’s “partisan allegiance” to influence PCL-R scores in adversarial proceedings.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  • Alterman, A. I., Cacciola, J. S., & Rutherford, M. J. (1993). Reliability of the Revised Psychopathy Checklist in substance abuse patients. Psychological Assessment, 5, 442–448.

    Article  Google Scholar 

  • Amenta, A. (2005). The assessment of sexual offenders for civil commitment proceedings: An analysis of report content. Unpublished doctoral dissertation. Sam Houston State University.

  • Archer, R. P., Buffington-Vollum, J. K., Stredny, R. V., & Handel, R. W. (2006). A survey of psychological test use patterns among forensic psychologists. Journal of Personality Assessment, 87, 84–94.

    Article  PubMed  Google Scholar 

  • Boccaccini, M. T., & Brodsky, S. L. (2002). Believability of expert and law witnesses: Implications for trial consultation. Professional Psychology: Research & Practice, 33, 384–388.

    Article  Google Scholar 

  • Borum, R., Otto, R., & Golding, S. (1993). Improving clinical judgment and decision making in forensic evaluation. Journal of Psychiatry & Law, 21, 35–76.

    Google Scholar 

  • Brodsky, S. L. (1991). Testifying in court: Guidelines and maxims for the expert witness. Washington, DC: American Psychological Association.

    Google Scholar 

  • Brennan, R. L. (2001). Generalizability theory. New York: Springer.

    Google Scholar 

  • Campbell, T. (2004). Assessing sex offenders: Problems and pitfalls. Springfield, IL: Thomas.

    Google Scholar 

  • Campbell, T. (2006).The validity of the Psychopathy Checklist-Revised in adversarial proceedings. Journal of Forensic Psychology Practice, 6, 43–53.

    Article  Google Scholar 

  • Committee on Ethical Guidelines for Forensic Psychologists. (1991). Specialty Guidelines for Forensic Psychologists. Law and Human Behavior, 15, 655–665.

    Google Scholar 

  • Cornell, D. G. (1987). Role conflict in forensic clinical psychology: A reply to Arcaya. Professional Psychology: Research & Practice, 18, 429–432.

    Article  Google Scholar 

  • DeMatteo, D., & Edens. J. F. (2006). The role and relevance of the Psychopathy Checklist-Revised in court: A case law survey of U.S. courts (1991–2004). Psychology, Public Policy, and Law, 12, 215–241.

    Article  Google Scholar 

  • Doren, D. M. (2002). Evaluating sex offenders: A manual for civil commitment and beyond. London: Sage Publications.

    Google Scholar 

  • Edens, J., Marcus, D., Lilienfeld, S., & Poythress, N. (2006). Psychopathic, not psychopath: Taxometric evidence for the dimensional structure of psychopathy. Journal of Abnormal Psychology, 115, 131–144.

    Article  PubMed  Google Scholar 

  • Gacono, C., & Hutton, H. (1994). Suggestions for the clinical and forensic use of the Hare Psychopathy Checklist–Revised (PCL-R). International Journal of Law and Psychiatry, 17, 303–317.

    Article  PubMed  Google Scholar 

  • Garb, H. N., & Boyle, P. (2003). Understanding why some clinicians use pseudoscientific methods: Findings from research on clinical judgment. In S.O. Lilienfeld, S.J. Lynn, & J. M. Lohr (Eds.), Science and pseudoscience in clinical psychology. (pp. 17–38). New York. Guilford.

    Google Scholar 

  • Grisso, T. (1998). Forensic evaluation of juveniles. Sarasota, Florida: Professional Resources Press.

    Google Scholar 

  • Hans, V. P. (1986). An analysis of public attitudes towards the insanity defense. Criminology, 24, 393–414.

    Article  Google Scholar 

  • Hare, R. D. (1991). The Hare Psychopathy Checklist Revised. Toronto, Ontario, Canada: Multi-Health Systems.

    Google Scholar 

  • Hare, R. D. (2003). The Hare Psychopathy Checklist Revised – Second Edition. Toronto, Ontario, Canada: Multi-Health Systems.

    Google Scholar 

  • Hemphill, J. F., Templeman, R., Wong, S., & Hare, R. D. (1998). Psychopathy and crime: Recidivism and criminal careers. In D. Cooke, A. Forth, & R. Hare, (Eds.), Psychopathy: Theory, research, and implications for society (pp. 375–398). Dordrecht, Netherlands: Kluwer Academic.

    Google Scholar 

  • Kroner, D. G., & Mills, J. F. (2001). The accuracy of five risk appraisal instruments in predicting institutional misconduct and newconvictions. Criminal Justice and Behavior, 28, 471–489.

    Article  Google Scholar 

  • Lally, S. J. (2003). What tests are acceptable for use in forensic evaluations? A survey of experts. Professional Psychology: Research and Practice, 5, 491–498.

    Article  Google Scholar 

  • Marcus, D. K., John, S., & Edens, J. F. (2004). A taxometric analysis of psychopathic personality. Journal of Abnormal Psychology, 113, 626–635.

    Article  PubMed  Google Scholar 

  • McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30–46.

    Article  Google Scholar 

  • Miller, H. A., Amenta, A. E., & Conroy, M. A. (2005). Sexually violent predator evaluations: Empirical evidence, strategies for professionals, and research directions. Law and Human Behavior, 29, 29– 54.

    Article  PubMed  Google Scholar 

  • Murrie, D. C., & Warren, J. I. (2005).Clinician variation in rates of legal sanity opinions: Implications for self-monitoring. Professional Psychology: Research and Practice, 36, 519–524.

    Article  Google Scholar 

  • Murrie, D.C., Marcus, D.K., Douglas, K.S., Salekin, R.T., Lee, Z., & Vincent, G., (in press). Youth with psychopathy features are not a discrete class: A taxometric analysis. Journal of Child Psychology and Psychiatry.

  • Otto, R. K. (1989). Bias and expert testimony of mental health professionals in adversarial proceedings: A preliminary investigation. Behavioral Sciences and the Law, 7, 267–273.

    Article  Google Scholar 

  • Otto, R. K. (2006). Discussion of the Forensic Specialty Guidelines. St. Petersburg, Florida: Panel presented at the annual meeting of the American Psychology-Law Society.

    Google Scholar 

  • Otto, R. K., & Heilbrun, K. (2002). The practice of forensic psychology: A look toward the future in light of the past. American Psychologist, 57, 5–18.

    Article  PubMed  Google Scholar 

  • Patrick C. (Ed.) (2006). Handbook of psychopathy. New York: Guilford.

    Google Scholar 

  • Porter, S., Woodworth, M., Earle, J., Drugge, J., & Boer, D. P. (2003). Characteristics of violent behavior exhibited during sexual homicides by psychopathic and non-psychopathic murderers. Law and Human Behavior, 27, 459–470.

    Article  PubMed  Google Scholar 

  • Rogers, R. (1987). Ethical dilemmas in forensic evaluations. Behavioral Sciences & the Law, 5, 149–160.

    Article  Google Scholar 

  • Rutherford, M., Cacciola, J. S., Alterman, A. I., McKay, J. R., & Cook, T. G. (1999). The 2–year test-retest reliability of the Psychopathy Checklist-Revised in methadone patients. Assessment, 6, 285–291.

    Article  PubMed  Google Scholar 

  • Salekin, R. T., Rogers, R., & Sewell, K. W. (1996). A review and meta-analysis of the psychopathy checklist and psychopathy checklist-revised: Predictive validity of dangerousness. Clinical Psychology: Science and Practice, 3, 203–215.

    Article  Google Scholar 

  • Schlank A. (Ed.) (2001). The sexual predator: Legal issues, clinical issues, special populations (2nd Ed). Kingston, NJ: Civic Research Institute, Inc.

    Google Scholar 

  • Shalverson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.

    Google Scholar 

  • Shuman, D. W., & Greenberg, S. A. (2003). The expert witness, the adversary system, and the voice of reason: Reconciling impartiality and advocacy. Professional Psychology: Research & Practice, 34, 219–224.

    Article  Google Scholar 

  • Silver, E., Cirincione, C., & Steadman, H. (1994). Demythologizing inaccurate perceptions of the insanity defense. Law and Human Behavior, 18, 63–70.

    Article  Google Scholar 

  • Texas Health & Safety Code § 841.000 – 841.150 (2000).

  • Walsh, T., & Walsh, Z. (2006). The evidentiary introduction of Psychopathy Checklist-Revised assessed psychopathy in U.S. courts: Extent and Appropriateness. Law and Human Behavior, 30, 493–507.

    Article  PubMed  Google Scholar 

  • Winick, B. J., & LaFond, J. Q. (2003). Protecting society from sexually dangerous offenders: Law, justice, and therapy. Washington, DC: American Psychological Association.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Sam Houston State University, Box 2447, Huntsville, TX, 77341, USA

    Daniel C. Murrie, Marcus T. Boccaccini, Jeremy T. Johnson & Chelsea Janke

Authors
  1. Daniel C. Murrie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marcus T. Boccaccini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeremy T. Johnson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chelsea Janke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marcus T. Boccaccini.

About this article

Cite this article

Murrie, D.C., Boccaccini, M.T., Johnson, J.T. et al. Does Interrater (Dis)agreement on Psychopathy Checklist Scores in Sexually Violent Predator Trials Suggest Partisan Allegiance in Forensic Evaluations?. Law Hum Behav 32, 352–362 (2008). https://doi.org/10.1007/s10979-007-9097-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10979-007-9097-5

Keywords

  • Psychopathy
  • PCL-R
  • Bias
  • Forensic evaluation
  • Sexually violent predator
  • Sex offender civil commitment