Abstract
In a series of articles in 2023, Leonhard has presented criticisms of extant approaches to the use and interpretation of performance validity tests (PVTs) in malingering determinations in neuropsychological assessments. This paper reviews the relevant details of the articles relevant commentary. In the first article, Leonard describes statistical issues. In the second, he moves on to methodological ones. His third article responds to commentaries on the first two articles. The fourth article gives implications for court, and whether standard approaches to the use of PVTs in malingering determinations meet expected scientific standards according to the SCOTUS decision of Daubert. Among the criticisms he raises, he queries: (a) whether PVTs are independent, in that they are too collinear; (b) classificatory algorithms do not use all relevant statistics; (c) the studies in the field do not typically present confidence intervals; (d) the studies on the validity of available PVTs are circular because they typically use PVTs to define malingering status; (e) classification accuracy statistics are overinflated by typically excluding indeterminate cases; and (f) validation studies reflect a high risk of bias, as demonstrated for one PVT reviewed. Both Leonhard and the present article call for more research on these questions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
None, no participants with gathered data.
References
Ali, S., Crişan, I., Abeare, C. A., & Erdodi, L. A. (2022). Cross-cultural performance validity testing: Managing false positives in examinees with limited English proficiency. Developmental Neuropsychology, 47(6), 273–294. https://doi.org/10.1080/87565641.2022.2105847
Baak, M., Koopman, R., Snoek, H., & Klous, S. (2020). A new correlation coefficient between categorical, ordinal and interval variables with Pearson characteristics. Computational Statistics & Data Analysis, 152, 107043. https://doi.org/10.1016/j.csda.2020.107043
Ben-Porath, Y. S., & Tellegen, A. (2020a). Minnesota Multiphasic Personality Inventory-3 (MMPI-3): Manual for administration, scoring, and interpretation. University of Minnesota Press.
Ben-Porath, Y. S., & Tellegen, A. (2020b). Minnesota Multiphasic Personality Inventory-3 (MMPI-3): Technical manual. University of Minnesota Press.
Berthelson, L., Mulchan, S. S., Odland, A. P., Miller, L. J., & Mittenberg, W. (2013). False positive diagnosis of malingering due to the use of multiple effort tests. Brain Injury, 27(7–8), 909–916. https://doi.org/10.3109/02699052.2013.793400
Boone, K. B. (2009). The need for continuous and comprehensive sampling of effort/ response bias during neuropsychological examinations. The Clinical Neuropsychologist, 23(4), 729–741. https://doi.org/10.1080/13854040802427803
Boone, K. B. (2013). Clinical Practice of Forensic Neuropsychology. Guilford Press.
Bossuyt, P. M. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., Altman, D. G., Hooft, L., Korevaar, D. A., & Cohen, J. F. (2015). STARD 2015 – An updated list of essential items for reporting diagnostic accuracy studies. BMJ - Clinical Research, 351, 1–9. https://doi.org/10.1136/bmj.h5527
Bush, S. S. (2023). Questioning what we thought we knew: Commentary on Leonhard’s performance validity assessment articles. Neuropsychology Review, 33(3), 624–627. https://doi.org/10.1007/s11065-023-09603-5
Crişan, I., May, N., Giromini, L., Roth, R. M., & Erdodi, L. A. (2023). Replicating the classification accuracy of the Verbal Paired Associates and Visual Reproduction recognition trials as embedded performance validity tests. Neuropsychology. https://psycnet.apa.org/doi/10.1037/neu0000934
Daubert v. Merrell Dow Pharmaceuticals, Inc. (1993). 509 U.S. 579.
Denning, J. (2021). When 10 is enough: Errors on the first 10 items of the Test of Memory Malingering (TOMMe10) and administration time predict freestanding performance validity tests (PVTs) and underperformance on memory measures. Applied Neuropsychology: Adult, 28(1), 35–47. https://doi.org/10.1080/23279095.2019.1588122
Denning, J. H. (2012). The efficiency and accuracy of the Test of Memory Malingering Trial 1, errors on the first 10 items of the Test of Memory Malingering, and five embedded measures in predicting invalid test performance. Archives of Clinical Neuropsychology, 27(4), 417–432. https://doi.org/10.1093/arclin/acs044
Dorociak, K. E., Schulze, E. T., Piper, L. E., Molokie, R. E., & Janecek, J. K. (2018). Performance validity testing in a clinical sample of adults with sickle cell disease. The Clinical Neuropsychologist, 32(1), 81–97. https://doi.org/10.1080/13854046.2017.1339830
Erdodi, L. A. (2023a). From “below chance” to “a single error is one too many”: Evaluating various thresholds for invalid performance on two forced choice recognition tests. Behavioral Sciences and the Law, 41, 445–462. https://doi.org/10.1002/bsl.2609
Erdodi, L. A. (2023b). Multivariate models of performance validity: The Erdodi Index captures the dual nature of non-credible responding (continuous and categorical). Assessment, 30(5), 1467–1485. https://doi.org/10.1177/10731911221101910
Erdodi, L. A. (2023c). Cutoff elasticity in multivariate models of performance validity assessment as a function of the number of components and aggregation method. Psychological Injury and Law, 16(4), 328–350. https://doi.org/10.1007/s12207-023-09490-4
Erdodi, L. A. (2024). Seeing clearly in the twilight: The clinical and forensic relevance of the indeterminate/ borderline range in multivariate models of performance validity assessment. Accepted for publication.
Faust, D. (2023). Invited Commentary: Advancing but not yet advanced: Assessment of effort/malingering in forensic and clinical settings. Neuropsychology Review, 33(3), 628–642. https://doi.org/10.1007/s11065-023-09605-3
Fischer, J. E., Bachmann, L. M., & Jaeschke, R. (2003). A readers’ guide to the interpretation of diagnostic test properties: Clinical example of sepsis. Intensive Care Medicine, 29(7), 1043–1051. https://doi.org/10.1007/s00134-003-1761-8
Giromini, L., & Erdodi, L. A. (2023). Assessing the credibility of clinical presentations using performance and symptom validity tests: Current trends and future directions—Part 2. Psychology & Neuroscience, 16(3), 217–224. https://doi.org/10.1037/pne0000325
Giromini, L., Young, G., & Sellbom, M. (2022). Assessing negative response bias using self-report measures: New articles, new issues. Psychological Injury and Law, 15(1), 1–21. https://doi.org/10.1007/s12207-022-09444-2
Green, P. (2003). Word Memory Test for Windows: User’s manual and program. Green’s Publishing.
Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6(3), 218–224. https://doi.org/10.1037/1040-3590.6.3.218
Greve, K. W., Bianchini, K. J., & Doane, B. M. (2006). Classification accuracy of the Test of Memory Malingering in traumatic brain injury: Results of a known-group analysis. Journal of Clinical and Experimental Neuropsychology, 28(7), 1176–1190. https://doi.org/10.1080/13803390500263550
Greve, K. W., Binder, L. M., & Bianchini, K. J. (2009). Rates of below-chance performance in forced choice symptom validity tests. The Clinical Neuropsychologist, 23, 534–544. https://doi.org/10.1080/13854040802232690
Holcomb, M. J., Roth, R. M., Tyson, B. T., & Erdodi, L. A. (2022). Critical item (CR) analysis expands the classification accuracy of performance validity tests based on the forced choice paradigm – Replicating previously introduced CR cutoffs within the Word Choice Test. Neuropsychology, 36(7), 683–694. https://doi.org/10.1037/neu0000834
Hood, E. D., Boone, K. B., Miora, D. S., Cottingham, M. E., Victor, T. L., Zeigler, E. A., Zeller, M. A., & Wright, M. J. (2022). Are there differences in performance validity test scores between African American and white American neuropsychology clinic patients? Journal of Experimental Neuropsychology, 44(1), 31–41. https://doi.org/10.1080/13803395.2022.2069230
Hromas, G., Rolin, S., & Davis, J. J. (2022). Racial differences in positive findings on embedded performance validity tests. Applied Neuropsychology: Adult. https://doi.org/10.1080/23279095.2022.2146504
Jewsbury, P. (2023). Inference with multiple tests: Commentary to Leonhard. Neuropsychology Review, 33(3), 643–652. https://doi.org/10.1007/s11065-023-09604-4
Larrabee, G. J. (2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17(3), 410–425. https://doi.org/10.1076/clin.17.3.410.18089
Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22(4), 666–679. https://doi.org/10.1080/13854040701494987
Larrabee, G. J. (2014). False-positive rates associated with the use of multiple performance and symptom validity tests. Archives of Clinical Neuropsychology, 29, 364–373.
Larrabee, G. J., Millis, S. R., & Meyers, J. E. (2009). 40 plus or minus 10, a new magical number: Reply to Russell. The Clinical Neuropsychologist, 23(5), 841–849. https://doi.org/10.1080/13854040902796735
Larrabee, G. J., Rohling, M. L., & Meyers, J. E. (2019). Use of multiple performance and symptom validity measures: Determining the optimal per test cutoff for determination of invalidity, analysis of skew, and inter-test correlations in valid and invalid performance groups. The Clinical Neuropsychologist, 33(8), 1354–1372.
Lawson, A. K., Fitzgerald, L. F., & Collinsworth, L. L. (2023). Computerized test interpretation of the MMPI-2 in the forensic context: A time to use your head and not the formula? Psychological Injury and Law, 16(2), 199–212. https://doi.org/10.1007/s12207-022-09465-x
Leonhard, C. (2023a). Review of statistical and methodological issues in the forensic prediction of malingering from validity tests: Part I: Statistical issues. Neuropsychology Review, 33(3), 581–603. https://doi.org/10.1007/s11065-023-09601-7
Leonhard, C. (2023b). Review of statistical and methodological issues in the forensic prediction of malingering from validity tests: Part II – Methodological issues. Neuropsychological Review, 33(3), 604–623. https://doi.org/10.1007/s11065-023-09602-6
Leonhard, C. (2023c). Quo Vadis forensic neuropsychological malingering determinations Reply to Drs. Bush, Faust, and Jewsbury. Neuropsychological Review, 33(3), 653–657. https://doi.org/10.1007/s11065-023-09606-2
Leonhard, C., & Leonhard, C. (2024). Neuropsychological malingering determinations: Science or fiction of lie detection? Georgia Law Review, 58(2), 483.
Lippa, S. M. (2018). Performance validity testing in neuropsychology: A clinical guide, critical review, and update on a rapidly evolving literature. The Clinical Neuropsychologist, 32(3), 391–421. https://doi.org/10.1080/13854046.2017.1406146
Marshall, P. S., & Schroeder, R. W. (2022). Validity assessment in patients with psychiatric disorders. In R. W. Schroeder & P. K. Martin (Eds.), Validity assessment in clinical neuropsychological practice: Evaluating and managing noncredible performance (pp. 319–337). Guilford Press.
Martin, P. K., & Schroeder, R. W. (2020). Base rates of invalid test performance across clinical non-forensic contexts and settings. Archives of Clinical Neuropsychology, 35(6), 717–725. https://doi.org/10.1093/arclin/acaa017
Martin, P. K., Schroeder, R. W., & Odland, A. P. (2015). Neuropsychologists’ validity testing beliefs and practices: A survey of North American professionals. The Clinical Neuropsychologist, 29(6), 741–776. https://doi.org/10.1080/13854046.2015.1087597
Merten, T., Dandachi-FitzGerald, B., Hall, V., Bodner, T., Giromini, L., Lehrner, J., González-Ordi, H., Santamaría, P., Schmand, B., & Di Stefano, G. (2022). Symptom and performance validity assessment in European countries: An update. Psychological Injury and Law, 15(2), 116–127. https://doi.org/10.1007/s12207-021-09436-8
Merten T., & Merckelbach, H. (2020). Factitious disorder and malingering. In J. R. Geddes, N. C. Andreasen, & G. M. Goodwin (Eds.), New Oxford textbook of psychiatry (pp. 1342–1349). Oxford University Press.
Rogers, R. (2018). An introduction to response styles. In R. Rogers, & S. D. Bender (Eds.), Clinical assessment of malingering and deception (4th ed., pp. 3–17). Guilford Press.
Rogers, R., & Bender, S. D. (2018). Clinical assessment of malingering and deception (4th ed.). Guilford Press.
Roor, J. J., Peters, M. J. V., Dandachi-FitzGerald, B., & Ponds, R. W. H. M. (2023). Performance validity test failure in the clinical population: A systematic review and meta-analysis of prevalence rates. Neuropsychology Review. https://doi.org/10.1007/s11065-023-09582-7
Schroeder, R. W., & Martin, P. K. (2022). Validity assessment in clinical settings: How it differs from forensic settings and why it is important. In R. W. Schroeder & P. K. Martin (Eds.), Validity assessment in clinical neuropsychological practice: Evaluating and managing noncredible performance (pp. 3–10). Guilford Press.
Sherman, E. M. S., Slick, D. J., & Iverson, G. L. (2020). Multidimensional malingering criteria for neuropsychological assessment: A 20-year update of the malingered neuropsychological dysfunction criteria. Archives of Clinical Neuropsychology, 35(6), 735–764. https://doi.org/10.1093/arclin/acaa019
Shura, R. D., Ord, A. S., & Worthen, M. D. (2022). Structured Inventory of Malingered Symptomatology: A psychometric review. Psychological Injury and Law, 15(1), 64–78. https://doi.org/10.1007/s12207-021-09432-y
Slick, D. J., Hopp, G., Strauss, E., & Thompson, G. B. (1997). Victoria Symptom Validity Test Version 1.0: Professional manual. Psychological Assessment Resources, Inc.
Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. Clinical Neuropsychologist, 13(4), 545–561. https://doi.org/10.1076/1385-4046(199911)13:04;1-Y;FT545
Soble, J. R. (2021). Future directions in performance validity assessment to optimize detection of invalid neuropsychological test performance: Special issue introduction. Psychological Injury and Law, 14(4), 227–231. https://doi.org/10.1007/s12207-021-09425-x
Soble, J. R., Cerny, B. M., Rhoads, T., DeBoer, A. B., Sharp, D. W., Ovsiew, G. P., Phillips, M. S., Pesanti, S. D., Jennette, K. J., & Resch, Z. J. (2023). Do demographic factors influence detection of invalid neuropsychological test performance using common performance validity tests? A Multisite Investigation. Neuropsychology, 37(2), 218–232. https://doi.org/10.1037/neu0000880
Sweet, J. J., Heilbronner, R. L., Morgan, J. E., Larrabee, G. J., Rohling, M. L., Boone, K. B., & Participants, C. (2021). American academy of clinical neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 35(6), 1053–1106. https://doi.org/10.1080/13854046.2021.1896036
Tombaugh, T. N. (1996). TOMM: The Test of Memory Malingering manual. Multi-Health Systems.
Tyson, B. T., Shahein, A., Abeare, C. A., Baker, S. D., Kent, K., Roth, R. M., & Erdodi, L. A. (2023). Replicating a meta-analysis: The search for the optimal Word Choice Test cutoff continues. Assessment, 30(8), 2476–2490. https://doi.org/10.1177/10731911221147043
Tyson, B. T., & Shahein, A. (2023). Combining accuracy scores with time cutoffs improves the specificity of the Word Choice Test. Psychology and Neuroscience, 16(3), 244–253. https://doi.org/10.1037/pne0000315
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leefang, M. M. G., Sterne, J. A. C., & Bossuyt, P. M. M. (2011). QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine, 155(8), 529–536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009
Whitman, M. R., & Ben-Porath, Y. S. (2024). Applying the Daubert factors to MMPI-3 based testimony. Psychological Injury and Law. https://doi.org/10.1007/s12207-023-09493-1
Wodushek, T. R., & Greher, M. R. (2017). Performance validity testing in neuropsychology: Methods for measurement development and maximizing diagnostic accuracy. Journal of Psychiatric Practice, 23(3), 214–220. https://doi.org/10.1097/PRA.0000000000000233
Young, G. (2014). Malingering, feigning, and response bias in psychiatric/psychological injury: Implications for practice and court. Springer Publishing. https://doi.org/10.1007/978-94-007-7899-3
Young, G. (2015). Malingering in forensic disability-related assessments: Prevalence ± 15%. Psychological Injury and Law, 8(3), 188–199. https://doi.org/10.1007/s12207-015-9232-4
Young, G. (2021). The call for aid (cry for help) in psychological injury and law: Reinterpretation, mechanisms, and a call for research. Psychological Injury and Law, 14(3), 185–200. https://doi.org/10.1007/s12207-021-09414-0
Young, G. (2022). Interpreting symptom validity test fails in forensic disability and related assessments: When the cry for help for one fail makes sense. Applied Neuropsychology: Adult. https://doi.org/10.1080/23279095.2022.2107929
Young, G., & Goodman-Delahunty, J. (2021). Revisiting Daubert: Judicial gatekeeping and expert ethics in court. Psychological Injury and Law, 14(4), 304–315. https://doi.org/10.1007/s12207-021-09428-8
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Human Subjects
None.
Conflict of Interest
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
There is no data collection and no funding of any sort. There are no conflicts of interest to disclose.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Young, G., Erdodi, L. Forensic Prediction of Malingering from Performance Validity Tests: Review of Leonhard (2023, a, b, c). Psychol. Inj. and Law (2024). https://doi.org/10.1007/s12207-024-09504-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s12207-024-09504-9