Abstract
Forensic neuropsychological examinations with determination of malingering have tremendous social, legal, and economic consequences. Thousands of studies have been published aimed at developing and validating methods to diagnose malingering in forensic settings, based largely on approximately 50 validity tests, including embedded and stand-alone performance validity tests. This is the first part of a two-part review. Part I explores three statistical issues related to the validation of validity tests as predictors of malingering, including (a) the need to report a complete set of classification accuracy statistics, (b) how to detect and handle collinearity among validity tests, and (c) how to assess the classification accuracy of algorithms for aggregating information from multiple validity tests. In the Part II companion paper, three closely related research methodological issues will be examined. Statistical issues are explored through conceptual analysis, statistical simulations, and through reanalysis of findings from prior validation studies. Findings suggest extant neuropsychological validity tests are collinear and contribute redundant information to the prediction of malingering among forensic examinees. Findings further suggest that existing diagnostic algorithms may miss diagnostic accuracy targets under most realistic conditions. The review makes several recommendations to address these concerns, including (a) reporting of full confusion table statistics with 95% confidence intervals in diagnostic trials, (b) the use of logistic regression, and (c) adoption of the consensus model on the “transparent reporting of multivariate prediction models for individual prognosis or diagnosis” (TRIPOD) in the malingering literature.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Availability of Data and Materials
No parts of this work have been presented elsewhere. However, in compliance with data sharing requirements, data and analysis presented in Fig. 4 and Table 3 have been publicly shared under the following Figshare DOIs: Fig. 4 Data and Analysis (chi square and rtet for all 2-validity-measure concordances), https://doi.org/10.6084/m9.figshare.14256470.v2; Table 3 Data and Analysis (Prediction of malingering from 7 validity measures), https://doi.org/10.6084/m9.figshare.14368727.v3.
Notes
The term “forensic” shall refer to any situation where there is an actual or potential legal question regarding the veracity of an examinee´s presentation such as when making determinations about the presence of sexual abuse, eligibility for disability benefits, fitness for fulfilling a particular role, such as being a parent or police officer, ability to stand trial, presence of neurocognitive and/or psychiatric conditions that may affect guilt, innocence, or sentencing of a criminal defendant, tort situations involving questions of the veracity of neurocognitive abilities and related conditions. Such actual or potential legal questions arise most frequently in medicolegal settings, but may arise also in clinical contexts (cf. Sherman et al., 2020, p. 9; Sweet et al., 2021, p. 1059).
In Martin et al. (2020, pp. 99–100; Table 5), 53 TOMM studies are listed with an additional two studies listed on p. 112 for a total of 55 studies. However, there is inconsistency in how studies from articles that report multiple studies are listed in Martin et al. (2020). For consistency, in Parts I and II of the present review, each separate study, whether reported in a publication with other studies or by itself, is counted separately.
The terms “positive predictive power” and “posterior probability” are sometimes considered conceptually distinct in that positive predictive power is an operating characteristic of a test in a given setting, computed as the ratio of true positive cases over the total number of all examinees with a positive diagnostic finding. In comparison, posterior probability is the probability a given examinee with a positive test result is actually a true positive case (cf. Fletcher et al., 2014, p. 118). Despite this conceptual distinction, the computation of both classification accuracy statistics is identical. In this article, these terms will be used in keeping with their conceptual distinction.
Knowledge of the base rate of a condition is required for the calculation of classification accuracy statistics. Statistical modelling and analyses of hypothetical simulation data in this review will be based on commonly accepted estimates of base rates from the malingering literature. Examination of whether these estimates are tenable considering the findings of the present review is beyond the scope of this review.
This quantity may also be referred to as the conditional correlation of \({PVT}_{1}\) and \({PVT}_{2}\) conditioned on \({\mathrm{M}}^{-}\).
This quantity may also be referred to as the conditional correlation of \({PVT}_{1}\) and \({PVT}_{2}\), conditioned on \({\mathrm{M}}^{+}\).
This quantity may also be referred to as the unconditional correlation of \(P{VT}_{1}\) and \({PVT}_{2}\), that is, the correlation is not conditioned on malingering status.
In the original work (Larrabee et al., 2019, Table 4, page 1362) average absolute skew among 11 validity tests is reported as − .942. This value is arithmetically impossible because the average of absolute values cannot be a negative quantity. Also, while said Table 4 reports skew for 16 validity tests, said average skew computation is based on a subset of only 11 validity tests with no rationale given for exclusion of the other 5 validity tests. Therefore, in the present analysis, the average skew from all 16 validity tests was recalculated based on all 16 skew values as reported in the original work. This is the value given in the text above (average absolute skew = 1.00).
An additional method for estimating posterior probabilities from multiple PVTs may be Markov Chain Monte Carlo algorithms that correctly implement multivariate Bayesian modelling (cf. Al-Khairullah & Al-Baldawi, 2021). When directly compared to frequentist approaches, such as logistic regression, such models have been shown to yield diagnostic accuracy comparable to logistic regression (e.g., Wang et al., 2014; Witteveen et al., 2018).
References
Aita, S. L., Borgogna, N. C., Aita, L. J., Ogden, M. L., & Hill, B. D. (2020). Comparison of clinical psychologist and physician beliefs and practices concerning malingering: Results from a mixed methods study. Psychological Injury and Law, 13(3), 246–260. https://doi.org/10.1007/s12207-020-09374-x
Al-Khairullah, N. A., & Al-Baldawi, T. H. K. (2021). Bayesian computational methods of the logistic regression model. Journal of Physics: Conference Series, 1804(1), 012073. https://doi.org/10.1088/1742-6596/1804/1/012073
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders: DSM-IV-TR. (4th ed., text revision.). American Psychiatric Association.
Armistead-Jehle, P., & Gervais, R. O. (2011). Sensitivity of the test of memory malingering and the Nonverbal Medical Symptom Validity Test: A replication study. Applied Neuropsychology, 18(4), 284–290. https://doi.org/10.1080/09084282.2011.595455
Ashendorf, L., O’Bryant, S., & McCaffrey, R. (2003). Specificity of malingering detection strategies in older adults using the CVLT and WCST. The Clinical Neuropsychologist, 17(2), 255–262. https://doi.org/10.1076/clin.17.2.255.16502
Ashendorf, L., Constantinou, M., & McCaffrey, R. J. (2004). The effect of depression and anxiety on the TOMM in community-dwelling older adults. Archives of Clinical Neuropsychology, 19(1), 125–130. https://doi.org/10.1016/S0887-6177(02)00218-4
Baak, M., Koopman, R., Snoek, H., & Klous, S. (2020). A new correlation coefficient between categorical, ordinal and interval variables with Pearson characteristics. Computational Statistics & Data Analysis, 152, 107043. https://doi.org/10.1016/j.csda.2020.107043
Babikian, T., Boone, K. B., Lu, P., & Arnold, G. (2006). Sensitivity and specificity of various digit span scores in the detection of suspect effort. The Clinical Neuropsychologist, 20(1), 145–159. https://doi.org/10.1080/13854040590947362
Bashem, J. R., Rapport, L. J., Miller, J. B., Hanks, R. A., Axelrod, B. N., & Millis, S. R. (2014). Comparisons of five performance validity indices in bona fide and simulated traumatic brain injury. The Clinical Neuropsychologist, 28(5), 851–875. https://doi.org/10.1080/13854046.2014.927927
Bayman, E. O., & Dexter, F. (2021). Multicollinearity in logistic regression models. Anesthesia and Analgesia, 133(2), 362–365. https://doi.org/10.1213/ANE.0000000000005593
Becker, M. P., & Clogg, C. C. (1988). A note on approximating correlations from odds ratios. Sociological Methods and Research, 16(3), 40–424. https://doi.org/10.1177/0049124188016003003
Benitez-Silva, H., Buchinsky, M., & Rust, J. (2004). How large are the classification errors in the social security disability award process? (No. w10219; p. w10219). National Bureau of Economic Research. https://doi.org/10.3386/w10219
Berthelson, L., Mulchan, S. S., Odland, A. P., Miller, L. J., & Mittenberg, W. (2013). False positive diagnosis of malingering due to the use of multiple effort tests. Brain Injury, 27(7–8), 909–916. https://doi.org/10.3109/02699052.2013.793400
Bianchini, K. J., Greve, K. W., & Glynn, G. (2005). On the diagnosis of malingered pain-related disability: Lessons from cognitive malingering research. The Spine Journal, 5(4), 404–417. https://doi.org/10.1016/j.spinee.2004.11.016
Binder, L. M. (1993). Portland Digit Recognition Test manual (second). L. M. Binder.
Binder, L. M., Larrabee, G. J., & Millis, S. R. (2014). Intent to fail: Significance testing of forced choice test results. The Clinical Neuropsychologist, 28(8), 1366–1375. https://doi.org/10.1080/13854046.2014.978383
Binder, L. M., & Willis, S. C. (1991). Assessment of motivation after financially compensable minor head trauma. Psychological Assessment /, 3(2), 175–181. https://doi.org/10.1037/1040-3590.3.2.175
Bonett, D. G. (2007). Transforming odds ratios into correlations for meta-analytic research. American Psychologist, 62(3), 254–255. https://doi.org/10.1037/0003-066X.62.3.254
Bossuyt, P. M. M., Davenport, C., Deeks, J. J., Hyde, C., Leeflang, M. M. G., & Scholten, R. (2013). Chapter 11 Interpreting results and drawing conclusions. In J. J. Deeks, P. M. M. Bossuyt, & C. A. Gatsonis (Eds.), Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.9. The Cochrane Collaboration.
Buddin, W. H., Schroeder, R. W., Hargrave, D. D., Von Dran, E. J., Campbell, B., & C. J., Heinrichs, R. J., & Baade, L. E. (2014). An examination of the frequency of invalid forgetting on the Test of Memory Malingering. The Clinical Neuropsychologist, 28(3), 525–542. https://doi.org/10.1080/13854046.2014.906658
Bush, S. S., Ruff, R. M., Tröster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., Reynolds, C. R., & Silver, C. H. (2005). Symptom validity assessment: Practice issues and medical necessity: NAN Policy & Planning Committee. Archives of Clinical Neuropsychology, 20(4), 419–426. https://doi.org/10.1016/j.acn.2005.02.002
Chafetz, M. D. (2020). Deception is different: Negative validity test findings do not provide “evidence” for “good effort.” The Clinical Neuropsychologist, 1–37. https://doi.org/10.1080/13854046.2020.1840633
Chafetz, M. D. (2008). Malingering on the social security disability consultative exam: Predictors and base rates. The Clinical Neuropsychologist, 22(3), 529–546. https://doi.org/10.1080/13854040701346104
Chafetz, M. D. (2011). Reducing the probability of false positives in malingering detection of social security disability claimants. The Clinical Neuropsychologist, 25(7), 1239–1252. https://doi.org/10.1080/13854046.2011.586785
Chafetz, M. D., Bauer, R. M., & Haley, P. S. (2020). The other face of illness-deception: Diagnostic criteria for factitious disorder with proposed standards for clinical practice and research. The Clinical Neuropsychologist, 34(3), 454–476. https://doi.org/10.1080/13854046.2019.1663265
Chafetz, M. D., & Underhill, J. (2013). Estimated costs of malingered disability. Archives of Clinical Neuropsychology, 28(7), 633–639. https://doi.org/10.1093/arclin/act038
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015). Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD statement. Journal of Clinical Epidemiology, 68(2), 112–121. https://doi.org/10.1016/j.jclinepi.2014.11.010
Collins, J., & Huynh, M. (2014). Estimation of diagnostic test accuracy without full verification: A review of latent class methods. Statistics in Medicine, 33(24), 4141. https://doi.org/10.1002/sim.6218
Davenport, E. C., & El-Sanhurry, N. A. (1991). Phi/Phimax: Review and synthesis. Educational and Psychological Measurement, 51(4), 821–828. https://doi.org/10.1177/001316449105100403
Davis, J. J., Wall, J. R., & Whitney, K. A. (2012). Derivation and clinical validation of consistency indices on the test of memory malingering. Archives of Clinical Neuropsychology, 27(7), 706–715. https://doi.org/10.1093/arclin/acs078
de Rooij, M., & Weeda, W. (2020). Cross-validation: A method every psychologist should know. Advances in Methods and Practices in Psychological Science, 3(2), 248–263. https://doi.org/10.1177/2515245919898466
Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. (2000). The California Verbal Learning Test-Second Edition. The Psychological Corporation.
Digby, P. G. N. (1983). Approximating the tetrachoric correlation coefficient. Biometrics, 39(3), 753–757. https://doi.org/10.2307/2531104
Edens, J. F., Truong, T. N., & Otto, R. K. (2020). Classification accuracy of the rare symptoms and symptom combinations scales of the Structured Inventory of Malingered Symptomatology in three archival samples. Law and Human Behavior, 44(2), 167–177. https://doi.org/10.1037/lhb0000361
Ekström, J. (2011). The Phi-coefficient, the tetrachoric correlation coefficient, and the Pearson-Yule debate. UCLA Department of Statistics Papers. https://escholarship.org/uc/item/7qp4604r
Erdodi, L. A., & Rai, J. K. (2017). A single error is one too many: Examining alternative cutoffs on Trial 2 of the TOMM. Brain Injury, 31(10), 1362–1368. https://doi.org/10.1080/02699052.2017.1332386
Fischer, J. E., Bachmann, L. M., & Jaeschke, R. (2003). A readers’ guide to the interpretation of diagnostic test properties: Clinical example of sepsis. Intensive Care Medicine, 29(7), 1043–1051. https://doi.org/10.1007/s00134-003-1761-8
Fletcher, R. H., Fletcher, S. W., & Fletcher, G. S. (2014). Clinical epidemiology: The essentials: Vol. 5th edition. LWW.
Gates, K. M., Henry, T., Steinley, D., & Fair, D. A. (2016). A Monte Carlo evaluation of weighted community detection algorithms. Frontiers in Neuroinformatics, 10, 45. https://doi.org/10.3389/fninf.2016.00045
Glas, A. S., Lijmer, J. G., Prins, M. H., Bonsel, G. J., & Bossuyt, P. M. M. (2003). The diagnostic odds ratio: A single indicator of test performance. Journal of Clinical Epidemiology, 56(11), 1129–1135. https://doi.org/10.1016/S0895-4356(03)00177-X
Green, D., Rosenfeld, B., Belfi, B., Rohlehr, L., & Pierson, A. (2012). Use of measures of cognitive effort and feigned psychiatric symptoms with pretrial forensic psychiatric patients. International Journal of Forensic Mental Health, 11(3), 181–190. https://doi.org/10.1080/14999013.2012.723665
Green, P. W. (2003). Green’s word memory test for windows: User’s manual. Green’s Publishing.
Green, P. W. (2004). Green’s medical symptom validity test (MSVT) for microsoft windows: User’s manual. Green’s Publishing.
Green, P. W. (2011). Comparison between the Test of Memory Malingering (TOMM) and the Nonverbal Medical Symptom Validity Test (NV-MSVT) in adults with disability claims. Applied Neuropsychology, 18(1), 18–26. https://doi.org/10.1080/09084282.2010.523365
Green, P. W., Allen, L. M., & Astner, K. (1996). Manual for the Word Memory Test. Cognisyst N.C.
Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6(3), 218–224. https://doi.org/10.1037//1040-3590.6.3.218
Greiffenstein, M. F., Greve, K. W., Bianchini, K. J., & Baker, W. J. (2008). Test of Memory Malingering and Word Memory Test: A new comparison of failure concordance rates. Archives of Clinical Neuropsychology, 23(7), 801–807. https://doi.org/10.1016/j.acn.2008.07.005
Greve, K. W., Bianchini, K. J., Black, F. W., Heinly, M. T., Love, J. M., Swift, D. A., & Ciota, M. (2006a). Classification accuracy of the Test of Memory Malingering in persons reporting exposure to environmental and industrial toxins: Results of a known-groups analysis. Archives of Clinical Neuropsychology, 21(5), 439–448. https://doi.org/10.1016/j.acn.2006.06.004
Greve, K. W., Bianchini, K. J., & Doane, B. M. (2006b). Classification accuracy of the test of memory malingering in traumatic brain injury: Results of a known-groups analysis. Journal of Clinical and Experimental Neuropsychology, 28(7), 1176–1190. https://doi.org/10.1080/13803390500263550
Grimes, D. A., & Schulz, K. F. (2005). Epidemiology 3: Refining clinical diagnosis with likelihood ratios. The Lancet, 365(9469), 1500–1505. https://doi.org/10.1016/S0140-6736(05)66422-7
Hand, D. J., & Yu, K. (2001). Idiot’s Bayes: Not so stupid after all? International Statistical Review / Revue Internationale De Statistique, 69(3), 385–398. https://doi.org/10.2307/1403452
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants1, C. (2009). American Academy of Clinical Neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129. https://doi.org/10.1080/13854040903155063
Henry, G. K., Heilbronner, R. L., Suhr, G., & J., Wagner, E., & Drane, D. L. (2018). Illness perceptions predict cognitive performance validity. Journal of the International Neuropsychological Society : JINS, 24(7), 735–745. https://doi.org/10.1017/S1355617718000218
Jones, A. (2013). Test of memory malingering: Cutoff scores for psychometrically defined malingering groups in a military sample. The Clinical Neuropsychologist, 27(6), 1043–1059. https://doi.org/10.1080/13854046.2013.804949
Kaltenhauser, J., & Lee, Y. (1976). Correlation coefficients for binary data in factor analysis. Geographical Analysis, 8(3), 305–313. https://doi.org/10.1111/j.1538-4632.1976.tb00538.x
Lange, R. T., & Lippa, S. M. (2017). Sensitivity and specificity should never be interpreted in isolation without consideration of other clinical utility metrics. The Clinical Neuropsychologist, 31(6–7), 1015–1028. https://doi.org/10.1080/13854046.2017.1335438
Larrabee, G. J. (2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17(3), 410–425. https://doi.org/10.1076/clin.17.3.410.18089
Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22(4), 666–679. https://doi.org/10.1080/13854040701494987
Larrabee, G. J. (2014). False-positive rates associated with the use of multiple performance and symptom validity tests. Archives of Clinical Neuropsychology, 29(4), 364–373. https://doi.org/10.1093/arclin/acu019
Larrabee, G. J., Greiffenstein, M. F., Greve, K. W., & Bianchini, K. J. (2007). Redefining diagnostic criteria for malingering. In G. J. Larrabee (Ed.), Assessment of malingered neuropsychological deficits. Oxford University Press.
Larrabee, G. J., Millis, S. R., & Meyers, J. E. (2009). 40 Plus or minus 10, a new magical number: Reply to Russell. The Clinical Neuropsychologist, 23(5), 841–849. https://doi.org/10.1080/13854040902796735
Larrabee, G. J., Rohling, M. L., & Meyers, J. E. (2019). Use of multiple performance and symptom validity measures: Determining the optimal per test cutoff for determination of invalidity, analysis of skew, and inter-test correlations in valid and invalid performance groups. The Clinical Neuropsychologist, 33(8), 1354–1372. https://doi.org/10.1080/13854046.2019.1614227
Lawson, R. (2004). Small sample confidence intervals for the odds ratio. Communications in Statistics - Simulation and Computation, 33(4), 1095–1113. https://doi.org/10.1081/SAC-200040691
Lippa, S. M. (2018). Performance validity testing in neuropsychology: A clinical guide, critical review, and update on a rapidly evolving literature. The Clinical Neuropsychologist, 32(3), 391–421. https://doi.org/10.1080/13854046.2017.1406146
Macek-Jilkova, Z., Malov, S. I., Kurma, K., Charrat, C., Decaens, T., Peretolchina, N. P., Marche, P. N., Malov, I. V., & Yushchuk, N. D. (2021). Clinical and experimental evaluation of diagnostic significance of alpha-fetoprotein and osteopontin at the early stage of hepatocellular cancer. Bulletin of Experimental Biology and Medicine, 170(3), 340–344. https://doi.org/10.1007/s10517-021-05063-0
Martin, P. K., Schroeder, R. W., & Odland, A. P. (2015). Neuropsychologists’ validity testing beliefs and practices: A survey of North American professionals. The Clinical Neuropsychologist, 29(6), 741–776. https://doi.org/10.1080/13854046.2015.1087597
Martin, P. K., Schroeder, R. W., Olsen, D. H., Maloy, H., Boettcher, A., Ernst, N., & Okut, H. (2020). A systematic review and meta-analysis of the Test of Memory Malingering in adults: Two decades of deception detection. The Clinical Neuropsychologist, 34(1), 88–119. https://doi.org/10.1080/13854046.2019.1637027
Meyers, J. E., Miller, R. M., Thompson, L. M., Scalese, A. M., Allred, B. C., Rupp, Z. W., Dupaix, Z. P., & Junghyun Lee, A. (2014). Using likelihood ratios to detect invalid performance with performance validity measures. Archives of Clinical Neuropsychology, 29(3), 224–235. https://doi.org/10.1093/arclin/acu001
Meyers, J. E., & Volbrecht, M. E. (2003). A validation of multiple malingering detection methods in a large clinical sample. Archives of Clinical Neuropsychology, 18(3), 261–276. https://doi.org/10.1016/S0887-6177(02)00136-1
Midi, H., Sarkar, S. K., & Rana, S. (2010). Collinearity diagnostics of binary logistic regression model. Journal of Interdisciplinary Mathematics, 13(3), 253–267. https://doi.org/10.1080/09720502.2010.10700699
Millis, S. R., & Kler, S. (1995). Limitations of the Rey Fifteen-Item test in the detection of malingering. The Clinical Neuropsychologist, 9(3), 241–244. https://doi.org/10.1080/13854049508400486
Mittenberg, W., Patton, C., Canyock, E. M., & Condit, D. C. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical & Experimental Neuropsychology, 24(8), 1094. https://doi.org/10.1076/jcen.24.8.1094.8379
Moons, K. G. M., de Groot, J. A. H., Linnet, K., Reitsma, J. B., & Bossuyt, P. M. M. (2012). Quantifying the added value of a diagnostic test or marker. Clinical Chemistry, 58(10), 1408–1417. https://doi.org/10.1373/clinchem.2012.182550
Myers, W. C., Hall, R., Marshall, R., Tolou-Shams, M., & Wooten, K. (2016). Frequency and detection of malingering in homicide defendants undergoing criminal responsibility evaluations using the schedule for nonadaptive and adaptive personality: A feasibility study. SAGE Open, 6(2), 215824401663813. https://doi.org/10.1177/2158244016638131
Ostrowski, T. R., & Ostrowski, T. (2020). The basic four measures and their derivates in dichotomous diagnostic tests. International Journal of Clinical Biostatistics and Biometrics, 6(1). https://doi.org/10.23937/2469-5831/1510026
Oudman, E., Krooshof, E., van Oort, R., Lloyd, B., Wijnia, J. W., & Postma, A. (2020). Effects of Korsakoff Amnesia on performance and symptom validity testing. Applied Neuropsychology:ADult, 27(6), 549–557. https://doi.org/10.1080/23279095.2019.1576180
Pearson. (2009). Advanced clinical solutions for WAIS®-IV and WMW®-IV: Clinical and interpretive manual. Pearson.
Pepe, M. S. (2004). The statistical evaluation of medical tests for classification and prediction. Oxford University Press.
Preacher, K. J. (2001). Calculation for the chi-square test: An interactive calculation tool for chi-square tests of goodness of fit and independence [Computer software]. http://www.quantpsy.org/chisq/chisq.htm
Proeve, M. (2009). Issues in the application of Bayes’ theorem to child abuse decision making. Child Maltreatment, 14(1), 114–120. https://doi.org/10.1177/1077559508318395
Rees, L. M., Tombaugh, T. N., Gansler, D. A., & Moczynski, N. P. (1998). Five validation experiments of the test of memory malingering (TOMM). Psychological Assessment, 10(1), 10–20. https://doi.org/10.1037/1040-3590.10.1.10
Rey, André (1964). L’ examen clinique en psychologie (2. éd.). Presses universitaires de France.
Rondinelli, R. D., Genovese, E., Brigham, C. R., & American Medical Association. (2008). Guides to the evaluation of permanent impairment. (6th ed.). American Medical Association.
Schuerger, J. M., & Witt, A. C. (1989). The temporal stability of individually tested intelligence. Journal of Clinical Psychology, 45(2), 294–302. https://doi.org/10.1002/1097-4679(198903)45:2%3c294::AID-JCLP2270450218%3e3.0.CO;2-N
Schwartz, E. S., Erdodi, L., Rodriguez, N., Ghosh, J. J., Curtain, J. R., Flashman, L. A., & Roth, R. M. (2016). CVLT-II forced choice recognition trial as an embedded validity indicator: A systematic review of the evidence. Journal of the International Neuropsychological Society, 22(8), 851–858. https://doi.org/10.1017/S1355617716000746
Sci Stat. (n.d.). Odd Ratio Calculator. Retrieved September 23, 2020, from https://www.scistat.com/statisticaltests/odds_ratio.php
Shandera, A. L., Berry, D. T. R., Clark, J. A., Schipper, L. J., Graue, L. O., & Harp, J. P. (2010). Detection of malingered mental retardation. Psychological Assessment, 22(1), 50–56. https://doi.org/10.1037/a0016585
Sherman, E. M. S., Slick, D. J., & Iverson, G. L. (2020). Multidimensional malingering criteria for neuropsychological assessment: A 20-year update of the malingered neuropsychological dysfunction criteria. Archives of Clinical Neuropsychology, acaa019. https://doi.org/10.1093/arclin/acaa019
Silva, H., & Bernardino, J. (2022). Machine learning algorithms: An experimental evaluation for decision support systems. Algorithms, 15(4), 130–154. https://doi.org/10.3390/a15040130
Slick, D. J., & Sherman, E. M. S. (2012). Differential diagnosis of malingering and related clinical presentations. In E. M. S. Sherman & Brooks, B. L. (Eds.), Pediatric Forensic Neuropsychology (pp. 113–135).
Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545–561. https://doi.org/10.1076/1385-4046(199911)13:04;1-Y;FT545
Steyerberg, E. W. (2009). Clinical prediction models: A practical approach to development, validation, and updating. Springer.
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, Jd. F. (2001). Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis. Journal of Clinical Epidemiology, 54(8), 774–781. https://doi.org/10.1016/S0895-4356(01)00341-9
Sweet, J. J. (2009). Appendix B: Neuropsychological and psychological measures used to identify insufficient effort and malingering: A cross-referenced bibliography. In J. E. Morgan & J. J. Sweet (Eds.), Neuropsychology of malingering casebook (pp. 586–608). Psychology Press.
Sweet, J. J., Heilbronner, R. L., Morgan, J. E., Larrabee, G. J., Rohling, M. L., Boone, K. B., Kirkwood, M. W., Schroeder, R. W., & Suhr, J. A. (2021). American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 35(6), 1053–1106. https://doi.org/10.1080/13854046.2021.1896036
Tabachnick, B. G., Fidell, L. S., & Ullman, J. B. (2019). Using multivariate statistics (Seventh edition.). Pearson.
The Chinese University of Hong Kong. (n.d.). C.I. Calculator: Diagnostic Statistics. C.I. Calculator: Diagnostic Statistics. https://www2.ccrb.cuhk.edu.hk/stat/confidence%20interval/Diagnostic%20Statistic.htm#Formula
Tombaugh, T. N. (1996). Test of memory malingering: TOMM. Multi-Health Systems.
Underwood, J., De Francesco, D., Leech, R., Sabin, C. A., & Winston, A. (2018). Medicalising normality? Using a simulated dataset to assess the performance of different diagnostic criteria of HIV-associated cognitive impairment. PLoS ONE, 13(4), e0194760–e0194760. https://doi.org/10.1371/journal.pone.0194760
Victor, T. L., Boone, K. B., Serpa, K. G., Buehler, J., & Ziegler, E. A. (2009). Interpreting the meaning of multiple symptom validity test failure. The Clinical Neuropsychologist, 23(2), 297–313. https://doi.org/10.1080/13854040802232682
Wang, K.-J., Makond, B., & Wang, K.-M. (2014). Modeling and predicting the occurrence of brain metastasis from lung cancer by Bayesian network: A case study of Taiwan. Computers in Biology and Medicine, 47, 147–160. https://doi.org/10.1016/j.compbiomed.2014.02.002
Webb, G. I. (2017). Algorithm Evaluation. In C. Sammut & G. I. Webb (Eds.), Encyclopedia of Machine Learning (2nd ed., pp. 40–41). Springer US. https://doi.org/10.1007/978-0-387-30164-8_18
Witteveen, A., Nane, G. F., Vliegen, I. M. H., Siesling, S., & IJzerman, M. J. (2018). Comparison of logistic regression and Bayesian networks for risk prediction of breast cancer recurrence. Medical Decision Making, 38(7), 822–833. https://doi.org/10.1177/0272989X18790963
Wolfe, P. L., Millis, S. R., Hanks, R., Fichtenberg, N., Larrabee, G. J., & Sweet, J. J. (2010). Effort Indicators within the California Verbal Learning Test-II (CVLT-II). The Clinical Neuropsychologist, 24(1), 153–168. https://doi.org/10.1080/13854040903107791
Zadora, G., Martyna, A., Ramos, D., & Aitken, C. (2014). Statistical analysis in forensic science: Evidential value of multivariate physicochemical data. John Wiley & Sons Inc.
Zhou, X.-Hua., McClish, D. K., & Obuchowski, N. A. (2011). Statistical methods in diagnostic medicine (2nd ed.). Wiley.
Author information
Authors and Affiliations
Contributions
Not applicable because this manuscript has only a single author.
Corresponding author
Ethics declarations
Ethics Approval
Not applicable because this is a review and not a study where human or animal data were collected.
Competing Interests
The author declares no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Leonhard, C. Review of Statistical and Methodological Issues in the Forensic Prediction of Malingering from Validity Tests: Part I: Statistical Issues. Neuropsychol Rev 33, 581–603 (2023). https://doi.org/10.1007/s11065-023-09601-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11065-023-09601-7