Abstract
Forensic neuropsychological examinations to detect malingering in patients with neurocognitive, physical, and psychological dysfunction have tremendous social, legal, and economic importance. Thousands of studies have been published to develop and validate methods to forensically detect malingering based largely on approximately 50 validity tests, including embedded and stand-alone performance and symptom validity tests. This is Part II of a two-part review of statistical and methodological issues in the forensic prediction of malingering based on validity tests. The Part I companion paper explored key statistical issues. Part II examines related methodological issues through conceptual analysis, statistical simulations, and reanalysis of findings from prior validity test validation studies. Methodological issues examined include the distinction between analog simulation and forensic studies, the effect of excluding too-close-to-call (TCTC) cases from analyses, the distinction between criterion-related and construct validation studies, and the application of the Revised Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2) in all Test of Memory Malingering (TOMM) validation studies published within approximately the first 20 years following its initial publication to assess risk of bias. Findings include that analog studies are commonly confused for forensic validation studies, and that construct validation studies are routinely presented as if they were criterion-reference validation studies. After accounting for the exclusion of TCTC cases, actual classification accuracy was found to be well below claimed levels. QUADAS-2 results revealed that extant TOMM validation studies all had a high risk of bias, with not a single TOMM validation study with low risk of bias. Recommendations include adoption of well-established guidelines from the biomedical diagnostics literature for good quality criterion-referenced validation studies and examination of implications for malingering determination practices. Design of future studies may hinge on the availability of an incontrovertible reference standard of the malingering status of examinees.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
The term “forensic” shall refer to any situation where there is an actual or potential legal question regarding the veracity of an examinee’s presentation, such as when making decisions about the presence of sexual abuse, eligibility for disability benefits, fitness for fulfilling a particular role such as being a parent or police officer, ability to stand trial, presence of neurocognitive or psychiatric conditions that may affect guilt, innocence, or sentencing of a criminal defendant, tort situations involving questions of the veracity of neurocognitive abilities and related conditions. Such actual or potential legal questions arise most frequently in medicolegal settings, but may arise also in clinical contexts (cf. Sherman et al., 2020, p. 9; Sweet et al., 2021, p. 1059).
Throughout both parts of the review, the following naming conventions are used. Presence of malingering is denoted \({M}^{+}\) and the absence of malingering is denoted \({M}^{-}\). A positive finding on a PVT indicative of \({M}^{+}\) is denoted \({PVT}^{+}\), while a negative finding on a PVT indicative of \({M}^{-}\) is denoted \({PVT}^{-}\). Note that in prior writing (e.g., Chafetz, 2011; Larrabee, 2008), the wording “failure on” or “failing” a PVT is sometimes used to denote a positive finding on a PVT \({(PVT}^{+})\). “Passing a PVT” is sometimes used to denote a negative finding on a PVT \({(PVT}^{-})\) indicative of credible responding that would not lead to a determination of malingering \({(M}^{-})\). In keeping with the terminology from the medical diagnostics literature that a positive result on a test indicates presence of the attribute tested, these papers use the term “positive finding on a PVT” (\({PVT}^{+}\)) to indicate “failing” a PVT. “Negative finding on a PVT” (\(P{VT}^{-}\)) will indicate “passing” the PVT. Symbolically, this will be denoted by \({PVT}^{+}\) and \({PVT}^{-}\). Implications of these naming conventions are discussed in more detail in Part I.
A “confusion table” or “confusion matrix” is a 2 × 2 diagnostic classification table (see Figs. 1 and 2 in Part I). It is a special case of the 2 × 2 contingency table often used in psychological research to show the association of two binary variables. In a confusion table, columns reflect the criterion values (here, \({M}^{+}\) and \({M}^{-}\)), while rows reflect the predictor values (here, \({PVT}^{+}\) and \({PVT}^{-}\)).
In Martin et al. (2020, their Table 5, pp. 99–100), 53 TOMM studies are listed with additional two studies listed on p. 112 for a total of 55 studies. However, there is inconsistency in how studies from articles that report multiple studies are listed in Martin et al. (2020). For consistency, in the present review, each separate study, whether reported in a publication with other studies or by itself, is counted separately. There are also two studies among the original list not discussed in this review and one not in the original list that is discussed: Instead of Ashendorf et al. (2003), Ashendorf et al. (2004) was included because the former presents no TOMM results while the latter does. It was therefore assumed Ashendorf et al. (2003) had been mistakenly cited in Martin et al. (2020). Another study (Greiffenstein et al., 2008) was excluded because, upon examination, data related to TOMM validation referenced in Martin et al. (2020, p. 99) could not be found.
Notably, the call to use STARD was not repeated in the updated 2021 AACN consensus statement (Sweet et al., 2021).
References
Agency for Healthcare Research and Quality. (2012). Methods guide for medical test reviews. AHRQ Publication No. 12-EC017. Rockville, MD: Agency for Healthcare Research and Quality. https://effectivehealthcare.ahrq.gov/reports/final.cfm
American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (U.S.). (2014). Standards for educational and psychological testing.
American Psychological Association. (2017). Ethical principles of psychologists and code of conduct (2002, amended effective June 1, 2010, and January 1, 2017). Retrieved 9/11/2021 from. https://www.apa.org/ethics/code/
An, K. Y., Kaploun, K., Erdodi, L. A., & Abeare, C. A. (2017). Performance validity in undergraduate research participants: A comparison of failure rates across tests and cutoffs. The Clinical Neuropsychologist, 31(1), 193–206. https://doi.org/10.1080/13854046.2016.1217046
An, K. Y., Zakzanis, K. K., & Joordens, S. (2012). Conducting research with non-clinical healthy undergraduates: Does effort play a role in neuropsychological test performance? Archives of Clinical Neuropsychology, 27(8), 849–857. https://doi.org/10.1093/arclin/acs085
Armistead-Jehle, P., & Gervais, R. O. (2011). Sensitivity of the test of memory malingering and the Nonverbal Medical Symptom Validity Test: A replication study. Applied Neuropsychology, 18(4), 284–290. https://doi.org/10.1080/09084282.2011.595455
Ashendorf, L., Constantinou, M., & McCaffrey, R. J. (2004). The effect of depression and anxiety on the TOMM in community-dwelling older adults. Archives of Clinical Neuropsychology, 19(1), 125–130. https://doi.org/10.1016/S0887-6177(02)00218-4
Ashendorf, L., O’Bryant, S., & McCaffrey, R. (2003). Specificity of malingering detection strategies in older adults using the CVLT and WCST. The Clinical Neuropsychologist, 17(2), 255–262. https://doi.org/10.1076/clin.17.2.255.16502
Barhon, L. I., Batchelor, J., Meares, S., Chekaluk, E., & Shores, E. A. (2015). A comparison of the degree of effort involved in the TOMM and the ACS word choice test using a dual-task paradigm. Applied Neuropsychology: Adult, 22(2), 114–123. https://doi.org/10.1080/23279095.2013.863775
Bashem, J. R., Rapport, L. J., Miller, J. B., Hanks, R. A., Axelrod, B. N., & Millis, S. R. (2014). Comparisons of five performance validity indices in bona fide and simulated traumatic brain injury. The Clinical Neuropsychologist, 28(5), 851–875. https://doi.org/10.1080/13854046.2014.927927
Batt, K., Shores, E. A., & Chekaluk, E. (2008). The effect of distraction on the Word Memory Test and Test of Memory Malingering performance in patients with a severe brain injury. Journal of the International Neuropsychological Society : JINS, 14(6), 1074–1080. https://doi.org/10.1017/S135561770808137X
Bianchini, K. J., Greve, K. W., & Glynn, G. (2005). On the diagnosis of malingered pain-related disability: Lessons from cognitive malingering research. The Spine Journal, 5(4), 404–417. https://doi.org/10.1016/j.spinee.2004.11.016
Bolan, B., Foster, J. K., Schmand, B., & Bolan, S. (2002). A comparison of three tests to detect feigned amnesia: The effects of feedback and the measurement of response latency. Journal of Clinical and Experimental Neuropsychology, 24(2), 154–167. https://doi.org/10.1076/jcen.24.2.154.1000
Bossuyt, P. M. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., Altman, D. G., Hooft, L., Korevaar, D. A., & Cohen, J. F. (2015). STARD 2015 – An updated list of essential items for reporting diagnostic accuracy studies. BMJ - Clinical Research, 351, 1–9. https://doi.org/10.1136/bmj.h5527
Bossuyt, P. M. M., Reitsma, J. B., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Drummond, R., & de Vet, H. C. W. (2003). Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD initiative. BMJ, 326(7379), 41–44. https://doi.org/10.1136/bmj.326.7379.41
Buddin, W. H., Schroeder, R. W., Hargrave, D. D., Von Dran, E. J., Campbell, B., & C. J., Heinrichs, R. J., & Baade, L. E. (2014). An examination of the frequency of invalid forgetting on the Test of Memory Malingering. The Clinical Neuropsychologist, 28(3), 525–542. https://doi.org/10.1080/13854046.2014.906658
Bush, S. S., Ruff, R. M., Tröster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., Reynolds, C. R., & Silver, C. H. (2005). Symptom validity assessment: Practice issues and medical necessity: NAN Policy & Planning Committee. Archives of Clinical Neuropsychology, 20(4), 419–426. https://doi.org/10.1016/j.acn.2005.02.002
Chafetz, M. D. (2011). Reducing the probability of false positives in malingering detection of social security disability claimants. The Clinical Neuropsychologist, 25(7), 1239–1252. https://doi.org/10.1080/13854046.2011.586785
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015). Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD statement. Journal of Clinical Epidemiology, 68(2), 112–121. https://doi.org/10.1016/j.jclinepi.2014.11.010
Cottingham, M. E. (2013). Corrigendum. The Clinical Neuropsychologist, 27(4), iv–iv. https://doi.org/10.1080/13854046.2013.781348
Davis, J. J., Wall, J. R., & Whitney, K. A. (2012). Derivation and clinical validation of consistency indices on the test of memory malingering. Archives of Clinical Neuropsychology, 27(7), 706–715. https://doi.org/10.1093/arclin/acs078
Erdodi, L. A., & Rai, J. K. (2017). A single error is one too many: Examining alternative cutoffs on Trial 2 of the TOMM. Brain Injury, 31(10), 1362–1368. https://doi.org/10.1080/02699052.2017.1332386
Etherton, J. L., Bianchini, K. J., Greve, K. W., & Ciota, M. A. (2005). Test of memory malingering performance is unaffected by laboratory-induced pain: Implications for clinical use. Archives of Clinical Neuropsychology, 20(3), 375–384. https://doi.org/10.1016/j.acn.2004.09.007
Green, D., Rosenfeld, B., Belfi, B., Rohlehr, L., & Pierson, A. (2012). Use of measures of cognitive effort and feigned psychiatric symptoms with pretrial forensic psychiatric patients. International Journal of Forensic Mental Health, 11(3), 181–190. https://doi.org/10.1080/14999013.2012.723665
Green, P. W. (2011). Comparison between the Test of Memory Malingering (TOMM) and the Nonverbal Medical Symptom Validity Test (NV-MSVT) in adults with disability claims. Applied Neuropsychology, 18(1), 18–26. https://doi.org/10.1080/09084282.2010.523365
Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6(3), 218–224. https://doi.org/10.1037//1040-3590.6.3.218
Greiffenstein, M. F., Greve, K. W., Bianchini, K. J., & Baker, W. J. (2008). Test of memory malingering and word memory test: A new comparison of failure concordance rates. Archives of Clinical Neuropsychology, 23(7), 801–807. https://doi.org/10.1016/j.acn.2008.07.005
Greve, K. W., Bianchini, K. J., Black, F. W., Heinly, M. T., Love, J. M., Swift, D. A., & Ciota, M. (2006a). Classification accuracy of the Test of Memory Malingering in persons reporting exposure to environmental and industrial toxins: Results of a known-groups analysis. Archives of Clinical Neuropsychology, 21(5), 439–448. https://doi.org/10.1016/j.acn.2006.06.004
Greve, K. W., Bianchini, K. J., & Doane, B. M. (2006b). Classification accuracy of the test of memory malingering in traumatic brain injury: Results of a known-groups analysis. Journal of Clinical and Experimental Neuropsychology, 28(7), 1176–1190. https://doi.org/10.1080/13803390500263550
Greve, K. W., Etherton, J., Ord, J., Bianchini, K. J., & Curtis, K. (2009). Detecting malingered pain-related disability: Classification accuracy of the test of memory malingering. The Clinical Neuropsychologist, 23(7), 1250–1271. https://doi.org/10.1080/13854040902828272
Greve, K. W., Ord, J., Curtis, K., Bianchini, K. J., & Brennan, A. (2008). Detecting malingering in traumatic brain injury and chronic pain: A comparison of three forced-choice symptom validity tests. The Clinical Neuropsychologist, 22(5), 896–918. https://doi.org/10.1080/13854040701565208
Grote, C. L., Kooker, E. K., Garron, D. C., Nyenhuis, D. L., Smith, C. A., & Mattingly, M. L. (2000). Performance of compensation seeking and non-compensation seeking samples on the Victoria Symptom Validity Test: Cross-validation and extension of a standardization study. Journal of Clinical and Experimental Neuropsychology, 22(6), 709–719. https://doi.org/10.1076/jcen.22.6.709.958
Gupta, S. K. (2011). Intention-to-treat concept: A review. Perspectives in Clinical Research, 2(3), 109–112. https://doi.org/10.4103/2229-3485.83221
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants, C. (2009). American academy of clinical neuropsychology consensus conference statement on the neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129. https://doi.org/10.1080/13854040903155063
Huang, L., & Li, J. (2018). Weighted volume under the three-way receiver operating characteristic surface. Statistical Methods in Medical Research. https://doi.org/10.1177/0962280218812211
Iverson, G. L., Le Page, J., Koehler, B. E., Shojania, K., & Badii, M. (2007). Test of Memory Malingering (TOMM) scores are not affected by chronic pain or depression in patients with fibromyalgia. The Clinical Neuropsychologist, 21(3), 532–546. https://doi.org/10.1080/13854040600611392
Jasinski, L. J., Harp, J. P., Berry, D. T., Shandera-Ochsner, A. L., Mason, L. H., & Ranseen, J. D. (2011). Using symptom validity tests to detect malingered ADHD in college students. The Clinical Neuropsychologist, 25(8), 1415–1428. https://doi.org/10.1080/13854046.2011.630024
Jones, A. (2013). Test of memory malingering: Cutoff scores for psychometrically defined malingering groups in a military sample. The Clinical Neuropsychologist, 27(6), 1043–1059. https://doi.org/10.1080/13854046.2013.804949
Kanser, R. J., Rapport, L. J., Bashem, J. R., & Hanks, R. A. (2019). Detecting malingering in traumatic brain injury: Combining response time with performance validity test accuracy. The Clinical Neuropsychologist, 33(1), 90–107. https://doi.org/10.1080/13854046.2018.1440006
Kazdin, A. E. (2017). Research Design in Clinical Psychology (5th ed.). Pearson.
Kulas, J. F., Rinaldi, A. R., & Axelrod, B. N. (2014). Cross-Validation of Supplemental Test of memory malingering scores as performance validity measures. Psychological Injury and Law, 7(3), 236–244. https://doi.org/10.1007/s12207-014-9200-4
Larrabee, G. J. (2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17(3), 410–425. https://doi.org/10.1076/clin.17.3.410.18089
Larrabee, G. J. (2007). Malingering, research designs, and base rates. In G. J. Larrabee (Ed.), Assessment of malingered neuropsychological deficits (pp. 3–11). Oxford University Press. http://catdir.loc.gov/catdir/toc/ecip078/2007001084.html
Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: Relationship to likelihood ratios. The Clinical Neuropsychologist, 22(4), 666–679. https://doi.org/10.1080/13854040701494987
Larrabee, G. J., Greiffenstein, M. F., Greve, K. W., & Bianchini, K. J. (2007). Redefining diagnostic criteria for malingering. In G. J. Larrabee (Ed.), Assessment of malingered neuropsychological deficits (pp. 334–371). Oxford University Press.
Larrabee, G. J., Rohling, M. L., & Meyers, J. E. (2019). Use of multiple performance and symptom validity measures: Determining the optimal per test cutoff for determination of invalidity, analysis of skew, and inter-test correlations in valid and invalid performance groups. The Clinical Neuropsychologist, 33(8), 1354–1372. https://doi.org/10.1080/13854046.2019.1614227
Lippa, S. M. (2018). Performance validity testing in neuropsychology: A clinical guide, critical review, and update on a rapidly evolving literature. The Clinical Neuropsychologist, 32(3), 391–421. https://doi.org/10.1080/13854046.2017.1406146
Love, C. M., Glassmire, D. M., Zanolini, S. J., & Wolf, A. (2014). Specificity and false positive rates of the test of memory malingering, Rey 15-item test, and Rey word recognition test among forensic inpatients with intellectual disabilities. Assessment, 21(5), 618–627. https://doi.org/10.1177/1073191114528028
Martin, P. K., Schroeder, R. W., & Odland, A. P. (2015). Neuropsychologists’ validity testing beliefs and practices: A survey of North American Professionals. The Clinical Neuropsychologist, 29(6), 741–776. https://doi.org/10.1080/13854046.2015.1087597
Martin, P. K., Schroeder, R. W., Olsen, D. H., Maloy, H., Boettcher, A., Ernst, N., & Okut, H. (2020). A systematic review and meta-analysis of the Test of Memory Malingering in adults: Two decades of deception detection. The Clinical Neuropsychologist, 34(1), 88–119. https://doi.org/10.1080/13854046.2019.1637027
McCaul, C., Boone, K. B., Ermshar, A., Cottingham, M., Victor, T. L., Ziegler, E., Zeller, M. A., & Wright, M. (2018). Cross-validation of the dot counting test in a large sample of credible and non-credible patients referred for neuropsychological testing. The Clinical Neuropsychologist, 32(6), 1054–1067. https://doi.org/10.1080/13854046.2018.1425481
Meyer, G. J. (2003). Guidelines for reporting information in studies of diagnostic test accuracy: The STARD initiative. Journal of Personality Assessment, 81(3), 191–193. https://doi.org/10.1207/S15327752JPA8103_01
Meyers, J. E., Miller, R. M., Thompson, L. M., Scalese, A. M., Allred, B. C., Rupp, Z. W., Dupaix, Z. P., & Junghyun Lee, A. (2014). Using likelihood ratios to detect invalid performance with performance validity measures. Archives of Clinical Neuropsychology, 29(3), 224–235. https://doi.org/10.1093/arclin/acu001
Morse, C. L., Douglas-Newman, K., Mandel, S., & Swirsky-Sacchetti, T. (2013). Utility of the Rey-15 recognition trial to detect invalid performance in a forensic neuropsychological sample. The Clinical Neuropsychologist, 27(8), 1395–1407. https://doi.org/10.1080/13854046.2013.832385
O’Bryant, S., Gavett, B., McCaffrey, R., O’Jile, J., Huerkamp, J., Smitherman, T., & Humphreys, J. (2008). Clinical utility of trial 1 of the Test of Memory Malingering (TOMM). Applied Neuropsychology, 15(2), 113–116. https://doi.org/10.1080/09084280802083921
Oudman, E., Krooshof, E., van Oort, R., Lloyd, B., Wijnia, J. W., & Postma, A. (2020). Effects of Korsakoff Amnesia on performance and symptom validity testing. Applied Neuropsychology:ADult, 27(6), 549–557. https://doi.org/10.1080/23279095.2019.1576180
Powell, M. R., Gfeller, J. D., Hendricks, B. L., & Sharland, M. (2004). Detecting symptom- and test-coached simulators with the test of memory malingering. Archives of Clinical Neuropsychology, 19(5), 693–702. https://doi.org/10.1016/j.acn.2004.04.001
Price, K. L., DeSantis, S. M., Simpson, T., McRae-Clark, B. K., Saladin, M. E., Baker, N. L., Wagner, M. T., & Brady, K. T. (2011). The impact of clinical and demographic variables on cognitive performance in methamphetamine-dependent individuals in rural South Carolina. The American Journal on Addictions, 20(5), 447–455. https://doi.org/10.1111/j.1521-0391.2011.00164.x
Rees, L. M., Tombaugh, T. N., Gansler, D. A., & Moczynski, N. P. (1998). Five validation experiments of the test of memory malingering (TOMM). Psychological Assessment, 10(1), 10–20. https://doi.org/10.1037/1040-3590.10.1.10
Reitsma, J. B., Rutjes, A. W. S., Whiting, P. F., Vlassov, V. V., Leeflang, M. M. G., & Deeks, J. J. (2009). Assessing Methodological Quality. In J. J. Deeks, P. M. M. Bossuyt, & C. A. Gatsonis (Eds.), Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0.0. The Cochrane Collaboration. Retrieved January 22, 2021, from http://srdta.cochrane.org/
Risk Assessment Workgroup. (2013). Assessing cardiovascular risk (p. 139). National Heart, Lung, and Blood Institute. Retrieved September 11, 2021, from https://www.nhlbi.nih.gov/sites/default/files/media/docs/risk-assessment.pdf
Roberson, C. J., Boone, K. B., Goldberg, H., Miora, D., Cottingham, M., Victor, T., Ziegler, E., Zeller, M., & Wright, M. (2013). Cross validation of the b test in a large known groups sample. The Clinical Neuropsychologist, 27(3), 495–508. https://doi.org/10.1080/13854046.2012.737027
Rogers, R. (2018). An introduction to response styles. In R. Rogers & S. D. Bender (Eds.), Clinical assessment of malingering and deception (Fourth edition., pp. 3–17). The Guilford Press.
Rosenfeld, B., Green, D., Pivovarova, E., Dole, T., & Zapf, P. (2010). What to do with contradictory data? Approaches to the integration of multiple malingering measures. International Journal of Forensic Mental Health, 9(2), 63–73. https://doi.org/10.1080/14999013.2010.499559
Ryan, J., Glass, L., Hinds, R., & Brown, C. (2010). Administration order effects on the test of memory malingering. Applied Neuropsychology, 17(4), 246–250. https://doi.org/10.1080/09084282.2010.499802
Schenk, K., & Sullivan, K. (2010). Do warnings deter rather than produce more sophisticated malingering? Journal of Clinical and Experimental Neuropsychology, 32(7), 752–762. https://doi.org/10.1080/13803390903512678
Schroeder, R. W., Buddin, W. H., Hargrave, D. D., VonDran, E. J., Campbell, E. B., Brockman, C. J., Heinrichs, R. J., & Baade, L. E. (2013). Efficacy of test of memory malingering trial 1, trial 2, the retention trial, and the Albany consistency index in a criterion group forensic neuropsychological sample. Archives of Clinical Neuropsychology, 28(1), 21–29. https://doi.org/10.1093/arclin/acs094
Shandera, A. L., Berry, D. T. R., Clark, J. A., Schipper, L. J., Graue, L. O., & Harp, J. P. (2010). Detection of malingered mental retardation. Psychological Assessment, 22(1), 50–56. https://doi.org/10.1037/a0016585
Sherman, E. M. S., Slick, D. J., & Iverson, G. L. (2020). Multidimensional malingering criteria for neuropsychological assessment: A 20-year update of the malingered neuropsychological dysfunction criteria. Archives of Clinical Neuropsychology, acaa019. https://doi.org/10.1093/arclin/acaa019
Slick, D. J., Hopp, G., Strauss, E., & Spellacy, F. J. (1996). Victoria Symptom Validity Test: Efficiency for detecting feigned memory impairment and relationship to neuropsychological tests and MMPI-2 validity scales. Journal of Clinical and Experimental Neuropsychology, 18(6), 911–922.
Slick, D. J., & Sherman, E. M. S. (2012). Differential diagnosis of malingering and related clinical presentations. In E. M. S. Sherman & Brooks, B. L. (Eds.), Pediatric Forensic Neuropsychology (pp. 113–135).
Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545–561. https://doi.org/10.1076/1385-4046(199911)13:04;1-Y;FT545
Smith, K., Boone, K., Victor, T., Miora, D., Cottingham, M., Ziegler, E., Zeller, M., & Wright, M. (2014). Comparison of credible patients of very low intelligence and non-credible patients on neurocognitive performance validity indicators. The Clinical Neuropsychologist, 28(6), 1048–1070. https://doi.org/10.1080/13854046.2014.931465
Sollman, M. J., Ranseen, J. D., & Berry, D. T. (2010). Detection of feigned ADHD in college students. Psychological Assessment, 22(2), 325. https://doi.org/10.1037/a0018857
Sullivan, K. A., & Elliott, C. (2012). An investigation of the validity of the MMPI-2 Response Bias Scale using an analog simulation design. The Clinical Neuropsychologist, 26(1), 160–176. https://doi.org/10.1080/13854046.2011.647084
Sweet, J. J., Heilbronner, R. L., Morgan, J. E., Larrabee, G. J., Rohling, M. L., Boone, K. B., Kirkwood, M. W., Schroeder, R. W., & Suhr, J. A. (2021). American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 35(6), 1053–1106. https://doi.org/10.1080/13854046.2021.1896036
Tombaugh, T. N. (1997). The Test of Memory Malingering (TOMM): Normative data from cognitively intact and cognitively impaired individuals. Psychological Assessment, 9(3), 260–268. https://doi.org/10.1037/1040-3590.9.3.260
Trikalinos, T., A., & Balion, C. M. (2012). Options for summarizing medical test performance in the absence of a “gold standard”. In Agency for Healthcare Research and Quality, Methods Guide for Medical Test Reviews (pp. 9–1–9–16). Agency for Healthcare Research and Quality.
Vanderslice-Barr, J., Miele, A., Jardin, B., & McCaffrey, R. (2011). Comparison of computerized versus booklet versions of the TOMM™. Applied Neuropsychology, 18(1), 34–36. https://doi.org/10.1080/09084282.2010.523377
Victor, T. L., Boone, K. B., Serpa, K. G., Buehler, J., & Ziegler, E. A. (2009). Interpreting the meaning of multiple symptom validity test failure. The Clinical Neuropsychologist, 23(2), 297–313. https://doi.org/10.1080/13854040802232682
Whiting, P. F. (2011). QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine, 155(8), 529. https://doi.org/10.7326/0003-4819-155-8-201110180-00009
Whiting, P. F., Rutjes, A. W. S., Reitsma, J. B., Bossuyt, P. M. M., & Kleijnen, J. (2003). The development of QUADAS: A tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology, 3(1), 1–13. https://doi.org/10.1186/1471-2288-3-25
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., & Mallett, S. (2013). A systematic review classifies sources of bias and variation in diagnostic test accuracy studies. Journal of Clinical Epidemiology, 6(10), 1093–1104. https://doi.org/10.1016/j.jclinepi.2013.05.014
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., & Bossuyt, P. M. M. (2011). QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Annals of Internal Medicine, 155(8), 529–536. https://doi.org/10.7326/0003-4819-155-8-201110180-00009
Williamson, K. D., Combs, H. L., Berry, D. T. R., Harp, J. P., Mason, L. H., & Edmundson, M. (2014). Discriminating among ADHD alone, ADHD with a comorbid psychological disorder, and feigned ADHD in a college sample. The Clinical Neuropsychologist, 28(7), 1182–1196. https://doi.org/10.1080/13854046.2014.956674
Yanez, Y. T., Fremouw, W., Tennant, J., Strunk, J., & Coker, K. (2006). Effects of severe depression on TOMM performance among disability-seeking outpatients. Archives of Clinical Neuropsychology, 21(2), 161–165. https://doi.org/10.1016/j.acn.2005.07.009
Zeeshan, M., Salam, B., Khalid, Q. S. B., Alam, S., & Sayani, R. (2018). Diagnostic accuracy of digital mammography in the detection of breast cancer. Cureus, 10(4). https://doi.org/10.7759/cureus.2448
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The author declares no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
No parts of this work have been presented elsewhere. However, in compliance with data sharing requirements, data and analysis presented in Fig. 1, Table 4, and in the Supplemental Appendix (use of QUADAS-2 by two raters on 20 TOMM validation studies) have been publicly shared under the following Figshare https://doi.org/10.6084/m9.figshare.14368739.v1.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Leonhard, C. Review of Statistical and Methodological Issues in the Forensic Prediction of Malingering from Validity Tests: Part II—Methodological Issues. Neuropsychol Rev 33, 604–623 (2023). https://doi.org/10.1007/s11065-023-09602-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11065-023-09602-6