Abstract
Professional organizations recommend inclusion of multiple performance (PVTs) and symptom validity tests (SVTs) (Bush et al., Archives of Clinical Neuropsychology, 20, 419–426, 2005; Heilbronner et al., The Clinical Neuropsychologist, 23(7), 1093–1129, 2009). However, to date, empirically driven recommendations for interpretation of multiple validity indicators are largely absent, and the generalizability of available psychometric data is questionable in clinical practice. The current aim is to provide base rate data and recommendations for interpretation of multiple validity indicators, assuming varying correlations between each PVT at a range of specificity and sensitivity rates. As an initial step, Monte Carlo methodology was validated across 24 embedded and stand-alone validity indicators in seven extant noncompensation-seeking clinical samples. Samples were comprised of patients with psychotic and nonpsychotic psychiatric disorders, as well as different neurological conditions. Strategies are outlined for clinical integration of base rate data for advanced administration and interpretation of multiple validity indicators. In light of the current findings, recommendations are provided to reduce false-positive rates associated with making determinations regarding noncredible test performance.
Similar content being viewed by others
References
Arnold, G., Boone, K. B., Lu, P., Dean, A., Wen, J., Nitch, S., & McPherson, S. (2005). Sensitivity and specificity of finger tapping test scores for the detection of suspect effort. The Clinical Neuropsychologist, 19(1), 105–120.
Axelrod, Meyers, & Davis. (2014). Finger tapping test performance as a measure of performance validity. The Clinical Psychologist, 28(5), 876–888.
Babikian, T., Boone, K. B., Lu, P., & Arnold, G. (2006). Sensitivity and specificity of various digit span scores in the detection of suspect effort. The Clinical Neuropsychologist, 20(1), 145–159.
Bashem, J. R., Rapport, L. J., Miller, J. B., Hanks, R. A., Axelrod, B. N., & Millis, S. R. (2014). Comparisons of five performance validity indices in bona fide and simulated traumatic brain injury. The Clinical Neuropsychologist, 28(5), 851–875.
Ben-Porath, Y. S., & Tellegen, A. (2008/2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): manual for administration, scoring, and interpretation. Minneapolis, MN: University of Minnesota Press.
Berthelson, L., Mulchan, S. S., Odland, A. P., Miller, L. J., & Mittenberg, W. (2013). False positive diagnosis of malingering due to the use of multiple effort tests. Brain Injury, 27(7–8), 909–916. doi:10.3109/02699052.2013.793400
Bilder, R. M., Sugar, C. A., & Hellemann, G. S. (2014). Cumulative false positive rates given multiple performance validity tests: commentary on Davis and Millis (2014) and Larrabee (2014). The Clinical Neuropsychologist. doi:10.1080/13854046.2014.969774
Binder, L. M., Iverson, G. L., & Brooks, B. L. (2009). To err is human: “abnormal” neuropsychological scores and variability are common in healthy adults. Archives of Clinical Neuropsychology, 24, 31–46. doi:10.1093/arclin/acn001
Boone, K. B. (2007). Assessment of feigned cognitive impairment. A neuropsychological perspective. New York: The Guilford Press.
Boone, K. B. (2009). The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations. The Clinical Neuropsychologist, 23, 729–741.
Boone, K. B. (2013). Clinical practice of forensic neuropsychology. New York: The Guilford Press.
Boone, K. B., & Lu, P. H. (2003). Noncredible cognitive performance in the context of severe brain injury. The Clinical Neuropsychologist, 17, 244–254.
Brooks, B. L., & Iverson, G. L. (2010). Comparing actual to estimated base rates of “abnormal” scores on neuropsychological test batteries: Implications for interpretation. Archives of Clinical Neuropsychology, 25, 14–21. doi:10.1093/arclin/acp100
Brooks, B. L., Strauss, E., Sherman, E. M. S., Iverson, G. L., & Slick, D. J. (2009). Developments in neuropsychological assessment: refining psychometric and clinical interpretive methods. Canadian Psychology, 50(3), 196–209. doi:10.1037/a0016066
Buddin, H.W., Schroeder, R.W., Hargrave, D.D., Von Dran, E.J., Campbell, E.B., Brockman, C.J., … Baade, L.E., et al. (2014). An examination of the frequency of invalid forgetting on the Test of Memory Malingering. The Clinical Neuropsychologist, 28(3), 525–42.
Bush, S. S., Ruff, R. M., Tröster, A. I., Barth, J. T., Koffler, S. P., Plisken, N. H., ... Silver, C. H. (2005). Symptom validity assessment: practice issues and medical necessity NAN policy and planning committee. Archives of Clinical Neuropsychology, 20, 419–426.
Busse, M., & Whiteside, D. (2012). Detecting suboptimal cognitive effort: classification accuracy of the Conner's continuous performance test-II, brief test of attention, and trail making test. The Clinical Neuropsychologist, 26(4), 675–687. doi:10.1080/13854046.2012.679623
Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., & Dahlstrom, W. G. (2001). MMPI-2 (Minnesota Multiphasic Personality Inventory-2): manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.
Button, K. S., Loannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafo, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews. Neuroscience, 14, 365–376.
Carrasco, R. M., Grups, J., Evans, B., Simco, E., & Mittenberg, W. (2013). Apparently abnormal Wechsler Memory Scale index score patterns in the normal population. Applied Neuropsychology: Adult, 0, 1–6. doi:10.1080/23279095.2013.816702
Crawford, J. R., Garthwaite, P. H., & Gault, C. B. (2007). Estimating the percentage of the population with abnormally low scores (or abnormally large score differences) on standardized neuropsychological test batteries: a generic method with applications. Neuropsychology, 21(4), 419–430. doi:10.1037/0894-4105.21.4.419
Curtis, K. L., Thompson, L. K., Greve, K. W., & Bianchini, K. J. (2008). Verbal fluency indicators of malingering in traumatic brain injury: classification accuracy in known groups. The Clinical Neuropsychologist, 22(5), 930–945. doi:10.1080/13854040701563591
Davis, J. J., & Millis, S. R. (2014). Examination of performance validity test failure in relation to the number of tests administered. The Clinical Neuropsychologist, 28(2), 199–214. doi:10.1080/13854046.2014.884633
Decker, S. L., Scheider, W. J., & Hale, J. B. (2012). Estimating base rates of impairment in neuropsychological test batteries: a comparison of quantitative models. Archives of Clinical Neuropsychology, 27, 69–84. doi:10.1093/arclin/acr088
Denning, J. H. (2012). The efficiency and accuracy of the Test of Memory Malingering Trial 1, errors on the first 10 items of the Test of Memory Malingering, and five embedded measures in predicting invalid test performance. Archives of Clinical Neuropsychology, 27(4), 417–432.
Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6, 218–224.
Gunner, J. H., Miele, A. S., Lynch, J. K., & McCaffrey, R. J. (2012). The Albany Consistency Index for the Test of Memory Malingering. Archives of Clinical Neuropsychology, 27(1), 1–9.
Heilbronner, R. L., Sweet, J. J., Morgan, J. L., Larrabee, G. J., Millis, S. R., & Participants, C. (2009). American Academy of Clinical Neuropsychology consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129. doi:10.1080/13854040903155063
IBM Corp. Released (2010). IBM SPSS Statistics for Windows, Version 19.0. Armonk, NY: IBM Corp.
Lange, R. T., Iverson, G. L., Brickell, T. A., Staver, T., Pancholi, S., Bhagwat, A., & French, L. M. (2013). Clinical utility of the Conners' Continuous Performance Test-II to detect poor effort in U.S. military personnel following traumatic brain injury. Psychological Assessment, 25(2), 339–352. doi:10.1037/a0030915
Larrabee, G. J. (2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17, 410–425.
Larrabee, G. J. (2005). Assessment of malingering. In G. J. Larrabee (Ed.), Forensic neuropsychology: a scientific approach. New York: Oxford University Press.
Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: relationship to likelihood ratios. The Clinical Neuropsychologist, 22(4), 666–679.
Larrabee, G. L. (2009). Malingering scales for the Continuous Recognition Memory Test and the Continuous Visual Memory Test. The Clinical Neuropsychologist, 23(1), 167–180.
Larrabee, G. J. (2014). False-positive rates associated with the use of multiple performance and symptom validity tests. Archives of Clinical Neuropsychology, 29(4), 364–373. doi:10.1093/arclin/acu019
Larrabee, G. J. (2007). Assessment of malingered neuropsychological deficits. New York: Oxford University Press.
Lezak, M. D., Howieson, D. B., Bigler, E. D., & Tranel, D. (2012). Neuropsychological assessment (5th ed.). Oxford: Oxford University Press.
Lu, P. H., Boone, K. B., Cozolino, L., & Mitchell, C. (2003). Effectiveness of the Rey-Osterrieth Complex Figure Test and the Meyers and Meyers recognition trial in the detection of suspect effort. Clinical Neuropsychologist, 17, 426–440.
Medici, R. (2013). The use of multiple performance validity tests. Journal of Forensic Practice, 13(1), 68–78. doi:10.1080/15228932.2013.746909
Meyers, J. E., Miller, R. M., Thompson, L. M., Scalese, A. M., Allred, B. C., Rupp, Z. W., ... Lee, A. J. (2014). Using likelihood ratios to detect invalid performance with performance validity measures. Archives of Clinical Neuropsychology, 29, 224–235.
Meyers, J. E., & Volbrecht, M. E. (2003). A validation of multiple malingering detection methods in a large clinical sample. Archives of Clinical Neuropsychology, 18, 261–276.
Mittenberg, W., Patton, C., Canyock, E. M., & Condit, D. C. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 4(8), 1094–1102.
Morey, L. C. (2007). Personality Assessment Inventory professional manual (2nd ed.). Lutz: Psychological Assessment Resources.
Nelson, N. W., Boone, K., Dueck, A., Wagener, L., Lu, P., & Grills, C. (2003). Relationship between eight measures of suspect effort. The Clinical Neuropsychologist, 17(2), 263–272.
O’Bryant, Engel, Kleiner, Vasterling, & Black. (2007). Test of memory malingering (TOMM) trial 1 as a screening measure for insufficient effort. The Clinical Neuropsychologist, 21(3), 511–21.
Ord, J.S., Boettcher, A.C., Greve, K.W., & Bianchini, K.J. (2010). Detection of malingering in mild traumatic brain injury with the Conners’ Continuous Performance Test–II. Journal of Clinical and Experimental Neuropsychology, 32(4). doi:10.1080/13803390903066881
Ortega, A., Labrenz, S., Markowitsch, H. J., & Piefke, M. (2013). Diagnostic accuracy of a Bayesian latent group analysis for the detection of malingering-related poor effort. The Clinical Neuropsychologist, 27(6), 1019–1042.
Ortega, A., Wagenmakers, E.-J., Lee, M. D., Markowitsch, H. J., & Piefke, M. (2012). A Bayesian latent group analysis for detecting poor effort in assessment of malingering. Archives of Clinical Neuropsychology, 27, 453–465.
Pearson. (2009). Advanced clinical solutions for use with WAIS-IV and WMS-IV. San Antonio: Pearson Education.
Powell, M. R., Locke, D. E., Smigielski, J. S., & McCrea, M. (2011). Estimating the diagnostic value of the trail making test for suboptimal effort in acquired brain injury rehabilitation patients. The Clinical Neuropsychologist, 25(1), 108–118. doi:10.1080/13854046.2010.532912
Reese, C. S., Suhr, J. A., & Riddle, T. L. (2012). Exploration of malingering indices in the Wechsler Adult Intelligence Scale-Fourth Edition Digit Span subtest. Archives of Clinical Neuropsychology, 27(2), 176–181. doi:10.1093/arclin/acr117
Root, J. C., Robbins, R. N., Chang, L., & Van Gorp, W. G. (2006). Detection of inadequate effort on the California Verbal Learning Test-Second edition: forced choice recognition and critical item analysis. Journal of the International Neuropsychological Society, 12(5), 688–696.
Schretlen, D. J., Testa, S. M., Winicki, J. M., Pearlson, G. D., & Gordon, B. (2008). Frequency and bases of abnormal performance by healthy adults on neuropsychological testing. Journal of the International Neuropsychological Society, 14, 436–445. doi:10.1017/S1355617708080387
Schroeder, R. W., & Marshall, P. S. (2011). Evaluation of the appropriateness of multiple symptom validity indices in psychotic and non-psychotic psychiatric populations. The Clinical Neuropsychologist, 25(3), 437–453. doi:10.1080/13854046.2011.556668
Schutte, C., & Axelrod, B. N. (2013). Use of embedded cognitive symptom validity measures in mild traumatic brain injury cases. In D. A. Carone & S. S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment and malingering (pp. 159–181). New York: Springer Publishing Company.
Silverberg, N. D., Hanks, R. A., Buchanan, L., Fichtenberg, N., & Millis, S. R. (2008). Detecting response bias with performance patterns on an expanded version of the Controlled Oral Word Association Test. The Clinical Neuropsychologist, 22(1), 140–157.
Slick, D. J. (1996). The Victoria Symptom Validity Test: a new clinical measure of response bias (Doctoral dissertation, University of Victoria, 1996). Dissertation Abstracts International, 59, 6114.
Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13, 465–473.
Sollman, M. J., & Berry, D. T. R. (2011). Detection of inadequate effort on neuropsychological testing: a meta-analytic update and extension. Archives of Clinical Neuropsychology, 26, 774–789.
Sugarman, M.A. & Axelrod, B.N. (2014). Embedded measures of performance validity using verbal fluency tests in a clinical sample. Applied Neuropsychology (online publication ahead of print).
Teichner, G., & Wagner, M. (2004). The Test of Memory Malingering: normative data from cognitively intact, cognitively impaired, and elderly patients with dementia. Archives of Clinical Neuropsychology, 19, 455–464.
Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM). North Tonowanda: Multi-Health Systems.
Victor, T. L., Boone, K. B., Serpa, J. G., Buehler, J., & Ziegler, E. A. (2009). Interpreting the meaning of multiple symptom validity test failure. The Clinical Neuropsychologist, 23, 297–313. doi:10.1080/13854040802232682
Walter, J., Morris, J., Swier-Vosnos, A., & Plisken, N. (2014). Effects of severity of dementia on a symptom validity measure. The Clinical Neuropsychologist, 28(7), 1197–1208.
Whiteside, D., Wald, D., & Busse, M. (2011). Classification accuracy of multiple visual spatial measures in the detection of suspect effort. The Clinical Neuropsychologist, 25(2), 287–301. doi:10.1080/13854046.2010.538436
Wolfe, P. L., Millis, S. R., Hanks, R., Fichtenberg, N., Larrabee, G. J., & Sweet, J. J. (2010). Effort indicators within the California Verbal Learning Test-II (CVLT-II). The Clinical Neuropsychologist, 24(1), 153–168. doi:10.1080/13854040903107791
Young, J.C., Caron, J.E., Baughman, B.C., & Sawyer R.J. (2012). Detection of suboptimal effort with symbol span: development of a new embedded index. Archives of Clinical Neuropsychology, 27(2). doi:10.1093/arclin/acr109
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Odland, A.P., Lammy, A.B., Martin, P.K. et al. Advanced Administration and Interpretation of Multiple Validity Tests. Psychol. Inj. and Law 8, 46–63 (2015). https://doi.org/10.1007/s12207-015-9216-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12207-015-9216-4