Skip to main content
Log in

Advanced Administration and Interpretation of Multiple Validity Tests

  • Published:
Psychological Injury and Law Aims and scope Submit manuscript

Abstract

Professional organizations recommend inclusion of multiple performance (PVTs) and symptom validity tests (SVTs) (Bush et al., Archives of Clinical Neuropsychology, 20, 419–426, 2005; Heilbronner et al., The Clinical Neuropsychologist, 23(7), 1093–1129, 2009). However, to date, empirically driven recommendations for interpretation of multiple validity indicators are largely absent, and the generalizability of available psychometric data is questionable in clinical practice. The current aim is to provide base rate data and recommendations for interpretation of multiple validity indicators, assuming varying correlations between each PVT at a range of specificity and sensitivity rates. As an initial step, Monte Carlo methodology was validated across 24 embedded and stand-alone validity indicators in seven extant noncompensation-seeking clinical samples. Samples were comprised of patients with psychotic and nonpsychotic psychiatric disorders, as well as different neurological conditions. Strategies are outlined for clinical integration of base rate data for advanced administration and interpretation of multiple validity indicators. In light of the current findings, recommendations are provided to reduce false-positive rates associated with making determinations regarding noncredible test performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  • Arnold, G., Boone, K. B., Lu, P., Dean, A., Wen, J., Nitch, S., & McPherson, S. (2005). Sensitivity and specificity of finger tapping test scores for the detection of suspect effort. The Clinical Neuropsychologist, 19(1), 105–120.

    Article  PubMed  Google Scholar 

  • Axelrod, Meyers, & Davis. (2014). Finger tapping test performance as a measure of performance validity. The Clinical Psychologist, 28(5), 876–888.

    Google Scholar 

  • Babikian, T., Boone, K. B., Lu, P., & Arnold, G. (2006). Sensitivity and specificity of various digit span scores in the detection of suspect effort. The Clinical Neuropsychologist, 20(1), 145–159.

    Article  PubMed  Google Scholar 

  • Bashem, J. R., Rapport, L. J., Miller, J. B., Hanks, R. A., Axelrod, B. N., & Millis, S. R. (2014). Comparisons of five performance validity indices in bona fide and simulated traumatic brain injury. The Clinical Neuropsychologist, 28(5), 851–875.

    Article  PubMed  Google Scholar 

  • Ben-Porath, Y. S., & Tellegen, A. (2008/2011). MMPI-2-RF (Minnesota Multiphasic Personality Inventory-2 Restructured Form): manual for administration, scoring, and interpretation. Minneapolis, MN: University of Minnesota Press.

  • Berthelson, L., Mulchan, S. S., Odland, A. P., Miller, L. J., & Mittenberg, W. (2013). False positive diagnosis of malingering due to the use of multiple effort tests. Brain Injury, 27(7–8), 909–916. doi:10.3109/02699052.2013.793400

    Article  PubMed  Google Scholar 

  • Bilder, R. M., Sugar, C. A., & Hellemann, G. S. (2014). Cumulative false positive rates given multiple performance validity tests: commentary on Davis and Millis (2014) and Larrabee (2014). The Clinical Neuropsychologist. doi:10.1080/13854046.2014.969774

    Google Scholar 

  • Binder, L. M., Iverson, G. L., & Brooks, B. L. (2009). To err is human: “abnormal” neuropsychological scores and variability are common in healthy adults. Archives of Clinical Neuropsychology, 24, 31–46. doi:10.1093/arclin/acn001

    Article  PubMed  Google Scholar 

  • Boone, K. B. (2007). Assessment of feigned cognitive impairment. A neuropsychological perspective. New York: The Guilford Press.

    Google Scholar 

  • Boone, K. B. (2009). The need for continuous and comprehensive sampling of effort/response bias during neuropsychological examinations. The Clinical Neuropsychologist, 23, 729–741.

    Article  PubMed  Google Scholar 

  • Boone, K. B. (2013). Clinical practice of forensic neuropsychology. New York: The Guilford Press.

    Google Scholar 

  • Boone, K. B., & Lu, P. H. (2003). Noncredible cognitive performance in the context of severe brain injury. The Clinical Neuropsychologist, 17, 244–254.

    Article  PubMed  Google Scholar 

  • Brooks, B. L., & Iverson, G. L. (2010). Comparing actual to estimated base rates of “abnormal” scores on neuropsychological test batteries: Implications for interpretation. Archives of Clinical Neuropsychology, 25, 14–21. doi:10.1093/arclin/acp100

    Article  PubMed  Google Scholar 

  • Brooks, B. L., Strauss, E., Sherman, E. M. S., Iverson, G. L., & Slick, D. J. (2009). Developments in neuropsychological assessment: refining psychometric and clinical interpretive methods. Canadian Psychology, 50(3), 196–209. doi:10.1037/a0016066

    Article  Google Scholar 

  • Buddin, H.W., Schroeder, R.W., Hargrave, D.D., Von Dran, E.J., Campbell, E.B., Brockman, C.J., … Baade, L.E., et al. (2014). An examination of the frequency of invalid forgetting on the Test of Memory Malingering. The Clinical Neuropsychologist, 28(3), 525–42.

  • Bush, S. S., Ruff, R. M., Tröster, A. I., Barth, J. T., Koffler, S. P., Plisken, N. H., ... Silver, C. H. (2005). Symptom validity assessment: practice issues and medical necessity NAN policy and planning committee. Archives of Clinical Neuropsychology, 20, 419–426.

    Article  PubMed  Google Scholar 

  • Busse, M., & Whiteside, D. (2012). Detecting suboptimal cognitive effort: classification accuracy of the Conner's continuous performance test-II, brief test of attention, and trail making test. The Clinical Neuropsychologist, 26(4), 675–687. doi:10.1080/13854046.2012.679623

    Article  PubMed  Google Scholar 

  • Butcher, J. N., Graham, J. R., Ben-Porath, Y. S., Tellegen, A., & Dahlstrom, W. G. (2001). MMPI-2 (Minnesota Multiphasic Personality Inventory-2): manual for administration, scoring, and interpretation. Minneapolis: University of Minnesota Press.

    Google Scholar 

  • Button, K. S., Loannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafo, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature Reviews. Neuroscience, 14, 365–376.

    Article  PubMed  Google Scholar 

  • Carrasco, R. M., Grups, J., Evans, B., Simco, E., & Mittenberg, W. (2013). Apparently abnormal Wechsler Memory Scale index score patterns in the normal population. Applied Neuropsychology: Adult, 0, 16. doi:10.1080/23279095.2013.816702

  • Crawford, J. R., Garthwaite, P. H., & Gault, C. B. (2007). Estimating the percentage of the population with abnormally low scores (or abnormally large score differences) on standardized neuropsychological test batteries: a generic method with applications. Neuropsychology, 21(4), 419–430. doi:10.1037/0894-4105.21.4.419

    Article  PubMed  Google Scholar 

  • Curtis, K. L., Thompson, L. K., Greve, K. W., & Bianchini, K. J. (2008). Verbal fluency indicators of malingering in traumatic brain injury: classification accuracy in known groups. The Clinical Neuropsychologist, 22(5), 930–945. doi:10.1080/13854040701563591

    Article  PubMed  Google Scholar 

  • Davis, J. J., & Millis, S. R. (2014). Examination of performance validity test failure in relation to the number of tests administered. The Clinical Neuropsychologist, 28(2), 199–214. doi:10.1080/13854046.2014.884633

    Article  PubMed  Google Scholar 

  • Decker, S. L., Scheider, W. J., & Hale, J. B. (2012). Estimating base rates of impairment in neuropsychological test batteries: a comparison of quantitative models. Archives of Clinical Neuropsychology, 27, 69–84. doi:10.1093/arclin/acr088

    Article  PubMed  Google Scholar 

  • Denning, J. H. (2012). The efficiency and accuracy of the Test of Memory Malingering Trial 1, errors on the first 10 items of the Test of Memory Malingering, and five embedded measures in predicting invalid test performance. Archives of Clinical Neuropsychology, 27(4), 417–432.

    Article  PubMed  Google Scholar 

  • Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6, 218–224.

    Article  Google Scholar 

  • Gunner, J. H., Miele, A. S., Lynch, J. K., & McCaffrey, R. J. (2012). The Albany Consistency Index for the Test of Memory Malingering. Archives of Clinical Neuropsychology, 27(1), 1–9.

    Article  PubMed  Google Scholar 

  • Heilbronner, R. L., Sweet, J. J., Morgan, J. L., Larrabee, G. J., Millis, S. R., & Participants, C. (2009). American Academy of Clinical Neuropsychology consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 23(7), 1093–1129. doi:10.1080/13854040903155063

    Article  PubMed  Google Scholar 

  • IBM Corp. Released (2010). IBM SPSS Statistics for Windows, Version 19.0. Armonk, NY: IBM Corp.

  • Lange, R. T., Iverson, G. L., Brickell, T. A., Staver, T., Pancholi, S., Bhagwat, A., & French, L. M. (2013). Clinical utility of the Conners' Continuous Performance Test-II to detect poor effort in U.S. military personnel following traumatic brain injury. Psychological Assessment, 25(2), 339–352. doi:10.1037/a0030915

    Article  PubMed  Google Scholar 

  • Larrabee, G. J. (2003). Detection of malingering using atypical performance patterns on standard neuropsychological tests. The Clinical Neuropsychologist, 17, 410–425.

    Article  PubMed  Google Scholar 

  • Larrabee, G. J. (2005). Assessment of malingering. In G. J. Larrabee (Ed.), Forensic neuropsychology: a scientific approach. New York: Oxford University Press.

    Google Scholar 

  • Larrabee, G. J. (2008). Aggregation across multiple indicators improves the detection of malingering: relationship to likelihood ratios. The Clinical Neuropsychologist, 22(4), 666–679.

    Article  PubMed  Google Scholar 

  • Larrabee, G. L. (2009). Malingering scales for the Continuous Recognition Memory Test and the Continuous Visual Memory Test. The Clinical Neuropsychologist, 23(1), 167–180.

  • Larrabee, G. J. (2014). False-positive rates associated with the use of multiple performance and symptom validity tests. Archives of Clinical Neuropsychology, 29(4), 364–373. doi:10.1093/arclin/acu019

    Article  PubMed  Google Scholar 

  • Larrabee, G. J. (2007). Assessment of malingered neuropsychological deficits. New York: Oxford University Press.

  • Lezak, M. D., Howieson, D. B., Bigler, E. D., & Tranel, D. (2012). Neuropsychological assessment (5th ed.). Oxford: Oxford University Press.

    Google Scholar 

  • Lu, P. H., Boone, K. B., Cozolino, L., & Mitchell, C. (2003). Effectiveness of the Rey-Osterrieth Complex Figure Test and the Meyers and Meyers recognition trial in the detection of suspect effort. Clinical Neuropsychologist, 17, 426–440.

    Article  PubMed  Google Scholar 

  • Medici, R. (2013). The use of multiple performance validity tests. Journal of Forensic Practice, 13(1), 68–78. doi:10.1080/15228932.2013.746909

    Article  Google Scholar 

  • Meyers, J. E., Miller, R. M., Thompson, L. M., Scalese, A. M., Allred, B. C., Rupp, Z. W., ... Lee, A. J. (2014). Using likelihood ratios to detect invalid performance with performance validity measures. Archives of Clinical Neuropsychology, 29, 224–235.

    Article  PubMed  Google Scholar 

  • Meyers, J. E., & Volbrecht, M. E. (2003). A validation of multiple malingering detection methods in a large clinical sample. Archives of Clinical Neuropsychology, 18, 261–276.

    Article  PubMed  Google Scholar 

  • Mittenberg, W., Patton, C., Canyock, E. M., & Condit, D. C. (2002). Base rates of malingering and symptom exaggeration. Journal of Clinical and Experimental Neuropsychology, 4(8), 1094–1102.

    Article  Google Scholar 

  • Morey, L. C. (2007). Personality Assessment Inventory professional manual (2nd ed.). Lutz: Psychological Assessment Resources.

    Google Scholar 

  • Nelson, N. W., Boone, K., Dueck, A., Wagener, L., Lu, P., & Grills, C. (2003). Relationship between eight measures of suspect effort. The Clinical Neuropsychologist, 17(2), 263–272.

    Article  PubMed  Google Scholar 

  • O’Bryant, Engel, Kleiner, Vasterling, & Black. (2007). Test of memory malingering (TOMM) trial 1 as a screening measure for insufficient effort. The Clinical Neuropsychologist, 21(3), 511–21.

    Article  PubMed  Google Scholar 

  • Ord, J.S., Boettcher, A.C., Greve, K.W., & Bianchini, K.J. (2010). Detection of malingering in mild traumatic brain injury with the Conners’ Continuous Performance Test–II. Journal of Clinical and Experimental Neuropsychology, 32(4). doi:10.1080/13803390903066881

  • Ortega, A., Labrenz, S., Markowitsch, H. J., & Piefke, M. (2013). Diagnostic accuracy of a Bayesian latent group analysis for the detection of malingering-related poor effort. The Clinical Neuropsychologist, 27(6), 1019–1042.

    Article  PubMed  Google Scholar 

  • Ortega, A., Wagenmakers, E.-J., Lee, M. D., Markowitsch, H. J., & Piefke, M. (2012). A Bayesian latent group analysis for detecting poor effort in assessment of malingering. Archives of Clinical Neuropsychology, 27, 453–465.

    Article  PubMed  Google Scholar 

  • Pearson. (2009). Advanced clinical solutions for use with WAIS-IV and WMS-IV. San Antonio: Pearson Education.

    Google Scholar 

  • Powell, M. R., Locke, D. E., Smigielski, J. S., & McCrea, M. (2011). Estimating the diagnostic value of the trail making test for suboptimal effort in acquired brain injury rehabilitation patients. The Clinical Neuropsychologist, 25(1), 108–118. doi:10.1080/13854046.2010.532912

    Article  PubMed  Google Scholar 

  • Reese, C. S., Suhr, J. A., & Riddle, T. L. (2012). Exploration of malingering indices in the Wechsler Adult Intelligence Scale-Fourth Edition Digit Span subtest. Archives of Clinical Neuropsychology, 27(2), 176–181. doi:10.1093/arclin/acr117

    Article  PubMed  Google Scholar 

  • Root, J. C., Robbins, R. N., Chang, L., & Van Gorp, W. G. (2006). Detection of inadequate effort on the California Verbal Learning Test-Second edition: forced choice recognition and critical item analysis. Journal of the International Neuropsychological Society, 12(5), 688–696.

    Article  PubMed  Google Scholar 

  • Schretlen, D. J., Testa, S. M., Winicki, J. M., Pearlson, G. D., & Gordon, B. (2008). Frequency and bases of abnormal performance by healthy adults on neuropsychological testing. Journal of the International Neuropsychological Society, 14, 436–445. doi:10.1017/S1355617708080387

    Article  PubMed  Google Scholar 

  • Schroeder, R. W., & Marshall, P. S. (2011). Evaluation of the appropriateness of multiple symptom validity indices in psychotic and non-psychotic psychiatric populations. The Clinical Neuropsychologist, 25(3), 437–453. doi:10.1080/13854046.2011.556668

    Article  PubMed  Google Scholar 

  • Schutte, C., & Axelrod, B. N. (2013). Use of embedded cognitive symptom validity measures in mild traumatic brain injury cases. In D. A. Carone & S. S. Bush (Eds.), Mild traumatic brain injury: Symptom validity assessment and malingering (pp. 159–181). New York: Springer Publishing Company.

    Google Scholar 

  • Silverberg, N. D., Hanks, R. A., Buchanan, L., Fichtenberg, N., & Millis, S. R. (2008). Detecting response bias with performance patterns on an expanded version of the Controlled Oral Word Association Test. The Clinical Neuropsychologist, 22(1), 140–157.

    Article  PubMed  Google Scholar 

  • Slick, D. J. (1996). The Victoria Symptom Validity Test: a new clinical measure of response bias (Doctoral dissertation, University of Victoria, 1996). Dissertation Abstracts International, 59, 6114.

    Google Scholar 

  • Slick, D. J., Sherman, E. M. S., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13, 465–473.

    Article  Google Scholar 

  • Sollman, M. J., & Berry, D. T. R. (2011). Detection of inadequate effort on neuropsychological testing: a meta-analytic update and extension. Archives of Clinical Neuropsychology, 26, 774–789.

    Article  PubMed  Google Scholar 

  • Sugarman, M.A. & Axelrod, B.N. (2014). Embedded measures of performance validity using verbal fluency tests in a clinical sample. Applied Neuropsychology (online publication ahead of print).

  • Teichner, G., & Wagner, M. (2004). The Test of Memory Malingering: normative data from cognitively intact, cognitively impaired, and elderly patients with dementia. Archives of Clinical Neuropsychology, 19, 455–464.

    Article  PubMed  Google Scholar 

  • Tombaugh, T. N. (1996). Test of Memory Malingering (TOMM). North Tonowanda: Multi-Health Systems.

    Google Scholar 

  • Victor, T. L., Boone, K. B., Serpa, J. G., Buehler, J., & Ziegler, E. A. (2009). Interpreting the meaning of multiple symptom validity test failure. The Clinical Neuropsychologist, 23, 297–313. doi:10.1080/13854040802232682

    Article  PubMed  Google Scholar 

  • Walter, J., Morris, J., Swier-Vosnos, A., & Plisken, N. (2014). Effects of severity of dementia on a symptom validity measure. The Clinical Neuropsychologist, 28(7), 1197–1208.

    Article  PubMed  Google Scholar 

  • Whiteside, D., Wald, D., & Busse, M. (2011). Classification accuracy of multiple visual spatial measures in the detection of suspect effort. The Clinical Neuropsychologist, 25(2), 287–301. doi:10.1080/13854046.2010.538436

    Article  PubMed  Google Scholar 

  • Wolfe, P. L., Millis, S. R., Hanks, R., Fichtenberg, N., Larrabee, G. J., & Sweet, J. J. (2010). Effort indicators within the California Verbal Learning Test-II (CVLT-II). The Clinical Neuropsychologist, 24(1), 153–168. doi:10.1080/13854040903107791

    Article  PubMed  Google Scholar 

  • Young, J.C., Caron, J.E., Baughman, B.C., & Sawyer R.J. (2012). Detection of suboptimal effort with symbol span: development of a new embedded index. Archives of Clinical Neuropsychology, 27(2). doi:10.1093/arclin/acr109

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anthony P. Odland.

Appendix

Appendix

Tables 6, 7, 8, 9, 10, 11, 12, and 13

Table 6 Aggregate false-positive rates for n correlated (r = 0.1) PVTs/SVTs with 85, 90, and 95 % specificities
Table 7 Aggregate false-positive rates for n correlated (r = 0.2) PVTs/SVTs with 85, 90, and 95 % specificities
Table 8 Aggregate false-positive rates for n correlated (r = 0.4) PVTs/SVTs with 85, 90, and 95 % specificities
Table 9 Aggregate false-positive rates for n correlated (r = 0.5) PVTs/SVTs with 85 %, 90 %, and 95 % specificities
Table 10 Aggregate false-positive rates for n correlated (r = 0.6) PVTs/SVTs with 85, 90, and 95 % specificities
Table 11 Aggregate false-positive rates for n correlated (r = 0.7) PVTs/SVTs with 85, 90, and 95 % specificities
Table 12 Aggregate false-positive rates for n correlated (r = 0.8) PVTs/SVTs with 85, 90, and 95 % specificities
Table 13 Aggregate false-positive rates for n correlated (r = 0.9) PVTs/SVTs with 85, 90, and 95 % specificities

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Odland, A.P., Lammy, A.B., Martin, P.K. et al. Advanced Administration and Interpretation of Multiple Validity Tests. Psychol. Inj. and Law 8, 46–63 (2015). https://doi.org/10.1007/s12207-015-9216-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12207-015-9216-4

Keywords

Navigation