Skip to main content
SpringerLink
Log in

Optimization of Performance Validity Test (PVT) Cutoffs across Healthy and Non-Referred Clinical Research Samples

  • Published:
Psychological Injury and Law Aims and scope Submit manuscript

Abstract

Performance validity tests (PVTs) effectively detect suboptimal test performance, but cutoff scores for a given test may differ between populations. This research examines how optimal PVT cutoffs differ in a collegiate research population when mental health and clinical conditions are considered. Healthy controls (n = 328) and non-referred students with self-reported clinical conditions (n = 42) were assigned to perform their best while others simulated ADHD symptoms (n = 123). PVT indices were derived from a stand-alone measure (Victoria Symptom Validity Test) and embedded measures (California Verbal Learning Test – Second Edition; Wechsler Adult Intelligence Scale – Fourth Edition, Digit Span). PVT cutoffs with the highest sensitivity, while maintaining adequate specificity, were identified when the control groups were considered together, and when students with reported clinical conditions were considered separately. Mean differences in PVT performances were found between the simulation group and control groups, but not between clinical and nonclinical controls. The optimized cutoffs differed for five of eight PVT indices when all controls were considered together versus the clinical control group, only. When discordance was observed, cutoffs tended to be lower (less stringent) for the non-referred clinical control group. Together, these optimized cutoffs tended to be more stringent than previously established cutoffs. This study suggests that PVT cutoffs may be responsibly altered in a research context in the presence of a clinical condition. Future research should investigate if PVT classification accuracies can be improved in clinical and forensic samples while considering clinical conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Notes

  1. Neither demographic comparisons nor between-group comparisons of mean PVT performance meaningfully changed when the HSG and CSG were combined for analyses.

References

  • American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders, fifth edition (DSM-5). Arlington, VA: American Psychiatric Association.

    Book  Google Scholar 

  • An, K. Y., Kaploun, K., Erdodi, L. A., & Abeare, C. A. (2017). Performance validity in undergraduate research participants: A comparison of failure rates across tests and cutoffs. The Clinical Neuropsychologist, 31(1), 193–206.

    Article  Google Scholar 

  • An, K. Y., Zakzanis, K. K., & Joordens, S. (2012). Conducting research with non-clinical healthy undergraduates: Does effort play a role in neuropsychological test performance? Archives of Clinical Neuropsychology, 27(8), 849–857.

    Article  Google Scholar 

  • Axelrod, B. N., Fichtenberg, N. L., Millis, S. R., & Wertheimer, J. C. (2006). Detecting incomplete effort with digit span from the Wechsler adult intelligence scale—Third edition. The Clinical Neuropsychologist, 20(3), 513–523. https://doi.org/10.1080/13854040590967117.

    Article  PubMed  Google Scholar 

  • Blanco, C., Okuda, M., Wright, C., Hasin, D. S., Grant, B. F., Liu, S. M., & Olfson, M. (2008). Mental health of college students and their non–college-attending peers: Results from the national epidemiologic study on alcohol and related conditions. Archives of General Psychiatry, 65(12), 1429–1437.

    Article  Google Scholar 

  • Brooks, B. L., & Ploetz, D. M. (2015). Embedded performance validity on the CVLT-C for youth with neurological disorders. Archives of Clinical Neuropsychology, 30(3), 200–206.

    Article  Google Scholar 

  • Clark, A. L., Amick, M. M., Fortier, C., Milberg, W. P., & McGlinchey, R. E. (2014). Poor performance validity predicts clinical characteristics and cognitive test performance of OEF/OIF/OND veterans in a research setting. The Clinical Neuropsychologist, 28(5), 802–825.

    Article  Google Scholar 

  • Considine, C. M., Weisenbach, S. L., Walker, S. J., McFadden, E. M., Franti, L. M., Bieliauskas, L. A., et al. (2011). Auditory memory decrements, without dissimulation, among patients with major depressive disorder. Archives of Clinical Neuropsychology, 26(5), 445–453.

    Article  Google Scholar 

  • Constantinou, M., Bauer, L., Ashendorf, L., Fisher, J. M., & McCaffrey, R. J. (2005). Is poor performance on recognition memory effort measures indicative of generalized poor performance on neuropsychological tests? Archives of Clinical Neuropsychology, 20(2), 191–198. https://doi.org/10.1016/j.acn.2004.06.002.

    Article  PubMed  Google Scholar 

  • Curtis, K. L., Greve, K. W., Bianchini, K. J., & Brennan, A. (2006). California verbal learning test: Indicators of malingered neurocognitive dysfunction. Assessment, 13(1), 46–61. https://doi.org/10.1177/1073191105285210.

    Article  PubMed  Google Scholar 

  • Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). CVLT-II: California verbal learning test: Adult version. New York, NY: Psychological Corporation.

    Google Scholar 

  • DeRight, J., & Jorgensen, R. S. (2015). I just want my research credit: Frequency of suboptimal effort in a non-clinical healthy undergraduate sample. The Clinical Neuropsychologist, 29(1), 101–117. https://doi.org/10.1080/13854046.2014.989267.

    Article  PubMed  Google Scholar 

  • Erdődi, L. A., Seke, K. R., Shahein, A., Tyson, B. T., Sagar, S., & Roth, R. M. (2017). Low scores on the grooved pegboard test are associated with invalid responding and psychiatric symptoms. Psychology & Neuroscience, 10(3), 325.

    Article  Google Scholar 

  • Frazier, T. W., Frazier, A. R., Busch, R. M., Kerwood, M. A., & Demaree, H. A. (2008). Detection of simulated ADHD and reading disorder using symptom validity measures. Archives of Clinical Neuropsychology, 23(5), 501–509.

    Article  Google Scholar 

  • Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6(3), 218–224. https://doi.org/10.1037/1040-3590.6.3.218.

    Article  Google Scholar 

  • Grote, C. L., Kooker, E. K., Garron, D. C., Nyenhuis, D. L., Smith, C. A., & Mattingly, M. L. (2000). Performance of compensation seeking and non-compensation seeking samples on the Victoria symptom validity test: Cross-validation and extension of a standardization study. Journal of Clinical and Experimental Neuropsychology, 22(6), 709–719. https://doi.org/10.1076/jcen.22.6.709.958.

    Article  PubMed  Google Scholar 

  • Jones, A. (2013). Victoria symptom validity test: Cutoff scores for psychometrically defined malingering groups in a military sample. The Clinical Neuropsychologist, 27(8), 1373–1394. https://doi.org/10.1080/13854046.2013.851740.

    Article  PubMed  Google Scholar 

  • Loring, D. W., Larrabee, G. J., Lee, G. P., & Meador, K. J. (2007). Victoria symptom validity test performance in a heterogenous clinical sample. The Clinical Neuropsychologist, 21(3), 522–531. https://doi.org/10.1080/13854040600611384.

    Article  PubMed  Google Scholar 

  • Martin, P. K., Schroeder, R. W., & Odland, A. P. (2015). Neuropsychologists’ validity testing beliefs and practices: A survey of north American professionals. The Clinical Neuropsychologist, 29(6), 741–776.

    Article  Google Scholar 

  • Odland, A. P., Lammy, A. B., Martin, P. K., Grote, C. L., & Mittenberg, W. (2015). Advanced administration and interpretation of multiple validity tests. Psychological Injury and Law, 8(1), 46–63.

    Article  Google Scholar 

  • Rogers, R. (2008). Clinical assessment of malingering and deception. New York, NY: The Guilford Press.

    Google Scholar 

  • Ross, T. P., Poston, A. M., Rein, P. A., Salvatore, A. N., Wills, N. L., & York, T. M. (2016). Performance invalidity base rates among healthy undergraduate research participants. Archives of Clinical Neuropsychology, 31(1), 97–104. https://doi.org/10.1093/arclin/acv062.

    Article  PubMed  Google Scholar 

  • Rowland, J. A., Miskey, H. M., Brearly, T. W., Martindale, S. L., & Shura, R. D. (2017). Word memory test performance across cognitive domains, psychiatric presentations, and mild traumatic brain injury. Archives of Clinical Neuropsychology, 32(3), 306–315.

    PubMed  Google Scholar 

  • Santos, O. A., Kazakov, D., Reamer, M. K., Park, S. E., & Osmon, D. C. (2014). Effort in college undergraduates is sufficient on the word memory test. Archives of Clinical Neuropsychology, 29(7), 609–613. https://doi.org/10.1093/arclin/acu039.

    Article  PubMed  PubMed Central  Google Scholar 

  • Schroeder, R. W., Twumasi-Ankrah, P., Baade, L. E., & Marshall, P. S. (2012). Reliable digit span: A systematic review and cross-validation study. Assessment, 19(1), 21–30.

    Article  Google Scholar 

  • Silk-Eglit, G. M., Lynch, J. K., & McCaffrey, R. J. (2016). Validation of Victoria symptom validity test cutoff scores among mild traumatic brain injury litigants using a known-groups design. Archives of Clinical Neuropsychology, 31(3), 231–245. https://doi.org/10.1093/arclin/acv108.

    Article  PubMed  Google Scholar 

  • Silk-Eglit, G. M., Stenclik, J. H., Gavett, B. E., Adams, J. W., Lynch, J. K., & McCaffrey, R. J. (2014). Base rate of performance invalidity among non-clinical undergraduate research participants. Archives of Clinical Neuropsychology, 29(5), 415–421. https://doi.org/10.1093/arclin/acu028.

    Article  PubMed  Google Scholar 

  • Slick, D. J., Hopp, G., Strauss, E., & Thompson, G. (1997). The Victoria symptom validity test. Odessa, FL: Psychological Assessment Resources.

    Google Scholar 

  • Slick, D. J., Sherman, E. M., & Iverson, G. L. (1999). Diagnostic criteria for malingered neurocognitive dysfunction: Proposed standards for clinical practice and research. The Clinical Neuropsychologist, 13(4), 545–561.

    Article  Google Scholar 

  • Slick, D. J., Tan, J. E., Strauss, E., Mateer, C. A., Harnadek, M., & Sherman, E. M. (2003). Victoria symptom validity test scores of patients with profound memory impairment: Nonlitigant case studies. The Clinical Neuropsychologist, 17(3), 390–394.

    Article  Google Scholar 

  • Spencer, R., Axelrod, B. N., Drag, L. L., Waldron-Perrine, B., Pangilinan, P. H., & Bieliauskas, L. A. (2013). WAIS-IV reliable digit span is no more accurate than age corrected scaled score as an indicator of invalid performance in a veteran sample undergoing evaluation for mTBI. The Clinical Neuropsychologist, 27(8), 1362–1372. https://doi.org/10.1080/13854046.2013.845248.

    Article  PubMed  Google Scholar 

  • Spencer, R., Tree, H., Drag, L., Pangilinan, P., & Bieliauskas, L. (2010). Extending reliable digit span with the WAIS-IV sequencing task: Preliminary results. Paper presented at the poster presented at the 8th annual meeting for the American Academy of clinical neuropsychology conference, Chicago, IL.

  • Tombaugh, T. N. (1997). The test of memory malingering (TOMM): Normative data from cognitively intact and cognitively impaired individuals. Psychological Assessment, 9(3), 260.

    Article  Google Scholar 

  • Trueblood, W., & Schmidt, M. (1993). Malingering and other validity considerations in the neuropsychological evaluation of mild head injury. Journal of Clinical and Experimental Neuropsychology, 15(4), 578–590. https://doi.org/10.1080/01688639308402580.

    Article  PubMed  Google Scholar 

  • Wechsler, D. (2001). Wechsler test of adult Reading: WTAR. Psychological Corporation.

  • Wechsler, D. (2008). Wechsler adult intelligence scale– Fourth edition (WAIS-IV): Technical manual. San Antonio, TX: NSC Pearson, Inc..

    Google Scholar 

  • Wechsler, D. (2009). The Wechsler individual achievement test-III. TX, US: The Psychological Corporation.

    Google Scholar 

  • Williamson, K. D., Combs, H. L., Berry, D. T., Harp, J. P., Mason, L. H., & Edmundson, M. (2014). Discriminating among ADHD alone, ADHD with a comorbid psychological disorder, and feigned ADHD in a college sample. The Clinical Neuropsychologist, 28(7), 1182–1196.

    Article  Google Scholar 

  • Wolfe, P. L., Millis, S. R., Hanks, R., Fichtenberg, N., Larrabee, G. J., & Sweet, J. J. (2010). Effort indicators within the California verbal learning test-II (CVLT-II). The Clinical Neuropsychologist, 24(1), 153–168. https://doi.org/10.1080/13854040903107791.

    Article  PubMed  Google Scholar 

  • Young, J. C., Sawyer, R. J., Roper, B. L., & Baughman, B. C. (2012). Expansion and re-examination of digit span effort indices on the WAIS-IV. The Clinical Neuropsychologist, 26(1), 147–159. https://doi.org/10.1080/13854046.2011.647083.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, Marquette University, Milwaukee, WI, USA

    David E. Marra, Morgan E. Nitta, Kathryn A. Ritchie, Elizabeth A. Marston & James B. Hoelzle

  2. Department of Clinical and Health Psychology, University of Florida, Gainesville, Florida, USA

    David E. Marra

  3. Department of Neurology, Medical College of Wisconsin, Wauwatosa, WI, USA

    Elisabeth M. Vogt

Authors
  1. David E. Marra
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elisabeth M. Vogt
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Morgan E. Nitta
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kathryn A. Ritchie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elizabeth A. Marston
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. James B. Hoelzle
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David E. Marra.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Control Group Instructions

You will be asked to do a number of different things today that test your attention, memory, and problem solving speed. Some of the things may be really easy for you, but some may be hard. Most people do not answer every question correctly or finish every item, but please try your best on all tasks.

Simulation Group Instructions

You will be asked to do a number of different things today that test your attention, memory, and problem solving speed.

Now, imagine that you are experiencing the following symptoms that make it difficult to complete your coursework.

  • Difficulty paying attention

  • Difficulty remembering information or forgetting important things

  • Feeling easily distracted

  • Losing assignments or other needed things (e.g. keys)

  • Difficulty staying organized

  • Feeling as though you cannot do things as quickly as other students

  • Feeling as though you get tired easily

  • Having trouble planning out your assignments or keeping track of your schedule

As a result of these symptoms, you are not doing well in school. Imagine that this test today will help you get academic accommodations (e.g., extra time on tests, notes from the professor, extra help) because you have not been getting extra help and you are starting to fail classes.

You need to fake or exaggerate these symptoms in order to be certain that you get accommodations. There is at least one test that you will take today that is designed to catch faking. Your goal is to fake symptoms, beat the test, and not let the examiner know you are faking. This will be challenging but it is important to do your best to deceive the examiner.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Marra, D.E., Vogt, E.M., Nitta, M.E. et al. Optimization of Performance Validity Test (PVT) Cutoffs across Healthy and Non-Referred Clinical Research Samples. Psychol. Inj. and Law 13, 66–76 (2020). https://doi.org/10.1007/s12207-020-09372-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12207-020-09372-z

Keywords

Search

Navigation