Skip to main content

Relative Rates of Invalidity for the Test of Memory Malingering and the Dot Counting Test Among Spanish-Speaking Patients Residing in the USA

Psychological Injury and Law volume 14, pages 269–280 (2021)Cite this article

Abstract

Assessment of performance validity in neuropsychological testing is essential to ensuring the accuracy of an examinee’s cognitive functioning. Although there are many well-validated and reliable performance validity tests (PVTs) available for use with English-speaking populations, there is currently a dearth of literature supporting their use in Spanish-speaking populations living in the USA. This study examined relative rates of invalid performance on three common performance validity tests (i.e., the Dot Counting Test [DCT] and Test of Memory Malingering [TOMM] Trials 1 and 2) in a cross-sectional sample of 112 clinically referred Spanish-speaking patients. Additionally, exploratory analysis of classification accuracy was also conducted for subtests of the Batería Neuropsicológica en Español (BNE) using each PVT as a reference standard. Rates of invalid performance ranged from 23.3 to 31.1% across the three independent PVTs. Exploratory analysis of BNE subtests indicated that Reliable Digit Span Total and Reliable Spatial Span Total and Backward demonstrated acceptable classification accuracy, albeit not across each reference standard. Given the elevated rates of invalidity in this clinically referred Spanish-speaking sample, the results of this study highlight the potential drawbacks of using cut-scores derived from and validated in English-speaking populations, even among some of the most well-validated and robust PVTs (e.g., DCT, TOMM), to interpret PVT results in Spanish-speaking people residing in the USA.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References 

  • Ardila, A., Rosselli, M., & Puente, A. E. (1994). Critical issues in neuropsychology. Plenum Press. https://doi.org/10.1007/978-1-4899-1453-8

    Book  Google Scholar 

  • Boone, K. B. (Ed.). (2007). Assessment of feigned cognitive impairment: A neuropsychological perspective. Guilford Press.

  • Boone, K. B., Lu, P., & Herzberg, D. (2002). The Dot Counting Test manual. Los Angeles, CA: Western Psychological Services.

  • Boone, K., Victor, T., Wen, J., Razani, J., & Ponton, M. (2007). The association between neuropsychological scores and ethnicity, language, and acculturation variables in a large patient population. Archives of Clinical Neuropsychology, 22(3), 355–365. https://doi.org/10.1016/j.acn.2007.01.010

    Article  PubMed  Google Scholar 

  • Bailey, K. C., Goatte, W., Ramos-Usuga, D., Rivera, D., & Arango-Lasprilla, J. C. (2021). Cross-validation of the utility of Test of Memory Malingering (TOMM) cut-offs in a large Colombian sample. Psychological Injury and Law. https://doi.org/10.1007/s12207-021-09407-z

    Article  Google Scholar 

  • Bailey, K. C., Webber, T. A., Phillips, J. I., Kraemer, L. D. R., Marceaux, J. C., & Soble, J. R. (2019). When time is of the essence: Preliminary findings for a quick administration of the Dot Counting Test. Archives of Clinical Neuropsychology. https://doi.org/10.1093/arclin/acz058

    Article  Google Scholar 

  • Benuto, L. T., & Leany, B. D. (2013). Assessing effort and malingering with the Hispanic client. In L. T. Benuto (Ed.), Guide to psychological assessment with Hispanics (p. 99–112). Springer Science + Business Media. https://doi.org/10.1007/978-1-4614-4412-1_7

  • Burton, V., Vilar-Lopez, R., & Puente, A. E. (2012). Measuring effort in neuropsychological evaluations of forensic cases of Spanish speakers. Archives of Clinical Neuropsychology, 27(3), 262–267. https://doi.org/10.1093/arclin/acs026

    Article  PubMed  Google Scholar 

  • Bush, S., Ruff, R., Troster, A., Barth, J., Koffler, S., Pliskin, N., Reynolds, C., & Silver, C. (2005). Symptom validity assessment: Practice issues and medical necessity. NAN Policy & Planning Committee. Archives of Clinical Neuropsychology, 20(4), 419–426. https://doi.org/10.1016/j.acn.2005.02.002

  • Cirlugea, O. (2014). Assessing effort and malingering with the Asian client. In L. T. Benuto, N. S. Thaler, & B. D. Leany (Eds.), Guide to psychological assessment with Asians (p. 165–180). Springer Science + Business Media. https://doi.org/10.1007/978-1-4939-0796-0_11

  • Cohen, C. D., Jennette, K. J., Rhoads, T., Williams, C. P., Hansen, N. D., Keezer, R. D., Resch, Z. J., Ovsiew, G. P., & Soble, J. R. (2021). All of the accuracy in half of the time: Assessing abbreviated versions of the Test of Memory Malingering in the context of verbal and visual memory impairment. The Clinical Neuropsychologist.

  •  Daugherty, J. C., Puente, A. E., Lozano-Ruiz, A., Perez-Garcia, M. (2020). Cultural accommodations for cutoff scores of embedded performance validity tests in a Spanish college population Applied Neuropsychology: Adult 1–8 https://doi.org/10.1080/23279095.2020.1846536

  • Daugherty, J. C., Querido, L., Quiroz, N., Wang, D., Hidalgo-Ruzzante, N., Fernandes, S., Pérez-García, M., & De los Reyes-Aragon, C. J., Pires, R., & Valera, E. (2021). The Coin in Hand-Extended Version: Development and validation of a multicultural performance validity test. Assessment, 28(1), 186–198. https://doi.org/10.1177/1073191119864652

    Article  PubMed  Google Scholar 

  • Denning, J. H. (2012). The efficiency and accuracy of the Test of Memory Malingering Trial 1, errors on the first 10 items of the Test of Memory Malingering, and five embedded measures in predicting invalid test performance. Archives of Clinical Neuropsychology, 27(4), 417–432. https://doi.org/10.1093/arclin/acs044

    Article  PubMed  Google Scholar 

  • Denning, J. H. (2021). When 10 is enough: Errors on the first 10 items of the Test of Memory Malingering (TOMMe10) and administration time predict freestanding performance validity tests (PVTs) and underperformance on memory measures. Applied Neuropsychology: Adult, 28(1), 35–47. https://doi.org/10.1080/23279095.2019.1588122

    Article  Google Scholar 

  • Artiola-i-Fortuny, L., Hermosillo Romo, D. H., Heaton, R. K., & Pardee, R. E., III. (1999). Manual de normas y procedimientos para la Batería Neuropsicológica en Español. Tucson, AZ: mPress.

  • Gasquoine, P. G., Weimer, A. A., & Amador, A. (2017). Specificity rates for non-clinical, bilingual, Mexican Americans on three popular performance validity measures. The Clinical Neuropsychologist, 31(3), 587–597. https://doi.org/10.1080/13854046.2016.1277786

    Article  PubMed  Google Scholar 

  • Green, P. (2003). Green’s Word Memory Test for Windows: User’s manual. Edmonton, Alberta, Canada: Green’s.

  • Green, R. F., & Martinez, J. H. (1968). Manual para la Escala de lnteligencia Wechsler para Adulto. Traduccion y adaptacion para Puerto Rico. Psychological Corporation.

    Google Scholar 

  • Greiffenstein, M. F., Baker, W. J., & Gola, T. (1994). Validation of malingered amnesia measures with a large clinical sample. Psychological Assessment, 6, 218–224. https://doi.org/10.1037/1040-3590.6.3.218

    Article  Google Scholar 

  • Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013). Applied logistic regression (3rd Ed.). John Wiley & Sons.

  • Kim, M., Han, H.-R., & Phillips, L. (2003). Metric equivalence assessment in cross-cultural research: Using an example of the Center for Epidemiological Studies-Depression Scale. Journal of Nursing Measurement, 11(1), 5–18. https://doi.org/10.1891/jnum.11.1.5.52061

    Article  PubMed  Google Scholar 

  • Kraemer, L. D. R., Soble, J. R., Phillips, J. I., Webber, T. A., Fullen, C. T., Highsmith, J. M., Alverson, W. A., & Critchfield, E. C. (2020). Minimizing evaluation time while maintaining accuracy: Cross-validation of the Test of Memory Malingering (TOMM) Trial 1 and first 10-item errors as briefer performance validity tests. Psychological Assessment, 32, 442–450.

    Article  Google Scholar 

  • Llorente, A. M. (2008). American population estimates, trends in American immigration, and neuropsychology: Influences on assessment and inferential processes with Hispanic populations. In A. M. Llorente (Ed.), Principles of neuropsychological assessment with Hispanics: Theoretical foundations and clinical practice (pp. 29–46). Springer. https://doi.org/10.1007/978-0-387-71758-6

  • Llorente, A. M., & Pontόn, M. O., Taussig, I. M., & Satz, P. (1999). Patterns of American immigration and their influence on the acquisition of neuropsychological norms for Hispanics. Archives of Clinical Neuropsychology, 14(7), 603–614.

    PubMed  Google Scholar 

  • Martin, P. K., & Schroeder, R. W. (2020). Base rates of invalid test performance across clinical non-forensic contexts and settings. Archives of Clinical Neuropsychology, 35(6), 717–725. https://doi.org/10.1093/arclin/acaa017

    Article  PubMed  Google Scholar 

  • Martin, P. K., Schroeder, R. W., Olsen, D. H., Maloy, H., Boettcher, A., Ernst, N., & Okut, H. (2020). A systematic review and meta-analysis of the Test of Memory Malingering in adults: Two decades of deception detection. The Clinical Neuropsychologist, 34(1), 88–119. https://doi.org/10.1080/13854046.2019.1637027

    Article  PubMed  Google Scholar 

  • Rivera Mindt, M., Byrd, D., Saez, P., & Manly, J. (2010). Increasing culturally competent neuropsychological services for ethnic minority populations: A call to action. The Clinical Neuropsychologist, 24(3), 429–453. https://doi.org/10.1080/13854040903058960

    Article  PubMed  Google Scholar 

  • Nijdam-Jones, A., & Rosenfeld, B. (2017). Cross-cultural feigning assessment: A systematic review of feigning instruments used with linguistically, ethnically, and culturally diverse samples. Psychological Assessment, 29(11), 1321–1336. https://doi.org/10.1037/pas0000438

    Article  PubMed  Google Scholar 

  • Nijdam-Jones, A., Rivera, D., Rosenfeld, B., & Arango-Lasprilla, J. C. (2017). A cross-cultural analysis of the Test of Memory Malingering among Latin American Spanish-speaking adults. Law and Human Behavior, 41(5), 422–428. https://doi.org/10.1037/lhb0000250

    Article  PubMed  Google Scholar 

  • Nijdam-Jones, A., Rivera, D., Rosenfeld, B., & Arango-Lasprilla, J. C. (2019). The effect of literacy and culture on cognitive effort test performance: An examination of the Test of Memory Malingering in Colombia. Journal of Clinical and Experimental Neuropsychology, 41(10), 1015–1023. https://doi.org/10.1080/13803395.2019.1644294

    Article  PubMed  Google Scholar 

  • Ovsiew, G. P., Carter, D. A., Rhoads, T., Resch, Z. J., Jennette K. J., & Soble, J. R. (2021). Concordance between standard and abbreviated administrations of the Test of Memory Malingering: Implications for streamlining performance validity assessment. Psychological Injury and Law.

  • Ovsiew, G. P., Resch, Z. J., Nayar, K., Williams, C. P., Soble, J. R. (2020). Not so fast! Limitations of processing speed and working memory indices as embedded performance validity tests in a mixed neuropsychiatric sample Journal of Clinical and Experimental Neuropsychology 1–12 https://doi.org/10.1080/13803395.2020.1758635

  • Pearson. (2009). Advanced clinical solutions for WAIS®-IV and WMS®-IV: Clinical and interpretive manual. San Antonio, TX: Pearson.

  • Peña, E. D. (2007). Lost in translation: Methodological considerations in cross-cultural research. Child Development, 78, 1255–1264. https://doi.org/10.1111/j.1467-8624.2007.01064.x

    Article  PubMed  Google Scholar 

  • Pliskin, J. I., Stern, S. D., Resch, Z. J., Saladino, K. F., Ovsiew, G. P., Carter, D. A., & Soble, J. R. (2020). Comparing the psychometric properties of eight embedded performance validity tests in the Rey Auditory Verbal Learning Test, Wechsler Memory Scale Logical Memory, and Brief Visuospatial Memory Test-Revised recognition trials for detecting invalid neuropsychological test performance. Assessment. https://doi.org/10.1177/1073191120929093

    Article  PubMed  Google Scholar 

  • Pontόn, M. O., & Ardila, A. (1999). The future of neuropsychology with Hispanic populations in the United States. Archives of Clinical Neuropsychology, 14(7), 565–580.

    Google Scholar 

  • Puente, A. E., & Ardila, A. (2000). Neuropsychological assessment of Hispanics. In E. Fletcher-Janzen, T. L. Strickland, & C. R. Reynolds (Eds.), Handbook of cross-cultural neuropsychology (pp. 87–104). Springer. https://doi.org/10.1007/978-1-4615-4219-3_7

  • Ramírez, R. M., Chirivella-Garrido, J., Caballero, M. C., Ferri-Campos, J., & Noé-Sebastián, E. (2004). Inteligencia, memoria y simulación: Correlaciones entre instrumentos de medida. Rev Neurol., 38(1), 28–33. https://doi.org/10.33588/rn.3801.2003533

  • Relevance 2050 Initiative. American Academy of Clinical Neuropsychology (AACN). https://theaacn.org/relevance-2050-initiative/

  • Reyes, A., LaBode-Richman, V., Salinas, L., & Barr, W. B. (2019). WHO-AVLT recognition trial: Initial validation for a new malingering index for Spanish-speaking patients. Applied Neuropsychology: Adult, 26(6), 564–572. https://doi.org/10.1080/23279095.2018.1470974

    Article  Google Scholar 

  • Rhoads, T., Resch, Z. J., Ovsiew, G. P., White, D. J., Abramson, D. A., & Soble, J. R. (2021). Every second counts: A comparison of four dot counting test scoring procedures for detecting invalid neuropsychological test performance. Psychological Assessment, 33(2), 133–141. https://doi.org/10.1037/pas0000970

    Article  PubMed  Google Scholar 

  • Rivera, D., Perrin, P. B., Weiler, G., Ocampo-Barba, N., Aliaga, A., Rodríguez, W., Rodríguez-Agudelo, Y., Aguayo, A., Longoni, M., Trapp, S., Esenarro, L., & Arango-Lasprilla, J. C. (2015). Test of Memory Malingering (TOMM): Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation, 37(4), 719–735. https://doi.org/10.3233/NRE-151287

    Article  PubMed  Google Scholar 

  • Robles, L., López, E., Salazar, X., Boone, K. B., & Glaser, D. F. (2015). Specificity data for the b Test, Dot Counting Test, Rey-15 Item Plus Recognition, and Rey Word Recognition Test in monolingual Spanish-speakers. Journal of Clinical and Experimental Neuropsychology, 37(6), 614–621. https://doi.org/10.1080/13803395.2015.1039961

    Article  PubMed  Google Scholar 

  • Ryan, C. (2013). Language use in the United States: 2011. American Community Survey Reports, U.S. Census Bureau. https://www2.census.gov/library/publications/2013/acs/acs-22/acs-22.pdf

  • Salazar, X. F., Po, H. L., Wen, J., and Boone, K. B. (2007). The use of effort tests in ethnic minorities and in non-English speaking and English as a second language populations. In K. B. Boone (Ed.), Assessment of feigned cognitive impairment: A neuropsychological perspective. New York: Guilford Press.

  • Schroeder, R. W., Twumasi-Ankrah, P., Baade, L. E., & Marshall, P. S. (2012). Reliable digit span: A systematic review and cross-validation study. Assessment, 19(1), 21–30. https://doi.org/10.1177/1073191111428764

    Article  PubMed  Google Scholar 

  • Singh, S., Hamilton, C. E., & Soble, J. R. (2021). Integrating culturally informed qualitative data in neuropsychological evaluations of non-native English speakers. Professional Psychology: Research and Practice. Advance online publication. https://doi.org/10.1037/pro0000389

  • Strutt, A. M., Scott, B. M., Lozano, V. J., Tieu, P. G., & Peery, S. (2012). Assessing sub-optimal performance with the Test of Memory Malingering in Spanish speaking patients with TBI. Brain Injury, 26(6), 853–863. https://doi.org/10.3109/02699052.2012.655366

    Article  PubMed  Google Scholar 

  • Sue, D. W. (2001). Multidimensional facets of cultural competence. The Counseling Psychologist, 29(6), 790–821. https://doi.org/10.1177/0011000001296002

    Article  Google Scholar 

  • Sweet, J. J., Heilbronner, R. L., Morgan, J. E., Larrabee, G. J., Rohling, M. L., Boone, K. B., Kirkwood, M. W., Schroeder, R. W., Suhr, J. A., & Conference Participants. (2021). American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering. The Clinical Neuropsychologist, 1–54. https://doi.org/10.1080/13854046.2021.1896036

  • Weiss, R. A., & Rosenfeld, B. (2012). Navigating cross-cultural issues in forensic assessment: Recommendations for practice. Professional Psychology: Research and Practice, 43, 234–240. https://doi.org/10.1037/a0025850

    Article  Google Scholar 

  • Tombaugh, T. N. (1996). TOMM: Test of Memory Malingering. Toronto, Ontario, Canada: Multi-Health Systems.

  • Tonks, J., Whitfield, C. K., Williams, W. H., Slater, A. M., & Frampton, I. J. (2020). “Must try harder.” Is effort and performance validity testing a necessary part of pediatric neuropsychological assessment? Applied Neuropsychology: Child, 9(2), 97–105. https://doi.org/10.1080/21622965.2018.1524766

  • U.S. Census Bureau. (2019). Quick facts United States. Census.gov. https://www.census.gov/quickfacts/fact/table/US/RHI725218

  • Vespa, J., Medina, L., & Armstrong, D. M. (2020). Demographic turning points for the United States: Population projections for 2020 to 2060. Current Population Reports, P25–1144, U.S. Census Bureau.

  • Vilar-López, R., Santiago-Ramajo, S., Gomez-Rio, M., Verdejo-Garcia, A., Llamas, J., & Perez-Garcia, M. (2007). Detection of malingering in a Spanish population using three specific malingering tests. Archives of Clinical Neuropsychology, 22(3), 379–388. https://doi.org/10.1016/j.acn.2007.01.012

    Article  PubMed  Google Scholar 

  • Vilar-López, R., Gomez-Rio, M., Santiago-Ramajo, S., Rodriguez-Fernandez, A., Puente, A., & Perez-Garcia, M. (2008). Malingering detection in a Spanish population with a known-groups design. Archives of Clinical Neuropsychology, 23(4), 365–377. https://doi.org/10.1016/j.acn.2008.01.007

    Article  PubMed  Google Scholar 

  • Webber, T. A., Bailey, K. C., Alverson, W. A., Critchfield, E. A., Bain, K. M., Messerly, J. M., O’Rourke, J. J. F., Kirton, J. W., Fullen, C., Marceaux, J. C., & Soble, J. R. (2018). Further validation of the Test of Memory Malingering (TOMM) Trial 1: Examination of false positives and convergence with other validity measures. Psychological Injury and Law, 11, 325–335.

    Article  Google Scholar 

  • Webber, T. A., Critchfield, E. A., & Soble, J. R. (2020). Convergent, discriminant, and concurrent validity of nonmemory-based performance validity tests. Assessment, 27(7), 1399–1415. https://doi.org/10.1177/1073191118804874

    Article  PubMed  Google Scholar 

  • Wechsler, D. (1997). WAIS-III administration and scoring manual. Psychological Corporation.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychiatry, University of Illinois College of Medicine, Chicago, IL, USA

    Tasha Rhoads, Sophie I. Leib, Zachary J. Resch, Karen S. Basurto, Kyle J. Jennette & Jason R. Soble

  2. Department of Psychology, Rosalind Franklin University of Medicine and Science, 3333 Green Bay Road, North Chicago, IL, 60064, USA

    Tasha Rhoads, Sophie I. Leib & Zachary J. Resch

  3. Department of Psychology, City University of New York, John Jay College of Criminal Justice, New York, NY, USA

    Liliam R. Castillo

  4. Department of Neurology, University of Illinois College of Medicine, Chicago, IL, USA

    Jason R. Soble

Authors
  1. Tasha Rhoads
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sophie I. Leib
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zachary J. Resch
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Karen S. Basurto
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liliam R. Castillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kyle J. Jennette
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jason R. Soble
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tasha Rhoads.

Ethics declarations

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Rhoads, T., Leib, S.I., Resch, Z.J. et al. Relative Rates of Invalidity for the Test of Memory Malingering and the Dot Counting Test Among Spanish-Speaking Patients Residing in the USA. Psychol. Inj. and Law 14, 269–280 (2021). https://doi.org/10.1007/s12207-021-09423-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12207-021-09423-z

Keywords

  • Performance validity
  • Neuropsychological assessment
  • Spanish speakers
  • Test of Memory Malingering
  • Dot Counting Test