Validation of a digitally delivered visual paired comparison task: reliability and convergent validity with established cognitive tests

Abstract

Alzheimer’s disease (AD) affects the memory and cognitive function of approximately 5.7 million Americans. Early detection subsequently allows for earlier treatment and improves outcomes. Currently, there exists a validated 30-min eye-tracking cognitive assessment (VPC-30) for predicting AD risk. However, a shorter assessment would improve user experience and improve scalability. Thus, the purposes were to (1) determine convergent validity between the 5-min web camera-based eye-tracking task (VPC-5) and VPC-30, (2) examine the relationship between VPC-5 and gold-standard cognitive tests, and (3) determine the reliability and stability of VPC-5. This prospective study included two healthy cohorts: older adults (65+ years, n = 20) and younger adults (18–46 years, n = 24). Participants were tested on two separate occasions. Visit 1 included the Montreal Cognitive Assessment (MoCA), Digit Symbol Coding test (DSC), NIH Toolbox Cognitive Battery (NIHTB-CB), VPC-30, and VPC-5. Visit 2 occurred at least 14 days later; participants completed the VPC-5, DSC, NIHTB-CB, and dual-task walking assessments (DT). VPC-30 significantly correlated with VPC-5 at the first (p < .001) and second (p = .001) time points. VPC-5 and DSC (p < .01) and Pattern Comparison Processing Speed Test (PSPAC) (p = .01) were also correlated on day 1. Significant associations existed between VPC-5 and DSC (p < .001), Flanker Inhibitory Control Test (p = .05), PSPAC (p < .001), and Picture Sequence Memory Test (p = .02) during day 14 testing session. The test retest reliability of VPC-5 was significant (p < .001). VPC-5 displayed moderate convergent validity with the VPC-30 and gold-standard measures of cognition, while demonstrating strong stability, suggesting it is a valuable assessment for monitoring memory function.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

References

  1. Alzheimer’s Association (2018) 2018 Alzheimer’s disease facts and figures. Alzheimers Dement 14(3):367–429

    Article  Google Scholar 

  2. Alzheimer’s Association (2015) 2015 Alzheimer’s disease facts and figures. Alzheimers Dement 11(3):332–384. https://doi.org/10.1016/j.jalz.2015.02.003

    Article  Google Scholar 

  3. Best JR, Liu-Ambrose T, Boudreau RM, Ayonayon HN, Satterfield S, Simonsick EM et al (2016) An evaluation of the longitudinal, bidirectional associations between gait speed and cognition in older women and men. J Gerontol Ser A Biol Sci Med Sci 71(12):1616–1623. https://doi.org/10.1093/gerona/glw066

    Article  Google Scholar 

  4. Bland A, Roiser J, Mehta M, Schei T, Boland H, Campbell-Meiklejohn D et al (2016) EMOTICOM: a neuropsychological test battery to evaluate emotion. Motiv Impulsiv Soc Cogn 10. https://doi.org/10.3389/fnbeh.2016.00025

  5. Bott N, Madero EN, Glenn J, Lange A, Anderson J, Newton D, Brennan A, Buffalo EA, Rentz D, Zola S (2018) Device-embedded cameras for eye tracking–based cognitive assessment: validation with paper-pencil and computerized cognitive composites. J Med Internet Res 20(7):e11143. https://doi.org/10.2196/11143

    Article  PubMed  PubMed Central  Google Scholar 

  6. Bott NT, Lange A, Rentz D, Buffalo E, Clopton P, Zola S (2017) Web camera based eye tracking to assess visual memory on a visual paired comparison task. Front Neurosci:11 https://doi.org/10.3389/fnins.2017.00370

  7. Brustio PR, Magistro D, Zecca M, Rabaglietti E, Liubicich ME (2017) Age-related decrements in dual-task performance: comparison of different mobility and cognitive tasks. A cross sectional study. PLoS ONE 12(7):e0181698. https://doi.org/10.1371/journal.pone.0181698

    Article  PubMed  PubMed Central  Google Scholar 

  8. Crowe SF, Benedict T, Enrico J, Mancuso N, Matthews C, Wallace J (1999) Cognitive determinants of performance on the digit symbol-coding test, and the symbol search test of the Wais-III, and the symbol digit modalities test: an analysis in a healthy sample. Aust Psychol 34(3):204–210. https://doi.org/10.1080/00050069908257455

    Article  Google Scholar 

  9. Crutcher MD, Calhoun-Haney R, Manzanares CM, Lah JJ, Levey AI, Zola SM (2009) Eye tracking during a visual paired comparison task as a predictor of early dementia. Am J Alzheimers Dis Other Dement 24(3):258–266. https://doi.org/10.1177/1533317509332093

    Article  Google Scholar 

  10. Danckert SL, Craik FIM (2013) Does aging affect recall more than recognition memory? Psychol Aging 28(4):902–909. https://doi.org/10.1037/a0033263

    Article  PubMed  Google Scholar 

  11. Donohue MC, Sperling RA, Salmon DP, Rentz DM, Raman R, Thomas RG et al (2014) The preclinical Alzheimer cognitive composite: measuring amyloid-related decline. JAMA Neurol 71(8):961–970. https://doi.org/10.1001/jamaneurol.2014.803

    Article  PubMed  PubMed Central  Google Scholar 

  12. Fagan JF (1970) Memory in the infant. J Exp Child Psychol 9(2):217–226. https://doi.org/10.1016/0022-0965(70)90087-1

    Article  PubMed  Google Scholar 

  13. Glenn J, Vincenzo J, Canella C, Binns A, Gray M (2015) Glenn 2015 - Habitual and maximal dual-task gait speeds among sedentary, recreationally active, and masters athlete late-middle aged adults. J Aging Phys Act 23(3):433–437

    Article  Google Scholar 

  14. Goldberg TE, Harvey PD, Wesnes KA, Snyder PJ, Schneider LS (2015) Practice effects due to serial cognitive assessment: implications for preclinical Alzheimer’s disease randomized controlled trials. Alzheimer's & Dementia 1(1):103–111. https://doi.org/10.1016/j.dadm.2014.11.003

    Article  Google Scholar 

  15. Hanninen T, Hallikainen M, Tuomainen S, Vanhanen M, Soininen H (2002) Prevalence of mild cognitive impairment: a population-based study in elderly subjects. Acta Neurol Scand 106(3):148–154. https://doi.org/10.1034/j.1600-0404.2002.01225.x

    CAS  Article  PubMed  Google Scholar 

  16. Hansen DW, Pece AEC (2005) Eye tracking in the wild. Comput Vis Image Underst 98(1):155–181. https://doi.org/10.1016/j.cviu.2004.07.013

    Article  Google Scholar 

  17. Harvey PD (2012) Clinical applications of neuropsychological assessment. Dialogues Clin Neurosci 14(1):91–99

    PubMed  PubMed Central  Google Scholar 

  18. Hausdorff JM, Rios DA, Edelberg HK (2001) Gait variability and fall risk in community-living older adults: a 1-year prospective study. Arch Phys Med Rehabil 82(8):1050–1056. https://doi.org/10.1053/apmr.2001.24893

    CAS  Article  PubMed  Google Scholar 

  19. Heaton RK, Akshoomoff N, Tulsky D, Mungas D, Weintraub S, Dikmen S, Beaumont J, Casaletto KB, Conway K, Slotkin J, Gershon R (2014) Reliability and validity of composite scores from the NIH toolbox cognition battery in adults. J Int Neuropsychol Soc 20(06):588–598. https://doi.org/10.1017/S1355617714000241

    Article  PubMed  PubMed Central  Google Scholar 

  20. Kremen WS, Jak AJ, Panizzon MS, Spoon KM, Franz CE, Thompson WK, Jacobson KC, Vasilopoulos T, Vuoksimaa E, Xian H, Toomey R, Lyons MJ (2014) Early identification and heritability of mild cognitive impairment. Int J Epidemiol 43(2):600–610. https://doi.org/10.1093/ije/dyt242

    Article  PubMed  Google Scholar 

  21. Lagun D, Manzanares C, Zola SM, Buffalo EA, Agichtein E (2011) Detecting cognitive impairment by eye movement analysis using automatic classification algorithms. J Neurosci Methods 201(1):196–203. https://doi.org/10.1016/j.jneumeth.2011.06.027

    Article  PubMed  PubMed Central  Google Scholar 

  22. Lin Y-T, Lin R-Y, Lin Y-C, Lee GC (2013) Real-time eye-gaze estimation using a low-resolution webcam. Multimed Tools Appl 65(3):543–568. https://doi.org/10.1007/s11042-012-1202-1

    Article  Google Scholar 

  23. Maruff P, Thomas E, Cysique L, Brew B, Collie A, Snyder P, Pietrzak R (2009) Validity of the CogStat Brief battery: relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, Schizophrenia, and AIDS Dementia Complex. Arch Clin Neuropsychol 24:165–1178. https://doi.org/10.1093/arclin/acp010

    Article  PubMed  Google Scholar 

  24. McCulloch KL, Mercer V, Giuliani C, Marshall S (2009) Development of a clinical measure of dual-task performance in walking: reliability and preliminary validity of the walking and remembering test. J Geriatr Phys Ther 32:8

    Article  Google Scholar 

  25. Montero-Odasso M, Casas A, Hansen KT, Bilski P, Gutmanis I, Wells JL, Borrie MJ (2009) Quantitative gait analysis under dual-task in older people with mild cognitive impairment: a reliability study. J NeuroEng Rehabil 6(1):35. https://doi.org/10.1186/1743-0003-6-35

    Article  PubMed  PubMed Central  Google Scholar 

  26. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I et al (2005) The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment: MOCA: a Brief Screening Tool For MCI. J Am Geriatr Soc 53(4):695–699. https://doi.org/10.1111/j.1532-5415.2005.53221.x

    Article  Google Scholar 

  27. Petridis S, Giannakopoulos T, Spyropoulos CD (2013) Unobtrusive low cost pupil size measurements using web cameras. ArXiv:1311.7327 [Cs]. Retrieved from http://arxiv.org/abs/1311.7327

  28. Rose S (1980) Enhancing visual recognition memory in preterm infants. Dev Psychol 16(2):85–92. https://doi.org/10.1037/0012-1649.16.2.85

    Article  Google Scholar 

  29. Salthouse TA (1996) The processing-speed theory of adult age differences in cognition. Psychol Rev 103(3):403–428

    CAS  Article  Google Scholar 

  30. Sekuler R, Kahana MJ, McLaughlin C, Golomb J, Wingfield A (2005) Preservation of episodic visual recognition memory in aging. Exp Aging Res 31(1):1–13. https://doi.org/10.1080/03610730590882800

    Article  PubMed  Google Scholar 

  31. Smith-Ray RL, Irmiter C, Boulter K (2016) Cognitive training among cognitively impaired older adults: a feasibility study assessing the potential improvement in balance 4. https://doi.org/10.3389/fpubh.2016.00219

  32. Vivero V, Barreira N, Penedo MG, Cabrero D, Remeseiro B (2010) Directional gaze analysis in webcam video sequences. In: Campilho A, Kamel M (eds) Image Analysis and Recognition, vol 6112, pp 316–324. https://doi.org/10.1007/978-3-642-13775-4_32

    Google Scholar 

  33. Ward A, Arrighi HM, Michels S, Cedarbaum JM (2012) Mild cognitive impairment: disparity of incidence and prevalence estimates. Alzheimers Dement 8(1):14–21. https://doi.org/10.1016/j.jalz.2011.01.002

    Article  PubMed  Google Scholar 

  34. Yogev-Seligmann G, Hausdorff JM, Giladi N (2008) The role of executive function and attention in gait. Mov Disord 23(3):329–342. https://doi.org/10.1002/mds.21720

    Article  PubMed  Google Scholar 

  35. Zola SM, Manzanares CM, Clopton P, Lah JJ, Levey AI (2013) A behavioral task predicts conversion to mild cognitive impairment and Alzheimer’s disease. Am J Alzheimers Dis Other Dement 28(2):179–184. https://doi.org/10.1177/1533317512470484

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michelle Gray.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Verify currency and authenticity via CrossMark

Cite this article

Gills, J.L., Glenn, J.M., Madero, E.N. et al. Validation of a digitally delivered visual paired comparison task: reliability and convergent validity with established cognitive tests. GeroScience 41, 441–454 (2019). https://doi.org/10.1007/s11357-019-00092-0

Download citation

Keywords

  • Cognition
  • Visual paired comparison
  • Dementia
  • Alzheimer’s disease