, Volume 41, Issue 4, pp 441–454 | Cite as

Validation of a digitally delivered visual paired comparison task: reliability and convergent validity with established cognitive tests

  • Joshua L. Gills
  • Jordan M. Glenn
  • Erica N. Madero
  • Nick T. Bott
  • Michelle GrayEmail author
Original Article


Alzheimer’s disease (AD) affects the memory and cognitive function of approximately 5.7 million Americans. Early detection subsequently allows for earlier treatment and improves outcomes. Currently, there exists a validated 30-min eye-tracking cognitive assessment (VPC-30) for predicting AD risk. However, a shorter assessment would improve user experience and improve scalability. Thus, the purposes were to (1) determine convergent validity between the 5-min web camera-based eye-tracking task (VPC-5) and VPC-30, (2) examine the relationship between VPC-5 and gold-standard cognitive tests, and (3) determine the reliability and stability of VPC-5. This prospective study included two healthy cohorts: older adults (65+ years, n = 20) and younger adults (18–46 years, n = 24). Participants were tested on two separate occasions. Visit 1 included the Montreal Cognitive Assessment (MoCA), Digit Symbol Coding test (DSC), NIH Toolbox Cognitive Battery (NIHTB-CB), VPC-30, and VPC-5. Visit 2 occurred at least 14 days later; participants completed the VPC-5, DSC, NIHTB-CB, and dual-task walking assessments (DT). VPC-30 significantly correlated with VPC-5 at the first (p < .001) and second (p = .001) time points. VPC-5 and DSC (p < .01) and Pattern Comparison Processing Speed Test (PSPAC) (p = .01) were also correlated on day 1. Significant associations existed between VPC-5 and DSC (p < .001), Flanker Inhibitory Control Test (p = .05), PSPAC (p < .001), and Picture Sequence Memory Test (p = .02) during day 14 testing session. The test retest reliability of VPC-5 was significant (p < .001). VPC-5 displayed moderate convergent validity with the VPC-30 and gold-standard measures of cognition, while demonstrating strong stability, suggesting it is a valuable assessment for monitoring memory function.


Cognition Visual paired comparison Dementia Alzheimer’s disease 



  1. Alzheimer’s Association (2018) 2018 Alzheimer’s disease facts and figures. Alzheimers Dement 14(3):367–429CrossRefGoogle Scholar
  2. Alzheimer’s Association (2015) 2015 Alzheimer’s disease facts and figures. Alzheimers Dement 11(3):332–384. CrossRefGoogle Scholar
  3. Best JR, Liu-Ambrose T, Boudreau RM, Ayonayon HN, Satterfield S, Simonsick EM et al (2016) An evaluation of the longitudinal, bidirectional associations between gait speed and cognition in older women and men. J Gerontol Ser A Biol Sci Med Sci 71(12):1616–1623. CrossRefGoogle Scholar
  4. Bland A, Roiser J, Mehta M, Schei T, Boland H, Campbell-Meiklejohn D et al (2016) EMOTICOM: a neuropsychological test battery to evaluate emotion. Motiv Impulsiv Soc Cogn 10.
  5. Bott N, Madero EN, Glenn J, Lange A, Anderson J, Newton D, Brennan A, Buffalo EA, Rentz D, Zola S (2018) Device-embedded cameras for eye tracking–based cognitive assessment: validation with paper-pencil and computerized cognitive composites. J Med Internet Res 20(7):e11143. CrossRefPubMedPubMedCentralGoogle Scholar
  6. Bott NT, Lange A, Rentz D, Buffalo E, Clopton P, Zola S (2017) Web camera based eye tracking to assess visual memory on a visual paired comparison task. Front Neurosci:11
  7. Brustio PR, Magistro D, Zecca M, Rabaglietti E, Liubicich ME (2017) Age-related decrements in dual-task performance: comparison of different mobility and cognitive tasks. A cross sectional study. PLoS ONE 12(7):e0181698. CrossRefPubMedPubMedCentralGoogle Scholar
  8. Crowe SF, Benedict T, Enrico J, Mancuso N, Matthews C, Wallace J (1999) Cognitive determinants of performance on the digit symbol-coding test, and the symbol search test of the Wais-III, and the symbol digit modalities test: an analysis in a healthy sample. Aust Psychol 34(3):204–210. CrossRefGoogle Scholar
  9. Crutcher MD, Calhoun-Haney R, Manzanares CM, Lah JJ, Levey AI, Zola SM (2009) Eye tracking during a visual paired comparison task as a predictor of early dementia. Am J Alzheimers Dis Other Dement 24(3):258–266. CrossRefGoogle Scholar
  10. Danckert SL, Craik FIM (2013) Does aging affect recall more than recognition memory? Psychol Aging 28(4):902–909. CrossRefPubMedGoogle Scholar
  11. Donohue MC, Sperling RA, Salmon DP, Rentz DM, Raman R, Thomas RG et al (2014) The preclinical Alzheimer cognitive composite: measuring amyloid-related decline. JAMA Neurol 71(8):961–970. CrossRefPubMedPubMedCentralGoogle Scholar
  12. Fagan JF (1970) Memory in the infant. J Exp Child Psychol 9(2):217–226. CrossRefPubMedGoogle Scholar
  13. Glenn J, Vincenzo J, Canella C, Binns A, Gray M (2015) Glenn 2015 - Habitual and maximal dual-task gait speeds among sedentary, recreationally active, and masters athlete late-middle aged adults. J Aging Phys Act 23(3):433–437CrossRefGoogle Scholar
  14. Goldberg TE, Harvey PD, Wesnes KA, Snyder PJ, Schneider LS (2015) Practice effects due to serial cognitive assessment: implications for preclinical Alzheimer’s disease randomized controlled trials. Alzheimer's & Dementia 1(1):103–111. CrossRefGoogle Scholar
  15. Hanninen T, Hallikainen M, Tuomainen S, Vanhanen M, Soininen H (2002) Prevalence of mild cognitive impairment: a population-based study in elderly subjects. Acta Neurol Scand 106(3):148–154. CrossRefPubMedGoogle Scholar
  16. Hansen DW, Pece AEC (2005) Eye tracking in the wild. Comput Vis Image Underst 98(1):155–181. CrossRefGoogle Scholar
  17. Harvey PD (2012) Clinical applications of neuropsychological assessment. Dialogues Clin Neurosci 14(1):91–99PubMedPubMedCentralGoogle Scholar
  18. Hausdorff JM, Rios DA, Edelberg HK (2001) Gait variability and fall risk in community-living older adults: a 1-year prospective study. Arch Phys Med Rehabil 82(8):1050–1056. CrossRefPubMedGoogle Scholar
  19. Heaton RK, Akshoomoff N, Tulsky D, Mungas D, Weintraub S, Dikmen S, Beaumont J, Casaletto KB, Conway K, Slotkin J, Gershon R (2014) Reliability and validity of composite scores from the NIH toolbox cognition battery in adults. J Int Neuropsychol Soc 20(06):588–598. CrossRefPubMedPubMedCentralGoogle Scholar
  20. Kremen WS, Jak AJ, Panizzon MS, Spoon KM, Franz CE, Thompson WK, Jacobson KC, Vasilopoulos T, Vuoksimaa E, Xian H, Toomey R, Lyons MJ (2014) Early identification and heritability of mild cognitive impairment. Int J Epidemiol 43(2):600–610. CrossRefPubMedGoogle Scholar
  21. Lagun D, Manzanares C, Zola SM, Buffalo EA, Agichtein E (2011) Detecting cognitive impairment by eye movement analysis using automatic classification algorithms. J Neurosci Methods 201(1):196–203. CrossRefPubMedPubMedCentralGoogle Scholar
  22. Lin Y-T, Lin R-Y, Lin Y-C, Lee GC (2013) Real-time eye-gaze estimation using a low-resolution webcam. Multimed Tools Appl 65(3):543–568. CrossRefGoogle Scholar
  23. Maruff P, Thomas E, Cysique L, Brew B, Collie A, Snyder P, Pietrzak R (2009) Validity of the CogStat Brief battery: relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, Schizophrenia, and AIDS Dementia Complex. Arch Clin Neuropsychol 24:165–1178. CrossRefPubMedGoogle Scholar
  24. McCulloch KL, Mercer V, Giuliani C, Marshall S (2009) Development of a clinical measure of dual-task performance in walking: reliability and preliminary validity of the walking and remembering test. J Geriatr Phys Ther 32:8CrossRefGoogle Scholar
  25. Montero-Odasso M, Casas A, Hansen KT, Bilski P, Gutmanis I, Wells JL, Borrie MJ (2009) Quantitative gait analysis under dual-task in older people with mild cognitive impairment: a reliability study. J NeuroEng Rehabil 6(1):35. CrossRefPubMedPubMedCentralGoogle Scholar
  26. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I et al (2005) The Montreal cognitive assessment, MoCA: a brief screening tool for mild cognitive impairment: MOCA: a Brief Screening Tool For MCI. J Am Geriatr Soc 53(4):695–699. CrossRefGoogle Scholar
  27. Petridis S, Giannakopoulos T, Spyropoulos CD (2013) Unobtrusive low cost pupil size measurements using web cameras. ArXiv:1311.7327 [Cs]. Retrieved from
  28. Rose S (1980) Enhancing visual recognition memory in preterm infants. Dev Psychol 16(2):85–92. CrossRefGoogle Scholar
  29. Salthouse TA (1996) The processing-speed theory of adult age differences in cognition. Psychol Rev 103(3):403–428CrossRefGoogle Scholar
  30. Sekuler R, Kahana MJ, McLaughlin C, Golomb J, Wingfield A (2005) Preservation of episodic visual recognition memory in aging. Exp Aging Res 31(1):1–13. CrossRefPubMedGoogle Scholar
  31. Smith-Ray RL, Irmiter C, Boulter K (2016) Cognitive training among cognitively impaired older adults: a feasibility study assessing the potential improvement in balance 4.
  32. Vivero V, Barreira N, Penedo MG, Cabrero D, Remeseiro B (2010) Directional gaze analysis in webcam video sequences. In: Campilho A, Kamel M (eds) Image Analysis and Recognition, vol 6112, pp 316–324. CrossRefGoogle Scholar
  33. Ward A, Arrighi HM, Michels S, Cedarbaum JM (2012) Mild cognitive impairment: disparity of incidence and prevalence estimates. Alzheimers Dement 8(1):14–21. CrossRefPubMedGoogle Scholar
  34. Yogev-Seligmann G, Hausdorff JM, Giladi N (2008) The role of executive function and attention in gait. Mov Disord 23(3):329–342. CrossRefPubMedGoogle Scholar
  35. Zola SM, Manzanares CM, Clopton P, Lah JJ, Levey AI (2013) A behavioral task predicts conversion to mild cognitive impairment and Alzheimer’s disease. Am J Alzheimers Dis Other Dement 28(2):179–184. CrossRefGoogle Scholar

Copyright information

© American Aging Association 2019

Authors and Affiliations

  1. 1.Exercise Science Research CenterUniversity of ArkansasFayettevilleUSA
  2. 2.VP Clinical Development, Neurotrack Technologies, Inc.University of ArkansasRedwood CityUSA
  3. 3.Neurotrack Technologies, Inc.Redwood CityUSA

Personalised recommendations