Journal of NeuroVirology

, Volume 25, Issue 1, pp 72–84 | Cite as

Empiric neurocognitive performance profile discovery and interpretation in HIV infection

  • Daniela Gomez
  • Christopher Power
  • M. John Gill
  • Noshin Koenig
  • Roberto Vega
  • Esther FujiwaraEmail author


The measurement and determinants of HIV-associated neurocognitive disorders (HAND) are under intense debate. We used latent profile analysis (LPA) and machine learning to define neurocognitive performance profiles and identify their associated risk factors in HIV patients receiving antiretroviral therapy (ART). Neurocognitive performance was assessed by a multidomain neuropsychological test battery. LPA was used to define individual neurocognitive profiles. Random forest analyses (RFA) identified the most important factors distinguishing each profile. Three profiles emerged from the LPA: profile 1 (P1, n = 159) achieved the highest performance, while profile 2 (P2, n = 163) had lowered executive functions and verbal memory, and profile 3 (P3, n = 59) was globally impaired. RFA achieved good prediction (area under the curve ≥ 0.80) only for global impairment (P3). Non-North American descent was the dominant predictor of P3, followed by factors coinciding with non-North American descent (female sex and toxoplasma seropositivity). Additional predictors included unemployment, current depressive symptoms, lower nadir CD4, and longstanding HIV. Restricting analyses to North Americans pointed to the additional importance of ART achieving high CSF levels and older age in prediction of P3. HAND diagnoses were most common in the globally impaired profile (P3 = 89.8%), followed by the group with reduced higher-order neurocognitive performance (P2 = 16.6%). Thus, implementation of LPA and RFA empirically distinguished three distinct neurocognitive performance profiles in this HIV-infected cohort while also highlighting potential risk factors and their relative importance to neurocognitive impairment. These data-driven analytical methods pointed to discernible demographic, HIV- and treatment-related risk factor constellations in patients born outside and within North America that might influence diagnostic and therapeutic decisions.


HIV-associated neurocognitive disorders Neuropsychology Mixture modeling Machine learning Risk factors 



The authors would like to thank the patients and the staff at the Southern Alberta Clinic for their participation in the study. The authors further thank Roger A. Dixon, Kirstie McDermott, and Matthew Johnson for their statistical help.

Funding information

This study was supported by the Canadian Institutes of Health Research (CIHR) Emerging Team Grant (RFN: 125271) (CP, MJG, EF).

Compliance with ethical standards

The study was approved by the University of Calgary, Conjoint Health Research Ethics Board, CHREB13-0615.

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

13365_2018_685_MOESM1_ESM.pdf (207 kb)
ESM 1 (PDF 207 kb)
13365_2018_685_MOESM2_ESM.pdf (126 kb)
ESM 2 (PDF 126 kb)


  1. Aldenderfer MS, Blashfield RK (1984) Cluster analysis. Sage, Beverly HillsCrossRefGoogle Scholar
  2. Antinori A, Arendt G, Becker JT, Brew BJ, Byrd DA, Cherner M, Clifford DB, Cinque P, Epstein LG, Goodkin K, Gisslen M, Grant I, Heaton RK, Joseph J, Marder K, Marra CM, McArthur JC, Nunn M, Price RW, Pulliam L, Robertson KR, Sacktor N, Valcour V, Wojna VE (2007) Updated research nosology for HIV-associated neurocognitive disorders. Neurology 69:1789–1799CrossRefGoogle Scholar
  3. Asahchop EL, Akinwumi SM, Branton WG, Fujiwara E, Gill MJ, Power C (2016) Plasma microrna profiling predicts HIV-associated neurocognitive disorder. AIDS 30:2021–2031CrossRefGoogle Scholar
  4. Basavaraju A (2016) Toxoplasmosis in HIV infection: an overview. Trop Parasitol 6:129–135CrossRefGoogle Scholar
  5. Berrett AN, Gale SD, Erickson LD, Brown BL, Hedges DW (2017) Toxoplasma gondii moderates the association between multiple folate-cycle factors and cognitive function in U.S. adults. Nutrients 9:E564Google Scholar
  6. Bharti AR, McCutchan A, Deutsch R, Smith DM, Ellis RJ, Cherner M, Woods SP, Heaton RK, Grant I, Letendre SL (2016) Latent toxoplasma infection and higher Toxoplasma gondii immunoglobulin G levels are associated with worse neurocognitive functioning in HIV-infected adults. Clin Infect Dis 63:1655–1660CrossRefGoogle Scholar
  7. Brandt J, Benedict RH (2001) Hopkins Verbal Learning Test - Revised. Psychological Assessment Resources, Inc, LutzGoogle Scholar
  8. Breiman L (2001) Random forests. Mach Learn 45:5–32CrossRefGoogle Scholar
  9. Canizares S, Cherner M, Ellis RJ (2014) HIV and aging: effects on the central nervous system. Semin Neurol 34:27–34CrossRefGoogle Scholar
  10. Carey CL, Woods SP, Gonzalez R, Conover E, Marcotte TD, Grant I, Heaton RK, HNRC Group (2004) Predictive validity of global deficit scores in detecting neuropsychological impairment in HIV infection. J Clin Exp Neuropsychol 26:307–319CrossRefGoogle Scholar
  11. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357CrossRefGoogle Scholar
  12. Cherner M, Suarez P, Lazzaretto D, Fortuny LA, Mindt MR, Dawes S, Marcotte T, Grant I, Heaton R, Group H (2007) Demographically corrected norms for the Brief Visuospatial Memory Test - Revised and Hopkins Verbal Learning Test - Revised in monolingual Spanish speakers from the U.S.-Mexico border region. Arch Clin Neuropsychol 22:343–353Google Scholar
  13. Crane HM, Van Rompaey SE, Dillingham PW, Herman E, Diehr P, Kitahata MM (2006) A single-item measure of health-related quality-of-life for HIV-infected patients in routine clinical care. AIDS Patient Care STDs 20:161–174CrossRefGoogle Scholar
  14. Cristiani SA, Pukay-Martin ND, Bornstein RA (2004) Marijuana use and cognitive function in HIV-infected people. J Neuropsychiatr Clin Neurosci 16:330–335CrossRefGoogle Scholar
  15. Cysique LA, Vaida F, Letendre S, Gibson S, Cherner M, Woods SP, McCutchan JA, Heaton RK, Ellis RJ (2009) Dynamics of cognitive change in impaired HIV-positive patients initiating antiretroviral therapy. Neurology 73:342–348CrossRefGoogle Scholar
  16. Cysique LA, Heaton RK, Kamminga J, Lane T, Gates TM, Moore DM, Hubner E, Carr A, Brew BJ (2014) HIV-associated neurocognitive disorder in Australia: a case of a high-functioning and optimally treated cohort and implications for international neuroHIV research. J Neurovirol 20:258–268CrossRefGoogle Scholar
  17. Dawes S, Suarez P, Casey CY, Cherner M, Marcotte TD, Letendre S, Grant I, Heaton RK, H.I.V. Neurobehavioral Research Center Group (2008) Variable patterns of neuropsychological performance in HIV-1 infection. J Clin Exp Neuropsychol 30:613–626CrossRefGoogle Scholar
  18. De Francesco D, Underwood J, Post FA, Vera JH, Williams I, Boffito M, Sachikonye M, Anderson J, Mallon PW, Winston A, Sabin CA, Group Ps (2016) Defining cognitive impairment in people-living-with-HIV: the POPPY study. BMC Infect Dis 16:617CrossRefGoogle Scholar
  19. Delis DC, Kaplan E, Kramer JH (2001) Delis-Kaplan Executive Function System (D-KEFS). The Psychological Corporation, San AntonioGoogle Scholar
  20. Devlin KN, Giovannetti T (2017) Heterogeneity of neuropsychological impairment in HIV infection: contributions from mild cognitive impairment. Neuropsychol Rev 27:101–123CrossRefGoogle Scholar
  21. Dufouil C, Richert L, Thiebaut R, Bruyand M, Amieva H, Dauchy FA, Dartigues JF, Neau D, Morlat P, Dehail P, Dabis F, Bonnet F, Chene G, Group ACAS (2015) Diabetes and cognitive decline in a French cohort of patients infected with HIV-1. Neurology 85:1065–1073CrossRefGoogle Scholar
  22. Ene L, Marcotte TD, Umlauf A, Grancea C, Temereanca A, Bharti A, Achim CL, Letendre S, Ruta SM (2016) Latent toxoplasmosis is associated with neurocognitive impairment in young adults with and without chronic HIV infection. J Neuroimmunol 299:1–7CrossRefGoogle Scholar
  23. Fazeli PL, Crowe M, Ross LA, Wadley V, Ball K, Vance DE (2014) Cognitive functioning in adults aging with HIV: a cross-sectional analysis of cognitive subtypes and influential factors. J Clin Res HIV AIDS Prev 1:155–169CrossRefGoogle Scholar
  24. Fellows RP, Byrd DA, Morgello S, Manhattan H. I. V. Brain Bank (2013) Major depressive disorder, cognitive symptoms, and neuropsychological performance among ethnically diverse HIV+ men and women. J Int Neuropsychol Soc 19:216–225CrossRefGoogle Scholar
  25. Fine EM, Delis DC, Holdnack J (2011) Normative adjustments to the D-KEFS trail making test: corrections for education and vocabulary level. Clin Neuropsychol 25:1331–1344CrossRefGoogle Scholar
  26. Flensborg Damholdt M, Shevlin M, Borghammer P, Larsen L, Ostergaard K (2012) Clinical heterogeneity in Parkinson’s disease revisited: a latent profile analysis. Acta Neurol Scand 125:311–318CrossRefGoogle Scholar
  27. Frndak SE, Smerbeck AM, Irwin LN, Drake AS, Kordovski VM, Kunker KA, Khan AL, Benedict RH (2016) Latent profile analysis of regression-based norms demonstrates relationship of compounding MS symptom burden and negative work events. Clin Neuropsychol 30:1050–1062CrossRefGoogle Scholar
  28. Gale SD, Brown BL, Erickson LD, Berrett A, Hedges DW (2015) Association between latent toxoplasmosis and cognition in adults: a cross-sectional study. Parasitology 142:557–565CrossRefGoogle Scholar
  29. Gisslen M, Price RW, Nilsson S (2011) The definition of HIV-associated neurocognitive disorders: are we overestimating the real prevalence? BMC Infect Dis 11:356CrossRefGoogle Scholar
  30. Hair JF (2006) Multivariate data analysis, 6th edn. Pearson Prentice Hall, Upper Saddle RiverGoogle Scholar
  31. Hothorn T, Buhlmann P, Dudoit S, Molinaro A, van der Laan MJ (2006) Survival ensembles. Biostatistics 7:355–373CrossRefGoogle Scholar
  32. Jernigan TL, Archibald SL, Fennema-Notestine C, Taylor MJ, Theilmann RJ, Julaton MD, Notestine RJ, Wolfson T, Letendre SL, Ellis RJ, Heaton RK, Gamst AC, Franklin DR Jr, Clifford DB, Collier AC, Gelman BB, Marra C, JC MA, JA MC, Morgello S, Simpson DM, Grant I, Group C (2011) Clinical factors related to brain structure in HIV: the CHARTER study. J Neurovirol 17:248–257CrossRefGoogle Scholar
  33. Kohler S, Hamel R, Sistermans N, Koene T, Pijnenburg YA, van der Flier WM, Scheltens P, Visser PJ, Aalten P, Verhey FR, Ramakers I (2013) Progression to dementia in memory clinic patients without dementia: a latent profile analysis. Neurology 81:1342–1349CrossRefGoogle Scholar
  34. Kongs SK, Thompson LL, Iverson GL, Heaton RK (2000) Winsconsin Card Sorting Test-64 card version. Psychological Assessment Resources, OdessaGoogle Scholar
  35. Kroenke K, Spitzer RL, Williams JB (2001) The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 16:606–613CrossRefGoogle Scholar
  36. Kuhn M, Johnson K (2013) Applied predictive modeling. Springer, New YorkCrossRefGoogle Scholar
  37. Letendre S, Marquie-Beck J, Capparelli E, Best B, Clifford D, Collier AC, Gelman BB, JC MA, JA MC, Morgello S, Simpson D, Grant I, Ellis RJ, CHARTER Group (2008) Validation of the CNS penetration-effectiveness rank for quantifying antiretroviral penetration into the central nervous system. Arch Neurol 65:65–70CrossRefGoogle Scholar
  38. Lojek E, Bornstein RA (2005) The stability of neurocognitive patterns in HIV infected men: classification considerations. J Clin Exp Neuropsychol 27:665–682CrossRefGoogle Scholar
  39. Maki PM, Martin-Thormeyer E (2009) HIV, cognition and women. Neuropsychol Rev 19:204–214CrossRefGoogle Scholar
  40. Maki PM, Rubin LH, Valcour V, Martin E, Crystal H, Young M, Weber KM, Manly J, Richardson J, Alden C, Anastos K (2015) Cognitive function in women with HIV: findings from the Women’s Interagency HIV study. Neurology 84:231–240CrossRefGoogle Scholar
  41. McCombe JA, Vivithanaporn P, Gill MJ, Power C (2013) Predictors of symptomatic HIV-associated neurocognitive disorders in universal health care. HIV Med 14:99–107CrossRefGoogle Scholar
  42. McGuinness B, Barrett SL, McIlvenna J, Passmore AP, Shorter GW (2015) Predicting conversion to dementia in a memory clinic: a standard clinical approach compared with an empirically defined clustering method (latent profile analysis) for mild cognitive impairment subtyping. Alzheimers Dement (Amst) 1:447–454Google Scholar
  43. McLachlan GJ, Peel D (2000) Finite mixture models. Wiley, New YorkCrossRefGoogle Scholar
  44. Molsberry SA, Cheng Y, Kingsley L, Jacobson L, Levine AJ, Martin E, Miller EN, Munro CA, Ragin A, Sacktor N, Becker JT, Neuropsychology Working Group of the Multicenter ACS (2018) Neuropsychological phenotypes among men with and without HIV disease in the Multicenter AIDS Cohort Study. AIDS 32:1679–1688Google Scholar
  45. Muthen BO, Muthen LK (1998-2015) Mplus user’s guide, 7th edn. Muthen & Muthen, Los AngelesGoogle Scholar
  46. Nightingale S, Winston A, Letendre S, Michael BD, McArthur JC, Khoo S, Solomon T (2014) Controversies in HIV-associated neurocognitive disorders. Lancet Neurol 13:1139–1151CrossRefGoogle Scholar
  47. Nylund KL, Asparouhov T, Muthen B (2007) Decising on the number of classes in latent class analysis and growth mixture modeling: a Monte Carlo simulation study. Struct Equ Model 14:535–569CrossRefGoogle Scholar
  48. Patel SM, Thames AD, Arbid N, Panos SE, Castellon S, Hinkin CH (2013) The aggregate effects of multiple comorbid risk factors on cognition among HIV-infected individuals. J Clin Exp Neuropsychol 35:421–434CrossRefGoogle Scholar
  49. Public Health Agency of Canada (2015) HIV and AIDS in Canada: surveillance report to December 31, 2014. Minister of Public Works and Government Services Canada, OttawaGoogle Scholar
  50. R Core Team (2017) R: a language and environment for statistical computing. R Foundation for Statistical Computing, ViennaGoogle Scholar
  51. Rubin LH, Sundermann EE, Cook JA, Martin EM, Golub ET, Weber KM, Cohen MH, Crystal H, Cederbaum JA, Anastos K, Young M, Greenblatt RM, Maki PM (2014) Investigation of menopausal stage and symptoms on cognition in human immunodeficiency virus-infected women. Menopause 21:997–1006CrossRefGoogle Scholar
  52. Rubin LH, Cook JA, Springer G, Weber KM, Cohen MH, Martin EM, Valcour VG, Benning L, Alden C, Milam J, Anastos K, Young MA, Gustafson DR, Sundermann EE, Maki PM (2017) Perceived and post-traumatic stress are associated with decreased learning, memory, and fluency in HIV-infected women. AIDS 31:2393–1401CrossRefGoogle Scholar
  53. Sacktor N, Skolasky RL, Seaberg E, Munro C, Becker JT, Martin E, Ragin A, Levine A, Miller E (2016) Prevalence of HIV-associated neurocognitive disorders in the Multicenter AIDS Cohort Study. Neurology 86:334–340Google Scholar
  54. Saylor D, Dickens AM, Sacktor N, Haughey N, Slusher B, Pletnikov M, Mankowski JL, Brown A, Volsky DJ, McArthur JC (2016) HIV-associated neurocognitive disorder—pathogenesis and prospects for treatment. Nat Rev Neurol 12:309CrossRefGoogle Scholar
  55. Schretlen D, Testa SM, Pearlson GD (2010) Calibrated Neuropsychological Normative System. Psychological Assessment Resources, LutzGoogle Scholar
  56. Smith A (1973) Symbol Digit Modalities Test. Western Psychological Services, Los AngelesGoogle Scholar
  57. Strobl C, Boulesteix AL, Kneib T, Augustin T, Zeileis A (2008) Conditional variable importance for random forests. BMC Bioinf 9:307Google Scholar
  58. Strobl C, Malley J, Tutz G (2009) An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. Psychol Methods 14:323–348CrossRefGoogle Scholar
  59. Thames AD, Kuhn TP, Williamson TJ, Jones JD, Mahmood Z, Hammond A (2017) Marijuana effects on changes in brain structure and cognitive function among HIV+ and HIV− adults. Drug Alcohol Depend 170:120–127CrossRefGoogle Scholar
  60. Tierney SM, Sheppard DP, Kordovski VM, Faytell MP, Avci G, Woods SP (2017) A comparison of the sensitivity, stability, and reliability of three diagnostic schemes for HIV-associated neurocognitive disorders. J Neurovirol 23:404–421CrossRefGoogle Scholar
  61. Tripathi A, Jerrell JM, Liese AD, Zhang J, Rizvi AA, Albrecht H, Duffus WA (2013) Association of clinical and therapeutic factors with incident dyslipidemia in a cohort of human immunodeficiency virus-infected and non-infected adults: 1994-2011. Metab Syndr Relat Disord 11:417–426CrossRefGoogle Scholar
  62. Trites RL (1977) Grooved Pegboard. Royal Ottawa Hospital, OttawaGoogle Scholar
  63. Underwood J, Robertson KR, Winston A (2015) Could antiretroviral neurotoxicity play a role in the pathogenesis of cognitive impairment in treated HIV disease? AIDS 29:253–261CrossRefGoogle Scholar
  64. van Gorp WG, Hinkin C, Satz P, Miller E, Weisman J, Holston S, Drebing C, Marcotte TD, Dixon W (1993) Subtypes of HIV-related neuropsychological functioning: a cluster analysis approach. Neuropsychology 7:62–72CrossRefGoogle Scholar
  65. Vance DE, Fazeli PL, Dodson JE, Ackerman M, Talley M, Appel SJ (2014) The synergistic effects of HIV, diabetes, and aging on cognition: implications for practice and research. J Neurosci Nurs 46:292–305CrossRefGoogle Scholar
  66. Vuong QH (1989) Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica 57:307–333CrossRefGoogle Scholar
  67. Wilkinson GS, Robertson GJ (2006) Wide Range Achievement Test (4th ed.). Psychological Assessment Resources, LutzGoogle Scholar
  68. Winston A, Arenas-Pinto A, Stohr W, Fisher M, Orkin CM, Aderogba K, De Burgh-Thomas A, O’Farrell N, Lacey CJ, Leen C, Dunn D, Paton NI, Team PT (2013) Neurocognitive function in HIV infected patients on antiretroviral therapy. PLoS One 8:e61949CrossRefGoogle Scholar
  69. Woods SP, Rippeth JD, Frol AB, Levy JK, Ryan E, Soukup VM, Hinkin CH, Lazzaretto D, Cherner M, Marcotte TD, Gelman BB, Morgello S, Singer EJ, Grant I, Heaton RK (2004) Interrater reliability of clinical ratings and neurocognitive diagnoses in HIV. J Clin Exp Neuropsychol 26:759–778CrossRefGoogle Scholar
  70. Wright EJ, Grund B, Cysique LA, Robertson KR, Brew BJ, Collins G, Shlay JC, Winston A, Read TR, Price RW, International Network for Strategic Initiatives in Global HIVTSSG (2015) Factors associated with neurocognitive test performance at baseline: a substudy of the INSIGHT Strategic Timing of Antiretroviral Treatment (START) trial. HIV Med 16(Suppl 1):97–108CrossRefGoogle Scholar

Copyright information

© Journal of NeuroVirology, Inc. 2018

Authors and Affiliations

  • Daniela Gomez
    • 1
  • Christopher Power
    • 1
    • 2
  • M. John Gill
    • 2
    • 3
  • Noshin Koenig
    • 3
  • Roberto Vega
    • 4
  • Esther Fujiwara
    • 1
    Email author
  1. 1.Department of Psychiatry, 1E.1 WC Mackenzie Health Sciences CentreUniversity of AlbertaEdmontonCanada
  2. 2.Department of MedicineUniversity of CalgaryCalgaryCanada
  3. 3.Southern Alberta HIV Clinic, Alberta Health ServicesCalgaryCanada
  4. 4.Department of Computing ScienceUniversity of AlbertaEdmontonCanada

Personalised recommendations