AIDS and Behavior

, 15:1902

Performances on the CogState and Standard Neuropsychological Batteries Among HIV Patients Without Dementia

Authors

    • Washington University School of Medicine
  • John S. K. Kauwe
    • Department of BiologyBrigham Young University
  • Robert Paul
    • Department of PsychologyUniversity of Missouri
  • Karen Tashima
    • Brown University School of Medicine
  • David F. Tate
    • Brown University School of Medicine
  • Pragna Patel
    • Division of HIV/AIDS PreventionCenters for Disease Control and Prevention
  • Charles C. J. Carpenter
    • Brown University School of Medicine
  • David Patty
    • Department of BiologyBrigham Young University
  • John T. Brooks
    • Division of HIV/AIDS PreventionCenters for Disease Control and Prevention
  • David B. Clifford
    • Washington University School of Medicine
Brief Report

DOI: 10.1007/s10461-011-0033-9

Cite this article as:
Overton, E.T., Kauwe, J.S.K., Paul, R. et al. AIDS Behav (2011) 15: 1902. doi:10.1007/s10461-011-0033-9

Abstract

HIV-associated neurocognitive disorders remain prevalent but challenging to diagnose particularly among non-demented individuals. To determine whether a brief computerized battery correlates with formal neurocognitive testing, we identified 46 HIV-infected persons who had undergone both formal neurocognitive testing and a brief computerized battery. Simple detection tests correlated best with formal neuropsychological testing. By multivariable regression model, 53% of the variance in the composite Global Deficit Score was accounted for by elements from the brief computerized tool (P < 0.01). These data confirm previous correlation data with the computerized battery. Using the five significant parameters from the regression model in a Receiver Operating Characteristic curve, 90% of persons were accurately classified as being cognitively impaired or not. The test battery requires additional evaluation, specifically for identifying persons with mild impairment, a state upon which interventions may be effective.

Keyword

HIV infectionneurocognitive functionHIV-associated dementiaCogStateSUN study

Resumen

Los trastornos neurocognitivos asociados con el virus de inmunodeficiencia humana (VIH) son altamente prevalentes pero difíciles de diagnosticar, particularmente entre individuos que no presentan signos de demencia en el momento del examen clínico. Para determinar si una breve batería de pruebas computarizada correlaciona con resultados de pruebas neurocognitivas, identificamos a 46 individuos infectados con VIH que se habían sometido a pruebas neurocognitivas así como a esta prueba computarizada. Los resultados preliminares indicaron que las pruebas de detección simples mostraron la mejor correlación con los exámenes neurocognitivos. Mediante el uso de un modelo de regresión multivariable fuimos capaces de explicar el 53% de la varianza del Global Deficit Score (GDS, puntuación compuesta del déficit global) con componentes de la breve batería computarizada (P < 0.01). Estos datos confirman estudios anteriores que analizaron la correlación entre el GDS y la batería computarizada. Utilizando los cinco parámetros significativos del modelo de regresión en una curva del Receiver Operating Characteristic (ROC), el 90% de las personas fueron clasificadas correctamente como individios que presentaban deterioro cognitivo o no. Se necesitan estudios adicionales de esta batería de pruebas, en particular para identificar a personas con un deterioro leve, el cual podría ser especialmente interesante debido a que intervenciones clínicas, psicológicas o farmacéuticas podrían ser eficaces.

Introduction

With the development of combination antiretroviral therapy (cART), the prevalence of HIV-associated dementia (HAD) has declined but less severe HIV-associated neurocognitive disorders (HAND) have become prominent [1]. The recognition of subtler manifestations of HIV on cognition, including mild neurocognitive disorder (MND) and asymptomatic neurocognitive impairment (ANI), proves to be challenging in a busy clinic environment, particularly given the ability of individuals to compensate for mild impairment. While these latter entities are more difficult to recognize than HAD, timely diagnosis is critical since even ANI can impair capacity to adhere to medical regimens, safely operate a motor vehicle, complete basic activities of daily life, and maintain employment [25]. Furthermore, with the transformation of HIV into a chronic medical illness, advanced age and HIV infection may act synergistically to increase the prevalence of HAND [6]. Recent cohort studies have reported the prevalence of HAND to range from 39 to 69% of subjects on cART [7, 8]. The need for brief and psychometrically sound methods to evaluate neurocognitive function is important to identify HAND and limit progressive impairment in one’s capacity to complete activities of daily life independently [25].

Some available screening tools used to diagnose neurocognitive impairment, such as the International AIDS Dementia Screen or the Mini-Mental Status Exam, are insensitive to early manifestations of functional impairment [911]. Conversely, formal neurocognitive testing is time-consuming and requires special training and thus, cannot realistically be used in the typical outpatient setting, where the necessary time, staffing, space, and funding are generally not available. CogState, a commercially available product (http://www.cogstate.com), is a computerized cognitive test battery designed to measure psychomotor performance, attention, memory, and executive functioning: domains frequently impaired in persons with early neurocognitive disorders [12, 13]. The battery consists of brief tasks in the form of card games to minimize language and cultural differences. It has previously been validated in persons with HAD [13] and been used in clinical cohort studies to measure change in neurocognitive function and performance [14]. In this analysis, we sought to determine the correlation between CogState and formal neuropsychological testing to detect neurocognitive impairment in a subset of healthy, autonomous HIV-infected persons who are followed in two cohort studies at the Washington University Outpatient HIV Clinic.

Methods

Washington University in St. Louis is a site for two prospective cohort studies evaluating complications of HIV and HIV therapy including cognitive function, one with a validated traditional neurocognitive testing battery and the other with CogState. To determine the utility of the CogState computerized battery as a screening tool, subjects enrolled in both studies were identified to compare the two neurocognitive batteries. Subjects were eligible for participation in this study if they completed an assessment in CHARTER within 6 months of the baseline SUN assessment. Enrollment criteria for the studies have been outlined previously [15, 16]. Both studies were approved by the Washington University IRB and all subjects provided written informed consent.

SUN Study

The Study to Understand the Natural History of HIV and AIDS in the Era of Effective Therapy (The SUN Study) is a CDC-funded multi-site prospective observational cohort monitoring complications of antiretroviral therapy and HIV [15]. At baseline and each 6 month visit thereafter, participants are evaluated for neurocognitive ability using CogHealth© CogState Ltd., Melbourne, Australia [13]. The computerized battery requires 12–15 min to complete and consists of six individual tests: two tests of detection evaluating simple reaction times (DET1 and DET2) assess psychomotor function and speed of processing domains; a test of identification as a choice reaction time (IDN) assesses the domains of visual attention and vigilance; the one back test (ONB) assesses the domains of attention and working memory; the monitoring test is a measure of divided attention (MON) that assesses the domain of attention; and the associate learning test (ASSL) assesses the domain of visual learning and memory. Each test is scored based on time to complete the task (speed) and error rate (accuracy).

CHARTER Study

The CNS HIV Antiretroviral Therapy Effects Research (CHARTER) study is an NIH-funded multi-site cohort to explore HIV neurological complications in the context of emerging antiviral treatments such as cART [16, 17]. Participants receive comprehensive neuromedical, neurocognitive, and laboratory examinations. The neurocognitive battery performed in CHARTER requires approximately 1 h to assess the following domains: attention/psychomotor speed (Trailmaking Test Part A, Symbol Digit Test and Symbol Search Test from WAIS-III, Paced Auditory Serial Addition Task); fine motor speed skills (Dominant and Non-dominant Hand Pegboard Test); learning and memory (Brief Visuospatial Memory Test–Revised, Hopkins Verbal Learning Test–Revised, Figure Memory Learning Test and Story Memory Learning test); executive functioning including working memory (Wisconsin Card Sorting Test, Letter-Number Sequence from WAIS-III), fluency (Controlled Oral Word Association Test and Animal Category Test), and set shifting/response inhibition (Trailmaking Test Part B) [1830].

Statistical Analysis

Data from the computerized battery were evaluated for normality of data distribution; reaction time measures were log10 transformed due to a positive skew of the distribution and accuracy measures were transformed using Arcsine-root transformation [31]. The raw scores from the neurocognitive tests from CHARTER were converted into T scores corrected for demographic data to minimize the impact of education, age, race/ethnicity, and sex. The Global Deficit Score (GDS) is a composite score calculated by converting the T scores from the CHARTER battery into one summary deficit score ranging from 0 (normal) to 5 (severe neurocognitive impairment) [32]. For the evaluation of construct validity, a correlation matrix was created to determine the correlation between the CogState measures and the CHARTER study standard neuropsychological measures, including the GDS, using Pearson’s correlation coefficient.

To determine the potential value of the CogState battery as a screening tool, we performed a stepwise linear regression analysis with GDS (log transformed to approximate a normal distribution) as the dependent variable and the CogState measures as independent variables. The stepwise regression was performed using the default selection method (forward in, backward out) in SAS. The stepwise regression was validated using 1,000 bootstrapping replicates. The significant Cogstate variables from the stepwise regression were used as independent variables in a multivariable regression model with GDS as the dependent variable to determine the model R2. The Receiver Operating Characteristic (ROC) curve was produced to determine the sensitivity and specificity of the combination of the significant factors identified from the stepwise regression model for predicting whether an individual’s GDS was greater than or less than 0.5, the threshold for neurocognitive impairment.

Results

Forty-six subjects enrolled in both cohort studies were eligible for the present analysis. Table 1 outlines the clinical parameters of the subjects at the time of baseline CogState evaluation between 2004 and 2006. Median CD4 count was 424.5 c/mm3 with 74% of the cohort on cART and 61% with HIV viral load <400 cp/ml. The median GDS, as calculated from the CHARTER study, for the cohort was 0.47 (range 0.00–2.79), with 24 subjects (52%) having normal function (GDS < 0.5) and 22 (48%) having mild to moderate impairment (GDS ≥ 0.5).
Table 1

Baseline demographics of study participants, the study to understand the natural history of HIV and AIDS in the era of effective therapy (The SUN Study) and the CNS HIV antiretroviral therapy effects research (CHARTER) study

Characteristic

n = 46

n (%)

Gender

 Male

33 (72%)

 Female

13 (28%)

Race

 Caucasian

19 (41%)

 African American

26 (57%)

 Hispanic

1 (2%)

Median age (range)

40 (21–62)

Median years since HIV diagnosis (range)

5.5 (0.5–23)

Education level

 Less than HS

6 (13%)

 HS/GED

22 (48%)

 Some college

12 (26%)

 College graduate

5 (11%)

 Unknown

1 (2%)

Current substance use

 Alcohol

26 (57%)

 Marijuana

3 (7%)

 Cocaine

2 (4%)

 IVDU

1 (2%)

Median nadir CD4 count (range)

255 (0–1,020)

Nadir CD4 count <200 c/mm3

15 (33%)

Median current CD4 count (range)

424.5 (79–1,300)

On cART

34 (74%)

Median current HIV VL (range)

136.5 (<50–309,000)

VL range (copies/ml)

 <400

28 (61%)

 400–999

1 (2%)

 1,000–9,999

6 (13%)

 10,000–99,999

10 (22%)

 >100,000

1 (2%)

Global deficit score

 Normal

24 (52%)

 Mild impairment

20 (43%)

 Moderate impairment

2 (4%)

HIV Human immunodeficiency virus, HS high school, GED general educational development, IVDU intravenous drug user, VL viral load

The correlation matrix showing the Pearson product-moment correlation coefficient between each COGSTATE index and the dependent variables from the CHARTER clinical exam, including GDS is shown in Table 2. The speed measures for both simple detection tests (DET1, DET2) and the identification task (IDN), a more complex reaction time task, correlated with the GDS and had the highest levels of correlation across the tests performed in the CHARTER battery. The accuracy components of the CogState battery generally correlated poorly with the individual neurocognitive tests in the CHARTER battery. The accuracy measures for the complex tasks of One-Back Memory (ONB) and the Measure of Divided Attention (MON) were the only accuracy measures that correlated with the GDS derived from the CHARTER testing.
Table 2

Correlation matrix between CogState tests and CHARTER neurocognitive tests

CHARTER

Domain measure

Median test score (IQR)

SUN study CogState tests

Detection1

Detection2

Identification

One Back

Monitoring

Associate learning

Speed

Accuracy

Speed

Accuracy

Speed

Accuracy

Speed

Accuracy

Speed

Accuracy

Speed

Accuracy

Attention/psychomotor speed

 Trailmaking test part A

−0.5 (−1.4, 0.1)

−0.42**

0.31

−0.42**

0.26

−0.38*

0.34

−0.16

0.33

−0.12

0.23

−0.05

0.17

 Digit symbol test (WAIS- III)

0.15 (−1.1, 1.0)

−0.44**

0.22

−0.47**

-0.03

−0.43*

0.16

−0.31

0.40*

−0.11

0.11

−0.13

−0.02

 Symbol search test (WAIS-III)

−0.45 (−1.3, 0.3)

−0.34*

−0.04

−0.31

0.12

−0.21

0.02

−0.37*

0.31

−0.23

0.01

−0.06

0.06

 PASAT-50number correct

−0.8 (−1.5, 0.2)

−0.53**

0.15

−0.53**

−0.08

−0.49**

−0.04

−0.22

0.45**

−0.18

0.25

−0.07

0.00

Fine motor speed skills

 Pegboard dominant hand

−0.3 (−1.2, 0.2)

−0.46**

0.14

−0.47**

0.09

−0.29

0.40*

−0.15

0.36*

−0.18

0.27

−0.12

0.15

 Pegboard non-dominant hand

−0.7 (−1.3, −0.1)

−0.52**

0.10

−0.33**

−0.09

−0.28

0.21

−0.22

0.35*

−0.12

0.24

−0.23

0.02

Learning and memory

 BVMT total learning

−0.2 (−1.2, 0.6)

−0.34

0.08

−0.11

0.21

−0.23

0.14

−0.32

0.32

−0.21

0.37*

−0.07

0.26

 BVMT delayed recall

−0.55 (−1.3, 0.4)

−0.33

0.10

−0.08

0.15

−0.22

0.17

−0.27

0.28

−0.12

0.38*

−0.12

0.19

 HVLTtotal learning

−0.1 (−1.4, 0.9)

−0.42**

0.15

−0.44**

−0.11

−0.20

0.42**

−0.03

0.08

−0.13

0.18

0.04

0.11

 HVLT delayed recall

−0.1 (−1.5, 1.4)

−0.42**

0.16

−0.44**

−0.11

−0.23

0.36*

0.04

−0.10

−0.05

0.13

−0.02

−0.01

 Figure memory learning

−0.65 (−1.4, 0.2)

−0.40*

−0.04

−0.35*

0.09

−0.41*

0.20

−0.34*

0.17

−0.09

0.37*

−0.21

0.15

Figure memory delayed recall

0.0 (−0.3, 0.4)

−0.22

0.03

−0.19

−0.22

−0.37*

0.15

−0.24

−0.01

−0.01

−0.02

−0.23

0.05

 Story memory learning

−0.85 (−1.8, 0.4)

−0.36

−0.04

−0.27

−0.26

−0.10

0.06

0.02

0.09

−0.12

0.17

−0.05

−0.10

Executive systems function

Working memory

  WCSTperseverative responses

−0.8 (−1.6, 0.1)

−0.22

0.04

−0.21

−0.23

−0.45**

0.05

−0.15

0.16

−0.08

0.15

−0.23

−0.08

 Letter-number sequence  (WAIS)

−0.6 (−1.3, 0.0)

−0.31

−0.12

−0.15

−0.23

−0.27

0.06

−0.09

0.06

−0.20

0.21

−0.27

−0.30

Fluency

 COWATtotal correct word

−0.45 (−1.2, 0.0)

−0.30

0.21

−0.28

−0.01

−0.10

0.28

0.00

−0.06

−0.05

0.13

−0.12

−0.29

 Animal category

−0.25 (−1.2, 0.6)

−0.16

0.15

−0.24

0.28

0.03

0.27

0.02

0.04

−0.18

0.04

0.06

−0.05

Set shifting/response inhibition

 Trailmaking test part B

−0.2 (−1.3, 0.4)

−0.40*

0.05

−0.54**

−0.01

−0.47*

0.27

−0.13

0.27

0.03

0.15

−0.08

0.06

 Global deficit score

0.47 (0.26, 0.89)

0.52**

−0.22

0.48**

−0.05

0.40*

−0.35

0.16

−0.35*

0.16

−0.40*

0.03

−0.15

Pearson’s product moment correlation was performed

* Denotes P value < 0.05. ** Denotes P value < 0.01

Wechsler adult intelligence scale, Paced Auditory Serial Addition Task, Brief Visuospatial Memory Test–Revised, Hopkins Verbal Learning Test–Revised, Wisconsin Card Sorting Test, Controlled Oral Word Association Test

All tests scores are reported as T scores except the GDS which is a summary score of the entire test battery with higher scores indicating greater impairment on a scale from 0 to 5 with 0–0.5 considered normal cognition

The stepwise regression analysis identified the following potential independent correlates with GDS: the accuracy and speed of the two simple detection tests (DET1lmn, DET2acc) and three more complex tasks: the accuracy of the associate learning (ASSLacc), the accuracy of monitoring tasks (MONacc), and the accuracy of the One Back test (ONBacc) (see Fig. 1). A regression model using GDS as the dependent variable and these measures as independent variables yielded a model R2 of 0.53 (P < 0.0001) indicating that approximately 53% of the variance in the GDS is explained by these five CogState variables. In the validation analyses using results from the stepwise regression analysis of 1,000 bootstrap replicates MONacc appeared in 96% of the replicates, DET1lmn appeared in 76% of the replicates, DET2acc appeared in 71% of the replicates, ASSLacc appeared in 57% of the replicates and ONBacc appeared in 51% of the replicates. A regression model using GDS as the dependent variable and MONacc, DETlmn and DET2acc (the three variables that appear in more than 70% of the replicates) yields a model R2 of 0.39 (P = 0.0002).
https://static-content.springer.com/image/art%3A10.1007%2Fs10461-011-0033-9/MediaObjects/10461_2011_33_Fig1_HTML.gif
Fig. 1

Correlation between parameters from the CogState computerized battery. Correlation between different components of the CogState battery and GDS from CHARTER testing was determined by Pearson product-moment correlation coefficient. The five components displayed here were included in the final multivariate regression model

To further explore the utility of the composite score derived from the computer battery tests to detect cognitive impairment based on GDS, we performed an area under ROC analysis using the combination of five significant test parameters identified from the multivariate regression model. The area under ROC curve for the specified model, including a composite of ASSLacc, ONBacc, MONacc, DET1lmn, and DET2acc was 0.90 (95% CI: 0.81–0.99, P < 0.0001; Fig. 2).
https://static-content.springer.com/image/art%3A10.1007%2Fs10461-011-0033-9/MediaObjects/10461_2011_33_Fig2_HTML.gif
Fig. 2

Receiver operating characteristic (ROC) curve of five parameters from the CogState computerized battery for Global Deficit Score from CHARTER. CHARTER refers to the area under the curve for the CogState composite score to predict dichotomized GDS score was 0.90 (95% CI 0.81–0.99)

Discussion

In this study, we compared a brief, self-administered computerized screening battery with formal neurocognitive assessment. The best correlations were with the simple reaction tests, which measure functional speed as represented numerically in the correlation matrix and visually in the scatterplots. Cognitive slowing is a prominent feature of HAND, and thus, these results are consistent with previous comparisons of the CogState battery to formal neurocognitive functioning although the associations are not as robust for less severe cognitive impairment [13]. With the exception of reaction tests, the other individual tests in the computerized battery correlated poorly with formal neurocognitive testing.

While these results illustrate that a single brief screening test may be insensitive to identifying people with neurocognitive impairment, particularly those with mild impairment, the modeling of several parameters from the CogState battery against the Global Deficit Score highlights that there is utility to a brief computerized neurocognitive screening tool. Notably, the use of the five significant test parameters from the regression analysis to create a composite score correctly classified 90% of individuals as cognitively impaired or not. While these results require additional validation, they suggest clinical applicability for such a computerized battery. Future prospective studies are needed to determine the sensitivity of the CogState battery to identify HAND. Identifying persons with either ANI or MND with a relatively simple screening tool is particularly important for prevention to be effective; the early recognition of cognitive impairment will provide opportunities for timely intervention as well as ongoing evaluations for progression and the need for additional services to manage activities of daily life. If the screening tool proves to be sensitive with established valid cutpoints, this battery could be used to identify persons with mild cognitive impairment. However, in the absence of cutpoints that denote clinical significance, the battery will not achieve optimal clinical utility.

We recognize the limitations of this analysis. It consists of a small group of individuals and lacks a control group. No persons with advanced cognitive impairment were included. Another limitation to these findings was the lack of correlation between the Associate Memory test and the measures of learning and memory. The CogState battery evaluated here was insensitive to these measures and thus may impact the ability of Cogstate to identify persons with impaired learning ability. Additionally, there was a lack of correlation between CogState indices and the Trailmaking Part B test, which is widely used to reflect executive function with important implications regarding one’s ability to perform activities of daily living independently [33]. Our data fail to confirm the previous work by Cysique et al. [13], in which the strongest correlations were with the Trail Making tests. However, their work focused on persons with HAD. While the Trailmaking Part B test does not necessarily capture all of the components of executive control important for daily living, our findings suggest that CogState may not provide critical information related to early decline in functional independence, though it should be noted that executive function is a heterogeneous construct.

In summary, we found that a compilation of the tests from a brief computerized screening tool for neurocognitive function was correlated to traditional neurocognitive testing among HIV-infected persons and a composite score of five parameters could discriminate cognitive impairment. These findings confirm previous reports of correlation between brief computerized CogState battery and standard neuropsychological examination [13], especially for identifying cognitive slowing, a central feature of HIV-associated neurocognitive disorders, although there were differences regarding different domains potentially related to differences in the severity of impairment in the studies [34]. Nevertheless, additional research is needed to fully evaluate the utility of this battery, specifically to assess the composite score based upon the significant parameters identified in the regression analysis.

Acknowledgments

The authors would like to express our appreciation to all the SUN study and CHARTER participants. The CNS HIV Anti-Retroviral Therapy Effects Research (CHARTER) is supported by award N01 MH22005 from the National Institutes of Health.

Conflicts of interest

No authors have conflicts of interest regarding this research.

Copyright information

© Springer Science+Business Media, LLC 2011