Introduction

With the advent of combined antiretroviral therapy (cART) in the mid-1990s, life expectancy of patients with human immunodeficiency virus (HIV) has improved, including among those with cognitive impairment. While the phenotype is milder, the prevalence of HIV-associated neurocognitive disorders (HAND) has remained remarkably high and HAND is estimated to occur in 30–60 % of HIV-positive (HIV+) individuals (Grant 2008). HAND includes a wide spectrum of cognitive impairment ranging from a mild, asymptomatic form to those who have severe difficulties with everyday function (McArthur et al. 2010). The disease course is also variable with patients' cognition worsening over time, stabilizing, or even improving, often depending on compliance with cART and possibly on the degree of penetration of the cART across the blood–brain barrier.

Patients with HAND have impairments in everyday functioning. They have increased dependence with instrumental activities of daily living (IADL) (Heaton et al. 2004), are more likely to be unemployed (van Gorp et al. 1999; Benedict et al. 2000), and are more likely to have impaired driving (Marcotte et al. 1999, 2004). They also have lower medication adherence (Hinkin et al. 2002; Andrade et al. 2005), increasing an individual patient's risk for adverse virologic and clinical outcomes (Gifford et al. 2000). Finally, patients with all stages of HAND have an increased mortality risk (Ellis et al. 1997; Power et al. 2009).

The Frascati criteria for diagnosing HAND combines functional and neurocognitive assessments and stratifies individuals into three categories: (1) Patients with asymptomatic neurocognitive impairment (ANI) exhibit impairment on neuropsychological testing in at least two cognitive domains but do not have functional difficulties; (2) Those with mild neurocognitive disorder (MND) also have impairment on neuropsychological testing in at least two domains but demonstrate mild functional difficulties; and (3) Patients with HIV-associated dementia (HAD) exhibit marked impairment on neuropsychological testing and functional impairment that interferes with day-to-day activities (Antinori et al. 2007). Diagnosing a patient with HAND therefore requires an accurate assessment of their functional abilities. Current functional assessments are based principally on self-report or are too time consuming to administer in a research or clinical setting.

The Computer Assessment of Mild Cognitive Impairment (CAMCI)Footnote 1 was designed and validated to test mild cognitive impairment in older, HIV-negative (HIV−) patients (Saxton et al. 2009) and has been modified for use in an HIV+ population. Becker et al. (2011) demonstrated that the CAMCI is sensitive to mild forms of cognitive impairment in an HIV+ population.

This investigation sought to determine if the CAMCI could be used to assess functional impairment in an HIV+ population, thereby allowing its use to diagnose HAND and differentiate cognitive status among HIV+ individuals. We hypothesized that subjects with HAND will perform worse on CAMCI neuropsychological and functional tests compared with HIV− control subjects and that subjects with HAD will perform worse than those with ANI and MND.

Methods

Standard protocol approvals, registrations, and patient consents

This study was reviewed and approved by the institutional review board at Johns Hopkins University. Written informed consent was obtained from all patients participating in the study.

Participants

One hundred fourteen HIV+ individuals and 38 HIV− individuals were seen at the General Clinical Research Clinic at Johns Hopkins Hospital in Baltimore, Maryland from 2007 to 2008. HIV+ patients were recruited from the Infectious Disease (Moore) Clinic for HIV care at Johns Hopkins Hospital. We also identified prospective participants through our other clinical research studies at Johns Hopkins, specifically the Northeast AIDS Dementia cohort (McArthur et al. 2004) and the Oxidative Stress Cohort (Mohamed et al. 2010). Demographically matched HIV− individuals were recruited from inner city Baltimore via flier distributed throughout Johns Hopkins Hospital. In addition, we asked our HIV+ participants if they knew family or friends who were HIV− who would consider participating in the investigation, thus explaining the overall low education level and high frequency of drug abuse. Inclusion criteria were the following: adults 18 years or older, ability to provide written informed consent, and ability to ambulate at first clinic visit. Exclusion criteria were the following: history or current opportunistic central nervous system infection, history or current schizophrenia, current severe affective disorder believed to explain a subject's cognitive impairment, history of a chronic neurological disorder including multiple sclerosis and epilepsy, and current intoxication on illegal drugs or alcohol. Using these exclusion criteria, a small number of cases were excluded.

All participants underwent a clinical assessment in which basic demographic data were obtained as well as a complete medical, psychiatric, and neurologic history. Substance abuse information was ascertained using a self-assessment questionnaire (Valcour et al. 2004). Alcohol binges were defined as five or more drinks within 2–4 h. A neurologic examination was also performed on the HIV+ individuals. Laboratory testing confirmed HIV serostatus. Hepatitis C status was confirmed with laboratory testing for HIV+ individuals and by self-report for HIV− individuals. Laboratory testing for the HIV+ individuals also included CD4 T cell counts and HIV RNA levels in the serum. Urine toxicology was performed at the time of the visit to determine acute intoxication. Seventy-eight HIV+ subjects also underwent a lumbar puncture and subsequent testing of CD4 T cell counts and HIV RNA levels in the cerebrospinal fluid (CSF). Depression symptomatology was tested using the Beck Depression Inventory (Beck and Beamesderfer 1974) and a score greater than 16 was defined as depression. The North American Reading Test was also administered as a measure of premorbid intellectual abilities.

Classification of cognitive status

Neuropsychological testing was also performed on all subjects. Tests included semantic and verbal fluency tests (McCarthy and Warrington 1990), Timed Gait test (Robertson et al. 2006), Grooved Pegboard test (Klove 1963), Trail Making tests A and B (Reitan and Wolfson 1985), Digit Symbol Substitution test (Wechsler 1981), the Rey Auditory Verbal Learning test (Rey 1941), the Computerized Reaction time portion of the California Computerized Assessment Package (CALCAP) (Miller et al. 1991), the Odd Man Out test (Flowers and Robertson 1985), and the WAIS III Letter–Number Sequencing test (Wechsler 1997). To assess functional status, we completed the Karnofsky functional performance scale for each subject (Karnofsky et al. 1948) and the Function/Quality of Life scale (Stewart and Ware 1993). Based on these neuropsychological and functional tests, HIV+ subjects were classified into HAND category using the Frascati criteria.

CAMCI testing

The CAMCI collects data in a standardized fashion on a Tablet PC. The testing takes approximately 25 min and score reports are available immediately. The CAMCI has been validated for detection of mild cognitive impairment among non-demented, community-dwelling adults 60 years and older with a sensitivity of 86 % and a specificity of 94 % when compared to adjudication diagnosis based on paper and pencil tests (Saxton et al. 2009). The CAMCI utilized in our study was modified for use in an HIV population. Items removed from the original CAMCI included many of the items that were not scored but rather included to provide additional information to the clinician. Specifically, the removed items include a few of the orientation questions, the self-reported questions regarding depression, anxiety, and alcohol use, and the individual perception of memory questions. The verbal recognition neuropsychiatric test was also removed. Items added to the original CAMCI included more trials in the Simple Reaction Time task and the Digit Span Forward and Reverse tasks.

The modified version of the CAMCI administered to each subject consisted of seven neuropsychological tests and six functional tasks. At the beginning of the CAMCI administration, research staff type in the participants demographic information and start the testing. The approximately 25-min program then runs independently with the participant interacting with the computer with a stylus. The computer asks the participant the date and then begins with self-report questions involving previous computer use and driving. The neuropsychological tests then begin. Directions come up on the screen and are spoken aloud for each test. The tasks include Simple Reaction Time, Recurring Picture, Go/No Go rule 1 and rule 2, Word Recall, and Digit Span Forward and Digit Span Reverse. Together, these tasks assess attention, executive function, different forms of learning and memory, and speed of processing.

The functional task consists of a simulated supermarket–shopping trip and includes following directions to the supermarket (Shopping Trip Directions task) (Video 1), remembering four items to purchase at the market (Shopping List task), and subjects are told that they must remember to stop at the bank (Errand—bank) and post office (Errand—post office) (Video 2) on their way to the supermarket. When the task begins, the image on the screen indicates that they are in a residential neighborhood as seen in first person through the windshield of a car on the road. Written directions to the supermarket are located on the bottom right hand corner of the screen. While driving, they pass a few other cars as well as people on bikes. While following the directions, the sign for the bank will come into view on the left hand side of the road and the subject needs to touch the stylus to the sign in order to stop at the bank. At the bank, they must perform a transaction with the automated teller machine (ATM). After leaving the bank, the post office comes into view on the right hand side of the road and the subject must remember to touch the stylus to the post office sign.

Once at the supermarket, the computer presents to the subject pictures of supermarket items and the subject must select which items they were supposed to purchase (completion of the Shopping List task) (Video 3). Following this assisted recall task, there is an incidental recall task in which the subject is asked to choose from among other pictures what other people or objects they noted during the driving task (for example, the bus that drove past the subject's car).

Statistical analysis

Subjects were divided according to their HIV serostatus, and chi-squared or analysis of variance (ANOVA) was used to compare performance between HIV− and HIV+ subjects on the neuropsychological and CAMCI tests. Comparisons of each neuropsychological and CAMCI test were also made between HIV− and HIV+ subjects who were classified into HAND status based on the Frascati criteria. Again, chi-squared or ANOVA was used as appropriate. Results were considered significant if the p value was ≤0.05. Sensitivity and specificity of each portion of the CAMCI was determined, using the Frascati criteria diagnosis as the gold standard. Regression analysis sought to correlate CAMCI scores with scores on functional measurements. As there were some significant differences between the group demographics, we used a general linear model to fit the CAMCI test as a function of HIV serostatus, adjusting for baseline hepatitis C status, depression symptomatology according to the Beck Depression Inventory, and North American Reading Test (NART) scores. Pairwise comparisons were made using Tukey's test and Dunnett's T test. Stata 10.0 software (StataCorp LP, College Station, TX) and SAS (SAS Institute, Cary NC) were used for the statistical analysis.

Results

Participants

Table 1 contains the demographic characteristics of the 38 HIV− subjects and 114 HIV+ subjects. The HIV+ subjects are shown as a group and also classified into their HAND diagnosis of normal cognition, ANI, MND, and HAD. When comparing HIV− to HIV+ subjects, there were no differences in age, gender, education, race, history of computer use, history of ATM use, or history of driving, though HIV+ subjects were less likely to be currently driving (p = 0.01). The groups were also similar in their alcohol and illicit drug use, with differences noted only in heroin use in the past 6 months (18 % of the HIV− group versus 6 % of the HIV+ group, p = 0.02). Finally, HIV− and HIV+ subjects had different rates of hepatitis C infection (27 % of HIV− subjects and 59 % of HIV+ subjects, p < 0.001), depression symptomatology (mean Beck Depression Inventory score of 5.5 in HIV− subjects and 10.3 in HIV+ subjects, p < 0.00), and NART scores (mean NART score of 100.2 in HIV− subjects and 95.9 in HIV+ subjects, p = 0.02). When comparing HIV− subjects and the HIV+ subjects classified into their HAND diagnosis, there were no differences in age, gender, education, race, history of computer use, history of ATM use, history of driving, and current driving rates. The differences in hepatitis C status (p = 0.02) and depression rates remained (p < 0.01). There was no difference in NART scores between all of the groups (p = 0.13).

Table 1 Demographic information
Full size table

Subjects who were HIV+ and hepatitis C positive were older (mean age of 48.2 versus 44.9, p < 0.01), had less education (12.2 years of school versus 13.4 years of school, p < 0.01), had lower NART scores (93.9 versus 98.8, p < 0.01), were more likely to have depressive symptomatology (31.3 versus 10.6 %, p < 0.01), and were more likely to be African-American (p = 0.03). In addition, among subjects who were HIV+, approximately 4 % of the normal cognition group had depressive symptomatology whereas approximately 58 % of those with HAD had depressive symptomatology (p = 0.03).

Among the HIV+ patients, the mean CD4 count was 360.4 (standard deviation 206.2) with a mean plasma viral load of 32,987 copies/ml (standard deviation 104,305 copies/ml). The HIV+ groups, those with normal cognition, ANI, MND, and HAD, did not differ in CD4 count, HIV viral load in the plasma, HIV viral load in the CSF, or in the number of subjects who had obtained complete viral suppression with undetectable viral loads in the plasma and the CSF. Fewer total subjects completed lumbar punctures but there were no differences in the lumbar puncture rate between the HIV+ groups (p = 0.55). While there was a trend toward increased serum viral load among those with more advanced HAND, the differences were not significant (p = 0.43). In addition, there was no difference in the antiretroviral central nervous system penetration effectiveness (CPE index) (Letendre et al. 2008) comparing all four groups (p = 0.88) or in the pairwise comparisons.

Neuropsychological testing

When comparing HIV− subjects to HIV+ subjects, there were significant differences in tests of reaction time and executive function (Table 2). Specifically, HIV− subjects performed better on the CALCAP, Choice reaction time test (434 versus 471.6 s, p = 0.03), and the WAIS III Letter–Number Sequencing Test (10.2 versus 8.9, p = 0.03). In contrast, the Rey Auditory Verbal Learning Test recognition score was lower among the HIV− subjects (11.7 versus 13.5, p = 0.03), though HAD patients had worse performance than HIV+, normal cognition patients (p < 0.01).

Table 2 Neuropsychological tasks
Full size table

CAMCI neuropsychological tests

Three of the neuropsychological tests that are part of the modified CAMCI differentiated between HIV− and HIV+ subjects (Table 3). HIV− subjects performed better than HIV+ subjects on the Forward Digit span task (score of 7.6 versus 6.8, p = 0.02), Backward Digit span task (score of 6.2 versus 4.7, p < 0.01), and Word Recall task (score of 3.7 versus 2.4, p < 0.01). These differences were no longer significant when controlling for the demographic differences of hepatitis C infection and Beck Depression Inventory score. When controlling for the differences in NART scores, the Forward and Backward digit span, as well as Word Recall test, retained their significance (p < 0.01 for all three tests).

Table 3 Performance on CAMCI tasks
Full size table

When comparing HIV− subjects and the HIV+ subjects classified into HAND stages, the same neuropsychological tests showed significant differences between the groups. The Forward Digit span score was different among the five groups, averaging 7.6 in HIV− patients compared to only 6.2 in patients with HAD (p = 0.01). Pairwise comparisons indicated that the differences between the HIV− and HAD subjects, as well as between the HIV+, normal cognition and HAD subjects, were significant. The Backward Digit span task was also different between the five groups, following the same generally downward trend between the HIV− subjects and the HAD subjects (score of 6.2 versus score of 4.4, p < 0.01). Pairwise comparisons indicated a significant difference between the HIV− subjects and the MND and HAD subjects and between the HIV+, normal cognition group and the MND subjects. Word Recall scores were also different between the five groups, decreased from 3.7 in HIV− subjects to 2.1 in HAD subjects (p < 0.01). The pairwise comparisons indicated a significant difference between the HIV− subjects and all four categories of HIV+ subjects. When the general linear model controlled for hepatitis C infection rates and depression symptomatology, differences between the five groups for each of these tests were no longer their significant.

CAMCI functional tasks

Four of the functional tasks in the CAMCI were able to distinguish between HIV− and HIV+ subjects (Table 3). Specifically, HIV− subjects performed better on the Shopping Trip Directions task (16.2 versus 15.1, p = 0.05), Shopping List task (3.7 versus 3.1, p = 0.01), the Errands—bank (79 versus 53 %, p < 0.01), and Errands—post office tasks (79 versus 56 %, p < 0.01) (Fig. 1). After controlling for hepatitis C status and depression symptomatology, the Shopping Trip Directions task and the Shopping List task were no longer significant. The Errands—bank and Errands—post office tasks retained their significance (p < 0.01 for both tasks).

Fig. 1
figure 1

CAMCI Functional Directions. Four of the CAMCI functional tasks demonstrated significant differences in performance between HIV− and HIV+ subjects as well as between HIV− subjects and HIV+ subjects classified into HAND criteria. NML = normal cognition; ANI = asymptomatic neurocognitive impairment; MND = minor neurocognitive disorder; HAD = HIV-associated dementia

Full size image

The same functional tasks were significant when comparing HIV− patients and HIV+ patients classified by HAND diagnosis. There was a significant difference between the five groups on the Shopping Trip Directions task (p = 0.03) though there were no differences in the pairwise comparisons. The Shopping List task score was different between the five groups, decreasing from HIV− to HIV+ normal cognition to those with HAD (p = 0.02). The pairwise comparisons further noted a significant difference between the HIV− subjects and the HAD subjects. Similarly, the subjects' ability to remember to stop at the bank and post office (Errands—bank and Errands—post office) worsened from HIV− to HAD (bank, p = 0.01; post office, p = 0.03) (Fig. 1). The pairwise comparisons for both these tasks indicated a significant difference between the HIV− subjects and the HIV+ normal cognition subjects. Using the general linear model to control for hepatitis C status and depressive symptomatology when comparing the HIV− subjects to the HIV+ subjects classified into HAND diagnosis resulted in a loss of significance for each of these functional tasks.

Many of the CAMCI functional tasks correlated with conventional measures of functional performance. The Shopping Trip Directions task positively correlated with the Karnofsky performance scale (r = 0.19, p = 0.04), the Columbia Medication Management Test (r = 0.38, p < 0.001), and the San Diego Finance test (r = 0.30, p < 0.001). The Shopping List Task positively correlated with the Karnofsky performance scale (r = 0.27, p < 0.001) and the Columbia Medical Management Test (r = 0.23, p = 0.03). In addition, both the Errands—bank and the Errands—post office task positively correlated with the Karnofsky performance scale (r = 0.21, p = 0.02; r = 0.25, p = 0.01, respectively) and the Columbia Medication Management Test (r = 0.22, p = 0.03; r = 0.27, p = 0.01, respectively).

The area under the curve analysis indicates that forward digit span, forward digit span size, and the functional driving task differentiate between HIV+ individuals with and without HAD. Each of these tests achieved good sensitivity, but relatively poor specificity. For example, a forward digit span raw score cutoff of greater than or equal to 6 gives a sensitivity of 63 % and a specificity of 19 %, a forward span size raw score cutoff of greater than or equal to 5 gives a sensitivity of 80 % and a specificity of 9 %, and a raw score on the Shopping Trip Directions task of greater than or equal to 13 gives a sensitivity of 75 % and a specificity of 18 %.

Discussion

CAMCI is a computer-based screening tool that includes performance-based measures of functional impairment. Three of the CAMCI neuropsychological tests and four of the CAMCI functional tests differentiated between HIV− and HIV + subjects. The neuropsychological tasks of Forward and Backward Digit span, as well as Word Recall, separated the groups, as did the functional tasks of following directions to the supermarket (Shopping Directions task), remembering four items to purchase (Shopping List task), and remembering to stop at the bank and post office enroute to the supermarket (Errands—bank and Errands—post office). In many of these tasks, the HAD subjects had the worst performance. The CAMCI neuropsychological tests and functional tests also correlated with traditional measures of functional impairment. The CAMCI, therefore, could be a useful objective test of functional performance in HIV+ individuals. We cannot exclude, however, that differences in hepatitis C status, depression, and NART scores contributed to these findings.

With the exception of the CAMCI, the current gold standard of functional assessments are limited by being either time consuming to perform, or by needing to rely on patients' self-report, or both. For example, the Karnofsky performance scale, IADL scale of Lawton and Body, and the Katz/ADL Lawton self-maintenance scale are each assessments that require time and expertise to administer and are based on self-report from either the patient or the provider. Schifitto et al. demonstrated poor correlation between these more self-reported measures of functional impairment and neuropsychological outcomes involving HAND (Schifitto et al. 2001). Other researchers provided objective functional measures through rigorous laboratory assessments including testing abilities to shop, cook, vocational skills, and management of finances and medications (Heaton et al. 1995). While these objective measures were an improvement over subjective functional assessments, such large-scale rigorous laboratory measures could be impractical for both clinical trials and clinical practice.

The CAMCI is a potential alternative to these more cumbersome assessments. We sought to investigate the CAMCI in the HIV population and to compare CAMCI measures in HIV− and HIV+ individuals and in HIV+ individuals with and without HAND, Because there is no biomarker for HAND that can be used as a gold standard, the gold standard for HAND in practice is to use a clinical diagnosis combining neuropsychological test data, functional assessments, and other clinical assessments. These assessments (excluding the CAMCI data) were all used to diagnose HAND from the initiation of the study, so it would not be possible to compare the accuracy of the CAMCI results to the accuracy of the standard neuropsychological tests. Future studies with a limited neuropsychological test battery and functional assessments can be designed from the start to be compared to the CAMCI results.

The sensitivity and specificity of the Shopping Trip Directions task compares favorably to a sensitivity and specificity of 80 and 57 % for the International HIV Dementia Scale (Sacktor et al. 2005). The sensitivity and specificity of CAMCI subtests are also comparable to the sensitivity (71 %) and specificity (46 %) of the Grooved Pegboard non-dominant hand test, an established test for HIV dementia (using a cutoff of 1.5 SD below the age- and education-adjusted mean).

The CAMCI's utility was demonstrated by the correlation between the CAMCI testing and traditional measures of functional impairment as well as the sensitivity and specificity of individual tasks. These measures demonstrate that regardless of hepatitis C status and depression symptomatology the CAMCI tests both correlate with previously validated measures and discriminate between HIV− and HIV+ status. In addition, the CAMCI discriminates between those that are cognitively intact and those that have HAD in a manner that is easy to assess (i.e., a brief computer test).

The CAMCI results also indicate that there may be lower than expected performance in the group that was categorized as HIV+ with normal cognition. The pairwise comparisons are notable for the significant differences between the HIV− subjects and the HIV+ normal cognition subjects on the tasks of Word Recall and remembering to stop at the bank and post office (Errands—bank and Errands—post office). Furthermore, the HIV+ normal cognition group performed poorly on these tasks, with scores closer to subjects with HAND. The HIV+ normal cognition group, therefore, may have subtle cognitive difficulties that are being picked up by these CAMCI tasks. Future research looking at the HIV+ normal cognition group is needed to determine the significance of these results.

This study is limited by the demographic differences between the groups. After controlling for hepatitis C status and depression symptomatology, there were no longer differences between many of the CAMCI scores of the different groups. Subjects with hepatitis C, however, were older and less educated than those without hepatitis C. Older patients and those with less education have both been shown to have poorer performance on cognitive testing and have greater rates of dementia. Therefore, by controlling for hepatitis C serology, we have factored out a potentially large influence on the cognitive test results.

Furthermore, the external validity of our study is challenged by the severity of cognitive disease in our cohort. Many of our cohort patients were recruited because of their concerns and complaints regarding their cognitive status. They therefore do not represent the typical population of individuals with HAND that one would encounter in clinical practice. Later assessments can focus on determining standard scores in a general population and seek to assess the positive predictive value of the CAMCI in a general HIV+ population.

The CAMCI may be a useful objective measure of functional performance in the HIV+ population. While the results are limited by the demographic differences, the CAMCI may be able to assist clinicians in the diagnosis of HAND. This new tool therefore provides an opportunity for closer clinical follow-up and the earlier initiation of treatments specific for HAND.