Introduction

Accessible and self-administered tools are urgently needed to screen for, monitor and treat the cognitive impairment (CI) experienced by almost half of patients living with multiple sclerosis (MS) [1]. MS is a chronic inflammatory and neurodegenerative disorder afflicting three times more women than men. Its first symptoms begin prior to age 50 in over 90% cases, in the prime of patients’ productive lives. CI afflicts individuals with both relapsing and progressive forms of MS [1, 2]. Worsening of CI is in turn predictive of loss of employment, and loss of quality of life (QOL), affecting function in all spheres of activities of daily living [3,4,5]. Furthermore, early CI predicts subsequent functional decline. Loss of information processing speed is the most common type of CI in MS [6], and over time, the Symbol Digit Modalities Test (SDMT) has been established as the most sensitive test for detection of loss of processing speed even early in the MS disease course [7]. Consequently, SDMT is the mainstay for both CI screening as well as measuring outcomes, including as a component of the widely used three-part Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS) [8], among other batteries for MS [9].

Recently cognitive remediation trials, now targeting specific cognitive domains impacted by MS [10], have shown efficacy, most notably the landmark MEMREHAB trial that resulted in improvements in verbal memory [11,12,13,14]. Unfortunately, there is a shortage of cognitive therapists, and access to qualified providers may be limited for many patients with mobility or cognitive impairments who are living outside of urban centers and specialized MS care centers. Even for patients with access to MS Centers, management of CI and other domains affected by MS often takes a back seat to the need to discuss and monitor an increasingly complex array of disease modifying therapies (DMTs). To date, therefore, care delivery systems targeting cognitive function in MS are overwhelmingly inaccessible or inconvenient.

Game-based technologies, especially when deployed remotely, may play a substantial role in bridging this unmet need [15]. The purpose of the current pilot study was to evaluate the feasibility of treating processing speed in patients with MS using a tablet-based, videogame-like digital treatment.

Methods

Participants and Study Setting

A total of 21 participants with MS were recruited from the University of California, San Francisco Multiple Sclerosis and Neuroinflammation Center between January and March 2017. Inclusion criteria included: age 18 years or older; a diagnosis of MS by 2010 Revised McDonald criteria [16]; internet connectivity available in the home or work environment; and general personal concerns about cognition. Exclusion criteria included: visual, dexterity or cognitive deficit so severe that it precluded the use of a tablet-based tool. Participants completed a baseline neurological and cognitive evaluation. Then, participants utilized an in-home tablet-based tool for 25 min a day, 5 days a week, for 4 weeks, after which they returned for a repeat in-clinic evaluation.

Standard Clinical and Cognitive Measures

  • Demographic (age, gender, ancestry, education, employment) variables were obtained from all participants, and MS type, duration since first symptoms, Neurostatus Expanded Disability Status Scale (EDSS) [17] and MS DMT were obtained from the medical record for MS participants. The neurological evaluation included:

  • MS Functional Composite 4 (MSFC4) components, as outlined by Cohen et al. [18].

    • Walking speed: Timed 25 Foot Walk (T25FW).

    • Dexterity: nine-hole peg test (9HPT).

    • Vision: Sloan low-contrast letter acuity test (LCVA).

    • Cognition: the paced auditory serial addition task (PASAT) was replaced by the SDMT [19] as the SDMT is more congenial for patients and clinicians, rapid, and forms a component of the BICAMS. Serial versions of the test were used to minimize practice effects.

  • Paper and pencil cognitive tests.

    • Brief International Cognitive Assessment for Multiple Sclerosis (BICAMS), a standardized, internationally validated battery requiring 15 min or less.

      • Information processing speed: SDMT (as above; written version was administered to allow adequate comparison with the digital tools) [19].

      • Verbal memory (immediate recall): CVLT II Trials 1–5 [20]. Serial versions of the test were used to minimize practice effects.

      • Visual memory (immediate recall): Brief Visuospatial Memory Test Revised (BVMT-R) [21].

  • Patient-reported mood was assessed using the 14-item Hospital Anxiety and Depression Scale (HADS) [22, 23], a self-report instrument containing seven questions probing anxiety and seven questions for depression, each scored separately in a Likert fashion (0 through 3). Scores 0–7 are categorized as normal, 8–10 mild, 11–14 moderate, and 15–21 severe [24]. A threshold score of 8 or greater on the HADS depression subscale provides a sensitivity of 90% and specificity of 87.3% for major depression, and on the anxiety subscale provides a sensitivity of 88.5% and a specificity of 80.7% for generalized anxiety disorder only [23].

Digital Cognitive Measures

The Cogstate computerized MS battery consists of 4 game-like tasks presented on a web-based platform (www.cogstate.com) and requires about 15 min for administration. It measures several neuropsychological constructs with considerable construct and criterion validity [25]. Cogstate is a widely used platform which can be used for both multi-center clinical trials and for screening in clinical practice settings; validity of data is ensured by expected item accuracy and outlier detection. Cogstate has been deployed in the evaluation of CI in MS [15]. The Cogstate battery has a normative database from > 50,000 participants (ages 10–99), and for any measures, scores of one standard deviation or more below the age-based normative data are considered to have a mild impairment; having one or more impaired scores counts as an impaired assessment. Four tests were completed in the following order:

  • Detection task (DET): a reaction time task assessing psychomotor function. The subject presses the ‘Yes’ key as quickly as possible when the central card turns face-up. The face-up card displayed is always the same joker card. The primary outcome on this task is reaction time.

  • Identification task (IDN): a choice reaction time task assessing visual attention. A card is turned over in the center of the screen, and the subject should respond ‘Yes’ if the face-up card is red, or ‘No’ if it is black. Jokers are used again to ensure that playing cards presented in the next task were not previously seen. The primary outcome on this task is reaction time.

  • One card learning (OCL): assesses visual recognition memory and attention. Cards are sequentially shown and subjects are instructed to respond ‘Yes’ if the face-up card has appeared in the task before, and ‘No’ if it has not yet appeared. Normal playing cards are displayed without jokers. The primary outcome on this task is accuracy of responses.

  • One back task (ONB): assesses working memory and attention. Subjects are instructed to respond ‘Yes’ if the face-up card is exactly the same as the immediately previous card, or ‘No’ if it is not. The primary outcome on this task is reaction time.

  • The UCSF match test: match is a 2-min test of executive functions and processing speed that is based on the SDMT but delivered on a tablet using the TabCAT software platform (memory.ucsf.edu/TabCAT). Respondents are shown a number/symbol key at the bottom of the screen. Using this key as a reference, they are asked to tap the symbol that corresponds to a series of number cues as quickly and accurately as possible. In comparison to SDMT, it places less demand on motor functions and literacy because subjects tap rather than write their responses; also, it can be self-administered. Performance is scored by the total correct in 2-min. The Match shows expected correlations with traditional neuropsychological tests and regional gray matter volumes [26].

Digital Treatment for Cognitive Deficits

Project: EVO™ is an investigational digital treatment developed by Akili Interactive Labs. It uses the Selective Stimulus Management Engine (SSME™) engine, designed to improve attention and inhibitory control through a video game-like interface. The SSME™ engine involves simultaneous engagement in visual targeting and continuous motor tasks in an adaptive, autonomous algorithm that continuously pushes an individual’s cognitive control performance within the context of multi-tasking interference. This enables the administration of a personalized treatment experience specific to the needs of each individual patient. The Project: EVO™ investigational digital treatment has shown efficacy in a randomized, controlled trial of 348 children and adolescents diagnosed with ADHD on the predefined primary endpoint, a change in the Attention Performance Index (API), a composite score from the Test of Variables of Attention (T.O.V.A.®) system, a computerized performance test used to objectively measure attention and inhibitory control. Based on the results of the study, Akili filed AKL-T01 with the U.S. Food and Drug Administration (FDA) for clearance as a novel treatment for children and adolescents with ADHD. For the current study, we defined adherence as completion of 75% or more of prescribed days of training, i.e., 15 days or more.

Ethical Approvals

All procedures performed in studies involving human participants were in accordance with the ethical standards of the UCSF Institutional Review Board and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.

Statistical Analyses

To evaluate feasibility of this type of intervention in patients with MS, we defined adherence as completing ≥ 75% of prescribed days, i.e., at least 15 days, over the course of the treatment period. To account for delays in obtaining WiFi and initiating the study, and for travel and other factors, we expanded the timeframe to 5 weeks (35 days). To determine the improvement on the scores of processing speed between baseline and return visit, either left-tailed or right-tailed paired-samples t tests were performed. To evaluate the effect of baseline demographic and clinical variables on improvement in SDMT over the pilot study duration, simple linear regression was used to assess each variable’s prediction performance on SDMT change. Then, in analyses derived from random forest algorithms, the values of each feature were randomly permuted and a new mean squared error (MSE) was calculated in each simple linear regression model. Each feature’s importance was measured by the increase in the MSE. This procedure was repeated 20 times for each variable. Finally, to evaluate predictors of adherence to treatment, we used simple linear regression. All statistical analyses were performed in R 3.5.0.

Results

Participant Characteristics

The baseline demographic and clinical characteristics of the participants are described in Table 1. SDMT was not associated with age, sex, education, EDSS, disease duration, depression or anxiety at baseline (p > 0.05 for each).

Table 1 Demographic, clinical and cognitive characteristics of participants (N = 21)
Full size table

Feasibility of In-Home Digital Treatment for Cognitive Deficits: User Experience

Of 21 MS participants enrolled (mean [SD] SDMT z score: − 0.21 [1.16]), 18 completed and returned for their 4-week visit. Reasons for non-adherence included logistical (n = 1), vertigo induced by the game, (n = 1), and physical discomfort due to prolonged use of the tablet (n = 1). The 18 who completed the study played an average of 19.7 days (median [SD]: 20.5 [8.4]) per month. Overall, 78% of these 18 participants completed ≥ 75% of prescribed days, i.e., at least 15 days (Fig. 1), and 50% (n = 9) completed all 20 days or more.

Fig. 1
figure 1

Participant adherence to game protocol

Full size image

Predictors of Persistence with the Training Sessions

Lower SDMT scores at baseline were associated with greater number of days played over the 4 weeks (p = 0.003, R2 = 0.44, Supplementary Fig. 1; after adjusting for employment, a possible proxy for free time, partial R2 for SDMT = 0.30), suggesting an association between cognitive deficits and motivation to complete the sessions. In fact, lower scores across the other BICAMS cognitive tests (BVMT, CVLTII, p < 0.001 for each) as well as higher anxiety scores (HADS, p = 0.015) were also associated with greater number of days played.

Changes in Cognitive Scores after 4 Weeks of Digital Treatment

Over the 4-week period, scores improved significantly on our primary outcome, the SDMT (paired t test, p = 0.003, N = 18) (Fig. 2). In fact, the mean increase of 3.6 points was just shy of the 4-point clinically meaningful threshold established for trials assessing cognition in MS [7, 27].

Fig. 2
figure 2

Comparison of baseline and 1-month SDMT scores in 18 participants who completed the study (left-tailed paired t test, p = 0.003). Predictors of Improvement in the SDMT

Full size image

Scores also improved in 2/5 of the computer-based tests—specifically Match, which measures processing speed, and Cogstate DET, which measures psychomotor function—(p = 0.006 for each) (Table 2), showing encouraging consistency across tests of processing speed.

Table 2 Comparison of cognitive measures before and after 4 weeks of use of the Akili Interactive Project: EVO™ investigational digital treatment, in adults with MS
Full size table

When we evaluated the contribution of baseline clinical and demographic factors to SDMT improvement over the pilot study duration, three variables were identified: gender (male), employment (not employed) and anxiety category (borderline/abnormal) (Supplementary Fig. 2).

Discussion

In the current feasibility study, we report high enthusiasm for a videogame-like digital treatment for cognition, in a cohort of older adults with MS with a high baseline level of function. We observed 86% retention after 4 weeks, and in these 18 participants, 78% adhered to the treatment. These adherence rates for self-managed computerized cognitive rehabilitation at home are consistent with previous reports [28]. Interestingly, while we did not enroll patients with specific levels of CI, both cognitive deficits and anxiety at baseline were associated with greater use of the treatment over the course of the study, indicating feasibility of such an intervention in a group with mild to moderate impairments. Further, we confirmed that improvements were consistent across several tests of processing speed, and could be detected during tests that can be self-administered at home. Finally, we also identified specific features, such as pronounced vertigo, which might preclude use of video games as a digital therapeutic, and form a basis for exclusion from participation in larger trials.

Individuals with MS have demonstrated enthusiasm for using digital technologies to: (1) access information regarding MS, (2) pursue routine and rehabilitation care (e.g., through telemedicine-based web programs), (3) monitor their MS (e.g., through smartphone or activity trackers) and importantly (4) participate in research. Additionally, MS patients have enthusiastically embraced home-based care strategies [29]. Here, we extend these observations to report adherence to an in-home, videogame-based tool designed to improve processing speed. Recently, a videogame approach was deployed through 12-weeks using the adaptive PositScience BrainHQ® tool in 135 patients with MS and SDMT z scores of −1 or less. Training was reported to result in a significant cognitive composite score improvement in the 74 participants randomized to the PositScience BrainHQ® tool vs. in the 61 patients playing non-specific video games [15]. There was no improvement in individual cognitive tests, but SDMT was not included as an outcome measure. The tool tested in the current study is also unsupervised, but much less time-intensive (25 min daily for 4 weeks, vs. 1 h daily for 12 weeks), suggesting possible advantages in terms of overall burden to patients, and hence adherence, as demonstrated by adherence reported in the respective studies. Additionally, we found improvements in processing speed using several different tests, while we detected no change in other domains of cognition, suggesting a specific and fairly robust effect. The PositScience BrainHQ® exercises do not employ a multi-task interference paradigm, which could contribute to differences in effects.

Although the SDMT is more psychometrically sound than the PASAT [30], replacing the PASAT with the SDMT in the MSFC has not always been supported by the data [31]. The current feasibility study is also limited by small sample size, and lack of a control group to confirm that the improvements in SDMT could not simply be explained by learning effects or placebo. However, other similar studies with intervention control patient groups have demonstrated improvement on the SDMT for the treatment group [28]. From prior investigations, the current mean increase (3.6 points) almost meets the clinically meaningful threshold established for trials assessing cognition in MS (4 points [27]), and is substantially greater than anticipated learning effects in a similar time period [32]. A larger blinded, randomized and controlled study is underway.

Conclusions

In summary, the current feasibility study lends further support to the role that videogame-based digital treatments may play for ameliorating CI in MS [15], and its potential advantages [tablet-based, game-based, short and convenient session duration (25 min) over a fairly short timeframe (4 weeks)] support the expansion of the study to a larger, controlled trial evaluating both efficacy and sustained effects. The anticipated impact of a home-based training program to ameliorate CI in MS is large, given the scarcity of rehabilitation programs, and the improved accessibility, cost effectiveness, and rapid deployment that are afforded by remote trials.