1 Introduction

Every year, there are over 600,000 cases of new or first strokes and over two million cases of traumatic brain injury in the United States alone [1, 2]. This constantly increasing group of patients require frequent neurorehabilitation therapy but may be unable to see clinicians as much as required. Taking into consideration recent technological developments such as low-cost tablet computers and cloud computing, the field of telemedicine has risen to meet the growing demand for available care at home settings [3]. Telemedicine applications allow for patients to perform clinician-mediated therapy and recover from the comfort of their own home, circumventing the need to travel to a specialized therapy clinic which often proves difficult due to various constraints. Moreover, the availability of telemedicine options has been shown to increase patient compliance [4] as well as be effective in treating apraxia of speech, motor control degradation in the arm, and other complications of stroke and injury [5, 6].

The importance of telemedicine also connects with the rapidly developing field of neuroergonomics. Neuroergonomics is the study of how the brain behaves in natural environments, in so-called ecological situations [7, 8]. Several studies have already proven the validity of this school of thought in areas ranging from air traffic control to interaction with virtual reality [9–11]. Recording the brain using a teletherapy application at home bypasses sterile lab environments and more closely simulates real-world use of the system.

Although computerized methods for cognitive rehabilitation designed for patients to use on demand are being continuously developed [12–14], most evidence of the efficacy of similar treatments is limited to behavioral or standardized psychological tests [15–17]. Moreover, these results are valid only for regimens developed by specialized clinicians, and studies have found that some commercial applications do not necessarily provide comparable results [18, 19]. These outline the need for controlled and independent assessment of the capabilities of cognitive rehabilitation tools and specifically neural correlates of the paradigms in use.

In this pilot study, we aimed to perform an initial assessment of select cognitive tasks in Constant Therapy (©2014 Constant Therapy, constanttherapy.com), an iPad-based cognitive rehabilitation app which is already in active use by clinicians and patients to augment traditional therapy. Constant Therapy provides over sixty tasks targeting various areas of cognition [20]. Two tasks targeting attention and working memory have been selected for the proof of concept experiment demonstrating that past studies related to cortical activation and task expertise are applicable to the tablet-based therapy paradigm [11]. We hypothesize that in line with earlier studies, there will be a quantifiable difference in the brain activation of healthy subjects when performing the same task at different difficulties [21]. This hypothesis was chosen for this pilot study to test if task-induced differences in brain activation could be detected in the Constant Therapy neurorehabilitation tool.

To test this hypothesis, we chose to use the portable and accessible neuroimaging modality of functional near-infrared spectroscopy (fNIRS). fNIRS is a safe, wearable and relatively low cost optical measuring system of neural activity related to cognitive state and workload [11, 22, 23]. Compared to the traditional neuroimaging systems such as electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), fNIRS has a balanced trade-off in spatial and temporal resolution, fast setup, silent operation, minimal restrictions, and a miniaturized, battery operated, and wearable sensor set that can be potentially used in a home setting, making it ideal for use with telemedicine brain monitoring in conjunction with therapy apps [24]. By including these physiological signals into the clinician’s assessment of patient recovery, more effective therapy can be applied, which can therefore improve patient outcomes.

2 Method: Brain Activation Measurements During Tablet Therapy Application Use

2.1 Participants

Ten participants between the ages of 18 and 35 (five females, mean age = 23.2 ± 2.8 years) volunteered for this study. All confirmed that they met the eligibility requirements of being right-handed with vision correctable to 20/20, did not have a history of brain injury or psychological disorder, and were not on medication affecting brain activity. Prior to the study all participants signed consent forms approved by the Institutional Review Board of Drexel University.

2.2 Experimental Procedure

The experiment was performed in a single session lasting for approximately one hour. The sessions were held in a specially prepared subject room in our lab building free from distraction and outside interference (Fig. 1). Subjects were recorded using the fNIRS system as they performed the experimental protocol on a provided iPad 2 placed on a stand. The task application used for this experiment was Constant Therapy, a program that allows for clinicians to work with patients both directly and indirectly, and is notable for its ability to provide smart telemedicine even when subjects are at home. Each task was presented in randomized order and downloaded from the internet on the fly. The standard outcome measures of the program were in behavioral performance, the accuracy and latency of each trial.

Fig. 1.
figure 1

Subject operating application on iPad with fNIRS headband attached. Data acquisition apparatus and program visible on table and stand beyond subject.

Full size image

For the experiment, participants sat on a chair next to a desk with an iPad placed in a comfortable position before them, and were fitted with the fNIRS headband. Subjects were instructed to minimize head movement and only use their dominant (right) hands to interact with the iPad. The iPad was muted and screen brightness was set to maximum for consistency and clear view, and the iPad stand attached to the cover was set the same way for each subject. During the experiments the tablet screen was mirrored to a separate computer using Reflector 2 (©2016 Squirrels, LLC) and recorded using Microsoft Expression Encoder 4 (©2016 Microsoft Corporation).

The experiment was in two parts, one for each of the two tasks chosen, Symbol Matching and Pattern Recreation, as described below. Each of the tasks was given at two preselected difficulties, easy and hard, for a total of four block types throughout the experiment. In between blocks, subjects rested for 30 s to allow for their cortical oxygenation to return to resting levels, and a two-minute break was given in between the Symbol Matching and Pattern Recreation (Fig. 2). The exact order of blocks and whether Symbol or Pattern was presented first was balanced between subjects. The entire session took approximately 45 min.

Fig. 2.
figure 2

Sample timeline of experiment blocks showing order and timing. Difficulties are easy and hard presented in mixed order.

Full size image

2.2.1 Symbol Matching Task

The first task used was called Symbol Matching, which tested the subject’s attention (Fig. 3). In this task, subjects are shown a target symbol on the left part of the screen, and must select every panel exactly matching the target in shape and color, followed by hitting the “Check Answers” button to proceed. Clicking on an incorrect panel or hitting “Check Answers” without selecting every correct choice was considered an error. Subjects completed nine repetitions of this task in each block, and performed four blocks of each difficulty for a total of eight blocks of Symbol Matching. Blocks were presented in a pseudo-random order.

Fig. 3.
figure 3

Screenshots of easy and hard difficulties of Symbol Matching task

Full size image

2.2.2 Pattern Recreation Task

The second task used was Pattern Recreation, which tested the subject’s working memory (Fig. 4). This task presents the user with a sequence of panels that flash one at a time, which the user must then recreate from memory. The number of panels in each trial varied from four to six depending on whether it was an easy or hard block. Hitting an incorrect panel or hitting one out of order was considered an error. If at any point the subject could not remember the remainder of the sequence, they were permitted to hit the “Repeat” button as many times as needed, which would then play back any remaining unpressed panels. Like before, subjects completed nine repetitions of each trial per block, and four blocks per difficulty for a total of eight. Blocks were again presented in a pseudo-random order.

Fig. 4.
figure 4

Screenshots of instructional phase and in-progress step of Pattern Recreation task

Full size image

2.3 Data Acquisition

We used a continuous wave fNIRS system (fNIR Devices LLC, fnirdevices.com) that was described previously in [11]. COBI Studio [22] software was used to acquire data from the headband and visualize and record light intensity at two wavelengths, 730 and 850 nm, at a rate of 2 Hz. The headband itself (Fig. 5) is a soft, flexible, and lightweight foam pad containing four LED light sources and ten near infrared light-sensitive detectors placed in a grid with a fixed 2.5 cm source detector distance, which combined to form a total of 16 distinct optodes (measurement areas). With the inclusion of ambient channels, 730 nm, and 850 nm, a total of 3 channels per optode, 48 channels data overall were collected. Time synchronization markers were used to align the signal with the start and end times of each block throughout the experiment session.

Fig. 5.
figure 5

fNIRS sensor pad and diagram of optode layout. Sensors (S), Detectors (D), and Optode numbers are as labelled.

Full size image

2.4 Data Analysis

The outcome measures for the behavioral data were the accuracy (percentage correct) and latency (time to completion) for each trial within all blocks. For the neuroimaging data, raw light intensity measurements at 730 nm, 850 nm, and ambient light recorded with COBI Studio were analyzed in fnirSoft software [25]. First, raw light intensity data were low pass filtered with an order 20 Hamming finite impulse response filter with a cutoff frequency of 0.1 Hz to attenuate high frequency noise, respiration, and cardiac cycle effects. Data were also run through a sliding-window motion artifact rejection (SMAR) algorithm in order to eliminate motion artifacts and saturated channels [26]. Relevant blocks for each task were isolated and extracted using the manual markers. As the outcome measure, oxygenation that is oxygenated-hemoglobin (HbO) minus deoxygenated-hemoglobin (HbR) values for each of the sixteen optodes were calculated using the Modified Beer-Lambert Law with respect to a ten second local baseline at the beginning of each block.

For statistical analysis, a repeated measures linear mixed model analysis with Bonferroni correction was performed for both behavioral measures and average oxygenation changes for each block across difficulty conditions (alpha = 0.05).

3 Results

3.1 Behavioral Measures

The behavioral results of symbol matching task are shown in Fig. 6. As expected, both accuracy (F1,451.2 = 9.87, p < 0.002) and latency (F1,438.5 = 920.53, p < 0.001) were significantly different between easy and hard task conditions. Similar results were found for the pattern matching task which are shown in Fig. 7. Again, as expected, both accuracy (F1,524.1 = 132.24, p < 0.001) and latency (F1,557.3 = 422.983, p < 0.001) were significantly different across easy and hard conditions.

Fig. 6.
figure 6

Symbol Matching task behavioral results of accuracy (left) and latency (right) (**p < 0.002, ***p < 0.001). Error bars are the standard error of the mean (SEM).

Full size image
Fig. 7.
figure 7

Pattern Recreation task behavioral results of accuracy (left) and latency (right) (***p < 0.001). Error bars are SEM.

Full size image

3.2 fNIRS Measures

Average oxygenation changes showed the high contrast between task conditions. As expected, the higher level of activity observed for difficult conditions indicated a higher level of effort for those conditions. Significant differences were found for all but one optode between Symbol Matching tasks (Fig. 8, Table 1), but only one optode was found to provide a significant difference in cortex oxygenation while subjects performed Pattern Matching (Fig. 9, Table 1).

Fig. 8.
figure 8

Oxygenation for all sixteen optodes during Symbol Matching task (*p < 0.05, **p < 0.01, ***p < 0.001). Error bars are SEM.

Full size image
Table 1 Mixed model statistical analysis results for Symbol and Pattern tasks
Full size table
Fig. 9.
figure 9

Oxygenation for all sixteen optodes during Pattern Recreation task (*p < 0.05). Error bars are SEM.

Full size image

4 Discussion

In this study, we aimed to perform an initial proof of concept neuroergonomic evaluation of a clinical teletherapy tool, Constant Therapy, using a wearable and safe fNIRS neuroimaging system. Anterior prefrontal cortex activity during performance of the two cognitive tasks (attention and working memory) yielded high contrast differences in oxygenation changes across the task conditions (easy vs. hard).

We found that healthy subjects performing the two selected tasks of a tablet-based therapy application performed statistically worse (lower accuracy and higher latency) on the hard difficulty as compared to the easy one. Moreover, using functional near-infrared spectroscopy of the prefrontal cortex we found measurable differences in the cortical oxygenation of the subjects performing these tasks, demonstrating that consistent neural correlates exist, and can be potentially applied to a wider population using this system. This agrees with previous studies done using fNIRS on cognitive workload while performing challenging tasks, and demonstrates that higher mental effort is correlated with higher brain activation [10, 11, 24, 27–29].

This pilot study was limited by the number of subjects, as well as the time constraints we allowed for each experimental session. Given longer sessions or multiple recordings taken over several days, we would likely see more significant contrast in both task conditions and the longitudinal plasticity differences. This was exacerbated by our limitation of only using two distinct tasks. We also had to limit the range of difficulties for each task, as it was necessary to keep the test engaging but not too challenging in order not to overload the subjects during repetitive performance [30]. In future studies, more of the wide variety of tasks available could be tested, which will provide a richer information base for use of fNIRS measurements in conjunction with Constant Therapy in the field.

This pilot study provided initial proof that fNIRS-based neuroimaging techniques could be used to assess brain function changes during performance of the tasks and also potentially assess the efficacy of cognitive therapy applications during actual clinical use. Although we controlled as many aspects of the procedure as possible for the pilot study, the program itself was unaltered from the commercially available version, and therefore demonstrates the ecological validity of our results. In addition, this was the first neuroimaging study applied specifically with the Constant Therapy rehabilitation application. The results encourage future studies that can focus on cross sectional and longitudinal designs, including clinical populations, to further assess the utility of fNIRS to add value to a clinician’s ability to track and triage a patient’s recovery using quantitative brain measures. Moreover, if brain measures are integrated into the task design’s automated difficulty adjustment algorithms alongside the current smart analytics, they can improve the efficiency [31] and recovery speed as well as patients’ compliance [32].

In summary, this study demonstrates that fNIRS is a good candidate to enhance design (usability) and/or clinical field use of cognitive rehabilitation systems.