Introduction

According to the 4th International Conference on Concussion in Sport, concussion is defined as a brain injury involving complex pathophysiological processes affecting the brain, induced by biomechanical forces (McCrory et al. 2013). Commonly referred to as mild traumatic brain injuries (MTBI), these injuries are frequently caused by sports and recreation activities (Langlois et al. 2006). Recent estimates suggest up to 3.8 million concussions are sustained in the USA each year, and children and adolescents make up the majority of this category due to the large numbers of participants in youth sports (Halstead and Walter 2010).

Sustaining a concussion can result in immediate effects including loss of consciousness (LOC), post-traumatic amnesia (PTA), and physical neurological abnormalities (McCrea et al. 2002). McCrory et al. (2013) found that there are a range of symptoms associated with concussion, including physical signs such as headaches, cognitive deficits such as “feeling in a fog,” emotional lability, and slowed reaction time, along with loss of consciousness, amnesia, sleep disturbances, and behavioral changes such as irritability.

For the majority of individuals experiencing concussion, recovery is relatively swift, with young adults reporting full recovery in 6–10 days and adolescents taking only slightly longer (Williams et al. 2015). For some individuals, however, the effects of concussion can persist for much longer. Recent research finds that cognitive deficits may persist long-term as a result of a concussion, even without LOC or PTA (Bigler et al. 2013; Pertab et al. 2009; McCrea et al. 2002). Such symptoms are sometimes labeled “late effects.” Studies investigating long-term effects of concussion reveal lower scores on attention and memory measures in individuals who had a prior history of concussion compared to control samples (Belanger and Vanderploeg 2005; Moser and Schatz 2002). One study found that sufferers of childhood concussion reported long-term sequelae up to 23 years post-injury, implying that in some cases neurocognitive effects resulting from concussion may persist for a lifetime (Klonoff et al. 1993). Not only can these neurocognitive sequelae persist long-term, but they are also cumulative, meaning that individuals who sustain multiple concussions are more likely to have long-term effects and/or neurocognitive test results that are similar to individuals who have recently sustained a concussion (Moser et al. 2005). Adding to these fears is the finding that multiple concussions may contribute to tauopathy and chronic traumatic encephalopathy (McKee et al. 2009). Because children’s brains are continually maturing and developing, youths and teenagers may be at greater risk for long-term neurocognitive sequelae than adults (McCrory et al. 2004).

A landmark study conducted by Barth et al. (1989) first brought to light the importance of neuropsychological testing in individuals who sustained sports-related concussions. This 4-year prospective study demonstrated that neuropsychological testing could accurately detect neurocognitive changes that developed in athletes after sustaining mild traumatic brain injuries. Because the ability to process information may be dramatically reduced after concussions, and in particular repeated concussions (Gronwall and Wrightson 1974, 1975), experts recommend that medical professionals put in place “return to play” (RTP) guidelines that maximize the brain’s healing time and reduce the possibility of late effects as much as possible (Purcell 2009). Equally important to RTP guidelines are RTT or “return to think” guidelines which regulate when individuals are able to return to school and other cognitively taxing activities (McCrory et al. 2013).

Among the most commonly used assessment measures for neuropsychological effects of concussion is the Sport Concussion Assessment Tool-2 (SCAT-2; Eckner and Kutcher 2010). Although originally developed for sideline administration at sporting events, the tool has been demonstrated as useful in emergency rooms and other medical settings to track concussive symptoms (Luoto et al. 2014). The SCAT-2 is geared toward youth aged 10 and older and yields several scales of cognitive functioning. Among the most meaningful of these scales are a checklist of symptom severity and the Standardized Assessment of Concussion (SAC), a measure adapted from McCrea et al. (2000) to assess orientation, memory, and concentration.

The goal of this study is to examine to what extent the SCAT-2 predicts neuropsychological functioning in children after sustaining a concussion. Little previous work addresses the topic. Field et al. (2003) found that younger children may experience greater concussion-related impairments than older children, and McLeod et al. (2012) found that younger nonconcussed athletes had, on average, lower SCAT-2 scores (including SAC values) than older nonconcussed athletes, yet the differences between ages among concussed children were not assessed. Research with high school and college male athletes found the original Standardized Assessment of Concussion (from which the SCAT-2 SAC scale is derived) is a valid instrument for detecting immediate effects of concussion and mild traumatic brain injury among male older adolescents/emerging adults (Barr and McCrea 2001). A study conducted by Naunheim et al. (2008) correlated the SAC with performance on the Conners’ Continuous Performance Test II (CPT-II) of attention in adults and found that the SAC total score was significantly related to omissions and reaction time variables of the CPT-II at baseline and 3-h post-concussion intervals, but no published research has examined these relations among children.

In this study, the validity of the SCAT-2 in children will be assessed by examining relations between SAC and symptom severity measures from the SCAT-2 and multiple neuropsychological measures of attention and verbal memory among a sample of 57 youth ages 10–19 tested within 100 days of a sports-related concussion. We hypothesize that the SAC will predict performance on measures of attention and working memory. We also anticipate an inverse relation between symptom severity and both attention and working memory.

Methods

Participants

The study extracted available data of 57 eligible participants from a much larger data set of 832 individuals who were seen at Children’s of Alabama and the University of Alabama at Birmingham Concussion Clinic for concussion-related complaints. Criteria for inclusion included concussed athletes treated between May 2012 and January 2013, age between 10 and 19 years, completion of SCAT-2 assessments within 100 days of injury, and neuropsychological evaluations completed following SCAT-2 assessment. Concussion diagnoses were verified through medical record reviews. Seven hundred seventy-five individuals were excluded because they were not assessed during the specified time frame (n = 532), did not undergo neuropsychological testing (n = 221), and/or did not meet age inclusion criteria (n = 22). Eleven included participants had premorbid attention deficit hyperactivity disorder (ADHD) diagnoses but were retained in the sample due to a lack of significant differences between the groups with and without ADHD. SCAT-2 assessments were conducted at patients’ initial concussion appointments. The University of Alabama at Birmingham Institutional Review Board approved the study protocol.

Measures

Sport Concussion Assessment Tool-Second Edition (SCAT-2; Eckner and Kutcher 2010)

The Sport Concussion Assessment Tool was developed in 2004 and revised to the SCAT-2 in 2008 (Eckner and Kutcher 2010; McCrory et al. 2005). The tool yields several scales aimed at determining symptomatology and cognitive functioning, including a symptom checklist, questions assessing loss of consciousness and unsteadiness, a Glasgow coma scale, a Maddocks score, a standardized measure of cognitive status, a balance examination, and a coordination task. Of these scales, two scores were included in the present study: symptom severity and Standardized Assessment of Concussion (SAC). These scores were chosen for inclusion to reflect the relationship between neuropsychological functioning with physical functioning and cognitive functioning, respectively. In this study, SCAT-2 scores were obtained by either a licensed physician or neuropsychologist.

Symptom severity is a subcomponent of the symptom inventory. It is calculated by summing the total patient-endorsed severity values (ranging from 0 to 6) for each of a set of 22 symptoms, creating a possible range of 0 to 132, with higher scores indicating more severe symptoms. The SAC is a mini mental status exam which incorporates components of orientation, immediate memory, concentration, and delayed recall. The scale includes five questions on orientation to time, a five-word recall list (both immediate and delayed recall are assessed), and asking the participant to state the months in reverse order. Scores range from 0 to 30, with higher scores indicating better cognitive functioning. The SCAT-2 has shown excellent face validity and fair test-retest reliability, though reliability statistics have not been extensively studied (Alla et al. 2009; Chan et al. 2013). Although the test itself has not undergone extensive reliability testing, a review of tools that the scales were adapted from demonstrated moderate to good reliability for each of the SAC, BES, and Symptom Score scales (Guskiewicz et al. 2013).

Conners’ Continuous Performance Test II Version 5. (CPT-II; Conners 1995)

This assessment is a computerized measure that assesses vigilance and attention and is often used as an aid for the diagnosis of attention deficit hyperactivity disorder (ADHD; Conners 1995). The task, sometimes termed a “go/no-go task,” takes approximately 15 min to complete and requires individuals to press a bar when a target is shown on the screen but ignore the bar when other stimuli are presented. Scores include T scores for omissions (a measure of inattention), commissions (a measure of impulsivity), reaction time, and reaction time standard error (consistency of response speed). Higher T scores indicate poorer performance, and for the reaction time scale, both abnormally high and abnormally low scores indicate inattention. This tool has demonstrated reliability and validity and is consistently used to differentiate between ADHD and control samples (Epstein et al. 2003).

California Verbal Learning Test-Children’s Version and California Verbal Learning Test-Second Edition (CVLT; Delis et al. 2000; Delis et al. 1994)

Both the adult and child versions of the CVLT assess verbal learning through memory tasks which involve list recall. Patients included in the study were given the CVLT-C if they were between the ages of 10 and 16 and the CVLT-II if they were between the ages of 17 and 19. The test requires participants to recite a word list over five trials, following a 2-min delay, following an “intrusion” second word list, and following a 20-min delay. Verbal memory is evaluated through free recall, recall following category cues, and forced-choice recognition. The score included for the purposes of this study is the verbal learning T score, with lower scores indicating poorer performance. The CVLT-II and CVLT-C both have strong reliability (α > 0.70 in multiple tests; Delis et al. 2000; Horton 1996).

Data Analyses

Distributions were examined prior to analyses and all statistical assumptions of normality were met. A small number of data points (4 %) were missing and handled using pairwise deletion. To address the hypothesis that better scores on SCAT-2 variables would be related to higher scores on measures of attention and memory, we examined bivariate correlations between SCAT-2 scores and the selected indices from the CPT-II and CVLT. To determine if any SCAT-2 variable predicted performance on the attention and verbal memory measures, multiple regression equations were performed to examine whether initial SAC and symptom severity scores predicted attention and memory performance over and above the influence of demographic variables (age, gender, and days between injury and neuropsychological testing). All assumptions of regression analyses were met, including normality, linearity, and homoscedasticity.

Results

Fifty-seven subjects aged 10–19 comprised the sample. Descriptive characteristics of the sample appear in Table 1. Descriptive statistics of task performance appear in Table 2.

Table 1 Descriptive sample characteristics (N = 57)
Full size table
Table 2 Descriptive data on neuropsychological test performance
Full size table

Age was not significantly related to performance on measures of the SCAT-2. Age was, however, significantly correlated with measures of attention, such that older individuals had more commissions (r = 0.28, p < 0.05) than younger participants. Gender was correlated with symptom severity, such that females had more severe symptoms at their initial assessments than males (r = 0.47, p < 0.01). Gender was not significantly correlated with any measures of attention or memory, and race was not significantly correlated with SCAT-2 scores, attention performance, or memory performance.

Concussion Measures and Attention

As shown in Table 3, initial correlational analyses revealed that the SCAT-2 SAC score was related to some aspects of CPT performance, such that worse SAC scores corresponded with more inattention (CPT omissions; r = −0.32, p < 0.05) and more impulsivity (CPT commissions; r = −0.36, p < 0.01). Symptom severity scores were not correlated with any CPT measures.

Table 3 Correlations between SCAT-2 scores and demographics, attention, working memory, and time to cognitive clearance
Full size table

To determine if SCAT-2 variables predicted CPT performance over and above demographic variables, two-step multiple regression analyses were performed on those relations shown significant in bivariate analyses. Age, gender, and time between injury and neuropsychological testing were entered in Step 1, and SCAT-2 measures were entered in Step 2 (Table 4). Results indicated that over and above the variance accounted for by demographics, poorer SAC performance still significantly predicted worse performance on scores indicating inattention (∆R 2  = 0.16) and impulsivity (∆R 2  = 0.22).

Table 4 Hierarchical multiple regression analyses predicting measures of inattention from SCAT-2 Scales (N = 53)
Full size table

Concussion Measures and Verbal Memory

Initial correlational analyses revealed that the SCAT-2 SAC score was significantly correlated with verbal memory, such that worse SAC scores corresponded with poorer CVLT total verbal learning performance (Table 3; r = 0.58, p < 0.001). Symptom severity scores were not significantly correlated with verbal memory. A two-step multiple regression analysis to predict CVLT performance was performed such that age, gender, and time between injury and neuropsychological testing were entered in Step 1, and SCAT-2 measures were entered in Step 2 (Table 5). Results indicated that over and above the variance accounted for by demographics, poorer SAC performance predicted worse performance on verbal learning (∆R 2  = 0.34).

Table 5 Hierarchical multiple regression analyses predicting verbal learning from SCAT-2 Scales (N = 55)
Full size table

Discussion

This study examined whether scores on the SCAT-2, a brief standardized symptom reporting and cognitive screening measure, are associated with attention and verbal learning aspects of neuropsychological functioning in concussed youths. Results offer support for the validity of the SCAT-2, as the SAC component of the SCAT was correlated with verbal memory along with several facets of attention, including inhibition and impulsivity. The finding that SAC scores correlated with certain aspects of neurocognitive function suggests that sideline assessments should be utilized as standard procedure for concussion and RTP/RTT protocols. Symptom severity scores were not associated with attention and working memory scores. These findings are consistent with previous research in adults that showed no relationship between a patient’s symptoms and performance on neuropsychological testing (Echemendia et al. 2001; Naunheim et al. 2008).

A possible concern with the validity of our findings is the influence of ADHD diagnoses among the sample. Medical and neuropsychological records review revealed that 11 participants (19 %) in the sample had a documented previous diagnosis of ADHD. To determine if the presence of ADHD affected performance on the attention and working memory measures, independent samples t tests were computed between groups with premorbid ADHD and those without documented ADHD diagnoses. Results indicated that no measures of the SCAT-2, CPT, or CVLT were significantly different between participants with ADHD and those without, perhaps due to psychotropic medication use among the sample. Future research would benefit from data on current medication usage and the difference between performance of children with ADHD on medication, children with ADHD not on medication, and children without ADHD. Alternatively, it may be that the result is due to the heterogeneity of concussions and how greatly mild traumatic brain injuries can alter performance on cognitive tasks requiring sustained attention. Another possible concern is the presence of other mental illnesses among the sample. Data were not available concerning premorbid learning disabilities, present emotional state, history of emotional and behavioral disturbances, or number of prior concussions for the sample. Future research would benefit from the addition of these variables to assess their influence on cognitive performance.

An additional limitation of the study findings stems from the use of the SCAT-2 instead of the newly revised SCAT3 (Sport Concussion Assessment Tool-Third Edition; Guskiewicz et al. 2013), which was published after data collection for this study initiated. Due to the dearth of studies demonstrating reliability statistics for the SCAT-2, Guskiewicz et al. (2013) conducted an extensive review of the tools used to build the SCAT-2. The SAC, BES, and Symptom Score scales were shown to have moderate to good reliability across 22 studies, and the SCAT-2 Total Score was shown to have little practicality and was thus removed. The SCAT3 and the associated Child SCAT3, developed for children between the ages of 5 and 12, were introduced at the 4th International Conference on Concussion in Sport and current guidelines recommend using the updated versions for concussion management (McCrory et al. 2013). Despite this recommendation, the SCAT-2 is still frequently used by providers (Zimmer et al. 2014). Further, the scales used in this study (SAC and symptom severity) were relatively unchanged in the most recent update. The symptom severity scale was completely unchanged, and the SAC went from 5 concentration items and 15 immediate memory items to 15 concentration items and 5 immediate memory items. Due to the similarity of the scales in the SCAT-2 and SCAT3, the findings from this study remain relevant for both science and practice.

The idea that a patient may experience neuropsychological deficits after physical symptoms have remitted, supported by our results, speaks to the utility of post-concussive neurocognitive testing, particularly for individuals who experience prolonged cognitive symptom recovery. While administrators and coaches may be inclined to allow individual athletes to RTP or RTT because they are not physically injured, they may still be experiencing significant cognitive deficits. For concussed children who are still cognitively symptomatic, a focused neuropsychological evaluation is needed to evaluate all areas of functioning. Among other things, such an evaluation may reveal that although one area of cognition may be impaired, other areas of functioning are intact. A multidisciplinary evaluation is needed to effectively evaluate the effects of concussion. In addition to these post-injury evaluations, preseason baseline assessments are also recommended for youth athletes in order to accurately assess cognitive changes post-concussion (McLeod et al. 2012).

Developing a protocol that includes immediate administration of a concussion assessment tool such as the SCAT-2 after a youth sustains a concussion and then using that tool to track recovery over time could greatly influence the patient’s cognitive health. By understanding concussion assessment scores such as the SAC and the implications of their neuropsychological relevance, clinicians and coaches alike may be more cautious about RTP/RTT, make appropriate referrals for dedicated neuropsychological evaluation for athletes with an abnormal SAC, and potentially mitigate long-term cognitive effects of concussion. Further, safe and appropriate RTP/RTT after concussion remains a complex issue, and focused neuropsychological evaluation as part of a multidisciplinary team approach should be utilized to determine accurate neurocognitive status and optimal recovery protocols.