There has been an explosion of research in the field of autism, as evidenced by a 24-fold increase in the number of papers published on the topic over the past three decades (Chakrabarti 2017). Despite this proliferation, many experts have noted that those severely affected by autism spectrum disorder (ASD) appear to have been understudied. A recent query of the National Database for Autism Research (NDAR) revealed that just 11% of the research participants with ASD have an IQ less than 85 and that even fewer participants were categorized as minimally verbal (Jack and Pelphrey 2017). As a result, the picture of ASD that emerges from the research literature seems to be at risk of bias toward the higher functioning end of a spectrum of impairments.
This important observation—that certain subsets of the ASD population have potentially been over- or under-represented in research—has not been systematically evaluated across a broad historical range of peer-reviewed publications. While this issue may have implications for all areas of autism research, the most proximal effects may be seen in autism treatment studies, as these most directly inform current clinical practice. In this study, we set out to evaluate the degree to which the more severely affected population has been included in treatment studies, assess how severity is measured and represented, and determine what direction changes have taken, if any, over the past two decades. We focused specifically on children, for whom early treatment experiences are highly influential and therefore are heavily affected by treatment choices.
Before delving into the treatment literature, it is important to first address the fact that severity in ASD has been variably and inconsistently defined over the decades. As a result, evaluating the nature and extent of the inclusion of the severely affected end of the spectrum requires sensitivity to how severity has been defined and operationalized in the field.
Historically, autism studies have diagnosed individuals with ASD by relying on a set of core symptoms, which include restricted or repetitive behaviors and impaired social communication. However, a wide variety of incommensurate measures have been employed across treatment studies, and many do not provide specific severity ratings, use different criteria, or are not cross referenceable. While some of these tools, such as the Autism Diagnostic Observation Schedule (ADOS) (Lord et al. 2008), use a semi-structured, standardized means of diagnosis, they were not necessarily designed to provide a fine-grained look at the nuances of ASD severity or thoroughly capture the aspects of severity that may be most clinically relevant: communication ability, adaptive functioning, and intelligence. While core symptoms are useful criteria for categorical diagnoses of ASD, they are less informative for some clinical, therapeutic, and educational interventions, where the degree of language, functional, and cognitive impairment may be more relevant than the presence or absence of repetitive, restricted behaviors or social communication deficits.
In the absence of a widely-accepted, uniformly-applied definition and measurement approach for ASD severity, investigators have had several options for determining parameters of severity. One approach is to search for other potentially novel domains to measure to enhance or replace a core symptom-based definition of ASD severity. For instance, Zablotsky et al. (2015) showed that higher scores on a composite measure of family impact were more predictive of parents’ ratings of their child’s ASD severity, while descriptions of ASD symptoms in and of themselves were less predictive of parents’ severity ratings. One implication is that a significant component of severity is functional and contextual in nature. This finding should inform the field’s approach to ASD severity.
Another approach to rigorously delineating ASD severity is to employ a definition informed by prior work in the field. Enough treatment research has been published in the last several decades to serve as an adequate source of raw material for an inquiry into patterns. Over time, investigators’ efforts may be converging in a telling albeit unintended fashion, a convergence that ideally should inform what the field considers a standard of ASD severity to be. In this study, we took an approach that allowed us to see patterns in the literature and begin to derive an operational definition of ASD severity that is clinically useful.
Additionally (and perhaps as a result of the preceding), researchers and clinicians have determined the severity of patients by assessing phenotypic characteristics that are beyond the realm of core ASD symptoms. Because the clinical presentation of ASD is quite heterogeneous, a number of challenges arise when attempting to further characterize an individual along the autism spectrum. Previous research has touted the importance of identifying specific subtypes within the autism spectrum as a means of developing a framework to better characterize, research, and treat those with an ASD diagnosis (Grzadzinski et al. 2013), but if those more severely affected by ASD are not included in research, we risk missing entire subtypes. For the purposes of the present study, we chose to focus on three domains of functioning that manifest in unique ways in the ASD population and play a significant role in driving clinical treatment and educational intervention selection: cognitive functioning, communication ability, and adaptive functioning.
The most recent surveillance data has indicated that around 31% of children with ASD in the United States have an IQ in the intellectual disability (ID) range (IQ ≤ 70) (Centers for Disease Control and Prevention 2018). Because of variability and inconsistency in the cognitive profile of this population, it can be common to either over- or underestimate the ability levels of a child with ASD. Verification of an ID diagnosis can allow for more targeted treatment and more specialized care in a clinical or community setting (Siegel and Gabriels 2014). Challenges to establishing a child’s cognitive abilities can be confounded by the presumption of interconnectedness of cognitive and communicative impairments. One study of minimally verbal children with ASD found that they actually exhibited a range of cognitive abilities, from profoundly impaired to average intelligence (Bal et al. 2016). These findings support the importance of primarily considering cognitive functioning and communication ability as distinct entities in the context of ASD severity.
It is estimated that approximately 30% of youth with ASD are minimally verbal (Tager-Flusberg and Kasari 2013). Core autism and related symptoms are typically more severe in minimally verbal individuals, and they often exhibit a range of challenging behaviors (Tager-Flusberg et al. 2017). There is a clear lack of consensus, however, as to how to best define ‘minimally verbal’ and measure communication ability for both clinical and research purposes. One significant barrier to this is a lack of valid and appropriate measures for directly assessing communication ability across the entirety of the autism spectrum (Tager-Flusberg and Kasari 2013). These challenges in assessment can ultimately lead to challenges in selecting appropriate treatments for this population. Researchers in the field maintain a long-term goal of developing a more robust understanding of communication ability across the autism spectrum in order to provide more comprehensive assessment and treatment of the heterogeneous autism phenotypes (Kasari et al. 2013). As such, for the domain of communication in particular, it is of significant value to better our understanding of not only who is represented in autism treatment studies, but how communication is being measured and reported.
Within the context of ASD, measurement of adaptive functioning (encompassing communication, daily living skills, and socialization, among other attributes) indicates the degree of support an individual requires in their everyday life. Children with ASD are typically more delayed in their adaptive functioning than their IQ measurement would suggest (Klin et al. 2007), and these adaptive functioning delays can have significant effects on the individual’s life course. Adaptive functioning has also been found to be closely related to core ASD symptom severity. For instance, an assessment of minimally verbal school-aged children with ASD found that more severe core autism symptoms (as measured by the ADOS) were associated with more impaired adaptive functioning (Frost et al. 2017). Additionally, an assessment of children with ASD in an inpatient setting found that greater impairments in adaptive functioning and coping skills (as measured by the Vineland Adaptive Behavior Scales) were associated with an increased severity of behavioral problems (Williams et al. 2017). Further, possessing higher levels of adaptive skills has been shown to decrease the likelihood of psychiatric hospitalization for children with ASD (Righi et al. 2017). These associations between adaptive skills and other facets of the autism phenotype provide support for adaptive functioning being a meaningful domain through which to examine autistic severity.
Examining a spectrum disorder such as ASD demands consideration of dimensional characteristics that individually and in aggregate make up an individual. However, nowhere is the potential harm of over- or understating these characteristics greater than in treatment studies, where inadequate measurement and reporting of dimensions of autistic severity prevents clinicians and researchers from being able to define treatment group characteristics and search for clinically meaningful treatment moderators. For these reasons, in this study we focused exclusively on treatment studies.
Previous research has begun to explore the complex landscape of autistic severity through participant representation in the scientific literature. Two early studies examined smaller subsets of the literature to broadly describe subject characteristics. Charman (1994) surveyed a decade of publications from the Journal of Autism and Developmental Disorders (JADD) and categorized each by study topic and reporting of IQ data. Eighty-one percent of the papers reported some indication of IQ, of which 74% focused on subjects with some degree of intellectual impairment (IQ < 70). The authors noted a trend over time toward providing more complete descriptive information on study subjects. While this analysis only explored one aspect of autistic severity (cognitive impairment), it provides some evidence of significant inclusion of severely-affected participants in earlier research in the field in one journal. A later study (Dunlap et al. 1999) focused specifically on treatment research and broadened the analysis of participant characteristics to publications from ten journals over an 18-year time period. While the authors found few distinct patterns, there was a notable upward trend in the proportion of research conducted in more “typical” contexts (i.e., schools, as opposed to clinical settings), suggesting that intervention research may have begun to encompass a more heterogeneous group of participants across the autism spectrum with a range in symptom severity.
Exploring more current research on the topic, Bebko et al. (2008) reviewed abstracts presented at the International Meeting for Autism Research (IMFAR) from 2004 to 2006 to examine trends in research topics, participants, and study design. Most pertinent to the present study were their findings on the reported functioning levels of study participants. Abstracts were categorized as including either “high-functioning,” “moderate/low-functioning,” or “mixed/unknown” participants based on criteria for cognitive and overall functioning. Results revealed a decrease over time in studies of moderate/low-functioning samples, despite an increase in the overall number of abstracts accepted. While this study provides some valuable insight into severity representation in autism research, the date range was limited. Furthermore, the focus on conference abstracts limited access to comprehensive descriptions of the study population that can be found in complete, published articles. However, the authors raised a number of notable concerns for future research, recognizing a trend toward less specificity in the characterization of research participants, and positing that our understanding of the autism spectrum as a whole has potential to become biased toward the higher functioning end.
Crosland et al. (2013) assessed autism intervention studies from three journals from 1995 to 2009 to examine descriptive features of both participants and setting characteristics reported as well as examine potential trends over time. General level of cognitive and adaptive functioning was categorized for each assessed study as either “typical range,” “mild to moderate disability,” or “severe to profound disability.” Level of communication ability was categorized as either “communicative,” “rudimentary,” or “nonverbal/non-communicative.” While the findings showed few trends overall, it was notable that the proportion of studies that included typically-functioning participants with autism increased over time, as measured in both cognitive and communicative functioning domains. Conversely, the percentage of studies containing participants with severe/profound cognitive functioning deficits and/or nonverbal status decreased over time. Though this analysis was limited to only three journals, it demonstrated that fewer people from the severely affected end of the spectrum were included in intervention literature over time.
Most recently, Jack and Pelphrey (2017) examined the severely affected phenotype within the context of neuroimaging research and found that those with autism and co-occurring intellectual disability, minimally verbal status, or developmental regression have been generally understudied. The authors cited methodological difficulties, as well as inconsistent definitions of severity, as contributing to this gap in the imaging research. While the scope of the review did not specifically include treatment studies, the proposed under-representation of the more severe phenotype highlighted how challenges in defining and assessing severity in the autism spectrum extend across a breadth of research areas.
To assess the representation of the severely affected ASD phenotype in the treatment literature, and the ways severity is conceptualized, defined, and reported, we examined the characteristics of all peer-reviewed, published treatment studies of children with ASD, excluding single case designs, through a systematic, multi-step literature search and subsequent analysis covering the period of 1991–2013. Our objectives were twofold:
To assess representation of the severely affected population over the past two decades in the autism literature. Due to the growing heterogeneity of the ASD population, we hypothesized that evidence of the inclusion of the severely affected in autism research would decrease over time. Rapid growth in the diagnosis of higher functioning individuals on the autism spectrum has caused the proportion of those more severely affected in the overall ASD population to decrease (Matson and Kozlowski 2011). This increased heterogeneity has thus led us to posit a similar trend within the treatment literature.
To examine how severity is represented in the literature as well as the criteria used to determine severity. As discussed in more detail in the “Methods” section, we have chosen to operationalize severity using three domains: communication ability, cognitive functioning, and adaptive functioning. Based on an awareness that even seminal treatment studies in the field have not always reported information on their samples’ communication ability, as well as a general lack of consensus in the field on the best measures of communication (Kasari et al. 2013), we hypothesized that communication ability would be the least frequently reported domain (relative to the cognitive and adaptive functioning domains) and have the greatest variability in the measures used to capture the domain.