Brief Report: Preliminary Evaluation of the Theory of Mind Inventory and its Relationship to Measures of Social Skills
- First Online:
- Cite this article as:
- Lerner, M.D., Hutchins, T.L. & Prelock, P.A. J Autism Dev Disord (2011) 41: 512. doi:10.1007/s10803-010-1066-z
- 587 Views
This study presents updated information on a parent-report measure of Theory of Mind (ToM), formerly called the Perception of Children’s Theory of Mind Measure (Hutchins et al., J Autism Dev Disord 38:143–155, 2008), renamed the Theory of Mind Inventory (ToMI), for use with parents of children with autism spectrum disorder (ASD). This study examines the responses of parents of adolescents with ASDs and explores the relationship of parental responses on the ToMI to measures of autistic symptoms and social skills. Descriptive statistics were compared to previous samples; correlations and regressions were conducted to examine the ToMI’s criterion-related validity with social skills and ASD symptoms. Results support use of the ToMI with adolescent samples and its relationship to social impairments in ASDs.
KeywordsAutism spectrum disorderTheory of MindSocial skillsScale evaluationPsychometricsValidity
Theory of Mind (ToM), or an individual’s ability to take another’s perspective, is considered a central domain of impairment among those with autism spectrum disorder (ASD; Baron-Cohen 2001). In 2008, Hutchins et al. reported on the development and psychometric evaluation of a novel measure of child perspective-taking: the Perceptions of Children’s Theory of Mind Measure—Experimental version (PCToMM-E). The PCToMM-E is a parent-informant measure of ToM, consisting of 33 statements designed to tap a wide range of social-cognitive understanding and skill sets that are believed to be subsumed under (or strongly related to) the construct of ToM. All statements (e.g., “My child understands that when people frown, they feel differently than when they smile”) are accompanied by a 20-unit continuum anchored by “strongly disagree” to “strongly agree”. Parents are asked to indicate the degree to which they agree with each statement by making a vertical hash mark at the appropriate point along the continuum. Each item is scored by ruler with higher values reflecting greater degrees of the construct.
Theory of Mind (ToM) is a broad and multifaceted construct (Astington and Baird 2005; Hutchins et al. 2008), including understanding mental-physical distinctions, physical versus mental perspectives, and making inferences about thoughts and emotions (Baron-Cohen 2001; Repacholi and Slaughter 2003). Crucially, ToM is related to social functioning and autistic symptoms (Frith et al. 1994). As Hutchins et al. (2008) described, the PCToMM-E included (but was not limited to) items which were face valid indicators of false belief understanding, the notion that seeing leads to knowing, first and second order beliefs, the appearance-reality distinction, the mental-physical distinction, the understanding of speech acts, the understanding of a variety of mental state terms, and the causes and consequences of mental states. In short, the PCToMM-E was designed to be a content valid index of ToM.
To evaluate the psychometric properties of the PCToMM-E, Hutchins et al. (2008) performed several tests of reliability and validity using a sample of parents of typically-developing children and children with ASD. The measure performed well and established strong test–retest reliability for short and long lags as well as convergent validity with scores on a measure of receptive vocabulary and on a ToM task battery. It also performed well among two contrasting group comparisons of construct validity, distinguishing typically developing children from age-matched children with ASD, and distinguishing younger from older typically developing children.
Besides the encouraging evidence surrounding its reliability and validity, measurement features found in the PCToMM-E have been described as advantageous for several reasons (see Peterson et al. 2009; Wellman and Liu 2004). First, it is used to gain estimates of ToM in typical (ages 2–12) and ASD samples (ages 2–12) across verbal abilities. It does not suffer from ceiling effects (see, e.g., Steele et al. 2003) even when mentalizing is relatively good. Because it is an informant measure, it is not complicated by child performance factors (e.g., memory, linguistic, attentional, motivational factors) and can be used as a repeated measure because it does not suffer from test-practice effects. Further, using interval scores generates continuous variable amenable to more statistically powerful analyses.
A recent revision of the PCToMM-E (now referred to as the Theory of Mind Inventory or ToMI) was similar except that it consists of 48 statements believed to more adequately tap advanced components of ToM, such as understanding sarcasm, irony, and counter-factual reasoning (Steele et al. 2003). Other items were omitted or revised based on analysis of improvement in Cronbach’s alpha. In addition, authors clarified the instructions to reduce error in responding (e.g., respondents sometimes circled anchors or indicated a response with an ‘X’ as opposed to a vertical hash mark, so correct and incorrect examples were provided).
This brief report provides a preliminary psychometric evaluation of the ToMI when administered to a sample of parents of adolescents with ASD. In lab-based evaluations, ToM has been shown to remain impaired, and to develop slowly and inconsistently over time among individuals with ASDs (Hutchins et al. 2008; Peterson et al. 2005). Because ToM competencies are highly variable in individuals with ASD and less strongly related to chronological age than in typically developing samples (Hutchins and Prelock 2008), we sought to explore the patterns of parent reporting on the ToMI in adolescents with ASDs (see Dissanayake and Macintosh 2003; Tager-Flusberg 2003; Watson et al. 1999) relative to previous younger samples using the PCToMM-E. As the previous literature clearly demonstrates a link between ToM, social skills, and ASD symptoms (Frith et al. 1994), we sought to examine the concurrent criterion validity of the ToMI by exploring the relationship between ToMI and parent-report of these constructs on well-validated measures. Crucially, we expected the social impairment component of ASD to be the most important predictor of ToM deficits (Frith et al. 1994), and so we explored this possibility using the aforementioned measures.
Forty families of children with ASDs were successfully recruited from community-based programs and workshops in rural and suburban Virginia. All parents completed a questionnaire for demographic information; a subset of parents completed an additional question regarding whether their child was able to read at a fourth grade level, while the remainder provided their child’s Individualized Education Plan. Only families of children who had received a previous diagnosis from a licensed diagnostic professional (i.e., Developmental Pediatrician, Psychiatrist, Clinical Psychologist) were invited to participate in these workshops. To support the validity of the ASD diagnosis, strict recommended cutoffs on the Social Communication Questionnaire (SCQ; Rutter et al. 2005) and Social Responsiveness Scale (SRS; Constantino and Gruber 2005) were used to screen participants. We acknowledge that the Autism Diagnostic Interview—Revised (ADI-R; Lord et al. 1994) and Autism Diagnostic Observation System (ADOS; Lord et al. 1999) are considered the “gold standard” diagnostic tools for ASD populations in research. However, high correlations between the SRS & ADI-R (Bishop and Norbury 2002) and between the SCQ and ADOS (Corsello et al. 2007), as well as the recommendation to use the SCQ in combination with another instrument (Rutter et al. 2005), led us to conclude that an individual simultaneously exceeding recommended cutoffs on both of these instruments met sufficient diagnostic criteria.
Of those recruited, thirty youth [24 (80%) male, six female; Mage = 14.17, SDage = 3.11] met these criteria. Their parents (Mage = 46.58, SDage = 7.80; Mincome = $69,000, SDincome = $28,450) included 24 (80%) mothers, two (7%) fathers, one (3%) grandfather, and three (10%) parents who did not identify their relationship to the child. Of these, 27 (90%) endorsed the SCQ item that their child “is able to talk using short phrases or sentences” (verbal ability) and 19 (63%) indicated that their child could read at least at a fourth grade level (reading ability).
Social Communication Questionnaire (Rutter et al. 2005). The SCQ is a 40-item (sample item: “Has she/he ever had any interests that preoccupy him/her and might seem odd to other people?”) parent report measure of ASD symptoms designed as a screening tool in research and clinical settings. The SCQ generates subscale scores for each of the core domains of ASD diagnosis: communication, reciprocal social interaction, and restricted/repetitive behaviors. The SCQ scores range from 0 to 40, with higher scores corresponding to higher levels of ASD symptoms and a recommended clinical cut-off score of 15. The SCQ has been shown to be a reliable instrument for assessing the likelihood of having ASD, and has demonstrated sufficient sensitivity and specificity to be considered valid in identifying individuals with ASDs above the age of eight (Corsello et al. 2007).
Social Skills Rating System—Parent (SSRS-P; Gresham and Elliot 1990). The SSRS-P is a parent informant measure consisting of 52 items (sample item: “Joins group activities without being told to”) tapping four domains: Cooperation, Assertion, Responsibility, and Self-control. The SSRS-P yields standard scores (M = 100, SD = 15) with higher scores indicating more developed social skills. The SSRS-P is currently the most extensively used parent-report measure for assessing social skills in adolescents with ASD (White et al. 2007); in a normative sample, the SSRS-P demonstrated excellent internal consistency (Gresham and Elliot 1990).
Social Responsiveness Scale (SRS; Constantino and Gruber 2005). The SRS is a parent-informant measure consisting of 65 items (sample item: “takes things too literally and doesn’t get the real meaning of a conversation”) tapping five subscales: Social Awareness, Social Cognition, Social Communication, Social Motivation, and Autistic Mannerisms. The SRS yields T-scores (M = 50, SD = 10), with higher scores corresponding to a higher degrees of social impairment. Good internal consistency and validity is reported for ASD populations (Constantino et al. 2003).
Data Analytic Plan
The mean and range on the ToMI were compared to the PCToMM-E values reported by Hutchins et al. (2008) with the expectation that scores would be comparable. Likewise, differences in ToMI scores between those with high and low reported verbal and reading ability were compared. Next, correlations between variables were explored. Finally, as we expected that SRS would best account for the variance in ToMI after controlling for SCQ and SSRS-P, a hierarchical multiple regression model was specified, with SCQ on Step 1, SSRS-P on Step 2, and SRS on Step 3.
Social Communication Questionnaire (SCQ)
Social Skills Rating System—Parent (SSRS-P)
Social Responsiveness Scale (SRS)
One-way ANOVAs indicated a significant difference in ToMI scores between children with high and low verbal (Mhigh = 12.11, SDhigh = 3.00; Mlow = 3.49, SDlow = 2.54), F(1, 29) = 22.73 (p < 0.001) and reading (Mhigh = 12.67, SDhigh = 3.19; Mlow = 8.77, SDlow = 3.98), F(1, 29) = 8.71 (p < 0.006) abilities. ToMI scores demonstrated excellent internal consistency (α = 0.98).
Correlation matrix showing relationships between overall scores for each measure
1. Theory of Mind Inventory (ToMI)
2. Social Communication Questionnaire (SCQ)
3. Social Skills Rating System—Parent (SSRS-P)
4. Social Responsiveness Scale (SRS)
Correlation matrix between ToMI and the subscales of each measure
Theory of Mind Inventory (ToMI)
Hierarchical multiple regression model predicting ToMI scores
Model 1 B
Model 2 B
Social Communication Questionnaire (SCQ)
[− 0.67, −0.18]
Social Skills Rating System—Parent (SSRS-P)
Social Responsiveness Scale (SRS)
[− 0.24, −0.06]
We explored parent ratings on the ToMI using a sample of ASD adolescents. We found that ToMI scores were similar to those found for the experimental version of the measure (i.e., PCToMM-E) which was validated on a younger sample of children with ASD. ToMI scores also discriminated effectively between those with high and low verbal ability. We also explored the relationship of ToMI scores to parent-reported autistic symptoms, social skills, and autism-related social impairments. We found that ToMI scores were positively correlated with social skills, and negatively correlated with ASD symptoms and autism-related social impairments. We also found that these correlations were equivalently explained by each of the subscales in the social skills domain, and by each of the subscales in the autism-related social impairment domain, but were only accounted for by reciprocal social interaction (e.g., “does she/he have any particular friends or best friends?”) in the ASD symptom domain. Finally, we expected that autism-related social impairments would best account for the variance in ToMI scores after controlling for autistic symptoms and social skills. Our results supported this prediction.
These findings extend the use of the ToMI as a valuable parent-report measure of ToM. First, they repeat the finding that ToM remains impaired at comparable levels among adolescents with ASDs and is related to verbal and communication ability (Frith et al. 1994), and that variability in ToM may increase over time (Peterson et al. 2005); no ceiling effects were observed. Second, results support the external validity of the ToMI, demonstrating its valid use in a rural and suburban sample of community-recruited adolescents with ASDs. Third, findings support the relationship of ToM to ASD (Frith et al. 1994; Tager-Flusberg 2003), social skills (Dissanayake and Macintosh 2003), and particularly the social impairments characteristic of ASDs (Baron-Cohen 2001), supporting the concurrent criterion validity of the measure. In addition, results support parents as reliable and accurate sources of information about their children’s ToM.
Several important limitations to the findings are noteworthy. First, while strict procedures were used to ensure the ASD diagnosis of adolescent participants, gold-standard research measures (ADOS, ADI-R) were not used to confirm diagnoses. Thus, this sample may be considered to only represent a sample “at-risk” for ASDs. Second, the sample was fairly small, drawn only from a rural and suburban sample, and not compared to a normative control. Larger, more geographically diverse samples compared with normative peer data will be crucial to future efforts to explore the validity and utility of the ToMI, and such efforts are currently underway. Third, ToM scores were not assessed using existing experimental batteries for adolescents with ASDs or standard tests of verbal and adaptive functioning (e.g., language; Astington and Baird 2005), so more conclusive criterion validity could not be achieved. Current research efforts are also underway to explore these relationships and examine convergent validity of the ToMI. Fourth, measures were only administered at a single time point, so stability over time could not be assessed. Additional analyses of data using a normative control, and examining criterion-related validity and test-retest reliability are the subject of further research.
Future research will expand upon these promising findings regarding the use of the ToMI with diverse populations and its relationship to related measures. In particular, the ToMI should be administered to a broad normative sample of parents of children with ASDs across the lifespan as well as typically developing children to establish standardized scores. Additionally, ToMI scores should be compared to scores on measures of other theoretically related constructs to continue to explore its concurrent criterion validity. Finally, the ToMI should be used as an outcome measure in interventions designed to improve ToM in individuals with ASDs.
This study supports the importance of the ToMI as a valid parent-report measure of ToM among parents of adolescents with ASDs. This is an important step in the advancement of ToM research in this population, that does not suffer from floor or ceiling effects and represents a helpful, easy-to-use alternative to existing ToM paradigms. These paradigms require the scheduling of child visits and yield (often ordinal) scores that may not evidence sufficient variability to be sensitive to development and individual differences in ToM. In a related vein, this study supports the notion that parents are sensitive to the relationship of mentalizing ability to social functioning in their children with ASDs. In summary, we propose the continued and expanded use of the ToMI in research settings to advance its potential for clinical application.
The authors acknowledge the generous support of time and resources by the Commonwealth Autism Services, particularly Dottie Narodny, Didi Zaryczny, and John Toscano, who helped us to collect much of these data. We also thank Amori Mikami for her support of this project and the primary research assistant, Nancy Chae, for the hours she contributed. Most importantly, we thank the participating families, who volunteered a considerable amount of time to contributing their responses to these surveys and without whom this study would not be possible. The sponsor of the study (Commonwealth Autism Services) had no role in study design, data interpretation, or writing of the report.