Measuring Theory of Mind in Children. Psychometric Properties of the ToM Storybooks
- First Online:
Although research on Theory-of-Mind (ToM) is often based on single task measurements, more comprehensive instruments result in a better understanding of ToM development. The ToM Storybooks is a new instrument measuring basic ToM-functioning and associated aspects. There are 34 tasks, tapping various emotions, beliefs, desires and mental-physical distinctions. Four studies on the validity and reliability of the test are presented, in typically developing children (n = 324, 3–12 years) and children with PDD-NOS (n = 30). The ToM Storybooks have good psychometric qualities. A component analysis reveals five components corresponding with the underlying theoretical constructs. The internal consistency, test–retest reliability, inter-rater reliability, construct validity and convergent validity are good. The ToM Storybooks can be used in research as well as in clinical settings.
KeywordsTheory-of-Mind Storybooks Validation Reliability Normal children Autism spectrum disorder
From the beginning of the last century, research has been undertaken on the social empathy of children (e.g. Butterworth and Light 1982; Piaget 1929; Selman 1980). However, this topic only attracted the full attention of developmental psychologists after Premack and Woodruff (1978) introduced the term Theory of Mind in their chimpanzee research. Under the flag of ‘Theory-of-Mind’ it has become one of the most prolific research areas in social developmental psychology. Theory-of-Mind (ToM) is the social cognitive ability to attribute mental states to oneself and others and to use these attributions in understanding, predicting and explaining behavior of others and oneself (Mitchell 1997). ToM is also referred to as ‘folk psychological abilities’ or as ‘mind reading skills’. It is a core human capacity needed to fully understand the social environment and for showing socially adequate behavior (Astington and Jenkins 1995).
After the pioneering work of Premack and Woodruff (1978), research in normal ToM development proceeded with Wimmer and Perner (1983) who aimed their research at the understanding of wrong beliefs in young children. This was soon followed by studies in deviant ToM development. Concerning the latter, a great deal of research has been aimed at children with autism, starting with the studies of Baron-Cohen et al. (1985). They formulated the assumption that children with autism lack a ToM and that this deficit can explain a crucial part of the social impairment of these children. Since then a considerable amount of research has been undertaken in both typically developing children and children with autism (for a review see Wellman et al. 2001; Baron-Cohen 1989, 2000, respectively).
The majority of ToM research in children focuses on the comprehension of false beliefs. A false belief (FB) is the ability of a child to predict the action of a second person, while the child knows that this second person has an incorrect belief about the situation. Well-known paradigms used to test this are the Maxi test, which is an unexpected transfer test (Wimmer and Perner 1983), and the Smarties test, which is an unexpected content test (Perner et al. 1987). In the Smarties test, a child has to predict what a second person will say what is in the Smarties container, given that the child has seen that a pencil has been put in it (he holds a true belief) whereas the second person has not witnessed this (he holds an incorrect belief). Children only succeed on such tasks if they acknowledge that people act according to their own beliefs, even if those beliefs are, according to the child, wrong: the second person will say that the container holds smarties (and not a pencil).
The mastering of FB is considered to provide stringent evidence of a mature ToM (Hala and Carpendale 1997). As a result, the question of how and when children appreciate FB has moved centre stage in research on social cognitive development (Russel 2005). In addition, FB comprehension appears to be a universal milestone that occurs around the age of four, across different cultures (e.g. Callaghan et al. 2005; Wellman et al. 2001). However, equating FB understanding with the possession of a ToM is too simplistic. ToM comprises far more than that, like for instance the understanding of desires and emotions (Astington 2001). In addition, various ToM precursors are also involved. Already in the first and second year of life, a child develops socio-cognitive skills important for later ToM understanding, such as understanding intentional actions, engaging in pretend play, joint attention and imitation (e.g. Callaghan et al. 2005; Colonessi 2005).
Lately, the focus of research has moved from specific FB understanding to a more developmental view (Wellman and Lagattuta 2000; Steele et al. 2003) aiming at a wide range of ToM components that children develop between their second and sixth year (Wellman and Lagattuta 2000). In this period, ToM evolves from a simple desire theory to a complete belief-desire theory, from true beliefs to false beliefs, and from the understanding of first-order beliefs to second-order beliefs. Which mechanisms underlie this development remains subject of discussion (for a review, see Astington and Gopnik 1991; Hala and Carpendale 1997; Leekam 1993) (for a discussion, see Astington 2001; Scholl and Leslie 2001; Wellman and Cross 2001; Wellman et al. 2001). Roughly, three viewpoints can be distinguished: the theory–theory view, the modular view and the simulation view. The theory–theory view assumes that the ability to form theories is an innate capacity, founded on a general learning mechanism. The child learns through hypothesis testing (Carruthers 1996; Gopnik 2003; Gopnik and Wellman 1992; Perner 1988, 1991, 1993, 1995; Wellman 1990; Wellman and Bartch 1988). The modular view assumes that ToM has a specific innate basis, part of which is modular and which is activated on the basis of maturation (Baron-Cohen and Ring 1994; Fodor 1983, 1992; German and Leslie 2000; Leslie 1987, 1992, 2000; Leslie et al. 2004). The simulation view emphasizes the aspect of putting oneself in another person’s shoes, and thus of truly ‘empathizing’, which is the ability to recognize, perceive and feel directly the emotion of another person (Gallese 2007; Gordon 1992, 1996; Harris 1992). Recently, a rapprochement seems to emerge between the different views on mindreading abilities, resulting in a more hybrid position combining both the theory–theory view and the simulation view (Keysers and Gazzola 2007; Stueber 2006).
Relatively regardless of the view one holds on the underlying nature of ToM, the majority of researchers broadly agree on a number of observable aspects or components that constitute ToM knowledge in children. In deciding which aspects to incorporate in the present study, we leaned heavily upon the work of Wellman (1990), not only focusing on core ToM components, like desires and beliefs, but also on associated aspects, like the recognition of emotions, perception knowledge and the difference between physical and mental entities. The result is a comprehensive test of ToM components and associated aspects.
Comprehensive ToM Tests
Test psychologists recommend the use of comprehensive instruments composed of multiple tasks. Since aggregation favors broader applicability and reliability, such instruments can reduce standard errors and make measurements more reliable and valid. The total score of such a test is a compound score; that is, a score built of different parts. Research on ToM has shown that compound scores are more stable, because they average over multiple factors and lead to a more accurate measurement of the underlying skill (Hughes et al. 2000). In using such scores, a more adequate diagnostic procedure might be attained, which can help in studying the potential nature and causes of ToM differences in children (Hughes and Dunn 1998). In addition to providing a single, quantitative measure of the level of ToM ability, it also allows investigators to compare different relevant ToM components or aspects in the same child and thus to discover how these aspects are related during the course of development.
In current research on ToM, such comprehensive tests are seldom used (for exceptions, e.g. Happé 1994; Tager-Flusberg 2003; Wellman and Liu 2004). On the contrary, most research is based on single task measurements involving single aspects of ToM. These assessments may be quick and efficient, but provide no information about the nature and coherence of different aspects of ToM, and the stability of ToM ability over time. Examples of comprehensive ToM tests are the ToM battery of Happé (1994), the Tom-Test of Steerneman and colleagues (Steerneman et al. 2002; Muris et al. 1999), the ToM tasks of Tager-Flusberg (2003), and the ToM tasks of Wellman and Liu (2004). The first three comprehensive tests incorporate both simple and more advanced aspects of ToM. The ToM battery of Happé (1994) incorporates first-order-belief tasks, first-order deception tasks; second-order-belief tasks and second-order deception tasks. The ToM tasks of Tager-Flusberg (2003) consist of three batteries tapping early (pretend and desire), middle (perception/knowledge, location-change FB, unexpected-contents FB and sticker hiding) and more advanced ToM aspects (second-order-belief, lies and jokes, traits and moral commitment). The Tom-Test (Steerneman et al. 2002; Muris et al. 1999) consists of three subscales tapping ToM precursors (e.g., recognition of emotions and pretense), first manifestations of a real ToM (e.g., first-order-belief and FB) and more advanced ToM aspects (e.g., second-order-belief and humor). The last comprehensive test, the ToM tasks of Wellman and Liu (2004), confines itself to simple ToM tasks only. The tasks tap various desires, diverse beliefs, knowledge access, content FB, explicit FB, belief emotion and real-apparent emotion.
The ToM Storybooks
Many ToM tests are aimed at testing school-aged children. However, ToM-problems often occur long before this age, as the CHAT (CHecklist for Autism in Toddlers; Baron-Cohen et al. 1992) and M-CHAT (Modified CHAT, Robins et al. 2001) illustrate by identifying potential ToM problems at the age of 18 months on. We did not have the intention to measure ToM functioning from this age on, but wanted to develop a comprehensive test that can be used to assess basic ToM functioning in an age range that is as wide as possible. The aspects we aim at are ToM aspects that normally develop in the preschool years, but that also show further refinements during the school age period. Therefore, in accordance with Wellman and Liu (2004), we decided not to include second-order-belief tasks or other more advanced ToM aspects. At the time of the instrument building, the test of Wellman and Liu had not yet been published; in contrast to the ToM battery of Happé and the ToM-test of Steerneman and colleagues. Since the latter two tests were considered too complex to be used in preschool children, we developed a new test, the ToM Storybooks. In the 2002 paper from the current authors (Serra et al. 2002) a preliminary version of the test was presented. The following requirements were set for the final version: the test must comprise a wide and representative range of ToM components, cover a broad age range in order to allow for direct comparisons between children of different ages based on a continuous developmental trajectory and, finally, be optimally accessible and attractive to the youngest age range in particular, since that is the age range where the most rapid developments in ToM occur.
In this paper, we present four studies on the validity and reliability of the ToM Storybooks. The first study presents the construction of the new ToM Storybooks. The second study is aimed at the content validity of the test. The third study addresses the reliability of the test. Is the internal consistency of the test items sufficiently high? Are measurements repeatable, what is the test–retest reliability? What is the correspondence between raters evaluating the answers of children? The fourth study is aimed at the construct validity of the test. Is there convergent validity; does this test correlate highly with other tests that are known to measure ToM? As regards divergent validity: do the results obtained with this test differ sufficiently from tests not aimed at ToM, like an intelligence test and a language test? Since research has already shown that ToM results correlate positively with verbal intelligence scores (e.g. Hughes et al. 1999) and language scores (de Villiers 2000; Happé 1995; Tager-Flusberg 2000), we expect the test to show a positive correlation with language and IQ-tests.
An important question regarding the validity of the ToM Storybooks is whether the test is able to distinguish typically developing children, from children with autism spectrum disorders. A related question is whether the results regarding validity and reliability obtained with typically developing children also hold for the children with autism spectrum disorders. The latter group is known to have ToM problems (Buitelaar et al. 1999; Dissanayake and Macintosh 2003; Hill and Frith 2004; Serra et al. 2002; Yirmiya et al. 1998). We aimed to test children with PDD-NOS. If the test is able to distinguish the ToM functioning of children with PDD-NOS from that of typically developing children, it is by definition also suitable for distinguishing children with more severe impairments in ToM functioning, for instance children with a more severe pervasive disorder like an autistic disorder.
Study 1: Development of the ToM Storybooks
We wanted to develop a comprehensive ToM test that assesses a variety of ToM components and associated aspects, which develop during the preschool years and also tend to further refine and increase during the early school years. The construction of the test is explained below.
Setting and Participants
We tested 324 typically developing children that came from preschools, kindergartens and elementary schools. All children had a Dutch linguistic background, and did not have any language acquisition problems that could have hampered their performance on the ToM tasks (for the effect of language on ToM performance see for instance Garfield et al. 2001; Lohmann and Tomasello 2003). Two Dutch language tests were used, depending on the age of the child. For 3–6 year olds, the Reynell was administered (test for receptive language comprehension; Van Eldik et al. 1997); and for 6–9 year olds, the TvK (Taaltest voor Kinderen, Language Test for Children; Van Bon 1982) was used (subtests ‘vocabulary’ and ‘sentence construction’). Language scores were available for 249 children (Reynell: n = 170, TvK: n = 79). Those children who did not receive a language test were older than 6 years and judged as having appropriate language skills by their teachers.
Distribution in age groups of the typically developing children
Age (in years)
Construction of the ToM Storybooks
Primarily based on Wellman’s work (1990), core ToM components like desires and beliefs were included, but also emotions and associated aspects like the distinction between physical and mental entities, and understanding that seeing leads to knowing were included.
Emotion recognition is an important aspect, since discriminating and labeling facial expression of emotions lay the foundation for the ability to respond empathically to others (Feshbach 1982). By the end of the first year, typically developing children respond differently to facial expressions of emotion in others (Baron-Cohen 1994). At 20 months they use emotion words like happy, angry, sad, and scared (Flavell et al. 1993). At 3 years, they understand desire-based emotions (Yuill 1984), and at 5 years belief-based emotions (Hadwin and Perner 1991).
Beliefs and desires are considered core components of ToM. At 1.5 years children understand that other people have desires (Repacholi and Gopnik 1997); at 2.5 years they have a desire theory (Wellman and Woolley 1990); at 3 years a simple desire belief theory, they understand first-order beliefs (Bartsch 1996; Wellman 1990; Wimmer and Perner 1983), and at 4 years a complete belief desire theory is established (Wellman 1990). Four-year-olds have a representational understanding of beliefs (Gopnik 1993; Gopnik and Astington 1988; Perner 1991). Finally, 4-year-olds can distinguish true and false beliefs (Hala and Carpendale 1997; Wellman 1990; Wimmer and Perner 1983).
Concerning the associated aspects, during their second year children comprehend the difference between physical and mental entities (Wellman 1990). At 3 years, they understand that seeing leads to knowing (Astington and Gopnik 1991; Pillow 1989; Pratt and Bryant 1990).
Origin of tasks used in the ToM Storybooks, ordered by age
Sort of task
Task based on research from
Comparison with other comprehensive ToM tests
Pons and Harris (2000)
Steerneman et al. (2002)
Desire resulting in action or emotion
Mental physical distinction (including close impostors)
Steerneman et al. (2002)
Belief resulting in action or emotion
First-order false belief
We developed 34 tasks in total. Task examples can be found in Appendix A. The order of tasks and the number of questions per task are described in Appendix B.
The 34 tasks follow each other in a natural way; they are interwoven in stories. The stories feature a main protagonist, named Sam. A coherent drawing style was used (for instance, Sam always wears the same cloths). Each task is illustrated with a full color picture. The drawings are enlivened by the use of toy doors that can be opened, magnetic emotion faces that can be placed on the characters, and patches of soft fur that can be caressed, if wanted.2 Transitions between tasks are also accompanied by drawings and text, to keep the story going and to avoid too much switching between tasks.
Correlations between the four parallel books within the ToM storybooks
Although we conceive of the storybooks featuring the character Sam as the default version of our ToM test, three additional versions of the test were developed, based on different protagonists. They are designed to be used in a time-serial design, preventing trivial learning effects that might result from mere repetition. In the present article, we will confine ourselves to the default version of the test, featuring Sam.
The test takes 40–50 min, including a short break. The child sits at the left side of the administrator, so it can see the drawings clearly (the drawings are on the left side of the book, while the accompanying text for the experimenter can be found on the right side). The drawings remain in front of the child during the questioning in order to prevent mistakes due to memory requirements (in agreement with Charman et al. 2001).
The 34 tasks consist of 95 questions, namely 77 ‘test questions’ and 18 ‘justification questions’ in total. The test questions (for instance, Where will Sam look for his rollerblades? In the toy trunk or in the box?), can be considered a quick and less thorough method of testing, since they do not require justifications from a child. The answers to these questions are coded as correct or incorrect (1 or 0 points; maximum score = 77). Because justifications are considered to better reflect the ToM knowledge of a child, most tasks also include such questions. Justification questions (for instance, Why will Sam look in the box?) result in 2, 1 or 0 points, depending on the correctness of the mental state terms spontaneously used by the child (maximum score = 36) (for the scoring procedure see the right four columns of Appendix B).
In order to enable the standardized evaluation of the justifications, a category system has been developed, based on the category system used by Rieffe (1998), on different categories from Wellman (1990), and on an exploration of the empirical data (the elaborate category system can be requested from author EB). Two rules of thumb are followed in scoring the justifications. First, a justification can only be scored if the preceding test question is answered correctly. Second, the correctness of categories varies over the different types of questions. For instance, a desire answer can only be considered a correct category if it was used within a desire task and not within a FB task. Therefore, for each justification question, correct answer categories are determined. They are chosen from 21 formulated justification categories; in Appendix C definitions of these categories can be found.
ToM sumscore as an estimation of ToM ability. To assess the properties of the test items in estimating the ToM ability, a one-parameter logistic model (OPLM; Verhelst et al. 1995) was used. The key idea in OPLM, a unidimensional Item Response Model, is that for each item the probability of responding correctly to the item can be described by a particular monotonic increasing function of ToM ability. In OPLM, the particular functions of the items may differ in the item location (some ToM items are more difficult to master than others), and in the item discrimination (some items discriminate children better in their ToM ability than others). For the justification questions, with three response categories (2 points, 1 point or 0 point), a polytomous OPLM was used.
The OPLM showed a good fit for the 95 ToM items (77 test questions + 18 justification questions), except for the three items of the inferred belief control task. For those items, a higher ToM ability did not result in a higher probability of giving a correct answer. Therefore, those items were eliminated from the ToM test. The OPLM of the 92 remaining items revealed a good fit for all items. All items contribute significantly to estimating the ToM ability. The correlation between OPLM ToM ability estimate and the ToM sumscore was 0.99. Thus, the ToM sumscore and the OPLM ToM ability estimate yield approximately the same results for ordering the children on their ToM ability. Therefore, we confine ourselves to a ToM sumscore; weighted values are not required. The testing with the ToM Storybooks results in a maximum total score of 110 points (ToM-total score), consisting of a maximum of 74 points for answers to the ‘test questions’ (3 inferred belief control questions are excluded) and a maximum of 36 points (= 18 × 2) for answers to the ‘justification questions’.
ToM Quotient Score
In addition to a total score, a ToM quotient score (ToMQ) can be calculated. Norms for the ToM sumscores were obtained by applying a non-linear smoothing method over the raw data. Smoothness of the estimated curve is induced by weighing neighbouring observed scores (see for instance Simonoff 1996; Härdle 1991). A Fourier-series tenth-order polynomial based on a Loess smoothing technique (locally weighted least squares estimate) has been applied, which enables us to calculate the conversion curve. This curve enables us to determine the value of the smoothed curve at any possible age between 3 and 11 years. The conversion curve was calculated with the help of the TableCurve 2D programme (Systat 2000).
Raw ToM sumscores were converted to Z-scores and converted to quotient scores (Wechsler 1981). This is a standardized normed score, with an average of 100 and a standard deviation of 15 (for more details on the norming procedure of the ToM Storybooks, we refer to Blijd-Hoogewys et al. submitted).
The ToM Storybooks have been developed with the aim of providing practitioners and researchers with a comprehensive ToM test assessing different basic ToM components and associated aspects. The test was administered to typically developing children.
The test consists of 34 tasks divided over six storybooks. It holds 74 test questions and 18 justification questions, resulting in a maximum total score of 110. The ToM sumscore can be considered a good estimation of ToM ability, as the OPLM results illustrated. Weighted values are not required. Also, a ToM quotient score can be calculated.
As far as typically developing children are concerned, the test focuses on the age range between three and six, given the rapid developments of ToM that occur in this period. However, the test has standardized norms and is applicable up to the age of 12. As a result, an 11-year-old child with ToM problems can be compared to a typically developing 5-year-old but also to a typically developing 11-year-old. Thus, the test is particularly suited for studies requiring age comparisons, based on the same instrument (e.g. cross-sectional research, assessment of clinical populations at various ages).
Criteria for Subgrouping
ToM total scores over age
Average ToM score
8 + 9
10 + 11
The dependence of the scores on age poses a number of problems for the analyses of reliability and validity. Hence, where needed, analyses were carried taking into account an age correction or by using distinct age groups. In the group of 324 typically developing children, we distinguished three age groups. The subdivisions were made based on theoretical expectations (expected levels of ToM functioning: low level, intermediate level and master level) and pragmatic grounds (approximate equal groups). The youngest (n = 87; 3–4.5 years old) represents the age at which ToM development is at its beginning, at least as measured with the ToM Storybooks. The eldest (n = 118; 6.5–11 years old) represents the age at which the ToM aspects measured with the ToM Storybooks is expected to have consolidated. The fastest growth of ToM is expected to occur in the intermediate age group (n = 119; 4.5–6.5 years old).
Because this article discusses different psychometric studies, each with different sub-studies, we enclose an overview of the statistical analyses performed and their results (see Table 9). These results are discussed in more detail below.
Study 2: Content Validity
Our test, the ToM Storybooks consists of different tasks on ToM components and associated aspects taken from literature. It contains tasks aimed at assessing five subtypes of abilities, namely emotion recognition, understanding of desires and beliefs, making the distinction between physical and mental entities, and seeing leads to knowing (compare Appendix A). To assess whether those subtypes are indeed present, a component analysis was performed.
We expected the subtypes to be correlated, and the correlation to depend on age. That is, the degree of differentiation is expected to be the largest at ages where ToM has rapid growth. Less differentiation, and hence greater correlations between subtypes, is to be expected in early stages of ToM.
Subjects and Method
The analyses were based on the 324 typically developing children from Study 1. We calculated composite variables for the ToM Storybooks: for the different tasks, means were calculated over theoretically similar items. This resulted in 21 composite variables (between brackets are the number of test questions + number of justification questions) (for example of the tasks see Appendix A): emotion recognition (5 + 0) and emotion naming (5 + 2) (parts from the emotion recognition tasks); desire action (3 + 1), desire emotion recognition (5 + 1), and desire emotion naming (5 + 0) (parts from the desire tasks); standard belief emotion recognition (2 + 0), standard belief emotion naming (2 + 1), standard belief action (3 + 1), changed belief action (1 + 1), not own belief action (1 + 1), not belief action (2 + 1), inferred belief (control) action (4 + 0), and (explicit) FB action (5 + 2) (parts from the belief tasks); mental physical senses (8 + 2), mental physical others (4 + 1), mental physical future (4 + 1), real imaginary (7 + 0), close impostor senses (4 + 0), close impostor others (2 + 1), and close impostor future (2 + 1) (parts from tasks aimed at the distinction between physical and mental entities); and finally, the variable seeing-is-knowing (1 + 1).
The scores of the three age groups (group 1: n = 87, 3–4.5 years old; group 2: n = 119, 4.5–6.5 years old; and group 3: n = 118; 6.5–11 years old) were analyzed using a simultaneous component analysis with equal pattern (SCA-P; Kiers and Ten Berge 1994). SCA-P, which is a variant of principal component analysis, estimates one pattern matrix for all three groups. As a result, the interpretation of the components (or factors) is equal for all groups, but the correlations between components and standard deviations of the component scores can differ across groups. To determine the number of components, the scree-test (Cattell 1966), the eigenvalue-greater-than-one rule (Kaiser 1960), and the substantive meaning of components, was used. Only minimum loadings of 0.400 were considered. Finally, composite variables had to show adequate specificity for their components. Subsequently, internal consistency reliability was calculated for the components found.
SCA-P structure matrix and component correlation matrix
(A) Structure matrix with correlations between 5 components and 21 composite variables
Standard belief emotion recognition
Standard belief emotion naming
Desire emotion recognition
Desire emotion naming
Mental physical senses
Mental physical others
Mental physical future
Close impostor senses
Close impostor others
Close impostor future
Standard belief action
Changed belief action
(Explicit) false belief action
Not own belief action
Not belief action
Inferred belief (control) action
(Explicit) FB action
(B) Component correlation matrix
Cronbach alphas, corrected for age, for these five components were calculated: component 1 (10 items, α = 0.79), component 2 (4 items, α = 0.47), component 3 (9 items, α = 0.80), component 4 (25 items, α = 0.62) and component 5 (14 items, α = 0.61). Since the scores on the justification items depended on the child’s answer on the related dichotomous items, justification items were not included in the calculation of the alphas, in order to avoid artifacts.
To assess the degree of differentiation in the three age groups, inter-factor correlations between the five components were computed within the three age groups. The correlations between the components were largest in the youngest group (average correlations, standard deviations of the correlations: M = 0.47, SD = 0.08), and comparable in the intermediate age group (M = 0.26, SD = 0.013) and in the eldest group (M = 0.26, SD = 0.15).
The component analysis resulted in a structure that largely corresponds with the underlying theoretical constructs from the test. The five components appeal to the five subtypes of abilities named in Appendix A) except for the composite variable ‘seeing leads to knowing’ which did not appear as a separate component. This is not surprising, since the composite variable consists of too few questions (only two). The internal consistency reliability is satisfying (although some Cronbach alphas are not >0.70), since it concerns alphas on subparts of the test each containing a limited number of items, that are also corrected for age. The inter-factor correlations are consistent with the expectations: they are high in the youngest children implying that ToM abilities are not (yet) differentiated.
Study 3: Reliability
In order to examine the reliability of the ToM Storybooks, we calculated the internal test consistency, test–retest reliability and inter-rater reliability. In addition, we examined the possibility of diminished test performance due to nuisance factors such as fatigue or boredom.
Subjects and Method
For the internal test consistency, the data of the 324 typically developing children from Study 1 were used. For the test–retest reliability, a subgroup of 45 typically developing children (age 3–7) was tested again, with the second administration occurring 2–3 weeks later. We presume that ToM ability remains relatively constant when reassessed after such a short period. The test–retest reliability was also measured for children with PDD-NOS (n = 18; age 5–9) (a subgroup from the clinical group that will be presented in Study 4, see Table 7), with the second administration after 1 week. In order to determine the inter-rater reliability of the justifications, the test results of a subsample of 10 children were randomly chosen from both research groups (n = 10 typically developing children and n = 10 children with PDD-NOS). For the analysis of possible diminishing test performance at the end of the test, the data of the 324 typically developing children was used.
The internal consistency was established by means of a Cronbach’s alpha. The test–retest reliability was established by means of a Pearson product-moment correlation coefficient. The inter-rater reliability was calculated on the basis of Cohen’s kappa’s. Five independent raters scored the justifications and the correlations between these five raters were calculated. This was done in two manners: a flexible manner by points awarded to the justifications (2, 1 or 0 points) (compare Appendix B) and a stringent manner by justification category chosen (compare Appendix C). To examine whether the test scores were affected by nuisance factors such as fatigue or boredom, it was checked if the results over the various storybooks showed a significant decline. Books 2–5 were considered, because they have a similar item structure (see Study 1). Since children could choose the order of the books, the average total score of the actual presentation of those books were compared. If nuisance played a part, the last presented book should result in a lower score than the first presented book.
Test performance from beginning to end of testing
Age group 1 (3–4.5 years)
Age group 2 (4.5–6.5 years)
Age group 3 (6.5–11 years)
Based on the minimum standard for reliability of 0.70 (Nunnally and Bernstein 1994, p. 265), the internal consistency (Cronbach’s α = 0.90) of the total score was good (0.90). This is an adequate value for a test aimed at young children and is consistent with findings from comparable research on standard and complex FB tasks (Hughes et al. 1999, 2000 obtained alphas of 0.83–0.84; Muris et al. 1999 obtained alphas of 0.84–0.92) and suggests that the different tasks measure the same underlying construct. Also, the test–retest reliability is good, both in typically developing children (r = 0.86) and in children with PDD-NOS (r = 0.98). This is consistent with findings from comparable research (r = 0.77: Hughes et al. 2000; r = 0.88: Muris et al. 1999). However, a significant increase in ToM total scores was found at the second measurement in typically developing children. Such a rise is not surprising, since it can be expected that young children learn from being tested (Grigorenko and Sternberg 1998). The average score rise (M = 6.86, SD = 10.33) is of the same magnitude as those obtained with most standard psychometric measures on cognitive skills for young children. For instance, a difference of six IQ points can also be found in test–retest research with intelligence tests (e.g. Tellegen et al. 2003). A similar observation has also been reported in ToM research in typically developing children (Muris et al. 1999). The children with PDD-NOS did not show such a rise. They seemed not to have learned from their former experience. This finding may form an important point of attention in evaluating children with suspected ToM problems.
The inter-rater reliability of scoring the justifications is high (Cohen’s Kappa >0.80, namely 0.81–0.97, even concerning the more stringent scoring criterion) (see also Charman et al. 2001; Muris et al. 1999). There were no differences in difficulty in judging the justifications of typically developing children versus children with PDD-NOS. There was also no evidence for a statistically significant negative effect on the test scores due to increasing fatigue or boredom during the test administration.
Study 4: Construct Validity
We tested both the convergent and divergent validity of the ToM Storybooks. Concerning convergent validity, correlations with three similar tests were calculated. Concerning divergent validity, correlations with language and intelligence tests were calculated. The latter can be considered moderator variables in performance on ToM tests, but should not be considered to be equal to ToM. Despite their diversity, we do expect to find a positive relationship between ToM scores and scores on a language test, since ToM questions make a relatively strong appeal to lexical and syntactic knowledge (see for instance Garfield et al. 2001; Lohmann and Tomasello 2003). We also expect a positive relationship with verbal IQ (Hughes et al. 1999).
Subjects and Method
Children were referred to an outpatient clinic for child and adolescent psychiatry. After an extensive psycho-diagnostic and psychiatric examination (which included parent interviews and play contacts with the child), the children were diagnosed as having PDD-NOS (pervasive developmental disorder not otherwise specified) according to DSM-IV criteria (APA 1994).
Test results of the children with PDD-NOS
Age (in years)
n = 2
n = 3
n = 4
n = 11
n = 5
n = 5
n = 30
Play and leisure time
In order to check the validity of the clinical diagnosis, two additional tests were administered: the Vineland adaptive behavior scales (VABS) (Sparrow et al. 1984; Dutch version: Researchgroup Developmental Disorders, State University Leiden, 1995) and the Children’s Social Behavior Questionnaire (CSBQ; Luteijn et al. 2000; Dutch version: VISK; Luteijn et al. 2002; Hartman et al. 2007). The VABS is an interview in which parents are questioned about the actual social behavior and skills of their child. We used parts of the expanded form of the VABS. For each child the discrepancy between the Vineland age equivalent (in months) and the chronological age (in months) was computed (VA-CA). The results showed that these children had large and negative discrepancy scores in receptive language, playing skills, interpersonal relationships and coping skills (see also Serra et al. 2002) as can be expected in children with pervasive developmental disorders. Their problems with expressive language and daily living skills (community) were less profound (Paul et al. 2004) (compare Table 7). The parents also filled in the CBSQ. This is a questionnaire in which parents report autism-related behavior. It can be used to facilitate selection of PDD samples for research purposes (Hartman et al. 2006). The CSBQ scores of our group are comparable to those known for children with HFA and PDD-NOS (compare Table V in Hartman et al. 2006) (total score, M = 48, SD = 18; ‘tuned’, M = 10, SD = 5; ‘social’, M = 13, SD = 5; ‘orientation’, M = 8, SD = 3; ‘understanding’, M = 5, SD = 3; ‘stereotyped behavior’, M = 6, SD = 3; ‘change, M = 2, SD = 2).
Next, all children participated in an extensive psychological examination which included the assessment of intelligence (Wechsler Intelligence Scale for Children-Revised: Wechsler 1974; Dutch version, 1986) and the level of language comprehension. Concerning the latter, two Dutch language tests were used, depending on the age of the child. For 3–6 year olds, the Reynell was administered (test for receptive language comprehension; Van Eldik et al. 1997); and for 6–9 year olds, the TvK (Taaltest voor Kinderen, Language Test for Children; Van Bon 1982) was used (subtests ‘vocabulary’ and ‘sentence construction’).
Concerning convergent validity, two additional questionnaires and one test were included. The CSBQ (Luteijn et al. 2000) measures, among other things, ToM related knowledge, namely in the subscale ‘difficulties in understanding social information’. The VABS questionnaire (Vineland adaptive behavior scales questionnaire; Frith et al. 1994; Dutch translation: Hoogewys et al. 1999) consists of 32 theoretically derived items aimed at discriminating between social behaviors for which mentalizing (ToM) is essential (Interactive Sociability Scale, abbreviated as IS scale) or not (Active Sociability Scale, abbreviated as AS scale, concerning social behaviors that can be acquired without mentalizing). Both CSBQ and VABS questionnaire were administered for the clinical group (n = 30 PDD-NOS, 4–8 years). Also a second ToM instrument was administered, namely the Tom-Test, a Dutch test that questions a wide variety of ToM aspects (Steerneman et al. 2002; see also Muris et al. 1999). In contrast with the ToM Storybooks, it also includes second-order-belief tasks. From the 30 children with PDD-NOS, 23 received the Tom-Test (age 4–8).
There were also four groups of typically developing children involved in Study 4. The first group is a subsample of 30 control children drawn from the 324 typically developing children from Study 1. They were matched on age and gender with the PDD-NOS group. This control group was used to make comparisons with the clinical group. The second is a subsample of 249 typically developing children (drawn from the group of typically developing children in Study 1; 3–9 years). For these children, language scores were available (Reynell: n = 170, TvK: n = 79; 59 boys and 48 girls). This control group was used to explore the relationship of ToM scores and language scores in typically developing children. The third is a subsample of 107 typically developing children (drawn from the 324 typically developing children in Study 1; 3–7 years). For these children, intelligence scores were available. They received a nonverbal intelligence test. Depending on the age of the child this consisted of the SON-R 2½-7 years (Snijders-Oomen Nonverbal intelligence scale: Tellegen et al. 1998) or the SON-R 5½-17 years (Snijders-Oomen Nonverbal intelligence scale—Revised: Snijders et al. 1988).3 This control group was used to explore the relationship between ToM scores and IQ scores in typically developing children. The fourth group is a subsample of 106 typically developing children (drawn from the 324 typically developing children in Study 1, 3–8 years; 54 boys and 52 girls). For these children, VABS questionnaire scores were available. This control group was used to explore the relationship between ToM Scores and VABS questionnaire scores in typically developing children.
With regard to convergent validity, we calculated Pearson product-moment correlations between the ToM Total score and the CSBQ subscales, the VABS questionnaire and the Tom-Test. Divergent validity was tested by comparing the ToM quotient scores with language scores and IQ scores by calculating Pearson product-moment correlations.
ToM results of children with PDD-NOS
MC, p < 0.001
MC, p < 0.001
Emotion recognition (14)
Real-mental items (0–24)
MC, p < 0.001
Real-imaginary items (0–8)
MC, p < 0.001
Close impostors (0–12)
MC, p = 0.01
Predicting action (0–5)
MC, p < 0.001
Predicting emotion (0–12)
Predicting action (0–26)
MC, p < 0.001
Predicting emotion (0–6)
MC, p < 0.001
Summary of the statistical methods
n = 324, 3–11 years old
3 age groups:
Simultaneous component analysis: 5 correlated factors
n = 87, 3–4.5 years old
n = 119, 4.5–6.5 years old
n = 118, 6.5–11 years old
n = 324, 3–11 years old
(α – (correlation ToM&age)b)/(1-(correlation ToM&age)b)
Cronbach’s α = 0.90
Test retest reliability
n = 45, 3–7 years old
No age correction applied
r = 0.86***
n = 22, 5–10 years old
No age correction applied
r = 0.98, ns increase
n = 10
n = 10
No age correction applied
n = 324, 3–11 years old
3 age groups:
n = 87, 3–4.5 years old
n = 119, 4.5–6.5 years old
n = 118, 6.5–11 years old
ToM SBb and CSBQc
n = 30, 4–8 years old
No age correction applied
SCd 3, r = −0.60‡; SC 4, r = −0.47‡
ToM SB and VABS-Qe
n = 106, 3–8 years old
ISf, r = 0.19†; ASg, r = 0.13**
n = 30, 4–8 years old
No age correction applied
IS, r = 0.35†; AS, r = 0.24**
ToM SB and ToM test
n = 23, 4–8 years old
No age correction applied
r = 0.79***
ToM SB and diagnosis
n = 30, 4–8 years old
n = 30, 4–8 years old
ToM quotient scores
M = 85.10 < M = 101.09***
ToM SB and language
n = 249, 3–9 years old
ToM quotient scores
Reynel, r = 0.47***; TvKh, r = 0.43***
ToM SB and IQ scores
n = 107, 3–7 years old
ToM quotient scores
PIQi, r = 0.47***
n = 30, 4–8 years old
ToM quotient scores
VIQj, r = 0.41* and PIQ, r = ns
The correlations of the ToM Storybooks with the VABS questionnaire subscores are significant for the IS scale (r = 0.19 and r = 0.35, for, respectively, typically developing children and children with PDD-NOS, p = 0.01, one-tailed) and show a trend for the AS scale (p = 0.06, one-tailed, for both typically developing children and children with PDD-NOS). Thus, a higher ToM score implies higher sociability.
The correlation of the ToM Storybooks with the Tom-Test is high (M = 47.09, SD = 9.74 versus M = 87.39, SD = 11.36, scores of, respectively, Tom-Test and ToM Storybooks, r = 0.79, p < 0.001, tested two-tailed). Children with PDD-NOS evidence ToM problems on both the ToM Storybooks and the Tom test.
The correlation with language comprehension in typically developing children varies for the different language tests from 0.43 to 0.47 (p ≤ 0.001, tested two tailed; a common variance of 18–22%) (see Table 9). Concerning IQ, in typically developing children only a performance IQ was obtained. The correlation with ToM-Q was 0.47 (p = 0.001, tested two-tailed; a common variance of 22%); while in children with PDD-NOS, there was no significant correlation with performance IQ (r = 0.07). However, the correlation of their verbal IQ with ToM-Q was 0.41 (p < 0.05, tested one-tailed; a common variance of 17%).
The results show that children with PDD-NOS evidence ToM problems. Children with PDD-NOS have problems with beliefs, both in predicting behaviors and emotions. In addition, they have problems on emotion recognition, real-imaginary, real-mental, close impostor, and desire-action tasks. These findings largely agree with the findings from Serra et al. (2002). Despite differences in p-values, the findings from both studies coincide. The only contrary finding is that beliefs used to predict actions were significantly more difficult for children with PDD-NOS than for typically developing children, whereas Serra and colleagues found the opposite. The finding from the present study, however, is more consistent with clinical expectations.
The construct validity of the ToM Storybooks is good, both for the convergent and the divergent validity. Concerning the convergent validity, substantial correlations with ToM-related tests were found. The correlations of the ToM total score with the CSBQ subscales were good. Average negative correlations were found with all subscales. The highest correlations were found for the subscale ‘difficulties in understanding social information’, which can be perceived of as related to ToM skills, and for the subscale ‘orientation problems in time, place, or activity’, which can be perceived of as related to executive functions. It is known that executive functions are somehow linked with ToM development (e.g. Carlson et al. 2002).
The results from the ToM Storybooks also correlated with the VABS supplementary items from Frith et al. (1994). We found significant correlations with the IS Scale (requiring ToM) and a trend for the AS Scale (not requiring ToM) with the ToM Storybooks, for both the typically developing group and the PDD-NOS group. Our results agree to a large extent with the results of Frith et al. (1994), except that the latter found significant differences for the AS only in the normal control group and for the IS only in the autistic group. Purely speculatively, the differences in results can be due to the restricted use of FB measurements (Smarties test and Three Boxes test instead of a comprehensive ToM test) in a more seriously affected group (children with an autistic disorder compared to children with PDD-NOS in our research).
With regard to the convergent validity, the correlation of the ToM Storybooks with the Tom-Test of Steerneman et al. (2002) is as expected. The ToM Storybooks test also has adequate discriminant validity. It can distinguish children with a normal ToM development from children with ToM problems, such as children with PDD-NOS. For future research, examining the applicability and discriminatory power of the ToM Storybooks, it is recommended to include children with an autistic disorder and other clinical groups, like for instance children with ADHD.
Finally, the correlations between children’s scores on the ToM Storybooks and language acquisition tests are high (>0.40). Also, correlations with IQ scores were inspected. The verbal IQ results of the children with PDD-NOS were somewhat lower than the normal sample, which is often seen in subsamples of children with autism (compare Joseph et al. 2002; Kraijer 2004; Siegel et al. 1996). As regards the correlations with IQ scores, our research showed a significant correlation with PIQ for the typically developing group (compare Muris et al. 1997, 1999; Carlson et al. 2002), but not for the PDD-NOS group. The latter group showed significant correlations with VIQ, consistent with findings of other researchers (r(230) = 0.43 in typically developing children, p < 0.001 in Hughes et al. 1999; r(52) = 0.61 in typically developing children and children with PDD-NOS, p < 0.001 in Muris et al. 1999). Final conclusions on the differences between these two groups cannot be drawn since different IQ tests were used. Due to the age limitations of IQ tests, different tests were used for the children with PDD-NOS in comparison with typically developing children. Moreover the PDD-NOS group was much smaller. As concerns future research on the relationship between IQ and ToM, the present authors recommend the use of a comprehensive ToM instrument, as was done in the present study and in studies of Hughes and colleagues, and Muris and colleagues.
Correlations between ToM Storybooks and IQ scores were notably high. However, this is not surprising. One could say that, if we look at intelligence in a broad way, comprehensive ToM instruments measure a specific aspect of intelligence, namely a kind of social intelligence. After all, these tests look into the logical reasoning of people and correlations of one type of intelligence with another are highly common. In addition, intelligence contributes to acquiring ToM skills, making it possible for children to understand connections between causes and results. In that view, comparison with IQ should perhaps not be considered as a test for divergent validity.
This article presented the construction and validation of the ToM Storybooks. It is a comprehensive ToM test, measuring different basic ToM components, but also associated aspects. In Study 1 the construction of this test was discussed. The test holds 34 tasks, spread over six storybooks. A ToM sumscore and a ToM quotient score can be calculated. In Study 2, analyses showed an agreement between the underlying theoretical constructs and the components found through component analysis. Study 3 looked into the reliability of the test. Internal consistency, test–retest reliability and inter-rater reliability were found good. Lastly, Study 4 assessed the construct validity of the ToM Storybooks. Convergent validity, based on two questionnaires and an additional ToM test, was good. The ToM-score had high correlations with language tests and IQ tests, as was expected.
It can be concluded that the validity and reliability of the ToM Storybooks complies with the requirements of an instrument of this sort. The separate findings are consistent with findings of other researches, but also agree with the more general findings of Wellman and colleagues on FB tasks (2001), which show that researchers can vary the tasks over an extended set of possibilities without influencing the performance of children. There is no indication that the medium in which ToM tasks is presented, in this case pictured storybooks, has affected the results in ways that reduce the test’s reliability.
A Critical Remark
The reliance of this kind of task on language comprehension with this kind of population, may lead to potential complications. Children with weak language comprehension undoubtedly will have more problems with successfully completing the test. The literature shows that there are strong relationships between language and ToM (Astington and Baird 2004; Astington and Jenkins 1999; de Villiers 2000; Tager-Flusberg 2000). In addition, many children with autism have language problems. In people with autism, ToM results are correlated to verbal mental age (Frith et al. 1991; Prior et al. 1990) and verbal skills (Happé 1995). However, early research has shown that language problems do not contribute to mental state impairment, because children with for instance semantic language impairment do not show such problems (Leslie and Frith 1988; Perner et al. 1989). On the other hand, the influence of language on ToM development should not be underestimated (Ruffman et al. 2003; Sparrevohn and Howie 1995), also in testing. Language is a medium through which children learn about beliefs (Astington 2001). Reading storybooks, for instance, form a rich source of mentalizing information for children (Dyer et al. 2000).
Potentialities of the ToM Storybooks
The test includes a wider range of ToM aspects commonly tested. It includes not only tasks on first-order beliefs and desires, but also tasks on associated aspects such as the distinction between mental and physical entities. It is a comprehensive test consisting of tasks with different developmental challenges. The primary advantage of this test over existing batteries is that it targets skills that develop in typically developing children prior to the age of five, and further refine and increase during the early school years. The test, however, is applicable beyond the age of five; it has norm scores up to the age of 12 years and thus allows for comparisons between children of widely varying age, which makes it particularly appropriate for comparison with clinical groups in which ToM development is delayed. As a consequence, this test may have potential for a range of applications to both fundamental and applied work. Moreover, since this study covers a wider age range than is normally included in ToM research, valid comparisons between older children with ToM problems and their age mates with normal ToM functioning can be made. We like to remark that the older age group included in this study is not intended for discrimination between typically developing children, but between older children with clinical diagnosis. Since older typically developing children have, as a group, a smaller range in ToM total scores, a lower ToM score on these simple ToM tasks is very informative. Because of the use of simple ToM tasks and a motivating storyline, the test might also be useful in the field of intellectual disability, where autism spectrum disorders and related ToM problems are common. However, for future research it is advisable to include more complicated ToM tasks, such as a second-order belief task (see for instance Hughes et al. 2000) and a ‘faux pas’ task (Baron-Cohen et al. 1999), so that older children with more subtle problems can also be detected.
The test–retest correlations of the typically developing children suggested a small learning effect. As stated before, this is consistent with findings from Muris et al. (1999). Grigorenko and Sternberg (1998) recommended that this effect—the learning potential of individual children—be included in normal diagnostics. In that case, the pretest–posttest difference can eventually be considered an estimation of learning abilities that are, at least in part, ToM specific. The absence of a comparable learning effect in specific groups of children, like we have found in children with PDD-NOS, could provide interesting information about the nature of ToM abilities in such children. In this line, further research on ToM might profit from dynamic testing—as opposed to static testing—where the learning potential of a child is quantified on the basis of his or her understanding and use of feedback given during testing (Grigorenko and Sternberg 1998). Dynamic indexes can represent a quality step-up compared with static indexes (Fabio 2005).
To conclude, one of the methodological strengths of the current test is that it has extended the limitations common in the majority of the researches done in the field of ToM. Most research has been undertaken in young children only (mostly up to 6 years, with a major focus on 3–4 year olds), has used only a few tasks (FB tasks, mainly single tasks) and considered small research groups (exceptions in the latter can be found in Charman et al. 2002; Hughes et al. 1999). The present research, aimed at constructing a new ToM Storybooks, used a wide range of tasks (not only FB tasks) and consisted of a substantial number of children over a wide age range. The test not only allows for comparisons on the basis of raw scores but standardized norms and norm scores are also available (Blijd-Hoogewys et al. submitted). In our opinion, the ToM Storybooks provide a comprehensive, valid and reliable instrument for researchers and clinicians who wish to measure Theory-of-Mind in young typically developing children, as well as children with an autism spectrum disorder from a broader age range.
Also children older than 5 years were tested, in order to determine the upper-age limit of the test. In addition, testing older children makes comparisons between children with and without ToM problems easier.
Non-graphical elements are distributed sparsely across the text; manipulation of these elements is not a necessary condition for answering the test. None of the children from clinical populations to which the test has been administered so far has shown any sign of disturbance or aversion for the non-graphical elements.
For pragmatic reasons, children with PDD-NOS were tested with the WISC. The division between verbal IQ and performance IQ can be very informative in children with autism spectrum disorders. Since the NT group also consists of children younger than 6 years, the WISC could not been applied and a nonverbal intelligence test was preferred.
The explanations are translated from Dutch examples. It is not unthinkable that there are nuances in English that we could not explain here.
This research was supported by a grant from the GUF Gratama Foundation. We would like to thank all children and their parents for participating; the numerous students helping in collecting data; late D. Kraijer, P. Tellegen, S. Begeer and K. McIntyre for their commentaries on this manuscript; and M. E. Timmerman for advising about the one-parameter logistic model and the factor analysis.
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
- American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: American Psychiatric Association.Google Scholar
- Astington, J. W., & Baird, J. A. (2004). Why language matters for theory of mind. International Society for the Study of Behavioural Development Newsletter, 45(1), 7–9.Google Scholar
- Astington, J. W., & Gopnik, A. (1991). Theoretical explanations of children’s understanding of the mind. British Journal of Developmental Psychology, 9, 7–31.Google Scholar
- Astington, J. W., & Jenkins, J. M. (1995). Theory of mind development and social understanding. Cognition and Emotion, 9, 151–165.Google Scholar
- Baron-Cohen, S. (1989). Theory of mind and autism: A fifteen year review. In S. Baron-Cohen, H. Tager-Flusberg, & D. Cohen (Eds.), Understanding other minds. Perspectives from developmental cognitive neuroscience (2nd ed.). New York: Oxford University Press.Google Scholar
- Baron-Cohen, S. (1994). How to build a baby that can read minds: Cognitive mechanisms in mindreading. Cahiers de Psychologie Cognitive/Current Psychology of Cognition, 13, 513–552.Google Scholar
- Baron-Cohen, S. (2000). Theory of mind and autism: A fifteen year review. In S. Baron-Cohen, H. Tager-Flusberg & D. J. Cohen (Eds.), Understanding other minds: Perspectives from developmental cognitive neuroscience (2nd ed., pp. 3–20). Oxford: Oxford University Press.Google Scholar
- Baron-Cohen, S., & Goodhart, F. (1994). The “seeing leads to knowing” deficit in autism: The Pratt and Bryant probe. British Journal of Developmental Psychology, 12, 397–402.Google Scholar
- Baron-Cohen, S., & Ring, H. (1994). A model of the mindreading system: Neuropsychological and neurobiological perspectives. In C. Lewis & P. Mitchell (Eds.), Children’s early understanding of mind (pp. 183–207). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
- Baron-Cohen, S., O’Riordan, M., Stone, V., Jones, R., & Plaisted, K. (1999). Recognition of faux pas by normally developing children and children with Asperger syndrome or high-functioning autism.Google Scholar
- Blijd-Hoogewys, E. M. A., van Geert, P. L. C., Serra, M., & Minderaa, R. B. (submitted for publication). A new way of developing standardized norm scores: Normalizing the ToM Story Books.Google Scholar
- Butterworth, G. E., & Light, P. (1982). Social cognition: Studies of the development of understanding. Brighton: Harvester Press.Google Scholar
- Carlson, S. M., Moses, L. J., & Breton, C. (2002). How specific is the relation between executive function and theory of mind? Contributions of inhibitory control and working memory. Infant and Child Development, 11, 73–92.Google Scholar
- Carruthers, P. (1996). Simulation and self-knowledge: A defence of theory–theory. In P. Carruthers & P. K. Smith (Eds), Theories of theories of mind (pp. 22–38). Cambridge University Press.Google Scholar
- Cattell, R. B. (1966). The meaning and strategic use of factor analysis. In R. B. Cattell (Ed.), Handbook of multivariate experimental psychology (pp. 174–243). Chicago: Rand McNally.Google Scholar
- Charman, T., Carroll, F., & Sturge, C. (2001). Theory of mind, executive function and social competence in boys with ADHD. Emotional and Behavioural Difficulties, 6(1), 31–49.Google Scholar
- Charman, T., Ruffman, T., & Clements, W. (2002). Is there a gender difference in false belief development? Social Development, 11(1), 1–10.Google Scholar
- Colonnesi, C. (2005). The emergence of a theory of mind from infancy to childhood: Related abilities at 12, 15 and 39 months of age. Doctoral Dissertation: Universitá di Roma ‘La Sapienza’.Google Scholar
- De Villiers, J. (2000). Language and theory of mind: What are the developmental relationships? In S. Baron-Cohen, H. Tager-Flusberg, & D. Cohen (Eds.), Understanding other minds: Perspectives from autism and developmental cognitive neuroscience (2nd ed., pp. 83–123). Oxford: Oxford University Press.Google Scholar
- Dissanayake, C., & Macintosh, K. (2003). Mind reading and social functioning in children with autistic disorder, Asperger’s disorder. In B. Repacholi & V. Slaughter (Eds.), Individual differences in theory of mind. Implications for typical and atypical development (pp. 213–240). Hove: Psychology Press.Google Scholar
- Dyer, J. R., Shatz, M., & Wellman, H. M. (2000). Young children’s storybooks as a source of mental state information. Cognitive Development, 15, 17–37.Google Scholar
- Feshbach, N. D. (1982). Sex differences in empathy, social behavior in children. In N. Eisenberg (Ed.), The development of prosocial behavior (pp. 315–338). New York: Academic Press.Google Scholar
- Flavell, J. H., Miller, P. H., & Miller, S. (1993). Cognitive development (pp. 100–116). Englewood Cliffs, NJ: Prentice Hall.Google Scholar
- Fodor, J. A. (1983). The modularity of mind. Cambridge, MA: MIT Press.Google Scholar
- Frith, U., Morton, J., & Leslie, A. (1991). The cognitive basis of a biological disorder: Autism. Trends in Neuroscience, 14, 433–438.Google Scholar
- Frith, U., Happé, F., & Siddons, F. (1994). Autism and theory of mind in everyday life. Social development, 3(2), 108–124.Google Scholar
- Gallese, V. (2007). Before and below ‘theory of mind’: Embodied simulation and the neural correlates of social cognition. Philosophical Transactions of the Royal Society of London. Series B, Biological sciences, 1480(362), 659–669.Google Scholar
- Garfield, J. L., Peterson, C. C., & Perry, T. (2001). Social cognition, language acquisition and the development of the theory of mind. Mind and Language, 16(5), 494–541.Google Scholar
- German, T. P., & Leslie, A. M. (2000). Attending to and learning about mental states. In P. Mitchell & K. J. Riggs (Eds.), Children’s reasoning and the mind (pp. 229–252). Hove: Psychology Press LTD.Google Scholar
- Gopnik, A. (1993). How we know our own minds: The illusion of first-person knowledge of intentionality. Brain and Behavioral Sciences, 16, 1–14.Google Scholar
- Gopnik, A. (2003). The theory theory as an alternative to the innateness hypothesis. In L. Antony & N. Hornstein (Eds.), Chomsky and his critics. New York: Basil Blackwell.Google Scholar
- Gopnik, A., & Wellman, H. M. (1992). Why the child’s theory of mind really is a theory. Mind and Language, 7, 145–171.Google Scholar
- Gordon, R. M. (1992). The simulation theory: Objections and misconceptions. Mind and language, 7, 11–34.Google Scholar
- Gordon, R. M. (1996). ‘Radical’ simulationism. In P. Carruthers & P. K. Smith (Eds.), Theories of theories of mind (pp. 11–21). Cambridge University Press.Google Scholar
- Grigorenko, E. L., & Sternberg, R. J. (1998). Dynamic testing. Psychological Bulletin, 124, 75–111.Google Scholar
- Hadwin, J., & Perner, J. (1991). Pleased and surprised: Children’s cognitive theory of emotion. British Journal of Developmental Psychology, 9, 215–234.Google Scholar
- Hala, S., & Carpendale, J. (1997). All in the mind: Children’s understanding of mental life. In S. Hala (Ed.), The development of social cognition (pp. 189–239). Hove: Psychology Press.Google Scholar
- Härdle, W. (1991). Smoothing techniques (with implementation in S). New York: Springer Verlag.Google Scholar
- Harris, P. L. (1992). From simulation to folk psychology: The case for development. Mind and Language, 7, 120–144.Google Scholar
- Hartman, C. A., Luteijn, E. F., Moorlag, H., De Bildt, A., & Minderaa, R. B. (2007). VISK: Vragenlijst voor Inventarisatie van Sociaal Gedrag van Kinderen, herziene handleiding 2007. [CSBQ: The Children’s Social Behavior Questionnaire, revised manual 2007]. Amsterdam: Harcourt test publishers.Google Scholar
- Hill, E. L., & Frith, U. (2004). Understanding autism: Insights from mind and brain. In U. Frith & E. Hill (Eds.), Autism: Mind and brain. New York: Oxford University Press.Google Scholar
- Hoogewys, E. M. A., van Geert, P. L. C., & Serra, M. (1999). Dutch translation of the Vineland adaptive behavior scales: Supplementary items. Groningen: Rijksuniversiteit Groningen.Google Scholar
- Hughes, C., Deater-Deckard, K., & Cutting, A. L. (1999). ‘Speak roughly to your little boy’? Sex differences in the relations between parenting and preschoolers’ understanding of mind. Social Development, 8(2), 143–160.Google Scholar
- Kaiser, H. F. (1960). The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20, 141–151.Google Scholar
- Kiers, H. A. L., & Ten Berge, J. M. F. (1994). Hierarchical relations between methods for simultaneous component analysis and a technique for rotation to a simple simultaneous structure. British Journal of Mathematical and Statistical Psychology, 47, 109–126.Google Scholar
- Kraijer, D. (2004). Autisme spectrumstoornissen en verstandelijke beperking [Autism spectrum disorders and mental retardation] (pp. 154–158). Amsterdam: Harcourt.Google Scholar
- Leekam, S. (1993). Children’s understanding of mind. In M. Bennet (Ed.), The child as psychologist. An introduction of social cognition (pp. 26–61). Hemel Hempstead: Harvester Wheatsheaf.Google Scholar
- Leslie, A. (1987). Pretence and representation: The origins of ‘theory of mind’. Psychological Review, 94, 412–426.Google Scholar
- Leslie, A. M. (1992). Pretense, autism, and the “theory of mind” module. Current Directions in Psychological Science, 1, 18–21.Google Scholar
- Leslie, A. M. (2000). How to acquire a ‘representational theory of mind’. In D. Sperber (Ed.), Metarepresentations: A multidisciplinary perspective (pp. 197–223). Oxford, UK: Oxford University Press.Google Scholar
- Leslie, A. M., & Frith, U. (1988). Autistic children’s understanding of seeing, knowing and believing. British Journal of Developmental Psychology, 6, 315–324.Google Scholar
- Luteijn, E. F., Minderaa, R. B., & Jackson, A. E. (2002). Vragenlijst voor Inventarisatie van Sociaal Gedrag bij Kinderen, Handleiding [The Children’s Social Behavior Questionnaire, Manual]. Lisse: Swets & Zeitlinger.Google Scholar
- Mitchell, P. (1997). Introduction to theory of mind. Children, autism and apes. Londen: Arnold.Google Scholar
- Muris, P., Steerneman, P., Meesters, C., Merckelbach, H., Horselenberg, R., van den Hogen, T., & van Dongen, L. (1999). The Tom-Test: A new instrument for assessing theory of mind in normal children and children with pervasive developmental disorders. Journal of Autism and Developmental Disorders, 29(1), 67–80.PubMedGoogle Scholar
- Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.Google Scholar
- Paul, R., Miles, S., Cicchetti, D., Sparrow, S., Klin, A., Volkmar, F., Coflin, M., & Booker, S. (2004). Adaptive behaviour in autism and pervasive developmental disorder—not otherwise specified: Microanalysis of scores on the Vineland adaptive behaviour scales. Journal of Autism and Developmental Disorders, 34(2), 223–228.PubMedGoogle Scholar
- Perner, J. (1988). Developing semantics for theories of mind: From propositional attitudes to mental representation. In J. Astington, P. L. Harris, & D. Olson (Eds.), Developing theory of mind (pp. 141–172). Cambridge: Cambridge University Press.Google Scholar
- Perner, J. (1991). Understanding the representational mind. Cambridge, MA: MIT Press.Google Scholar
- Perner, J. (1993). The theory of mind deficit in autism: Rethinking the metarepresentational theory. In S. Baron-Cohen, H. Tager-Flusberg, & D. Cohen (Eds.), Understanding other minds: Perspectives from autism (pp. 112–137). Oxford: Oxford University Press.Google Scholar
- Perner, J., Leekam, S., & Wimmer, H. (1987). Three-year-old’s difficulty with false belief: The case for conceptual deficit. British Journal of Developmental Psychology, 5, 125–137.Google Scholar
- Perner, J., Frith, U., Leslie, A. M., & Leekam, S. R. (1989). Exploration of the autistic child’s theory of mind: Knowledge, belief, and communication. Child Development, 60, 689–700.Google Scholar
- Piaget, J. (1929). The child’s conception of the world. London: Routledge and Kegan Paul.Google Scholar
- Pons, F., & Harris, P. (2000). Test of emotion comprehension—TEC. Oxford: University of Oxford.Google Scholar
- Repascholi, B. M., & Gopnik, A. (1997). Early reasoning about desires: Evidence from 14–18-months-olds. Developmental Psychology, 33, 12–21.Google Scholar
- Rieffe, C. (1998). The child’s theory of mind: Understanding desires, beliefs and emotions. Doctoral Dissertation: Free University Amsterdam.Google Scholar
- Ruffman, T., Slade, L., Rowlandson, K., Rumsey, C., & Garnham, A. (2003). How language relates to belief, desire, and emotion understanding. Cognitive Development, 18, 139–158.Google Scholar
- Russel, J. (2005). Justifying all the fuss about false belief. Trends in Cognitive Sciences, 9(7), 307–308.Google Scholar
- Selman, R. L. (1980). The growth of interpersonal understanding: Developmental and clinical analyses. New York: Academic Press.Google Scholar
- Simonoff, J. S. (1996). Smoothing methods in statistics. New York: Springer Verlag.Google Scholar
- Snijders, J. Th., Tellegen, P. J., & Laros, J. A. (1988). Snijders-Oomen Niet-verbale Intelligentietest SON-R 5½-17. Verantwoording en handleiding. [Snijders-Oomen Non verbal Intelligence Test SON-R 5½-17. Justification and manual.] Groningen: Wolters-Noordhoff.Google Scholar
- Sparrow, S. S., Balla, D. A., & Ciccheti, D. V. (1984). Vineland adaptive behaviour scales. Minnesota: American Guidance Services. Dutch version: Research group Developmental Disorders, State University Leiden, 1995 (expanded form).Google Scholar
- Steele, S., Joseph, R. M., & Tager-Flusberg, H. (2003). Brief report: Developmental change in theory of mind abilities in children with autism. Journal of Autism and Developmental Disorders, 33, 4.Google Scholar
- Steerneman, P., Meesters, C., & Muris, P. (2002). Tom-Test. Leuven-Apeldoorn: Garant.Google Scholar
- Stueber, K. R. (2006). Rediscovering empathy. Agency, folk psychology, and the human sciences. Cambridge, Massachusetts: The MIT Press.Google Scholar
- Systat. (2000). TableCurve (version 5.0 for Windows) [Computer software]. Mapleton (OR): AISN Software Inc.Google Scholar
- Tager-Flusberg, H. (2000). Language and understanding minds: Connections in autism. In S. Baron-Cohen, H. Tager-Flusberg, & D. J. Cohen (Eds.), Understanding other minds: Perspectives from developmental cognitive neuroscience (2nd ed., pp. 124–149). Oxford: Oxford University Press.Google Scholar
- Tager-Flusberg, H. (2003). Exploring the relationship between theory of mind and social-communicative functioning in children with autism. In B. Repacholi & V. Slaughter (Eds.), Individual differences in theory of mind. Implications for typical and atypical development (pp. 197–212). Hove: Psychology Press.Google Scholar
- Tellegen, P. J., Winkel, M., & Wijnberg-Williams, B. J. (1998). SON 2½-7. Handleiding. Lisse: Swets & Zeitlinger BV.Google Scholar
- Tellegen, P., Winkel, M., Wijnberg-Williams, B., & Laros, J. (2003). Snijders-Oomen Niet-verbale Intelligentietest. SON-R 2½-7. Handleiding en verantwoording [Snijders-Oomen Non-verbal Intelligence Test. SON-R 2½-7. Manual and justification]. Göttingen: Hogrefe Verlag.Google Scholar
- Van Bon, W. H. J. (1982). Taaltests voor kinderen [Language Test for children]. Lisse: Swets & Zeitlinger.Google Scholar
- Van Eldik, M. C. M., Schlichting, J. E. P. T., Lutje Spelberg, H. C., Van der Meulen, S. J., & Van der Meulen, B. F. (1997). Handleiding Reynell Test voor Taalbegrip (2e dr.) [Manual for the Reynell Language Apprehension Test]. Nijmegen: Berkhout/Lisse: Swets & Zeitlinger.Google Scholar
- Verhelst, N. D., Glas, C. A. W., & Verstralen, H. H. F. M. (1995). One-parameter logistic model OPLM. Arnhem: Cito.Google Scholar
- Wechsler, D. (1974). Wechsler intelligence scale for children-revised. New York: Psychological Corporation.Google Scholar
- Wechsler, D. (1981). Wechsler adult intelligence scale-revised. New York: The Psychological Corporation.Google Scholar
- Wellman, H. M. (1990). The child’s theory of mind. Cambridge, MA: MIT Press.Google Scholar
- Wellman, H., & Lagattuta, K. H. (2000). Developing understandings of mind. In S. Baron-Cohen, H. Tager-Flusberg, & D. J. Cohen (Eds.), Understanding other minds: Perspectives from developmental cognitive neuroscience (2nd ed., pp. 21–49). Oxford: Oxford University Press.Google Scholar
- Yeargin-Allsopp, M, Rice, C., Karapurkar, T., Doernberg, N., Boyle, C., & Murphy, C. (2003). Prevalensce of autism in a US metropolitan area. Journal of American Medical Association, 289(1) 49–55.Google Scholar
- Yuill, N. (1984). Young children’s coordination of motive and outcome in judgments of satisfaction and morality. British Journal of Developmental Psychology, 2, 73–81.Google Scholar