Journal of Autism and Developmental Disorders

, Volume 42, Issue 1, pp 23–34 | Cite as

Examination of the Properties of the Modified Checklist for Autism in Toddlers (M-CHAT) in a Population Sample

  • Brie Yama
  • Tom Freeman
  • Erin Graves
  • Su Yuan
  • M. Karen Campbell
Original Paper

Abstract

This study examines the following properties of the Modified Checklist for Autism in Toddlers (M-CHAT) in an unselected low-risk sample: (a) the maximum age for screen administration; (b) the positive screen rate in the absence of follow-up telephone interviews and; (c) the distributional properties of positive screens. Data came from a prospective cohort study (n = 1,604). Results suggest that the M-CHAT can appropriately be administered to children aged 20–48 months. Documented explanations provided by mothers during screening, appear to effectively identify potential screen misclassifications in the absence of the follow-up telephone interviews. This further emphasizes the importance of clinician expertise in verifying positive M-CHAT screens. Results have implications for the administration of the M-CHAT in clinical and research settings.

Keywords

Autism spectrum disorders Autism Modified checklist for autism in toddlers M-CHAT Toddlers Developmental screening 

Introduction

Autism Spectrum Disorders (ASDs) are a group of neurological disorders characterized by core deficits in 3 domains: social interaction, communication, and repetitive or stereotypic behaviours (American Psychiatric Association 2000; Newschaffer et al. 2007; World Health Organization 2006). The symptoms characterizing this disorder usually first appear during infancy or early childhood, with the trajectory of the disorder commonly following a steady course without remission (Baird et al. 2001, 2003; Filipek et al. 1999, 2000; Johnson 2008; Robins 2002; World Health Organization 2006). Epidemiological evidence estimates the current prevalence of ASDs to be 6 cases per 1,000 individuals, (Fombonne 2009; Newschaffer et al. 2007). The Standing Senate Committee on Social Affairs Science and Technology (2007) and the most recent estimates from the Centers for Disease Control and Prevention (2009) estimate prevalence as 1 in 110, making ASDs the most common serious childhood neurodevelopmental disorder. Given this, substantial interest in developing screening instruments and implementing screening programs exists.

A screening device is a quick means of identifying a condition or its associated risk factors; it identifies individuals for whom diagnostic testing for confirmation of a disease or a disorder is appropriate (Fletcher and Fletcher 2005; Oleckno 2002). With respect to screening for ASDs, there are currently 2 categories of instruments: broadband developmental instruments and ASD-specific screening devices (American Academy of Pediatrics 2006; Johnson and Myers 2007). While the value of broadband screens for developmental screening in the general population is established, a growing body of literature suggests that these screens may not be able to effectively differentiate children with ASDs from those with other developmental disorders (Glascoe et al. 2007; Johnson and Myers 2007; Pinto-Martin et al. 2008). This therefore suggests the importance of ASD-specific screens.

Although many ASD-specific screens exist (see reviews in Mawle and Griffiths 2006; American Academy of Pediatrics 2006; Johnson and Myers 2007), the Modified Checklist for Autism in Toddlers (M-CHAT) is among the most accessible (American Academy of Pediatrics 2006; Johnson and Myers 2007; Mawle and Griffiths 2006). It was developed by Robins et al. (2001) and is specifically designed to target the early detection of ASDs in toddlers through parental response to the 23 questions that comprise the screen (Dumont-Mathieu and Fein 2005; Robins et al. 2001). Repeated applications of the M-CHAT emphasize its promise and current status as one of the most respected instruments available for early ASDs screening (Snow and Lecavalier 2008).

The M-CHAT was originally tested on 1,293 children with approximately 10% of the sample screening positive based solely on the parental questionnaire (Robins and Dumont-Mathieu 2006; Robins et al. 2001). If a positive M-CHAT screen was obtained, parents were re-contacted via telephone and their responses were subsequently verified to reduce the occurrence of false-positives (Robins et al. 2001). The original positive predictive value (PPV) of the M-CHAT has been reported as 0.59–0.80, with variation accounted for by differences in: the populations screened, the application of the follow-up telephone interview, and the definition of what constitutes a positive screen (Snow and Lecavalier 2008; Eaves et al. 2006; Kleinman et al. 2008; Robins 2008). Positive predictive value is dependent upon the population prevalence of ASD. In addition to studies examining the validity of the M-CHAT, the screen has commanded the interest of other investigators studying ASDs that have elected to use the M-CHAT as a definitive outcome (Kuban et al. 2009; Limperopoulos et al. 2008).

As applications of the M-CHAT have become more widespread in the literature, additional variations have arisen in the conditions under which the screen is administered. In particular, the age-range to which the M-CHAT is being applied appears to be increasingly variable, ranging from 16 to 48 months despite the fact that the screen was originally developed using children between the ages of 18–25 months (non-selected population) and 18–30 months (high risk sample) (Robins et al. 2001). Additionally, while the follow-up telephone interview has been applied in many studies (Robins et al. 2001; Kleinman et al. 2008; Pandey et al. 2008; Robins 2008) its use is not unanimous (Snow and Lecavalier 2008; Limperopoulos et al. 2008). Despite these observations, to date, studies examining the properties of the M-CHAT have primarily focused on establishing the validity of the screen (Kleinman et al. 2008; Robins and Dumont-Mathieu 2006; Snow and Lecavalier 2008; Eaves et al. 2006).

Acknowledging the various applications and inconsistent administration of the M-CHAT in the literature, the purpose of the present study was to examine the performance of the M-CHAT when administered via survey methods in a population-based cohort with a view to its use as a measure in subsequent epidemiologic analyses. In order to do this, we specifically set out to: examine positive screen rates, in the absence of the M-CHAT Follow-up telephone Interview; to determine the general distributional properties of positive M-CHAT screens; and to compare these to published literature as well as to perform comparisons across age groups. The latter specific objective aimed to draw inferences regarding the age groups for which data collected in the manner may be applicable, both for further analysis in this data set or for future studies.

Methods

Sample and Data Source

All data came from the Prenatal Health Project (PHP), a longitudinal prospective cohort study initiated to study the determinants and outcomes of preterm birth. Detailed recruitment and data collection methods have been reported elsewhere (Sontrop et al. 2007, 2008). Briefly, 2,357 participants were enrolled in the PHP between 10 and 22 weeks gestation. Recruitment took place over a three year interval (January 30, 2002–November 8, 2004) at seven ultrasound clinics located throughout London, Ontario, Canada. These seven clinics perform the vast majority of prenatal ultrasounds in the city (the other three clinics are smaller satellites of the main clinics). Enrolment was unequal across the three years with slow enrolment initially and peak enrolment in year two. At the time of prenatal recruitment, subjects gave consent for abstraction of data from their medical records as well as consent for ongoing contact from the study team. Subjects provided their contact information as well as a secondary contact of a friend/relative who could be contacted if they moved.

Data were collected through telephone interviews (prenatal, n = 2,357; childhood, n = 1,604) and data extraction from hospital medical records (perinatal, n = 2,357). The childhood interviews took place over the interval of January 2006–May 2008. Attempts were made to contact all 2,357 participants; up to 10 calls were attempted to each subject. If lines were disconnected, tracing was attempted via the secondary contact. When contacted, parents were reminded of the PHP study and the caller ensured that the parent knew the child to which the study pertained (in the case of multiparous women). Children of participants ranged in age from 20 to 67 months at time of participation in the childhood interview survey. The analytic sample for this study included 1,604 participants who successfully completed all phases of data collection.

Ethics approval was obtained from the University of Western Ontario Review Board for Health Sciences Research Involving Human Subjects.

Measures

Many demographic, social and biological variables were collected in the PHP. The variables selected for analysis in this study are described below.

Participants were asked about their current marital status (married, common law, single or never married, separated or divorced, or widowed) and their parity (number of prior live births) at the time of the interview. Familial income was self-reported with 17 response options to sort participants into fine income categories. Following univariate analysis of the data, income was regrouped into the following categories: ≤$80,000 and >$80,000. Maternal education was self-reported according to a question with 8 response options and subsequently categorized as <high school, high school, or >high school. The Center for Epidemiologic Studies Depression Scale (CES-D) was administered to all participants to measure depression symptomatology (Radloff 1977). Potential scores ranged from 0 to 60 with a score ≥16 indicative of depressive symptomatology (Radloff 1977). Participants were classified as having a CESD <16 or >16 to indicate depressive symptomatology. Maternal age was self-reported and verified via data extracted from child birth hospital records.

Modified Checklist for Autism in Toddlers (M-CHAT) performance was the primary outcome in the present study. The M-CHAT is a simple 23 question screening tool developed by Robins et al. (1999) that targets the early detection of ASDs in toddlers. The tool relies on parental report of current skills and behaviours of the child (Robins et al. 2001). A positive M-CHAT screen was designated if either of the following criteria was met: responses to any 2 of the 6 critical questions reflected atypical patterns of development (Definition 1 Positive Screen) or responses to any 3 of the 23 questions reflected atypical development (Definition 2 Positive Screen) (Dumont-Mathieu and Fein 2005; Robins et al. 2001). The M-CHAT was administered within the telephone interview along with other health scales. The M-CHAT Follow-up Interview was not conducted for positive screens. Instead, qualitative notes recorded at the time the M-CHAT was administered were examined to verify positive screens. At the outset of the interview, participants were encouraged to seek clarification and/or provide elaboration regarding any of the questions asked throughout the interview. Accordingly, qualitative notes were recorded by interviewers if there was parental elaboration in response to specific M-CHAT questions, or in response to other portions of the survey, during the telephone interview.

Statistics and Analyses

Characteristic baseline differences between participants who completed all phases of data collection versus those who did not were assessed via chi-square analyses.

To identify the maximum age at which the M-CHAT scores could appropriately be utilized we first examined the frequency of negative and positive screens by age. Subsequently, we proceeded to look at the percentage of positive screens by age for each of the 23 questions which comprise the M-CHAT screen. Analyses were conducted within the following age categories: 20–32, 33–48 and >48 months. The lower and upper limits for assessment were based on the range of ages of the children participating in the study from which the data were secondarily selected. The cut-point of 32 months was selected after examining the maximum age of children screened in previous studies, for which the majority was between 30 and 32 months (Charman et al. 2001; Eaves et al. 2006; Robins 2002; Robins and Dumont-Mathieu 2006). Thus this grouping will allow for comparison to prevalence reported by these authors. The upper age range category (>48 months) was determined based on two criteria. First, after considering the maximum age for screening in all previous studies it would appear that 48 months is the maximum age to which the M-CHAT has previously been applied (Eaves et al. 2006). However, due to the convenience nature of our sample, we have data on subjects up to 67 months of age. Further, data from a multisite US prevalence study suggested that the median age in which children receive an ASD diagnosis was between 52 and 56 months (Rice 2007). Finally, it should be noted that screening at ages >48 months is of interest for detection of children who have late regression. Therefore, it was of interest to us to examine data in this age range.

To examine positive screen rates in the absence of the follow-up telephone interview, the qualitative notes referencing parental elaboration at the time of M-CHAT screening were scrutinized in detail by research and physician experts for each child who screened positive on the M-CHAT. Extraneous circumstances linked to positive screens were identified and documented as presumed false-positives. The reclassification of screening performance status (i.e. from positive to negative) was applied to the analytic sample. After accounting for the results of the above analyses a univariate analysis was conducted to investigate the distributional properties of M-CHAT screening performance.

All analyses were conducted using SAS version 9.1 software (SAS 2000).

Results

Characterization of the PHP Cohort According to Baseline Familial and Maternal Factors

Table 1 shows the characteristic baseline differences between members of the source cohort who completed the childhood interview of the PHP study versus those who did not complete this phase.
Table 1

Comparison of characteristics of the prenatal and perinatal variable frequencies for participants who completed versus did not complete the childhood interview

Variable

Participants who completed the first childhood interview (%) (n = 1,604)

Participants who did not complete the first childhood interview (%) (n = 754)

p-value (chi-square tests)

Maternal age at delivery

≤21 years

241 (32.01%)

599 (37.37%)

0.0113

>21 years

512 (67.99%)

1,004 (62.63%)

Missing

1

0

Maternal marital status

Married

1,300 (81.05%)

502 (66.76%)

<0.0001

Common law

209 (13.03%)

159 (21.14%)

Single or never married

81 (5.05%)

74 (9.84%)

Separated or divorced

14 (0.87%)

17 (2.26%)

Missing

0

1

Income level

<30,000

126 (8.19%)

142 (20.52%)

<0.0001

30,000 to 80,000

773 (50.26%)

345 (49.86%)

>80,000

639 (41.55%)

205 (29.62%)

Missing

66

61

Maternal education

<High school

46 (2.88%)

77 (10.28%)

<0.0001

High school

156 (9.75%)

203 (16.82%)

>High school

1,398 (87.38%)

749 (72.90%)

Missing

4

4

Maternal depression

CES-D score <16

1,384 (86.77%)

572 (76.78%)

<0.0001

CES-D score >16

211 (13.23%)

173 (23.22%)

Missing

9

8

Parity (prior livebirths)

0

599 (37.37%)

241 (32.01%)

0.0113

≥1

1,004 (62.63%)

512 (67.99%)

Missing

1

0

Compared to those who completed the childhood interview, participants who did not complete this phase were significantly younger at delivery, less likely to be married or living as common law, less likely to earn >$80,000 per year, less likely to have a post-secondary education, more likely to exhibit depression symtomatology, and more likely to have at least 1 previous pregnancy.

M-CHAT Performance and Age

M-CHAT performance according to child age group is presented in Table 2.
Table 2

M-CHAT performance according to age groups (n = 1,604)

Age range

M-CHAT performance

Total interviewed*

PHP study subjects**

Negative screen frequency (%)

Positive screen frequency (%)

20–32 months

591 (97.20%)

17 (2.80%)

608

848

33–48 months

908 (94.39%)

54 (5.61%)

962

1,387

>48 months

30 (88.24%)

4 (11.76%)

34

122

Total

1,529

75

1,604

2,357

* Interviewed subjects are categorized by age at interview

** PHP subjects (potentially available for interview) are categorized by age at the mid-point of follow-up (March 2007)

The proportion of children with positive M-CHAT screens increases with age. Specifically, 2.80% (n = 17) of the children aged 20–32 months screened positive on the M-CHAT; 5.61% (n = 54) of the children aged 33–48 months screened positive on the M-CHAT; and 11.76% (n = 4) of the children older than 48 months screened positive on the M-CHAT (Table 2).

The percentage of responses reflecting atypical development for each M-CHAT question varies with age (Table 3).
Table 3

Positive screen frequencies according to age category for each M-CHAT question

M-CHAT Question (response reflecting typical development)

# of Responses reflecting atypical development (% within each age group)

Overall frequency of responses reflecting atypical development (% in cohort) (n = 1,604)

20–32 months (n = 608)

33–48 months (n = 962)

>48 months (n = 33)

1

Does your child enjoy being swung, bounced on your knee, etc.? (yes)

14 (2.30%)

39 (4.05%)

4 (11.76%)

57 (3.55%)

♦2

Does your child taken an interest in other children? (yes)

6 (0.99%)

14 (1.46%)

0 (0.00%)

20 (1.25%)

3

Does your child like climbing on things, such as upstairs? (yes)

8 (1.32%)

19 (1.98%)

1 (2.94%)

28 (1.75%)

4

Does your child enjoy playing peek-a-boo/hide-and-seek? (yes)

5 (0.82%)

11 (1.14%)

0 (0.00%)

16 (1.00%)

5

Does your child ever pretend, for example, to talk on the phone or take care of dolls, or pretend other things? (yes)

4 (0.66%)

10 (1.04%)

0 (0.00%)

14 (0.87%)

6

Does your child ever use his/her index finger to point, to ask for something? (yes)

29 (4.77%)

61 (6.34%)

2 (5.88%)

92 (5.74%)

♦7

Does your child ever use his/her index finger to point, to indicate interest in something? (yes)

25 (4.11%)

53 (5.51%)

1 (2.94%)

79 (4.93%)

8

Can your child play properly with small toys (e.g. cars or bricks) without just mouthing, fiddling or dropping them? (yes)

4 (0.66%)

1 (0.10%)

0 (0.00%)

5 (0.31%)

♦9

Does your child ever bring objects to you (parent) to show you something? (yes)

3 (0.49%)

2 (0.21%)

0 (0.00%)

5 (0.31%)

10

Does your child look you in the eye for more than a second or two? (yes)

6 (0.99%)

9 (0.94%)

1 (2.94%)

16 (1.00%)

11

Does your child ever seem oversensitive to noise? (no)

97 (15.95%)

225 (23.39%)

15 (44.12%)

337 (21.016%)

12

Does your child smile in response to your face or your smile? (yes)

2 (0.33%)

5 (0.52%)

0 (0.00%)

7 (0.44%)

♦13

Does your child imitate you? (yes)

8 (1.32%)

22 (2.29%)

2 (5.88%)

32 (2.00%)

♦14

Does your child respond to his/her name when you call? (yes)

0 (0.00%)

3 (0.31%)

0 (0.00%)

3 (0.19%)

♦15

If you point at a toy across the room, does your child look at it? (yes)

5 (0.82%)

5 (0.52%)

0 (0.00%)

10 (0.62%)

16

Does your child walk? (yes)

1 (0.16%)

2 (0.21%)

0 (0.00%)

3 (0.19%)

17

Does your child look at things you are looking at? (yes)

3 (0.49%)

3 (0.31%)

0 (0.00%)

6 (0.37%)

18

Does your child make unusual finger movements near his/her face? (no)

20 (3.29%)

51 (5.30%)

5 (14.71%)

76 (4.74%)

19

Does your child try to attract your attention to his/her own activity? (yes)

10 (1.64%)

30 (3.12%)

0 (0.00%)

40 (2.49%)

20

Have you ever wondered if your child is deaf? (no)

12 (1.97%)

30 (3.12%)

1 (2.94%)

43 (2.68%)

21

Does your child understand what people say? (yes)

6 (0.99%)

3 (0.31%)

0 (0.00%)

9 (0.56%)

22

Does your child sometimes stare at nothing or wander with no purpose? (no)

47 (7.73%)

73 (7.59%)

2 (5.88%)

122 (7.61%)

23

Does your child look at your face to check your reaction when faced with something unfamiliar? (yes)

20 (3.29%)

41 (4.26%)

4 (11.76%)

65 (4.05%)

Total Number of Positive M-CHAT Screens (% of total positive screens)

17 (22.67%)

54 (72.00%)

4 (5.33%)

75

♦ Critical response items (Definition 1 Positive Screen: Responses to any 2 of the 6 critical questions reflect atypical development)

Compared to overall percentages for the cohort, the percentage of responses reflecting atypical development is elevated in the >48 months age group for the following questions: (1) Does your child enjoy being swung, bounced on your knee, etc.?; (11) Does your child ever seem oversensitive to noise?; (13) Does your child imitate you?; (18) Does your child make any unusual finger movements near his/her face?; and (23) Does your child look at your face to check your reaction when faced with something unfamiliar? (Table 3). Within children who screened positive on the M-CHAT, the distribution of positive M-CHAT screens according to age group is as follows: children aged 33–48 months have the highest proportion of positive M-CHATs (54/75 or 72.00%), while children >48 months have the lowest proportion of positive M-CHATs (4/75 or 5.33%), with children aged 20–32 months lying in between (17/75 or 22.67%) (Table 3).

Reclassification of Presumed False-Positive M-CHAT Screens

The sample was examined and reasonable criteria were applied to permit the reclassification of presumed false-positive M-CHAT screens (Table 4).
Table 4

Identification and rationale for reclassification of presumed false-positive cases (i.e. children who screened positive but likely should not have)

Age (months)

M-CHAT questions responses reflecting atypical development

Qualitative notes

Rationale for reclassification

35

(6) Point index finger to ask for something

(7) Point index finger to indicate interest in something

(11) Oversensitive to noise

(6) uses words

(7) uses words

Qualitative notes indicate the child is using words to ask for and/or indicate an interest in something, rather than pointing

If we interpret the notes to indicate that the child is naming or speaking to indicate interest or ask for an object, then 6, 7 are eliminated, and the child screens negative

34

(1) Enjoys being swung/bounced

(19) Child attracts parent’s attention to own activity

(20) Deafness suspected

(1) can’t do it because of a broken collar bone

Qualitative notes suggest the child’s ability does not truly reflect atypical development for question 1

If we interpret the notes to indicate that the child’s ability was modified due to injury, the child would consequently screen negative had he/she not had a broken collar bone

44

(6) Point index finger to ask for something

(7) Point index finger to indicate interest in something

(13) Child imitates parent

(6) verbal

(7) verbal

Qualitative notes indicate the child is using words to ask for and/or indicate an interest in something, rather than pointing

If we interpret the notes to indicate that the child is naming or speaking to indicate interest or ask for an object, then 6, 7 are eliminated, and the child screens negative

32

(4) Child enjoy peek-a-boo/hide-and-seek

(6) Point index finger to ask for something

(7) Point index finger to indicate interest in something

(10) Makes eye contact

(11) Oversensitive to noise

(15) Parent points, child looks

(23) Check parent’s reaction when faced with unfamiliar

Note: he is blind so results reflect this

The fact that the child is blind would prevent him from being able to perform most of the questions

30

(1) Enjoys being swung/bounced

(6) Point index finger to ask for something

(13) Child imitates parent

(6) verbal

Based on the previous decisions that if the child can verbalize a request and get a response, they won’t need to point

37

(6) Point index finger to ask for something

(11) Oversensitive to noise

(22) Stare and/or wander

(6) words, not any more

Based on the previous decisions that if the child can verbalize a request and get a response, they won’t need to point

A presumed-false positive was defined as a positive screen that occurred in the presence of qualitative data that strongly suggested an alternate explanation existed for the atypical response reported by the participant. Six cases of presumed false-positive M-CHAT screens were identified by study researchers and physicians. The qualitative notes indicate that the development and abilities of 4 of the children appeared to have exceeded the target range of the screen, prompting participants to provide responses which indicated atypical development even though this was likely not the case. Specifically, in response to the questions ‘does your child ever use his/her index finger to point to ask for something?’ and ‘does your child ever use his/her index finger to point, to indicate interest in something?’, participants indicated that their child was ‘using words’ instead of pointing to indicate interest in objects. This presumably caused these 4 children to screen positive on the M-CHAT, therefore, consensus among physicians and researchers led to the decision to reclassify these positive screens. In the other 2 probable misclassification cases, participants’ positive responses might be attributed to physical constraints of the children (childhood blindness and broken collar bone), which again presumably resulted in the positive M-CHAT screens.

Distribution of M-CHAT Scores

The sample used to examine the distribution of M-CHAT scores excluded children outside the applicable age range (20–48 months) and accounts for the reclassification of presumed false-positives (Table 5).
Table 5

Distribution of M-CHAT scores after restricting the age range and accounting for the reclassification of presumed false-positive M-CHAT screens

 

Total number of M-CHAT questions reflecting atypical development

Frequency

Percent (%)

 
 

0

938

59.75

 
 

1

396

25.22

 
 

2

172

10.96

 
 

3

30

1.91

 
 

4

20

1.27

 
 

5

4

0.25

 
 

6

3

0.19

 
 

7

1

0.06

 
 

8

2

0.13

 
 

9

0

0.00

 
 

10

1

0.06

 
 

11

1

0.06

 
 

12

1

0.06

 
 

13

0

0.00

 
 

14

0

0.00

 
 

15

0

0.00

 
 

16

0

0.00

 
 

17

0

0.00

 
 

18

0

0.00

 
 

19

0

0.00

 
 

20

1

0.06

 
 

21

0

0.00

 
 

22

0

0.00

 
 

23

0

0.00

 
 

Total

1,570

100.00

 

In total, 95.93% of the cohort had negative screening findings. Among this group 59.75% provided responses that reflected typical development for all 23 questions, 25.22% had a single response that reflected atypical development and 10.96% provided 2 responses that reflected atypical development (Table 5). Of the 65 children with positive M-CHAT screens, only 1 child screened positive exclusively according to a Definition 1 Positive Screen (responses to any 2 of the 6 critical questions reflect atypical development), while 64 of the children screened positive according to a Definition 2 Positive Screen (responses to any 3 of the 23 questions reflect atypical development) (Table 5). The majority of M-CHAT screen scores were clustered around the cut-off score of 3 (Definition 2 Positive Screen: responses to any 3 of the 23 questions reflect atypical development) (Table 5).

Discussion

This is the first study to blindly administer the M-CHAT in a research study involving a general population sample of children reflecting such a wide range of ages (20–67 months). Findings suggest that the M-CHAT should not be administered to children beyond the age of 48 months as results suggested that specific questions may not be age relevant. Qualitative explanations offered by parents during screening should also be documented and reviewed by clinical experts in interpreting M-CHAT performance as many positive screens are tightly clustered around the threshold differentiating between a negative versus a positive screen (Table 5).

Interpretation of Findings

The proportion of children with positive M-CHAT screens increased with age category (Table 2). The highest rate of positive screens occurred unexpectedly in the oldest group of children and may be indicative of an inappropriate application of the M-CHAT. Correspondingly, we suggest that some older children may have been more likely to screen positive on the M-CHAT, not because they were at a higher risk for ASD, but because the questions in the screen no longer reflected age relevant abilities. This hypothesis was consequently investigated further by examining responses for each question by age category. Marked variation among questions by age category was demonstrated. In particular, whether or not a child enjoyed being swung or bounced on their mother’s knee may no longer have been relevant to the determinant of ASD for a child who was >48 months (Question 1). Similarly, as a child’s development progressed it may have been natural for the child to imitate his/her mother less (Question 13) and/or to have reacted in ways that did not include looking at his/her mother’s face when faced with the unfamiliar (Question 23). Furthermore, there were negligible variations in the atypical development rates for the questions that inquired about the child’s walking ability (Question 16) and the child’s response to his/her name when called (Question 14). Arguably, the latter 2 questions were widely relevant to children 20 months of age and beyond. Therefore, questions that target transient developmental milestones may have been responsible for the increase in positive screens with age. Based on this, we suggest that the M-CHAT is most appropriate for children between the ages of 20–48 months. Analyses and discussion from this point forward reflect this recommendation.

After restricting the sample to include children between the ages of 20–48 months inclusive, 6 cases were designated for reclassification as they were identified as presumed false-positives (Table 4). The capacity to review and subsequently assess the qualitative notes that were collected at the time of interview was a substantial and novel asset in the present study. In the original conception of the M-CHAT, any participant with a child who screened positive on the M-CHAT received an additional M-CHAT follow-up interview to verify responses and the positive screen result (Robins et al. 1999, 2001). Accordingly, in the initial application of the M-CHAT by Robins et al. (2001), approximately 10% of the cohort screened positively on the M-CHAT screen alone; however, after the M-CHAT follow-up interview, the proportion of positive screens decreased to ~4.5% of the original sample (Robins et al. 2001, 2006). In the present study, after applying the age restriction as determined above, and reclassifying presumed false-positives based on review of qualitative notes, ~4.1% of our study sample had a positive M-CHAT screen. This concurs with the ~4.5% positive screen rate obtained with the M-CHAT Follow-up Interview in the study conducted by Robins et al. (2001). It is our belief that the incorporation of the qualitative notes and physician discretion to overrule positive screens were both important in accurately classifying positive screens in the absence of the follow-up interview. However, we also note that the initial positive screen rate in the present study, prior to the reclassification of the presumed false-positives, was lower than the rates found by Robins (2008) who, in a sample of 4,797 toddlers assessed at well-child visits at ages 16–26.9 months, found failure rates of 9.7% and 3.4% before and after the M-CHAT Follow-up Interview, respectively. In low risk samples of 4,265 children aged 16–23 months and 1,785 children aged 24–30 months, Pandey et al. (2008) report failure rates, respectively, of 6.5% and 7.2% before the M-CHAT Follow-up Interview and rates of 1.0% and 1.6%, respectively, after the M-CHAT Follow-up Interview. Differences in samples sources, unselected population versus clinical population may account for some of the difference.

The majority of the positive screens in this study had scores of 3 or 4, thus clustered around the threshold that differentiates between a positive and negative screening result. Correspondingly, the M-CHAT score distribution had a long tail with few children with the highest and most severe scores; only 10 children (0.6%) had scores of 6 or greater. Extreme cases, collectively, had minimal statistical value. Thus, treatment of M-CHAT scores as positive or negative, rather than as a continuous variable, was statistically sensible. It was also consistent with how the data would be used in epidemiologic studies as an indicator of an “outcome”. Although we were unable to assess the cut-offs which delineate Definition 1 and 2 Positive Screens in the current study, as definitive ASD diagnoses were not obtained, this was previously done by Snow and Lecavalier (2008). Based on the M-CHAT scores distribution, we expect concurrence.

Comparisons with Previous Research

In its conception, the M-CHAT screen was specifically designed to detect early symptoms of ASDs in children (Robins et al. 2001; Robins 2002). Accordingly, Robins et al. (2001) screened children between 18–25 months in an unselected sample and 18–30 months in a high-risk early intervention cohort (Robins et al. 2001). Additional studies administering the M-CHAT to identify children who should be referred for diagnostic evaluations have focused on the following age ranges: Pandey et al. (2008) screened children between the ages of 16–23 and 24–30 months; Robins (2008) screened children between the ages of 16–34 months; Snow and Lecavalier (2008) screened children between 18 and 48 months; and lastly, Eaves et al. (2006) screened children aged 17–48 months. The age range retrospectively determined to be appropriate in the current study concurs with the above. Therefore, even though the M-CHAT was designed with an emphasis on early detection, findings from the current study reflect recent unsubstantiated trends in the literature to suggest that children up to and including 48 months of age can be appropriately screened with this device.

A noteworthy follow-up study by Kleinman et al. (2008) performed initial screens on both high- and low-risk children aged 16–30 months (at Time 1) before proceeding to rescreen many of the children when they were between the ages of 42–54 months (at Time 2). In this study, 15 cases were identified as potential ‘misses’ (i.e. the children screened negative on the initial screen at Time 1, but screened positively at the point of rescreening at Time 2) (Kleinman et al. 2008). Each of these children received diagnostic evaluations, of whom 4 received no diagnoses, ASD or otherwise (Kleinman et al. 2008). Although Kleinman and colleagues did not provide explanations for why these children screened negative at Time 1, but positive at Time 2, it is perhaps possible that these particular cases could reflect children who screened positive at the latter time point because of reasons related to age appropriate behaviours, as was found in the current study.

In addition to its use in identifying children who should be referred for diagnostic evaluations, the M-CHAT has also been used as a definitive measure in and of itself to identify early autistic features in children, in research. Specifically, Limperopoulos et al. (2008) used the M-CHAT to assess early autistic symptoms in children with a history of very low birth weight. Kuban et al. (2009) conducted a similar study to test their hypothesis that children born preterm were more likely to screen positive on the M-CHAT for an ASD. Limperopoulos et al. (2008) administered the M-CHAT to children between 17 and 27 months, while Kuban et al. (2009) did not report the exact age range of children who were screened, instead stating that ‘77% of the participants underwent developmental assessment within 23.5–27.9 months; of the others, about half were assessed before 23.5 months, and the other half were assessed after 27.9 months.’ The findings pertaining to the establishment of age appropriateness of M-CHAT administration in the present study therefore also have implications for future applications of the M-CHAT in research settings.

Throughout the literature, the inclusion of the M-CHAT follow-up interview following a positive M-CHAT screen is sporadic (Kleinman et al. 2008). This represents a principal challenge in comparing specificity and sensitivity values obtained across studies. However, Kleinman and colleagues explicitly note that the ‘telephone interview’ does not need to be completed via phone, but can adequately be administered onsite immediately following the identification of a positive M-CHAT screen.

As this study used data from a secondary source, results from a follow-up telephone interview were not available. However, study findings suggest that the review of documented explanatory responses expressed by parents during M-CHAT administration may be sufficient to clarify responses in this study. The findings associated with this novel application of the M-CHAT may therefore have implications for previous studies that did not include the follow-up telephone interview as well as for future studies.

Strengths and Limitations

The principal strength of this study is that it was conducted within the framework of an ongoing prospective cohort study. Given that ASDs are not typically diagnosed until children are between 4 and 6 years old, many ASD studies are necessarily ‘case–control’ where groups are assembled for study on the basis of having, or not having, ASD. The screening literature in particular, shows the frequent assembly of high-risk groups to screen for ASD. This study is important because the M-CHAT was applied unselectively to each participant in the PHP cohort. Additionally, it uses comprehensive measures of maternal and child demographic, socioeconomic, physical and emotional health, and lifestyle variables. Accordingly, the above strengths of the PHP dataset supported its use in this study.

However, despite these strengths, there are also limitations attached to using secondary data from an ongoing longitudinal study. There is potential for bias in terms of participants who elected to remain in the PHP study. Those who dropped out, or who were lost to follow-up, might reasonably be expected to be younger and less educated and therefore might be anticipated to have a higher rate of “misinterpreted” questions when presented with questions from the M-CHAT (Schaus and Harrington 2010). The age of interview was predetermined and thus the youngest children screened for ASD were 20 months old. As noted above, the literature extensively documents the use of the M-CHAT in children starting at 16 months (Kleinman et al. 2008; Pandey et al. 2008; Robins 2008). Arguably, a substantial limitation is the absence of opportunity to conduct follow-up investigations to determine definitive ASDs diagnoses. As such, a positive screen cannot be inferred to indicate case ascertainment. In regards to the latter, the purpose of the present study was to examine whether the M-CHAT tool could be reasonably delivered by survey in a population-based cohort and not to assess the accuracy of case ascertainment. In fact, the gold standard for this study would be comparison of the survey-based score to the M-CHAT score as assessed using “usual” methodologies. Unfortunately, such data were not available.

The influence of social-desirability on responses is a potential limitation in all screening environments. The M-CHAT screen is dependent on how accurately a mother’s responses reflect her child’s development; however, having said this, the literature does note that parental report is essential to any attempt to administer pediatric screens (Robins 2002). While the potential for social desirably to influence parents’ responses must be acknowledged, we anticipate that this type of bias was minimal as the M-CHAT was administered via a telephone interview. Regarding this, the literature suggests that while telephone interviews retain the benefit of a live interviewer to prompt the participant, they are perceived as less personal than face-to-face interviews (Aday and Cornelius 2006).

Future Research Directions

While the present study supports the applicability of administering the M-CHAT to children between the ages of 20–48 months for screening and research purposes, the literature notably documents the use of the M-CHAT in children younger than 20 months (Kleinman et al. 2008; Pandey et al. 2008; Robins 2008). Future research is warranted to specifically investigate the lower age limit for screening. Additionally, it is of interest to conduct additional follow-up studies with children in the PHP cohort who had positive M-CHAT screens to determine longer-term definitive outcomes. To the best of our knowledge the M-CHAT has only recently been administered to an unselected general population sample in a single study (Robins 2008). Prior to making general recommendations about screening with the M-CHAT, additional studies comprehensively demonstrating its performance in general population samples are needed.

Conclusions

We conclude, based on comparability of prevalence of positive screens, that the M-CHAT can appropriately be administered to children between the ages of 20–48 months using survey methods in an unselected sample. This may be useful clinically, for case-finding, and may also have importance in future epidemiologic studies. Additionally, when used in conjunction with physician discretion, these findings suggest that documentation of the mother’s explanations during screening may be important in the detection of potential false-positives. The latter may be interpreted with some caution since some parents may be less likely to provide unsolicited elaboration. Nonetheless, these findings have potential public health importance. Additionally, although the M-CHAT was specifically designed for the early detection of ASDs, our findings indicate that with the addition of clinician discretion, the M-CHAT can successfully be administered by survey methods.

Notes

Acknowledgments

This study was supported by funding provided by the Canadian Institutes of Health Research. Funding for Brie Yama was provided by the Ontario Graduate Student—Science and Technology award.

References

  1. Aday, L. A., & Cornelius, L. J. (2006). Designing and conducting health surveys: A comprehensive guide, 3rd edn. San Francisco: Jossey-Bass.Google Scholar
  2. American Academy of Pediatrics. Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children With Special Needs Project Advisory Committee. (2006). Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics, 118, 405–420.Google Scholar
  3. American Psychiatric Association. (2000). Pervasive developmental disorders. In A. P. Association (Ed.). Diagnostic and statistical manual of mental disorders, Fourth editiontext revision (DSM-IV-TR)) (pp. 69–70). Washington DC.Google Scholar
  4. Baird, G., Cass, H., & Slonims, V. (2003). Diagnosis of autism. BMJ, 327(7413), 488–493.PubMedCrossRefGoogle Scholar
  5. Baird, G., Charman, T., Cox, A., Baron-Cohen, S., Swettenham, J., Wheelwright, S., et al. (2001). Current topic: Screening and surveillance for autism and pervasive developmental disorders. Archives of Disease in Childhood, 84(6), 468–475.PubMedCrossRefGoogle Scholar
  6. Centers for Disease Control and Prevention. (2009). Prevalence of Autism Spectrum Disorders—Autism and Developmental Disabilities Monitoring Network, United States, 2006. In: Surveillance Summaries, December 18, 2009; MMWR 2009:58(SS10), 1–20.Google Scholar
  7. Charman, T., Baron-Cohen, S., Baird, G., Cox, A., Wheelwright, S., Swettenham, J., et al. (2001). Commentary: The modified checklist for autism in toddlers. Journal of Autism and Developmental Disorders, 31(2), 145–148. (Commentary).PubMedCrossRefGoogle Scholar
  8. Dumont-Mathieu, T. M., & Fein, D. (2005). Screening for autism in young children: The modified checklist for autism in toddlers (M-CHAT) and other measures. Mental Retardation and Developmental Disabilities, 11, 253–262. (Review).CrossRefGoogle Scholar
  9. Eaves, L. C., Wingert, H., & Ho, H. H. (2006). Screening for autism: Agreement with diagnosis. Autism, 10(3), 229–242.PubMedCrossRefGoogle Scholar
  10. Filipek, P. A., Accardo, P. J., Ashwal, S., Baranek, G. T., Cook, E. H., Dawson, G., et al. (2000). Practice parameter: Screening and diagnosis of autism: Report on the quality of standards subcommittee of the American Academia of Neurology and the Child Neurology Society. Neurology, 55(4), 468–479.PubMedGoogle Scholar
  11. Filipek, P. A., Accardo, P. J., Baranek, G. T., Cook, E. H., Dawson, G., Gordon, B., et al. (1999). The screening and diagnosis of autistic spectrum disorders. Journal of Autism and Developmental Disorders, 29(6), 439–484.PubMedCrossRefGoogle Scholar
  12. Fletcher, R. W., & Fletcher, S. W. (2005). Fourth edition: Clinical epidemiology the essentials. Baltimore: Lippincott Williams and Wilkins.Google Scholar
  13. Fombonne, E. (2009). Epidemiology of pervasive developmental disorders. Pediatric Research, 65(4), 591–598.PubMedCrossRefGoogle Scholar
  14. Glascoe, F. P., Macias, M. M., Wegner, L. M., & Robertshaw, N. S. (2007). Can a broadband developmental-behavioral screening test identify children likely to have autism spectrum disorder? Clinical Pediatrics, 46(9), 801–805.PubMedCrossRefGoogle Scholar
  15. Johnson, C. P. (2008). Recognition of autism before age 2 years. Pediatrics in Review, 29(3), 86–96.PubMedCrossRefGoogle Scholar
  16. Johnson, C., & Myers, S. (2007). Identification and evaluation of children with autism spectrum disorders. Pediatrics, 120(5), 1183–1215.PubMedCrossRefGoogle Scholar
  17. Kleinman, J. M., Robins, D. L., Ventola, P. E., Pandey, J., Boorstein, H. C., Esser, E. L., et al. (2008). The modified checklist for autism in toddlers: A follow-up study investigating the early detection of autism spectrum disorders. Journal of Autism and Developmental Disorders, 38, 827–839.PubMedCrossRefGoogle Scholar
  18. Kuban, K. C. K., O’Shea, T. M., Allred, E. N., Tager-Flusberg, H., Goldstein, D. J., & Leviton, A. (2009). Positive screening on the modified checklist for autism in toddlers (M-CHAT) in extremely low gestational age newborns. The Journal of Pediatrics, 154(4), 535–540.e531.Google Scholar
  19. Limperopoulos, C., Bassan, H., Sullivan, N. R., Soul, J. S., Robertson, R. L., Jr., Moore, M., et al. (2008). Positive screening for autism in ex-preterm infants: Prevalence and risk factors. Pediatrics, 121(4), 758–765.PubMedCrossRefGoogle Scholar
  20. Mawle, E., & Griffiths, P. (2006). Screening for autism in pre-school children in primary care: Systematic review of english language tools. International Journal of Nursing Studies, 43(5), 623–636.PubMedCrossRefGoogle Scholar
  21. Newschaffer, C. J., Croen, L. A., Daniels, J., Giarelli, E., Grether, J. K., Levy, S. E., et al. (2007). The epidemiology of autism spectrum disorders. Annual Review of Public Health, 28(1), 235–258.PubMedCrossRefGoogle Scholar
  22. Oleckno, W. A. (2002). Essential epidemiology—principles and applications. Long Grove, IL: Waveland Press Inc.Google Scholar
  23. Pandey, J., Verbalis, A., Robins, D. L., Boorstein, H., Klin, A., Babitz, T., et al. (2008). Screening for autism in older and younger toddlers with the modified checklist for autism in toddlers. Autism, 12(5), 513–535.PubMedCrossRefGoogle Scholar
  24. Pinto-Martin, J. A., Young, L. M., Mandell, D. S., Poghosyan, L., Giarelli, E., & Levy, S. E. (2008). Screening strategies for autism spectrum disorders in pediatric primary care. Journal of Developmental and Behavioural Pediatrics, 29(5), 345–350.CrossRefGoogle Scholar
  25. Radloff, L. S. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1(3), 385–401.CrossRefGoogle Scholar
  26. Rice, C. E. (2007). Prevalence of autism spectrum disorders–autism and developmental disabilities monitoring network, six sites, United States, 2000. Morbidity and Mortality Weekly Report Surveillance Summary, 9(56), 1–11.Google Scholar
  27. Robins, D. L. (2002). The modified-checklist for autism in toddlers (M-CHAT): Early detection of autism spectrum disorders. Connecticut: University of Connecticut.Google Scholar
  28. Robins, D. L. (2008). Screening for autism spectrum disorders in primary care settings. Autism, 12(5), 537–556.PubMedCrossRefGoogle Scholar
  29. Robins, D. L., & Dumont-Mathieu, T. M. (2006). Early screening for autism spectrum disorders: Update on the modified checklist for autism in toddlers and other measures. Developmental and Behavioral Pediatrics, 26(2), S111–S119. (Review).CrossRefGoogle Scholar
  30. Robins, D. L., Fein, D., & Barton, M. L. (1999). Modified checklist for autism in toddlers (M-CHAT) Follow-Up Interview.Google Scholar
  31. Robins, D. L., Fein, D., Barton, M. L., & Green, J. A. (2001). The modified checklist for autism in toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Developmental Disorders, 31(2), 131–144.PubMedCrossRefGoogle Scholar
  32. SAS. (2000). Statistical analysis software (Version 8th): SAS Institute Inc.Google Scholar
  33. Schaus, K., & Harrington, J. W. (2010). Implementing the MCHAT in an urban community clinic. Abstract 1482.308, presented at Pediatric Academic Societies Annual Meeting. Canada: Vancouver.Google Scholar
  34. Snow, A. V., & Lecavalier, L. (2008). Sensitivity and specificity of the modified checklist for autism in toddlers and the social communication questionnaire in preschoolers suspected of having pervasive developmental disorders. Autism, 12(6), 627–644.PubMedCrossRefGoogle Scholar
  35. Sontrop, J. M., Avison, W. R., Evers, S. E., Speechley, K. N., & Campbell, M. K. (2008). Depressive symptoms during pregnancy in relation to fish consumption and intake of n-3 polyunsaturated fatty acids. Paediatric and Perinatal Epidemiology, 22(4), 389–399.PubMedCrossRefGoogle Scholar
  36. Sontrop, J. M., Campbell, M. K., Evers, S. E., Speechley, K. N., & Avison, W. R. (2007). Fish consumption among pregnant women in London, Ontario: Associations with socio-demographic and health and lifestyle factors. Canadian Journal of Public Health, 98(5), 389–394.Google Scholar
  37. The Standing Senate Committee on Social Affairs Science and Technology. (2007). Final report on: The enquiry on the funding for the treatment of autismpay now or pay later: Autism families in crisis.Google Scholar
  38. World Health Organization. (2006). F84. Pervasive developmental disorders. International Statistical Classification of Diseases and Related Health Problems. (10th (ICD-10) ed.). Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Brie Yama
    • 1
    • 2
    • 6
  • Tom Freeman
    • 3
  • Erin Graves
    • 1
    • 2
  • Su Yuan
    • 1
    • 2
    • 7
  • M. Karen Campbell
    • 1
    • 2
    • 4
    • 5
  1. 1.Department of Epidemiology and BiostatisticsThe University of Western OntarioLondonCanada
  2. 2.Division of Children’s Health and TherapeuticsChildren’s Health Research InstituteLondonCanada
  3. 3.Department of Family MedicineThe University of Western OntarioLondonCanada
  4. 4.Department of Obstetrics and GynaecologyThe University of Western OntarioLondonCanada
  5. 5.Department of PaediatricsThe University of Western OntarioLondonCanada
  6. 6.Chicago Medical SchoolRosalind Franklin University of Medicine and ScienceNorth ChicagoUSA
  7. 7.Faculty of MedicineUniversity of CalgaryCalgaryCanada

Personalised recommendations