Consultation-based Academic Interventions for Children with ADHD: Effects on Reading and Mathematics Achievement
- First Online:
- Cite this article as:
- DuPaul, G.J., Jitendra, A.K., Volpe, R.J. et al. J Abnorm Child Psychol (2006) 34: 633. doi:10.1007/s10802-006-9046-7
- 863 Views
The purpose of this investigation was to evaluate the relative efficacy of two consultation-based models for designing academic interventions to enhance the educational functioning of children with attention-deficit/hyperactivity disorder (ADHD). Children (N=167) meeting DSM-IV criteria for ADHD were randomly assigned to one of two consultation groups: Individualized Academic Intervention (IAI; interventions designed using a data-based decision-making model that involved ongoing feedback to teachers) and Generic Academic Intervention (GAI; interventions designed based on consultant-teacher collaboration, representing “consultation as usual”). Teachers implemented academic interventions over 15 months. Academic outcomes (e.g., standardized achievement test, and teacher ratings of academic skills) were assessed on four occasions (baseline, 3 months, 12 months, 15 months). Hierarchical linear modeling analyses indicated significant positive growth for 8 of the 14 dependent variables; however, trajectories did not differ significantly across consultation groups. Interventions in the IAI group were delivered with significantly greater integrity; however, groups did not differ with respect to teacher ratings of treatment acceptability. The results of this study provide partial support for the effectiveness of consultation-based academic interventions in enhancing educational functioning in children with ADHD; however, the relative advantages of an individualized model over “consultation as usual” have yet to be established.
KeywordsADHDSchool interventionConsultationAcademic outcomes
Children with Attention-deficit/hyperactivity disorder (ADHD) frequently exhibit significant impairment in academic achievement throughout their school years (American Psychiatric Association, 2000; DuPaul & Stoner, 2003). In fact, the strong association between ADHD symptoms, particularly inattention, and academic problems has been demonstrated across many investigations with both clinic-referred (e.g., Barkley, DuPaul, & McMurray, 1990) and school-referred (e.g., DuPaul et al., 2004) samples. Samples of children with ADHD typically function approximately one standard deviation below their classmates with respect to achievement test scores (for review see DuPaul & Stoner, 2003; Hinshaw, 1992). The academic difficulties that children with ADHD experience often begin in early elementary school and continue through high school and college (Barkley, Fischer, Edelbrock, & Smallish, 1990; Mannuzza, Gittelman-Klein, Bessler, Malloy, & LaPadula, 1993). Thus, it is not surprising that children with ADHD are at higher than average risk for grade retention and school drop-out (Barkley, 2006). The problems that children with ADHD may have with respect to reading and math achievement are particularly critical, given the importance of reading and math skills for success as an adult (Wirt et al., 2004).
The most effective treatments for ADHD include psychostimulant medication (e.g., methylphenidate) and contingency management strategies (Barkley, 2005; MTA Cooperative Group, 1999). Unfortunately, most intervention outcome studies focus almost exclusively on symptom reduction rather than enhancement of academic functioning. Thus, although effect sizes associated with school-based interventions for disruptive behavior are in the moderate range, only small effects (ES=.20) have been obtained for interventions addressing academic problems in this population (DuPaul & Eckert, 1997). In similar fashion, although stimulants and other medications may enhance academic productivity, long-term outcome studies indicate that these drugs have minimal impact on educational achievement. For example, the Multimodal Treatment of Children with ADHD (MTA) study found small, statistically non-significant effects (ES range from 0 to .20) on reading and mathematics achievement test scores for children receiving carefully titrated psychotropic medication either alone or in combination with psychosocial treatment at 14-months (MTA Cooperative Group, 1999) or 24-months (MTA Cooperative Group, 2004) after initiation of treatment.
The results of single-subject design studies support the efficacy of various academic interventions for children with ADHD (DuPaul & Eckert, 1998; DuPaul & Stoner, 2003). For example, DuPaul, Ervin, Hook, and McGoey (1998) found that ClassWide Peer Tutoring (CWPT) led to clinically significant improvements in academic performance and behavioral control for 18 students with ADHD who were placed in first through fifth grade general education classrooms. Moderate to large effects on children’s weekly math and spelling test scores were obtained with CWPT. In similar fashion, computer-assisted instruction has led to clinically significant gains in oral reading fluency (Clarfield & Stoner, 2005) and performance on curriculum-based measurement probes in mathematics (Mautone, DuPaul, & Jitendra, 2005; Ota & DuPaul, 2002) for small samples of children with ADHD. Studies examining these interventions with large samples in the context of a group research design have not been conducted, to date.
Conclusions regarding the effects of academic interventions on the achievement of children with ADHD are limited due to (a) small sample size of most investigations, (b) examination of outcomes over relatively short time periods (e.g., several weeks or months), and (c) use of a “one size fits all” approach wherein all participants receive the identical intervention regardless of differences in academic profiles. Further, prior studies have not involved classroom teachers in the design and planning of interventions. Given that teachers are critical to the implementation and integrity of classroom-based academic interventions, their absence from the planning process is likely to reduce the viability of treatment plans.
In most public school settings, teachers work with a consultant (e.g., school psychologist) to plan classroom interventions, with the final selection of treatment strategies based on teacher choice, presumably based on both perceived effectiveness and feasibility of the various intervention options. In the research literature, the predominant consultation model for addressing academic and behavioral concerns is a data-based decision making approach based on the tenets of behavior analysis (Daly, Witt, Martens, & Dool, 1997; Sheridan et al., 1996). In this behavioral consultation model, classroom teachers work with consultants to develop individualized interventions based on baseline data regarding a student’s academic strengths and weaknesses as well as important contextual variables (e.g., antecedent and consequent events prompting and maintaining academic behaviors). In addition, consultants monitor treatment implementation and teachers are provided with feedback designed to enhance intervention fidelity.
Comparison of behavioral consultation model and typical consultation in designing academic interventions
⦾Teacher provided information about child’s ADHD and related difficulties
⦾Teacher provided information about child’s ADHD and related difficulties
Problem Identification Interview
⦾Identify academic areas of concern
⦾Identify academic areas of concern
⦾Discuss current performance
⦾Discuss antecedent and consequent conditions
⦾Identify patterns to academic behavior problems
⦾Determine additional observation and data collection procedures
No Data Collection and Observation
Data Collection and Observation
⦾Complete Functional Academic Assessment
⦾Review permanent products
⦾Link assessment to intervention strategies
Problem Analysis Interview
⦾Provide menu of intervention options appropriate for teacher identified academic concerns
⦾Review functional academic assessment data
⦾Teacher choose intervention(s)
⦾Provide menu of intervention options as determined by teacher concerns, direct observations, and assessment data.
⦾Teacher chooses intervention(s)
⦾Establish progress monitoring plan
Provide Specific Intervention Plan
Provide Specific Intervention Plan
⦾Frequency, Materials, & Steps
⦾Frequency, Materials, & Steps
⦾Provide on-site training to teacher and/or students
⦾Progress Monitoring Procedures
Minimal or no contact with teacher
Contacts with Teacher During Implementation
⦾Weekly phone calls, emails, or face-to-face
No Progress Monitoring
⦾Linked to intervention
No monitoring of intervention integrity or feedback given to teacher
⦾Collected by consultant on several occasions
No Treatment Evaluation Interview
Treatment Evaluation Interview
⦾Review progress monitoring data
⦾Evaluate intervention effectiveness
⦾Review treatment integrity data with teacher
⦾Decision-making regarding continued implementation of intervention
Demographic and diagnostic characteristics by academic intervention and treatment group
t or χ2
t or χ2
Age (in months)
% ADHD Combined
% ADHD Inattentive
% ADHD Hyperactive
% Receiving Special Education
% Receiving Psychotropic Medication
A sample of 175 children (133 boys, 42 girls; M age = 104.3 months; SD=14.7), attending first through fourth grade in public elementary schools in eastern Pennsylvania participated in this study. Participants were referred by classroom teachers due to concerns regarding significant difficulties with ADHD symptoms and below average academic achievement (based on teacher perceptions of below grade level performance). The sample consisted primarily of White children (58%; 26.9% Hispanic; 11.4% Black) and families were primarily in the lower middle class and middle class range based on the Hollingshead Index (Hollingshead, 1975) with a mean Index score of 48.0 (SD=24.8). Elementary schools were located in urban, rural, and suburban settings.
To be identified as a child with ADHD for this study, both parent and teacher ratings on the ADHD Rating Scale-IV (DuPaul et al., 1998) exceeded the 90th percentile on either the Inattention or Hyperactive-Impulsivity subscales using appropriate age and gender norms. Additionally, children met DSM-IV-TR (American Psychiatric Association, 2000) criteria for one of the three ADHD subtypes based on parent interviews using the National Institute of Mental Health Diagnostic Interview Schedule for Children-IV (NIMH-DISC-IV; Shaffer, Fisher, & Lucas, 1998). Children with all three ADHD subtypes were included with the majority (65%) being combined type. The sample included children with co-morbid oppositional defiant disorder (38%) and conduct disorder (15%). A total of 51 students (29.1%) were receiving part-time special education services, while 50 children (28.5%) were receiving psychotropic medication. Some students were receiving more than one type of psychotropic medication and included psychostimulants (n=38), anti-depressants (n=8), and other medications (n=26).
Of the total sample, data were available for 167 children who were randomized to one of two educational consultation groups: Individualized Academic Intervention (IAI) or Generic Academic Intervention (GAI). Participants were included if they received at least one semester of consultation for either math or reading using an intent-to-treat methodology. A total of 54 of the 167 participants received intervention for both academic subjects and were included in both reading and math samples. Table 2 presents demographic data separately for consultation groups within both reading and math samples. The two groups did not differ with respect to gender, ethnicity, mother’s occupation, ADHD subtype, comorbid ODD or CD, and receipt of special education or psychotropic medication. The mean age (p < .01) and grade (p < .05) for children in the IAI group was significantly greater than for children in the GAI condition, for the math sample only. In the reading sample father’s occupation was significantly higher for the IAI group (p < .01).
The 8 students from the total sample that were not included in any analyses resulted from 3 participants moving to schools out of the area before intervention could begin, 2 teachers preferring behavioral interventions and choosing not to participate in consultation regarding academic concerns, 2 parents withdrawing consent for consultation with their child’s teacher, and 1 student moving to a school that declined participation. Fifty students did not receive consultation services during their second year of participation (semesters 2 and 3). For 48 of these cases, the teacher refused the services. In the other 2 cases, parents no longer wanted a consultant to work with their child’s teacher. As part of the intent-to-treat methodology, data were still collected and included in the analyses for these participants. Eighteen additional students did not receive either consultation services or data collection in the second year. These missing data were a result of 6 teachers and 3 principals refusing to allow data collection, 2 parents no longer wanting data collected on their child, 6 students moving out of the area, and 1 student changing schools without informing project staff of their new location. Group membership was not cited as a reason to decline consultation services at any time and chi-square analyses revealed no significant differences between consultation models for either the type of services received (consultation, data collection only, nothing) or the reason for refusal.
A total of 204 teachers across 52 schools participated in this study. Teachers were primarily female (87.7%), White (96.5%), and had either bachelor’s degrees (48%) or master’s degrees (52%). Most were general education teachers with 14.4% identifying themselves as teachers of special education classrooms. Once a student in a teachers’ classroom was randomized to a consultation model, all other students beginning participation within that classroom were then assigned to that group as well. This was done to avoid confusion and potential treatment contamination resulting from a teacher being asked to participate in two different models of consultation simultaneously as well as having more than one consultant. A total of 37 teachers had more than one student simultaneously in their classroom at some point in the study. For 26 of these teachers, this took place during the initial semester, indicating that these teachers referred multiple students. Given the number of schools and our status as consultants and guests we could not control for future classroom placement. Fifteen of the total number of teachers (7.35%), therefore, experienced working with consultants from both models at some point in the study. On only one occasion did a teacher work with consultants from both models simultaneously. After eliminating the 15 teachers shared between the treatment groups, chi-square analyses revealed no significant differences between the groups on gender, ethnicity, or highest degree earned. The IAI consultants, however, worked with more male teachers (n=17) than did the GAI (n=5; χ2(1)=8.15, p < .05).
Consultants consisted of 11 doctoral students in either a special education or school psychology training program. Consultants were appointed to either the GAI or IAI consultation models and at no point did they participate simultaneously in both groups. Children were assigned to each consultant at the start of their participation, and whenever possible, a consultant would retain their cases throughout the students’ participation. Consultants in the GAI group had a mean age of 24.00 (SD=1.83) years when they began implementing their assigned model and were primarily female (85.7%). The IAI group also consisted primarily of females (80.0%) and had a mean age of 26.20 (SD=3.19). None of the consultants were of a minority ethnic group.
The ADHD Rating Scale-IV (DuPaul et al., 1998) is a behavior rating scale that includes items directly related to the 18 symptoms of ADHD based on the DSM-IV-TR (American Psychiatric Association, 2000). Home and school versions are available for completion by parents and teachers, respectively. Items are scored on a 0 (never or rarely) to 3 (very often) basis. Normative data based on age and gender are available and the psychometric properties of this instrument are well established (DuPaul et al., 1998).
The Computerized NIMH Diagnostic Interview Schedule for Children (Parent Version) (CDISC 4.0; Shaffer et al., 1998) is a structured diagnostic interview that is administered using computer software. Current (present state) symptoms and symptoms over the past year are reported by parents on this interview. The Disruptive Behavior Disorders module was administered by a trained interviewer (i.e., doctoral student in school psychology) either in person or by phone. The entire CDISC was not administered due to time constraints (i.e., disruptive behavior disorder module took approximately 1 h to complete) and because the focus of the treatment outcome study was on externalizing difficulties (as well as academic achievement). Interviewers were trained by the research project coordinator who was a masters level psychologist. Diagnostic decisions based on this interview have been found to be highly reliable (Shaffer et al., 1998). All CDISC 4.0 interviews were audiotaped and a random sub-sample (21%) was reviewed by a second trained interviewer (i.e., doctoral student in school psychology) to assess inter-diagnostician agreement. Agreement was 100% across all interviews with respect to overall diagnosis and subtype designation.
Standard and raw scores on four subtests from the Woodcock-Johnson III Tests of Achievement (WJ-III; Woodcock, McGrew, & Mather, 2001) served as indicators of academic achievement. Reading subtests included reading fluency and reading comprehension. Measures of mathematic achievement included math fluency and math calculation scores. The WJ-III includes two versions, A and B, that have been shown to have strong psychometric properties (Mather & Woodcock, 2001). Both versions of this measure were utilized during this investigation.
Raw scores on four subscales of the Academic Competency Evaluation Scales (ACES; DiPerna & Elliott, 2000) served as measures of teacher perceptions of children’s academic skills and achievement related-behaviors. Subscales used in this study included Mathematics, Reading/Language Arts, Critical Thinking, and total Academic Enablers. Items on all four subscales were completed on a 1 (“never”) to 5 (“almost always”) Likert scale. The ACES has more than adequate levels of reliability and validity (DiPerna & Elliott, 1999, 2000). The academic skills portion of the ACES includes 11 items measuring Reading/Language Arts skills, 8 measuring Mathematics skills, and either 9 items for kindergarten through second grade or 14 items for third grade and above measuring Critical Thinking skills. Total scores range from 11 to 55 for Reading/Language Arts, 8 to 40 for Mathematics, and 9 to 45 or 14 to 70 for Critical Thinking skills. Thirty-eight items, with two additional items for those in third grade and above, measuring motivation, engagement, interpersonal skills, and study skills were totaled to comprise the Academic Enablers subscale. Scores for this scale range from 38 to 190, or 40 to 200, depending on grade-level. Each child’s primary teacher for reading and math completed the ACES at each assessment phase.
Consultants monitored treatment integrity using checklists that were created using the steps of the intervention plan given to each teacher (see Appendix A for an example of a treatment integrity checklist). The steps of the intervention plan were modified to include a space to record the occurrence of each step during an observation. Treatment integrity scores were then calculated as the percentage of steps completed correctly relative to the total number of steps in the prescribed plan. The mean treatment integrity collected across at least 3 separate observations per assessment phase was used as the dependent measure in these analyses.
An adapted version of the Behavior Intervention Rating Scale (BIRS; Elliot & VonBrock Treuting, 1991) provided a measure of intervention acceptability. The BIRS is typically used as a measure of acceptability of behavioral interventions implemented in the classroom and reasonable psychometric properties have been demonstrated (Elliot & VonBrock Treuting, 1991). In this study, the BIRS was adapted to reflect academic skills interventions as opposed to behavioral interventions. As with the original, the adapted BIRS was comprised of 24 items scored on a 1 (“Strongly Disagree”) to 6 (“Strongly Agree”) Likert scale and was completed by the teacher involved in the consultation process. The total raw score was used as the dependent measure indicating intervention acceptability and the scale was considered valid if 23 of the 24 items were completed. Possible scores on this measure ranged from 24 to 144 with higher scores indicating greater treatment acceptability. The internal consistency of the adapted BIRS for the present sample at the first assessment phase was in the acceptable range (α=.94).
Data collection. Following receipt of written parental consent, baseline data collection of all dependent measures took place over approximately a one-month period during the middle of the school year (December to February). Trained graduate students in school psychology, special education, and counseling psychology served as research assistants and administered the WJ-III. Forms A and B of the WJ-III were alternated each assessment period per child (i.e. approximately 50% of the participants received version A at baseline, the other 50% version B). The research assistants were blind to the purpose of the study and to the group membership of participating children. The ACES and BIRS were part of a larger packet of behavior rating scales distributed directly to teachers by research assistants on the day the child was being assessed. Teachers returned scales directly to the investigators by mail. A stipend of $50 was provided to teachers upon completion of the packet of rating scales. The WJ-III and ACES were collected on three additional occasions across two school years (3-months, 12-months, and 15 months following initiation of academic intervention). The BIRS was collected at the 3, 12, and 15-month time points. Consultants completed treatment integrity checklists at least 3 times per assessment phase (except for baseline) for each intervention.
Treatment groups. Children were randomly assigned to one of two educational consultation groups: Individualized Academic Intervention (IAI; n=80) and Generic Academic Intervention (GAI; n=88). All consultants were school psychology and special education doctoral students and were supervised by the first and second authors. Each consultant had completed or were completing relevant coursework in consultation and school-based intervention. Consultation was provided beginning in the second half of the year (approximately February) and continued through the next year, if the participant’s 2nd year teacher was willing. Overall, the consultation procedure lasted approximately 15 months.
GAI. Consultants for the GAI group collaborated with classroom teachers to design academic interventions based on teacher choice (i.e., “consultation as usual” control condition). Once a student and teacher were identified as participants, the consultant assigned to the case scheduled a meeting with the teacher to provide them with information regarding ADHD and its effects on school performance. Teachers were given two resource materials, an overview chapter on ADHD, taken from Pfiffner (1996) and a handout from the National Association of School Psychologists entitled ADHD Students in the Classroom: Strategies for Teachers (Brock, 1998), and their contents were reviewed. Teachers were also informed of the procedures involved in the consultation process and what they could expect regarding the number and duration of future meetings. Such meetings consisted of two consultation interviews, with additional meetings scheduled as needed. The GAI consultation group was designed to be a close approximation to what typically takes place in the school setting.
During the initial interview, academic areas of concern were identified, current performance was discussed, and goals for intervention were determined. The consultants set up a time for a second interview in which they returned with a menu of empirically supported intervention options addressing the goals targeted in the initial interview. After explaining each intervention the consultants allowed teachers to choose the intervention(s) the teachers believed was most appropriate for their classroom and each student’s needs. The consultants would then provide teachers with specific intervention plans detailing the steps of the interventions chosen and provide any materials the teachers did not already have on hand. Weekly contact with teachers was arranged by phone or email for the teachers to provide updates and address questions or concerns. Data on student progress were not collected and any changes in intervention were based solely on teacher report. Intervention integrity was monitored at least three times per intervention phase by a GAI consultant not assigned to the case and neither the teacher nor the consultant responsible for designing the intervention were provided with feedback regarding integrity, progress, or outcomes.
The above procedure was implemented for each teacher involved in the consultation process. When a child changed teachers (i.e. advancing to the next grade level), the procedure was repeated. On average, GAI consultants spent 5.29 h (SD=2.90) per case in the first semester, 2.48 h (SD=1.08) per case in the second semester, and 2.81 h (SD=2.11) per case in the third semester.
Both the initial and second interviews were audiotaped and procedural integrity was completed by masters level psychologists provided with a checklist of interview steps. Approximately 20% of the audiotapes were randomly chosen resulting in 95.7% integrity for the initial interview and 97.6% integrity for the second. Feedback was provided to the consultant by the supervisor on any steps that were not completed adequately (i.e., if integrity <100%).
IAI. The IAI treatment group also began with the same ADHD informational session and materials as were provided to the GAI group. This model differed from the GAI model by including the use of functional and academic assessment data to design interventions, and by providing modeling, prompting, and feedback to teachers to guide implementation of interventions (see Table 1). Consultants for the IAI group collaborated with classroom teachers to design academic interventions based on assessment data using a consultative problem-solving model involving three consultant-teacher interviews (Bergan & Kratochwill, 1990). Following the ADHD information session, consultants and teachers participated in a problem identification interview (PII). During the problem identification interview (PII) to identify academic areas of concern, antecedent conditions, student’s response to these conditions, as well as consequent conditions. Patterns to academic behavior problems were also identified and goals were set and prioritized. Teachers and consultants then agreed on additional observational procedures.
Before conducting the second meeting, or problem analysis interview (PAI), consultants in the IAI treatment group conducted functional academic assessments of the classroom. These were done to obtain information regarding teacher routines, behaviors, and procedures as well as student and peer behaviors (e.g., Daly et al., 1997). Consultants also reviewed student work products in comparison to peers as well as complete a basic skills assessment using curriculum based assessment data in the content areas of reading and math. Once these observation and assessment procedures were completed, consultants conducted the PAI with the teacher. In this meeting, specific interventions were discussed based on teacher input as well as direct observation and assessment data. After explaining each intervention, consultants allowed teachers to choose the interventions they felt were most appropriate for their classroom and each student’s needs. Consultants then provided teachers with specific intervention plans detailing the steps of the interventions chosen, trained teachers and students, if necessary, on the steps of the intervention, and provided any materials teachers did not already have on hand. In contrast to the GAI group, progress monitoring data were collected on a weekly basis by either the teacher or the consultant. The type of data collected was based on the goals and academic subject targeted and included such procedures as curriculum based probes in pre-reading skills, oral reading fluency, and/or math fluency, as well as comprehension and problem solving skills when appropriate.
Intervention integrity was also monitored at least 3 times per intervention phase in the IAI treatment group. In contrast to the GAI group, the consultants responsible for the interventions conducted integrity checks and provided teachers with feedback on a biweekly basis. A treatment evaluation interview (TEI) was also conducted approximately 4 weeks into the intervention. Consultants used visual analysis of the graphed displays of the progress monitoring data to determine the level of progress made (i.e. mastery, no progress, adequate progress, inadequate progress, motivation problems) (Browder et al., 1989). It was then determined whether it was appropriate to leave the plan in place, intensify or simplify the intervention, provide for improved antecedents, change the intervention, redefine the goals, or retrain the teacher and/or students in the procedures.
As in the GAI group, the above procedure was implemented for each teacher involved in the consultation process with the addition of discussing past performance and progress in the project with additional teachers. On average, IAI consultants spent 8.28 h (SD=2.80) per case in the first semester, 7.48 h (SD=3.18) per case in the second semester, and 5.27 h (SD=3.20) per case in the third semester. The number of hours per case was significantly greater (p < .001) than for GAI consultants in all three semesters.
Both the initial and second interviews were audiotaped and the first author determined procedural integrity using a checklist of interview steps. Approximately 20% of the audiotapes were randomly chosen resulting in 94.8% integrity for the PII and 96.6% integrity for the PAI. Procedural Integrity was also completed on audiotapes of the TEI’s and was found to be 91.3%. Feedback was provided to the consultant by the first author on any steps that were not completed adequately (i.e., integrity <100%).
Interventions. Both the GAI and IAI groups utilized a range of interventions including teacher-mediated, peer-mediated, computer-assisted, and self-mediated strategies (DuPaul & Stoner, 2003). No significant differences were found between the groups in terms of percentage use of any of the intervention types (teacher meditated χ2(1)=0.003; peer mediated χ2(1)=2.42; computer assisted χ2(1)=0.17; & self-mediated χ2(1)=0.63). Teachers were the most common mediator for both groups with 84.9% of GAI and 86.2% of IAI participants receiving at least one teacher mediated intervention across the 15-month period. Peer mediated interventions were the second most common (GAI=52.3%; IAI=64.8%) followed by self-mediated (GAI=15.1%; IAI=19.9%) and computer assisted (GAI=5.8%; IAI=7.4%).
Interventions focused on math and/or reading skills depending on the difficulties exhibited by specific children. The GAI and IAI consultation groups had access to the same intervention materials and all interventions were supported by empirical research, most typically in the area of learning disabilities. Common reading interventions for both groups included repeated readings (O’Shea, Sindelar, & O’Shea, 1985; Samuels, 1979; Tingstrom, Edwards, & Olmi, 1995), listening passage preview (Rathvon, 1999), collaborative strategic reading (Vaughn & Klinger, 1999), and group story mapping (Idol, 1987). In the area of mathematics, interventions such as cover-copy-compare (Skinner, Turco, Beatty, & Rasavage, 1989), reciprocal peer tutoring (Fantuzzo, King, & Heller, 1982), classwide student tutoring teams (Harper & Maheady, 1999), and schema-based problem solving (Jitendra & Hoff, 1996; Jitendra, Hoff, & Beck, 1999) were found in both groups.
Means and standard deviations for reading outcomes
WJ-III Reading Fluency Raw
WJ-III Reading Fluency Standard
WJ-III Passage Comp Raw
WJ-III Passage Comp Standard
Teacher Perceptions of Performance
ACES Reading & Language Arts
ACES Critical Thinking
ACES Academic Enablers
Percentage of Intervention Steps
Behavior Intervention Rating Scale
Means and standard deviations for mathematics outcomes
WJ-III Calculation Raw Score
WJ-III Calculation Standard Score
WJ-III Math Fluency Raw
WJ-III Math Fluency Standard
Teacher Perceptions of Performance
ACES Critical Thinking
ACES Academic Enablers
Percentage of Intervention Steps
Behavior Intervention Rating Scale
Hierarchical linear modeling analyses of achievement outcomes
Mean Intercept for GAI (γ00)
Mean Change in Intercept for IAI (γ01)
Mean Growth Rate for GAI (γ10)
Mean Change in Growth for IAI (γ11)
WJIII Calc (Stan)
WJIII Calc (Raw)
WJIII Math Fluency (Stan)
WJIII Math Fluency (Raw)
WJIII Reading Fluency (Stan)
WJIII Reading Fluency (Raw)
WJIII Passage Comp (Stan)
WJIII Passage Comp (Raw)
ACES Academic Enablers (Math)
ACES Math Skills
ACES Critical Thinking (Math)
ACES Academic Enablers (Rdg)
ACES Reading Skills
ACES Critical Thinking (Rdg)
Treatment Integrity (Math)
Treatment Integrity (Rdg)
Behavior Intervention Rating (Math)
Behavior Intervention Rating (Rdg)
Means and standard deviations for all dependent measures are presented for the reading and math samples in Tables 3 and 4, respectively. Separate hierarchical linear modeling analyses for each dependent variable were used to assess possible differences in academic growth between the two consultation groups (using an intent to treat methodology). At level one, individual trajectories (i.e., intercept (baseline value) and slope) were calculated for each participant. At level two, group level parameters of individual change were examined, including mean initial performance for GAI (γ00), difference in mean initial performance between GAI and IAI (γ01), mean growth rate (per assessment period) for GAI (γ10), and difference in mean growth rate between GAI and IAI (γ11). Because participants in the IAI math sample were significantly older than the children in the GAI math sample, age in months was used as a level 2 covariate for math analyses only. Although there also was a significant difference in grade level between math intervention groups, grade was not added as a covariate because it is an ordinal variable and was highly correlated (r=0.85) with age.
For all dependent measures, γ00 was statistically significant (p < .05), indicating that the GAI group started out at a non-zero level of performance (see Table 5). For all but one measure, γ01 was not statistically significant; thus, there was no significant difference in initial performance between the two treatment groups. The ACES Academic Enablers (Reading) score was significantly higher (p < .05) for IAI participants at baseline.
Statistically significant growth (p < .05) was obtained for 8 of the 14 dependent measures (see γ10 values in Table 5). Specifically, significant positive trajectories were found for WJ-III Calculation, Math Fluency, Reading Fluency, and Passage Comprehension raw scores, as well as WJ-III Reading Fluency standard scores, ACES Academic Enablers (both Math and Reading), and ACES Critical Thinking (Reading). Significant slopes were not obtained for WJ-III Calculation, Math Fluency, and Passage Comprehension standard scores or for ACES Math, Reading/Language Arts, and Critical Thinking (Math) scores. Contrary to prediction, γ11 values were not statistically significant and, thus, none of the slopes differed between the two consultation groups.
In order to estimate the magnitude of change, within-group effect sizes were calculated using the formula ((M15-mos – MBL)/pooled SD).1 Thus, these effect sizes represent change over baseline functioning in standard deviation units (Cohen, 1988; see Table 6). Effect sizes were in the small range (ES≤.50) for WJ-III Calculation (GAI only), Math Fluency, and Passage Comprehension standard scores (both groups), as well as for ACES Math (both groups), Math Critical Thinking (both groups), Math Academic Enablers (IAI only), Reading/Language Arts scores (GAI only), Reading Critical Thinking (GAI only), and Reading Academic Enablers (both groups). Alternatively, moderate effect sizes (.50 < ES < .80) were obtained for WJ-III Reading Fluency standard score (both groups), WJ-III Math Calculation standard score (IAI only), WJ-III Math Fluency raw score (IAI only), as well as ACES Reading/Language Arts (IAI only), ACES Reading Critical Thinking (IAI only), and ACES Math Academic Enablers (GAI only). Finally, large effect sizes (ES>.80) were found for WJ-III Math Calculation (both groups), Math Fluency (GAI only), Reading Fluency (both groups) and Passage Comprehension raw scores (both groups).
Effect sizes for change from baseline to 15-months
WJ-III Math Calculation Standard Score
WJ-III Math Calculation Raw Score
WJ-III Math Fluency Standard Score
WJ-III Math Fluency Raw Score
ACES Critical Thinking (Math)
ACES Math Academic Enablers
WJ-III Reading Fluency Standard Score
WJ-III Reading Fluency Raw Score
WJ-III Passage Comprehension Standard Score
WJ-III Passage Comprehension Raw Score
ACES Reading/Language Arts
ACES Critical Thinking (Reading)
ACES Reading Academic Enablers
At the 3-month assessment phase, IAI teachers implemented interventions with significantly (p < .001) greater integrity for both math (M=87.3%) and reading (M=86.6%) than did GAI teachers (M=64.8% for math; M=58.8% for reading). Treatment integrity slope was essentially flat across phases for both groups indicating that IAI integrity was higher than GAI integrity throughout intervention phases (see Table 5). Alternatively, the two groups did not differ with respect to treatment acceptability, which was uniformly positive (M item score ≈ 4.4 on 6-point Likert scale) regardless of group (see Tables 3 and 4).
The results of this study provide partial support for the effectiveness of consultation-based academic interventions in enhancing educational functioning in children with ADHD. Specifically, these findings appear to support academic consultation; however, the type of consultation model did not appear to make a difference, suggesting that the less time-consuming consultation approach may be sufficient. Positive growth trajectories were obtained for math and reading skills as well as teacher ratings of academic enablers (e.g., motivation, study skills) across a 15-month period. These positive trajectories are noteworthy, because children with ADHD are expected to experience greater academic difficulty over time given the interaction between their symptoms and increased academic challenges across grade levels. In fact, the slopes obtained in this study compare favorably with academic achievement growth rates found in the Multimodal Treatment of ADHD study (MTA Cooperative Group, 1999). In the MTA study, slopes (over a 14-month period) for reading achievement ranged from 0 (community treatment control group) to .20 (combined treatment group), while slopes for math achievement ranged from .13 (community treatment control group) to .20 (medication management and behavioral treatment groups). Obtained effect sizes were comparable or, in many cases, larger than the effect size of .20 found for classroom behavioral interventions found in a meta-analysis of school-based treatment for ADHD (DuPaul & Eckert, 1997).
Effect sizes representing change over a 15-month period ranged from small to large across the 14 dependent variables. For all but two of these variables (WJ-III Reading Fluency raw score, ACES Reading Academic Enablers), effect sizes were larger for children in the IAI group. Regardless of group, effect sizes were larger for ACES ratings and WJ-III raw scores than for WJ-III standard scores. These results indicate that improvement in reading fluency occurred in the context of both absolute (as represented by raw score) and relative (as represented by standard score) change. Alternatively, other skill areas appeared to change only in an absolute sense. Hechtman et al. (2004) found larger effect sizes (.73 to .94) for achievement than were obtained in the present study as a function of stimulant medication and/or multimodal psychosocial treatment. However, their results were obtained with a higher functioning sample (i.e., baseline academic achievement scores averaged >100 with very few participants scoring in the below average range) that was, at worst, minimally impaired in the academic domain.
Although positive growth was found for raw scores on all WJ-III subtests, the only subtest to show a significantly positive slope in standard scores was Reading Fluency. In fact, growth in this academic skill area was moderate in magnitude (ES=.58 & .45 for IAI and GAI, respectively). This result is not surprising given that reading fluency was the primary intervention target for nearly all of the children in the reading intervention sample. Significant change in reading fluency is particularly noteworthy given the importance of this skill in predicting overall reading performance (Hintze & Silberglitt, 2005). Reading comprehension scores presumably would increase with (a) greater focus on this skill area as an intervention target, and (b) longer-term use of specific interventions for reading fluency. In addition, measures that are more sensitive to short-term academic skill change (e.g., curriculum-based measurement) might have indicated more robust treatment effects.
Teachers in the IAI group delivered academic interventions with significantly greater integrity than teachers in the GAI group. This higher level of integrity was presumably related to teachers receiving data regarding student progress as well as specific feedback about the degree to which they were implementing interventions accurately. Similar rates of treatment integrity have been found for classroom behavioral interventions using a data-based, feedback approach with teachers (Sterling-Turner, Watson, & Moore, 2002).
Although interventions designed through the IAI approach were implemented with greater integrity than were interventions designed through the GAI approach, the two groups did not differ significantly with respect to academic growth or treatment acceptability. The lack of a relationship among treatment integrity, acceptability, and outcome is counterintuitive in relation to theoretical linkages among the three variables (Witt & Elliott, 1985). It should be noted, however, that, on average, GAI teachers implemented academic strategies with greater than 50% integrity, which is higher than previous findings for behavioral interventions under similar no-feedback conditions (Noell et al., 2005). Further, teachers in both groups reported implementing prescribed interventions at relatively high rates, even when not observed by a consultant. Thus, the present results may indicate that positive academic change (albeit small to moderate in magnitude) may be accomplished even when interventions are delivered with modest degrees of integrity.
The present findings must be interpreted with caution due to several limitations. First, a no-treatment control group was not included due to ethical concerns about withholding academic interventions from children over an extended period of time. The absence of a no-treatment control condition limits the degree to which we can attribute positive academic growth to the interventions employed in this study. Nevertheless, it is important to note that changes in academic skills obtained in this study were similar in magnitude to those found in prior school-based intervention studies (DuPaul & Eckert, 1997) and were greater in magnitude than for children in the community treatment control group of the MTA study (MTA Cooperative Group, 1999) over a similar period of time. Further, longitudinal studies have found deficits in academic achievement (based on achievement tests and/or parent report) that are consistent across school years for samples of children with ADHD followed into middle and high school (Fischer et al., 1990; Lambert, Hartsough, Sassone, & Sandoval, 1987; Latimer et al., 2003). For example, Fischer et al. found standard scores on achievement tests to be 0.5 to 1.0 standard deviations lower in adolescents diagnosed as children with ADHD relative to non-ADHD controls. Thus, based on prior research with this population, the expected trajectory for academic achievement in the absence of treatment would be flat (no change) or deteriorating, rather than improvement in achievement over time, as was found in this study.
A second limitation is that participants in this study had to meet diagnostic criteria for ADHD and also exhibit significant impairment in academic achievement. Thus, these results may not generalize to the ADHD population, as a whole. Given that the diagnosis of ADHD depends, in part, on symptoms being associated with significant academic and/or social impairment (American Psychiatric Association, 2000), the present sample most likely represents the population of children with ADHD who require academic interventions (i.e., a significant majority of the ADHD population).
Third, because teachers in this study voluntarily sought consultation, they may represent the portion of the teacher population that is motivated to implement classroom interventions. Thus, implementation integrity rates for teachers in both consultation groups may have been inflated by this selection bias.
Finally, the modest results for academic achievement found in this study are likely related to the effects of several other school and/or teacher variables (e.g., experience, instructional time), as well as individual participant characteristics. For example, it is possible that treatment effects could be moderated by ADHD subtype or comorbid conditions (e.g., ODD). Unfortunately, because breaking our sample down further based on individual characteristics would lead to small subsamples with limited power, we were not able to conduct additional analyses of potential treatment moderators.
Significant improvement in academic skills was found for a large sample of children with ADHD as a function of academic interventions delivered through school-based consultation. In contrast to our hypothesis, consultation as usual was essentially equivalent to a more individualized, data-based approach. This result, while surprising, is similar to outcomes found for assessment-based behavioral consultation (Beavers, Kratochwill, & Braden, 2004). Specifically, Beavers and colleagues found no difference in treatment effects for reading difficulties between a consultation approach using functional assessment and consultation without specific functional assessment data. Further, it should be noted that the GAI group in the present study was probably more intensive and data-based than what could be considered “consultation as usual.” In fact, the GAI condition included structured consultation interviews, implementation of empirically supported interventions, and assessment of outcomes, all of which may be missing from typical school-based consultation (Bramlett et al., 2002). Thus, perhaps a comprehensive, data-based model as employed in the IAI condition may not be necessary to bring about effective growth in academic skills. Additional study of these two approaches should be undertaken to discern possible differences in outcomes for more specific measures of academic performance (e.g., individual goal attainment and report card grades) as well as differences over longer periods of time.
The preparation of this manuscript was supported by NIMH Grant R01-MH62941. We gratefully acknowledge the efforts of all teachers and students who participated in this project as well as Lisa Marie Angello, Andrea Deatline-Buchman, Anuja Divatia, Lauren Dullum, Rebecca Eng, Karen Hailstones, Jilda Hodges, Jayne Leh, Stacy Martin, Jennifer Mautone, Erin Post, Eve Puhalla, Hillary Rogers, Timothy Scholten, Cotie Strong, Deanna Tipton, and Yan Ping Xin who served as research assistants for this study.