Introduction

School engagement is a student’s behavioral, cognitive, and emotional commitment to school (Fredericks et al., 2004). Low levels of school engagement are associated with academic failure, school dropout, and substance use (Bond et al., 2007; Carter et al., 2007; Fredericks et al., 2004). Youth with attention problems, including those with attention-deficit/hyperactivity disorder (ADHD), commonly experience serious problems in school functioning and engagement (Booster et al., 2007; DuPaul & Langberg, 2014) and are at elevated risk for these poor outcomes. School engagement is thought to be malleable (see Fredericks et al., 2004) and causally related to achievement (Reyes et al., 2012). To date, no interventions have been developed to specifically improve school engagement among youth with attention problems. Such interventions are important due to the unique impairments these youth experience that put them at risk for low school engagement and poor outcomes. Involving peers may be a promising approach to enhance their engagement. Guided by the deployment-focused model (DFM; Weisz et al., 2005), our goal was to iteratively develop and evaluate a school-based peer-supported intervention to improve engagement among middle school students with attention problems.

School Engagement

School engagement is a student’s commitment to school along three interrelated, dynamic facets: behavioral engagement, emotional engagement, and cognitive engagement (Fredricks et al., 2004; Jimerson et al., 2003). Behavioral engagement includes behaving in prosocial and nondisruptive ways in school; being involved in learning and academic work by expending effort, persistence, and concentration; and participating in school-related activities (Fredricks et al., 2004). Emotional engagement includes affective reactions to the classroom, school, and teacher and feelings of belonging to, valuing of, and identification with school (Fredricks et al., 2004). Cognitive engagement involves psychological effort exerted in learning by preferring a challenge and striving to obtain mastery and understanding of academic material (Fredricks et al., 2004). Cognitive engagement has also been defined by the extent to which a student uses strategies to accomplish academic tasks, including learning strategies (e.g., rehearsal and organization) and self-regulation skills to plan, monitor, and evaluate completion of academic tasks (Fredricks et al., 2004).

School engagement is associated with a variety of indicators of school achievement, including grades, school dropout, and test scores (see Appleton et al., 2008; Fredricks et al., 2004). School engagement decreases as students progress through school (Bond et al., 2007; Marks, 2000), and low engagement has been tied to substance use, risk behaviors, and mental health problems. For example, low school engagement among adolescents is longitudinally associated with greater odds of substance use; depressive symptoms, and not completing high school compared to students with higher school engagement (Bond et al., 2007). Cross-sectionally, low school engagement is associated with greater odds of substance use, depressed mood, suicidal ideation, physical fights, and engaging in sexual intercourse compared to higher levels of school engagement among adolescents (Carter et al., 2007).

Strategies to Enhance School Engagement

Researchers have identified various methods for enhancing school engagement for students. For example, enhancing students’ feelings of competence, autonomy, and relatedness promotes school engagement (see Fredricks et al., 2011; Wang & Eccles, 2013). Offering choices for school activities, connecting academic tasks to students’ personal interests and goals, clearly providing goals and expectations, and providing opportunities for students and staff to connect and work together enhance engagement (see Newman, 1981; Wang & Eccles, 2013). Further, teacher support is associated with increased student engagement (see Fredricks et al., 2004; Wang & Eccles, 2013). Lastly, peer acceptance is associated with greater school engagement, peer rejection is associated with lower school engagement (see Fredricks et al., 2004), and emotional support from peers is associated with school engagement among middle school students (Wang & Eccles, 2013).

One intervention designed to enhance school engagement is the Freshman Success (FS) Intervention (Flannery et al., 2020). Incorporating a 12-lesson instructional curriculum, students in upper-level grades were trained to support teacher-delivered curriculum to all ninth-grade students. Results indicated peers delivered their component of the intervention with high levels of fidelity (94% according to study raters). Feasibility data were not reported, but FS evidenced small positive effects on cognitive engagement and school attendance and a large effect on credits earned in ninth grade. Because the participants in this study were in the ninth grade, the amount of benefit possible may have been limited due to potentially already experiencing consequences of school disengagement. Further, they did not specifically evaluate the effects on an at-risk population, like those with attention problems, who may need more intensive intervention than typically developing students.

School Engagement and Youth with Attention Problems

For youth with attention problems, school impairment typically includes difficulties completing homework assignments, low and failing class grades, low standardized test scores, significant school absences and tardiness, grade retention, and high rates of school dropout (Booster et al., 2012; Bussing et al., 2010; DuPaul & Langberg, 2014; Kent et al., 2011). Thus, these students experience poor behavioral school engagement (e.g., school absences and tardies), decreased cognitive engagement (e.g., difficulty using learning strategies to achieve), and decreased emotional engagement (e.g., difficulties in peer relationships; Sibley et al., 2010). Thus, in addition to interventions aimed at improving school engagement for all students, an important focus of intervention work for youth with attention problems may involve developing and using skills to improve school functioning. Intervening at the middle school level may be particularly timely as this represents a time of increased challenge for youth with attention problems (Wolraich et al., 2005) due to increased expectations for independence in completing schoolwork (Evans et al., 2005).

Interventions for School Functioning

There are several school-based interventions designed to improve school functioning skills for youth with attention problems that do not explicitly target school engagement but may enhance school engagement by improving school functioning. Evidence indicates that organization training interventions, which focus on teaching organization skills, providing multiple opportunities to practice skills over time, and providing performance feedback, are among the most effective (Evans et al., 2018). Organization training interventions can be delivered by school staff (Challenging Horizons Program [CHP], Evans et al., 2016) or peers (i.e., Students Taking Responsibility and Initiative through Peer Enhanced Support (STRIPES); Sibley et al., 2020). Evaluations of CHP demonstrated improvement in organization and time management skills and grade point average (GPA) (Evans et al., 2016), but when delivered by school staff during the school day, staff demonstrated poor implementation, and few student gains (Evans et al., 2016). This lack of robust effects may be due to limitations of school staff time to deliver the intervention during the school day, suggesting that this approach may not be feasible for many schools. In the STRIPES program, high school-aged peers delivered an organization training, problem-solving, and goal setting intervention to ninth-grade students with attention problems (Sibley et al., 2020). Results of STRIPES indicated general acceptability of STRIPES but some limitations with integrity (Sibley et al., 2020). For example, students attended an average of just over five of the intended 16 after school sessions, and similarly low attendance at sessions regardless of whether schools chose to have sessions after school or during lunches (M = 5.83, SD = 4.04), although attendance was higher in a pullout model (M = 8.42, SD = 1.78). Evaluation of STRIPES indicates good acceptability and fidelity of peer-delivered interventions for students with attention problems.

One method to enhance feasibility and attendance could be to combine peer delivery with adult support. Students may receive interventions delivered with high levels of adherence by a highly available resource in schools (i.e., other students) that allows them to connect with peers and adults. Attendance may be enhanced by adult support without requiring staff time to serve as direct providers. In addition to the benefits of increased feasibility of peer-delivered intervention that saves school staff time, the potential to leverage peer delivery is compelling because it allows for direct intervention on school engagement. When peers deliver interventions, they may intervene directly to facilitate emotional school engagement by positively connecting students with each other. Also, peers highly engaged in school may be viewed as credible models in the same social context. Finally, because middle school precedes many risk-taking opportunities that can decrease school engagement and coincides with the start of increased academic struggles commonly experienced by youth with attention problems, intervention during middle school may be the most potent time for intervention.

TEAM

Capitalizing on the strengths of both peer- and adult-delivered interventions for youth, we developed Together Engaging and Achieving Meaningfully (TEAM), which aims to improve school engagement among middle school students with attention problems. Prior to Study One, TEAM Version One (see Supplementary Fig. 1) was initially designed to be a semester-long hybrid peer-delivered adult-supported school-based intervention. Based on the literature, it was designed to incorporate strategies to enhance school engagement (i.e., providing opportunities to become involved in school, forming positive connections with teachers and peers, and connecting academic tasks to students’ interests) and evidence-based intervention strategies to address impairment common for students with attention problems (i.e., organization training, problem-solving training). As initially designed, TEAM Version One includes two types of intervention sessions: (1) weekly “coaching sessions” led by peer coaches and (2) bi-weekly whole-group “skills and opportunities groups” led by the project clinician and school staff. Additionally, initial implementation support activities include two-hour coach training workshops prior to starting TEAM and bi-weekly coach supervision meetings with the project clinician. Descriptions of TEAM Version One activities and hypothesized mechanisms of change prior to the current study activities are described below. Subsequent iterations of TEAM (Versions Two and Three) were developed after each of the three studies based on study findings and these are described later in this manuscript and in the Supplementary Materials.

Coaching Sessions During weekly coaching sessions between student participant–coach pairs, coaches develop a supportive relationship with participants, set and check goals, and deliver an organization training intervention shown to be effective with middle school students with ADHD (Evans et al., 2016, 2021). The organization training program consists of goal setting and coaching, monitoring, and frequent feedback about the organization of participants’ binders according to an organization checklist developed in CHP. TEAM takes place during the school day (e.g., during overlapping “specials” period [e.g., PE, study hall]) or before the school day begins. The peer coach either goes to the guidance office where meetings take place, and the student participant is called to the office, or an adult facilitates both the coach and participant meeting. The peer coach then meets with the student participant in an office, where there is an adult close by for monitoring. Coaching sessions begin with the coach building rapport, introducing the organization system for the participant’s binder(s), and assisting the student’s implementation of the system. The coach provides information about tracking the organization system in each meeting using an organization checklist. Additionally, the coach facilitates the student setting a school-related SMART goal (i.e., a Specific, Measurable, Achievable, Relevant, and Time-bound) for the next week to enhance motivation. Subsequent sessions involve the coach checking the student participant’s use of the organization system, providing corrective feedback, checking progress on the goal, and setting new or additional goals. Coaching sessions are designed to last no longer than 20 min.

Coach Training Each coach participates in a two-hour-long training workshop. Training workshops teach coaches about the purpose of TEAM, including the definition of school engagement; components of TEAM; intervention sessions and activities; how to deliver core components of the coaching session, including rapport building, goal setting, and the organization training intervention; how to lead a coaching session; and how to complete the fidelity checklist. In training workshops, coaches practice delivering each intervention component, lead a role play coaching session, and they receive feedback. Training workshops also include training coaches on confidentiality and responding to safety concerns. Coaches take a quiz at the end of the training workshop regarding TEAM delivery and sign confidentiality pledges.

Supervision Coaches meet in group or individual meetings with the project clinician every two weeks to ask questions about implementation and ensure fidelity. Supervision meetings take place during coaches’ overlapping specials periods and are designed to last no longer than 30 min. In supervision meetings, coaches describe problems encountered in coaching sessions and problem-solve to address them.

Skills and Opportunities Groups Participants, coaches, and the project clinician meet every two weeks for one-and-a-half hours for after school skills and opportunities groups. During groups, student participants share school-related goals, their approach to reaching the goals, and describe their measures of success. The group engages in brief problem-solving regarding students’ goals. In the second half of each group session, a school staff member joins the group to provide information about a school or community activity (i.e., club, sports team/league) and leads the group in an activity related to it. They provide information about how to join the activity and skills relevant to be successful in the extracurricular activity and the classroom. School staff receive no formal training in leading a group. Prior to the group, the project clinician provides information about what they will be asked to do in the group and offers to meet with the school staff to provide additional support if desired.

TEAM Mechanism of Action

TEAM targets school engagement in two primary ways. First, one of the obstacles to school engagement for students with attention problems is academic failure and this is often due to failure to complete assignments and other tasks compromised by disorganization. Organization training, problem-solving, and goal setting focused on improving these common areas of failure to help students improve their academic functioning and therefore reducing this obstacle to school behavioral and cognitive engagement. The second mechanism of action involves enhancing personal connections with peers and school staff. This directly addresses school engagement by building relationships that should enhance emotional engagement through the relationship with the coach and the portion of the Skills and Opportunities groups that encouraged participation in school and community activities. With enhanced skills (to enhance success), and positive connections with peers and adults in schools, school engagement may increase. Increased school engagement is expected to result in improved school functioning (see Fig. 1).

Fig. 1
figure 1

Hypothesized mechanisms of team intervention

Full size image

Current Study

The current study is a development and implementation study of TEAM, an intervention developed to improve school engagement among middle school students with attention problems. This study was guided by the DFM of implementation (Weisz et al., 2005). The DFM guides researchers to focus on implementation in the intended intervention context and implement and evaluate treatments into clinical practice early in development (Weisz et al., 2005). The DFM guides researchers to take this approach to enhance the external validity of intervention development studies within clinical practice while maintaining strong research methodology (Weisz, 2015). One of the first steps in the DFM (Weisz et al., 2005) is to develop a draft of the treatment protocol based on the literature and clinical experience (described in the TEAM section above) and receive feedback from school partners to iteratively develop and refine TEAM. As such, in Study One we shared our protocol (TEAM Version One) in meetings with partners in community development team (CDT) meetings to gather feedback on TEAM and make revisions informed by their feedback. The next step involves conducting a case study to implement the intervention in the target context and evaluate how results can inform future development and implementation prior scaling up with larger trials. Thus, Study Two is a single-case pilot study with one student participant and one student coach. Finally, after iterative protocol revisions based on the case study the next step of the DFM is a small pilot implementation trial. Study Three is a non-randomized pilot with 10 student participants and six student coaches. In Study One, we addressed the lowing research question: In line with the DFM, what are flexible aspects of TEAM as initially designed that can be tailored to the intervention context based on staff and student feedback? In Study Two we addressed the following research question: How can TEAM be implemented in a way that is sensitive to the intervention context and maintains fidelity to the core intervention components? In Study Three we attended to the following research questions: (1) To what extent can TEAM be implemented with fidelity? (2) How feasible and acceptable is TEAM? (3) How satisfied are partners with TEAM implementation and effects? (4) What are the preliminary effects of TEAM and what are partners’ perceptions of the effects of TEAM? See Fig. 2 and Supplementary Fig. 1 for a timeline of current study activities and guiding research questions.

Fig. 2
figure 2

Brief description and sequence of TEAM revision over the course of this study

Full size image

Method

All study procedures were approved by the partnering school and University’s Institutional Review Board and took place in one rural middle school in the Midwestern USA. First, we describe the intervention context (i.e., the school in which we conducted this study), then we describe the participants and procedures for Study One (CDT meetings), Study Two (single-case pilot), and Study Three (non-randomized pilot).

Intervention Context

The current study was conducted in one middle school (grades five through eight) serving approximately 1,000 students and employing approximately 65 full-time teachers in the rural Midwestern USA. Most of the students in this school are White (approximately 96%), and approximately half qualify for free school lunch. Many students at this school are uninsured or under-insured. There are few licensed psychologists and child psychiatrists available in the community, and it is approximately 50 miles to the closest children’s hospital. As such, accessing evidence-based psychosocial intervention for students with attention problems can be difficult.

The project clinician in this study (the first author) was a doctoral student in clinical psychology in her final year of doctoral training. As part of her graduate training, she had multiple years of supervised experience providing school mental health services and conducting school mental health research in several schools, including one year in the current middle school providing school-based intervention, assessment, and consultation services. The project clinician was supervised by a licensed clinical psychologist (the second author) with expertise in school mental health services and research.

Study One: CDT Meetings

Research Question: What are flexible aspects of TEAM as initially designed that can be tailored to the intervention context based on staff and student feedback?

Method

Participants

Participants were three school staff and six fifth- through eighth-grade students at the participating middle school. Participants were not asked to provide their demographic information.

Procedure

Participants were recruited in the fall of the 2022–2023 academic year using emails to all staff and students at the participating middle school. Adult participants provided informed consent to participate. Caregivers of students provided informed consent, and students provided assent to participate. Adults were eligible to participate if they were staff members employed at the participating middle school. Students were eligible to participate if they were students enrolled in the participating middle school. CDT meetings were held with school staff and students in separate meetings. Three student CDT meetings were held during lunch, and two staff CDT meetings were held after school. At the start of each CDT meeting, the project clinician provided information about the purpose of the meeting, informed them their responses would be anonymous, informed them they did not have to answer questions, gave participants the opportunity to ask questions, and then asked the same set of questions regarding participants’ thoughts about the intervention and how the intervention could fit well within the participating middle school (see CDT Meeting Interview Guide in Supplementary Materials). Participants were encouraged to add to each other’s responses. The project clinician asked follow-up questions when appropriate. CDT meetings were audio recorded and transcribed by the first author.

Data Analytic Plan

Qualitative data from CDT meetings were analyzed by the first author using thematic analysis (see Braun & Clark, 2006). Themes were coded by examining the prevalence of themes at the data item level. Data were coded according to theoretical thematic analysis, to allow for a greater focus on the research questions of the current study compared to an inductive approach. A semantic approach was used to describe the data. Transcripts of interviews were evaluated to generate initial codes. Then, codes were extracted into themes. Candidate themes were reviewed and then defined and refined to ensure coherent themes were identified that fit within the data.

Results

Results from CDT meetings revealed seven themes regarding school partners’ perceptions of TEAM. Quotes from school partners representing each theme may be found in Table 1 of the Supplementary Materials.

  • Theme 1: It is Valuable to Connect to Peers and Adults in School. This theme reflects the participants’ reports of the value and importance of students connecting to adults and peers in the school.

  • Theme 2: It is Important to have a Peer Coach. Participants described benefits of having a mentor, as well as the increased relatability and credibility that peer coaches can provide.

  • Theme 3: There are Rewarding Aspects of TEAM. This theme reflects rewards and benefits of TEAM identified, including gaining skills and helping others.

  • Theme 4: Additional Things to Include or Emphasize in TEAM. This theme reflects strategies or activities that school partners named as being important to include or emphasize in TEAM, including emphasizing the brevity and frequency of coaching sessions, incorporating more activities, emphasizing students’ autonomy in TEAM, incorporating tangible rewards for progress, sending coaches to meet during study hall, and choosing activities that students find interesting or meaningful.

  • Theme 5: There are Barriers to TEAM Implementation. This theme reflects school partners’ views of the barriers to implementation, including students not participating due to being involved in other activities, not having time to participate, not wanting to participate, and/or displaying oppositional behaviors.

  • Theme 6: There are Characteristics of Strong Peer Coaches. This theme reflects school partners’ views of characteristics of students who would make strong peer coaches, including being a good role model, finding mentoring rewarding, caring about others, having a stronger skill set than younger students, being trustworthy, being responsible, being understanding, being kind, having some experience with attention/behavior problems, and not having a disciplinary history.

  • Theme 7: There are Strategies for Appropriately Matching Pairs. This theme reflects school partners’ views of how to match student–coach pairs, including by common experiences, common identities, common interests, and similar personalities.

Lessons Learned

Results from CDT meetings indicated that students and staff found aspects of TEAM acceptable and valuable. As a result of findings from CDT meetings, TEAM implementation was revised in the following ways: (1) Increasing flexibility and autonomy of allowing pairs to choose to hold meetings at a time that is preferable to them, including allowing for before school or after school meetings, (2) Prioritizing a lack of disciplinary history for coaches in exclusion criteria, and (3) Considering personality and common interests with school counselors when matching pairs. In addition, student–coach pairs were matched considering overlapping free periods in schedules in consideration of the logistical barriers mentioned. No revisions to TEAM content were made based off study findings.

Study Two: Single-Case Pilot

Research Question: How can TEAM be implemented in a way that is sensitive to the intervention context and maintains fidelity to the core intervention components?

Method

Participants

Participants included one fifth-grade male student participant with elevated attention and academic problems and one eighth-grade male student coach. Coaches were not asked to provide demographic information. The student participant was 11 years old, White, and not Hispanic/Latiné. His caregiver completed some college and reported a total household income between $25,000 and $49,999. The student participant was not taking medication but received community-based psychosocial intervention for ADHD.

Procedures

The student participant was recruited in the fall of 2022 when interested parents contacted the first author in response to study information distributed by school counselors. The student and his caregiver provided assent and consent to participate in the study. To be eligible to participate, the student had to be in fifth, sixth, or seventh grade at the participating middle school; display elevated levels of inattention according to the parent version of the ADHD Rating Scale-5 (DuPaul et al., 2016); display low levels of school engagement as reported by student- or teacher-report on the Engagement versus Disaffection for Learning scale (EvsD; Skinner et al., 2009); display academic difficulties (i.e., a GPA less than 2.5 for most recent grading period or in special education); and demonstrate an abbreviated full scale IQ of 80 or above on the Wechsler Abbreviated Scale of Intelligence-Second Edition (WASI-II; Wechsler, 2011). The student was not eligible if diagnosed with autism spectrum disorder (ASD) or had a history of any serious disciplinary referrals. The student, his caregiver, and the teacher of the class in which the student had the lowest grade completed measures at eligibility assessment. At the end of implementation, the student participated in an interview regarding his experience participating in TEAM (see Supplementary Materials for the interview guide). The student participated was compensated for his completing study measures at each time point.

The student coach was also recruited in the fall of 2022 when interested parents contacted the first author in response to study information distributed by school counselors. The student and his caregiver provided assent and consent to participate in the study. To be eligible to participate, the student coach had to be in eighth grade at the participating middle school, have a cumulative GPA of 3.0 or above, and have school counselor endorsement. The student was not eligible if he was diagnosed with ASD or had a history of any serious disciplinary referrals. The student coach and participant were matched by sex and the pairing was endorsed by school counselors. At the end of the pilot, the student coach was asked to complete study measures regarding acceptability and satisfaction. He also participated in an interview regarding his experience participating in TEAM (see Supplementary Materials for the interview guide). The coach was compensated for his participation.

School staff who participated in skills and opportunities groups were recruited via phone or email by the project clinician from a list of school staff leading school activities provided by the school principal.

Measures

Attention problems. Caregivers and teachers completed the ADHD Rating Scale-5 (ARS-5 Home and School Versions; DuPaul et al., 2016) at eligibility assessment (T1). The ARS-5 is a measure of ADHD symptoms and related impairment with respondents rating the frequency of each symptom in the last six months on a scale from “0,” (never or rarely), to “3,” (very often). Item scores are used to derive Inattention (IA) and Hyperactivity-Impulsivity (H/I) subscales. The test–retest reliability, construct validity, discriminant validity, and predictive validity of the ARS-5 are good (DuPaul et al., 2016). In this study,Footnote 1 internal consistency of the ARS School Version IA (α = 0.94) and H/I (α = 0.96) and ARS Home Version IA (α = 0.92) and H/I (α = 0.97) subscales were excellent.

School Engagement. Student participants completed the Engagement versus Disaffection for Learning (EvsD; Skinner et al., 2009) Student Report and teachers completed the EvsD Teacher Report. The EvsD measures behavioral and emotional engagement. The Behavioral Engagement subscale assesses students’ effort and persistence involved in learning. Emotional Engagement assesses emotion connected to motivation when learning. Responses are rated on a 4-point scale from “1” (not at all true) to “4” (very true), with higher scores indicating more engagement. Behavioral and emotional subscales are computed by averaging items for each subscale. A Total Engagement scale is computed by averaging across behavioral and emotional engagement subscales. Evidence supports the factor structure of the EvsD, and it has moderate to high test–retest reliability (Skinner et al., 2009). Internal consistency for the teacher-report Total (α = 0.92), Behavioral (α = 0.77), and Emotional (α = 0.90) and student-report Total (α = 0.96), Behavioral (α = 0.93), and Emotional (α = 0.93) Engagement scales were acceptable to excellent.

Demographics. Caregivers provided their child’s demographic information at T1.

Treatment Fidelity. Treatment fidelity was measured using research team-created fidelity assessment tools for coaching and skills and opportunities group sessions. Reviewers marked whether the core aspect of the intervention was delivered in each session (yes/no).

After each coaching session, coaches completed a fidelity checklist indicating whether they delivered each intervention component. The project clinician reviewed audio recordings of each coaching session and completed the fidelity checklist to calculate the coach’s fidelity. Trained research assistants also independently reviewed audio recordings and completed fidelity checklists for a portion of all individual coaching sessions to determine treatment adherence and competence and to calculate interrater agreement. After each skills and opportunities group, the project clinician completed a research team-created fidelity checklist indicating whether she and the participating school staff member engaged in each intervention component in the group. The project clinician reviewed audio recordings of each group session and completed the fidelity checklist to calculate the project clinician and school staff member’s fidelity delivering intervention components. Trained research assistants independently reviewed audio recordings and completed fidelity checklists for a portion of all group sessions to determine treatment adherence and to calculate interrater agreement.

Treatment Acceptability. The student participant and coach were asked to provide treatment acceptability ratings at the end of implementation in the form of open-ended written and interview questions conducted by the project clinician regarding the most and least favorable aspects of the intervention, perceived benefits of the intervention, perceived negative effects of the intervention, and recommendations for what to change about the intervention (see Supplementary Material for open-ended written and interview questions).

Attendance. Attendance for student participants and coaches at each coaching session and skills and opportunities group were recorded. Coaches’ attendance at supervision meetings was recorded.

Data Analytic Plan

Treatment fidelity was evaluated by computing average session fidelity for 60% of individual coaching sessions and 50% of group sessions that were held during the single-case pilot. Disagreement in fidelity ratings was resolved by the project clinician. The acceptability of TEAM was evaluated by the first author reviewing written responses to acceptability questions and transcripts of their interview responses using content analysis. Satisfaction with TEAM was measured by computing the student participant and coach’s enjoyment and usefulness ratings. Student attendance was evaluated by calculating the total number of coaching and skills and opportunities groups attended. The total number of coaching, group, and supervision sessions the coach attended was calculated.

Results

Fidelity

For coaching sessions, the coach’s fidelity was rated for the core intervention components including being encouraging, showing interest, being accurate in feedback, and setting goals. There was perfect agreement (κ = 1.00) between coders across sessions for each component. Fidelity was 100% for the being encouraging and showing interest component. For being accurate in feedback, the coach reached 66.67% fidelity. For setting goals, the coach reached 40% fidelity.

Based on feedback from the coach after the first session, coaching sessions were revised. The coach gave feedback that he thought it was important to do an ice-breaking activity in the first session, meaning he did not complete some of the required items. As a result, the first TEAM coaching session was revised to allow for an ice-breaking activity and not include any organization training or goal setting activities, as initially designed. As a result of less-than-ideal fidelity with goal setting and the organization check, it was decided that supervision with the coach would occur every week for approximately 20 min rather than what was initially planned as hour-long every-other-week.

For skills and opportunities groups, fidelity ratings were completed for the project clinician-delivered component and the school staff-delivered component. Fidelity was rated for each of the core intervention components: Practitioner—facilitating students identifying problems, facilitating the group identifying solutions, facilitating the student choosing a solution to try, checking in on problems and solutions from the previous group session (if applicable); Staff—showing interest in the students, providing information about the activity/club/group they are involved in, describing the academic or school skills needed to be successful in the activity/club/group, and leading the group in a fun activity related to it. There was perfect agreement across all components between coders for the first group session (κs = 1.00). There were not multiple ratings available for the second group session. Fidelity for the project clinician and staff members were 100% across all session components.

Feasibility

The student–coach pair decided to hold coaching sessions before the start of the school day. To facilitate the pair remembering to meet, the project clinician implemented email reminders and met the student participant at the bus to walk him to the office where sessions were held. Overall, this resulted in approximately 10 min of time per week to facilitate students remembering to meet. Over the semester, the pair attended six coaching sessions and missed no sessions (100% attendance rate). They (the participant and coach) attended two skills and opportunities groups and missed no groups (100% attendance rate). One group was held during the school day during overlapping free periods. The second group was held after school. The coach attended seven supervision meetings and missed no supervision meetings (100% attendance rate). Supervision meetings lasted between 5 and 20 min each week.

Acceptability

The coach reported most enjoying “getting to interact with the student and watch them improve.” The coach and student participant both reported that skills and opportunities groups were the most fun part of TEAM. Even though the student participant reported that “nothing” about TEAM was not fun, the coach reported “…waking up in the morning and having to leave [hanging out] with friends and come [to TEAM coaching sessions] then go back” was the least fun part of TEAM. He reported that he also “didn’t like missing gym [class].” He reported that it would have been more fun to meet during a class where “you might not miss the social aspect.” Both the student participant and the coach reported finding working with another student rewarding. The student participant reported, “it’s a fellow student so he knows what I’ve been through.” However, the coach reported it was challenging to manage when “he … didn’t understand everything completely” and got “distracted.” Both the student participant and coach reported enjoying skills and opportunities groups, but the coach reported they could be more enjoyable if they were “shorter,” and the student participant reported it was challenging to have them “after school…because [they] needed to coordinate pick up [with caregivers].”

Lessons Learned

Results from Study Two suggested the potential of TEAM for high levels of attendance, acceptability, and satisfaction. However, lessons learned from the single-case pilot emphasized the necessity of changing TEAM in the following ways (1) enhancing coach fidelity by revising supervision meetings with coaches to occur every week; (2) incorporate “booster” trainings in coach supervision meetings to review, practice, and refine coaches’ delivery of a coaching session component; and (3) revising the first TEAM coaching session content so that the first coaching session included an “ice breaker” or rapport-building focus and no organization training or goal setting activities.

Study Three: Pilot Implementation

Research Questions: (1) To what extent can TEAM be implemented with fidelity? (2) How feasible and acceptable is TEAM implemented? (3) How satisfied are partners with TEAM implementation and effects? (4) What are the preliminary effects of TEAM and what are partners’ perceptions of the effects of TEAM?

Method

Participants

Student participants were 10 middle school students (90.0% male) in fifth (50.0%), sixth (10.0%), and seventh (40.0%) grade with elevated attention and behavior problems and academic difficulties (see Table 1). Coaches were six eighth-grade students who evidenced high levels of prosocial behavior and academic achievement (see eligibility criteria).

Table 1 Demographics for non-randomized pilot
Full size table

Procedure

Ten student participants were recruited in the fall and winter of the 2022–2023 school year at the participating middle school. Students were recruited with mailed letters advertising the study to caregivers of all fifth- through seventh-grade students at the participating school. Interested parents contacted the research team and scheduled eligibility assessments. Students and their parents provided assent and consent to participate. Student participants were eligible to participate if they met the eligibility criteria described in the Single-Case Pilot Procedures. Student participants were allocated to one of two conditions: (1) TEAM or (2) control (community care, e.g., school and community supports as usual). Given the small sample size, we did not utilize random assignment because it is unlikely to produce two groups equivalent on key variables. Prior to allocation, students were ranked from low to high school engagement (i.e., EvsD) scores. Using this list participants were assigned to conditions by alternating assignment to the TEAM and control group. Two consecutively ranked students in the middle were sorted into the TEAM group to achieve the assignment of six participants in TEAM and four in control. Student participants, one of their caregivers, and one of their teachers completed measures at three time points: eligibility assessment (December 2022 to early March 2023, T1), the middle of implementation (April 2023, T2), and the end of the intervention (May 2023, T3). Teachers who completed study measures for each student participant were selected because they taught the course with the student’s lowest grade at eligibility. At the end of implementation, student participants in the TEAM group completed individual interviews where the project clinician asked questions regarding their experiences participating in the intervention. All student participants in the TEAM and control groups were compensated for completing study measures at each time point.

Six student coaches were recruited during the fall and winter of the 2022–2023 school year at the participating middle school. Coaches were recruited with mailed letters advertising the study to caregivers of all eighth-grade students at the participating school. Students and their parents provided assent and consent to participate. Coaches were eligible if they met the eligibility criteria described in the Single-Case Pilot Procedures. Coaches were matched with the six student participants in the TEAM condition in pairs based on sex, common interests, and with school counselors. At the end of the study, coaches were asked to complete study measures regarding acceptability and satisfaction with TEAM, and they completed individual interviews regarding their experiences participating in the intervention. Coaches were compensated for participating in the study.

School staff who participated in skills and opportunities groups were recruited via phone or email by the project clinician from a list of school staff leading school activities provided by the school principal.

Measures

The same measures used in the single-case pilot were used in Study Three. Additional measures were included and are described below.

Treatment acceptability was measured by collecting coach ratings on the School Intervention Rating Form (SIRF; Harrison et al., 2016) at the end of implementation. Items on the SIRF were adapted to the school level rather than the classroom level and to reflect coach-participant interactions. SIRF items are rated on a seven-point scale ranging from unfavorable (‘0’) to favorable (‘6’) ratings. The SIRF is organized into three factors: Suitability, Perceived Benefit, and Convenience. The Suitability factor reflects the degree to which intervention implementation is understood, acceptable, and reasonable and willingness to implement the intervention. Perceived Benefit represents perceptions of improvement and the severity of students’ behaviors. Convenience represents the degree to which resources must be in implementation and negative effects (Harrison et al., 2016).

Student participants were asked to provide ratings of their working alliance with their coaches at T2 and T3 using an adapted version of the Working Alliance Inventory-Short Revised (WAI-SR; Hatcher & Gillaspy, 2006). Items are rated on a seven-point scale from “never” to “always” (Hatcher & Gillaspy, 2006). There are three subscales of the WAI-SR: the Goal Scale, the Task Scale, and the Bond Scale. Higher scores reflect a stronger working alliance. In the current study, internal consistency of the Goal (α = 0.89), Task (α = 0.94), and Bond (α = 0.99) scales were good to excellent.

Satisfaction. Satisfaction with TEAM was measured by asking student participants and coaches to rate how satisfied they were with components of the intervention using a research team-created measure. Students were asked to rate how much they enjoyed components of TEAM and how useful they found components of TEAM using a four-point Likert scale, with higher scores reflecting higher levels of enjoyment and usefulness. Average enjoyment and usefulness scores were computed (range: 0 to 3 with higher scores indicating greater enjoyment and usefulness).

Caregivers and teachers completed the Children’s Organizational Skills Scale (COSS; Abikoff & Gallagher, 2009) at T1, T2, and T3. The COSS is a measure of organization, time management, and planning skills problems for children through age 13. Items are rated on a four-point scale from “1” “Hardly ever or never” to “4” “Just about all of the time.” The COSS produces a Total score and three subscales: Task Planning, Organized Actions, and Memory/Materials Management. Higher scores indicate greater organizational skills problems. The COSS has high test–retest reliability, and construct, predictive, and discriminant validity (Abikoff & Gallagher, 2009). The COSS subscales evidence moderate to high internal consistency (α = 0.64-0.91) (Abikoff & Gallagher, 2009; Molitor et al., 2017). In this study, internal consistency for the Parent Version (Total (α = 0.91), Task Planning (α = 0.92), Organized Action (α = 0.81), Memory & Materials Management (α = 0.96)) and Teacher Version scales (Total (α = 0.87), Task Planning (α = 0.91), Organized Action (α = 0.78), Memory & Materials Management (α = 0.97)) were moderate to excellent. In this study, one participant turned 14 prior to administration at T2 and T3. For this student, raw scores were converted to T scores for the oldest available age group.

Student participants completed the Psychological Sense of School Membership (PSSM; Goodenow, 1992) at T1, T2, and T3. The PSSM is an 18-item self-report rating scale that measures students’ perception of feeling accepted, respected, included, and supported by others in school (Goodenow, 1993). Items are rated on a five-point scale from “1” meaning “not at all true” to “5” meaning “completely true.” Responses are averaged to compute a total PSSM score. The PSSM evidences good construct validity and high levels of internal consistency (α = 0.80 to 0.88), and it is correlated with aspects of academic achievement and school engagement (Goodenow, 1993). Internal consistency of the PSSM was excellent (α = 0.96) in the current study.

Students’ grades were collected from school records at T1, T2, and T3. GPA was calculated for participants’ grades at each time point using the weighting system, where an A was equal to 4.0, B to 3.0, etc.

Data Analytic Plan

Treatment fidelity was evaluated by computing average session fidelity for 66% of all coaching and group sessions that were held. Disagreement in fidelity ratings was resolved by the first author. The acceptability of TEAM was evaluated by averaging coaches’ SIRF scores. Satisfaction with TEAM was measured by computing student participants’ and coaches’ enjoyment and usefulness ratings. Coaches and students’ responses to written and interview questions were transcribed and analyzed with content analysis conducted by the first author. Attendance was evaluated by calculating the average number of coaching and group sessions student participants and coaches attended. Cohen’s d effect sizes were the primary approach for evaluating effects of TEAM over the course of the intervention and compared to the control group. There was no variability in COSS Organized Action T scores across reporters and time points, so this variable was not included in analyses.

Results

Fidelity

For coaching sessions, coaches’ fidelity was rated for the core intervention components. There was strong to perfect agreement between coders across sessions for core components (i.e., being encouraging (κ = 1.00), showing interest (κ = 1.00), being accurate in feedback (κ = 0.90), and setting goals (κ = 0.94)). For being encouraging and showing interest, there was 100.00% fidelity across coaches. For being accurate in feedback, there was 90.00% fidelity across coaches. For setting goals, there was 76.67% fidelity across coaches.

For skills and opportunities groups, fidelity ratings were completed for the clinician-delivered component (i.e., the problem-solving skills intervention) and the school staff-delivered component (i.e., the school activity component). There was perfect agreement between coders for the first and second groups across all session components (κs = 1.00). Fidelity for the project clinician and staff were 100% across all session components.

Feasibility

Overall, participant–coach attendance at coaching sessions was high (85.37% attendance). Students missed coaching sessions due to school absences and unanticipated closure days. Reminders for coaching sessions utilized in the single-case pilot were utilized in this trial and required approximately 15 min of clinician time per week. All skills and opportunities groups were held after school, and attendance was variable. Overall, coach attendance at skills and opportunities groups was low (38.89% attendance) but was higher for student participants (77.78% attendance). Reasons for coaches missing groups were due to conflicting extracurricular activities. The primary reason for students missing skills and opportunities groups was school absences. The project clinician emailed reminders about staying after school, which required approximately 6 min of clinician time per week. Across the trial, clinician time required to train coaches, hold supervision meetings, hold skills and opportunities groups, facilitate coaching session attendance, and send reminders to students and caregivers amounted to 18 h of time over nine weeks of implementation (i.e., two hours per week).

Acceptability and Satisfaction

Coach ratings indicated relatively high levels of suitability (M = 5.50, SD = 0.49), convenience (M = 6.08, SD = 0.73), and perceived benefit (M = 4.87, SD = 0.27) of TEAM. Student participants endorsed high enjoyment (M = 2.69, SD = 0.44) and usefulness (M = 2.86, SD = 0.24). Coaches also reported high enjoyment (M = 1.79, SD = 0.64) and usefulness (M = 2.08, SD = 0.43). Student participants endorsed high levels of working alliance with peer coaches at T2 (MGoal = 4.80, SDGoal = 0.33; MTask = 4.61, SDTask = 0.61; MBond = 4.88, SDBond = 0.31) and T3 (MGoal = 4.71, SDGoal = 0.60; MTask = 4.67, SDTask = 0.82; MBond = 4.67, SDBond = 0.82).

Across coaches, getting to know the student they were paired with was named as the most enjoyable part of TEAM. For student participants, skills and opportunities groups were named as the most enjoyable part of TEAM. Coaches identified challenging aspects of TEAM being managing “awkward silences” at the beginning, managing students’ off-task behaviors, and delivering aspects of the organization training intervention. Goal setting and organization training were commonly named as the most helpful part of TEAM by student participants and coaches. Student participants reported no unhelpful or not fun parts of TEAM. Some coaches and participants reported wanting to hold skills and opportunities groups during the school day instead of after school. Another coach reported TEAM could be improved by allowing coaches to pick with whom they were paired.

Preliminary Effects

Means and standard deviations of outcome variables by each condition are reported in Table 2. Cohen’s d effect sizes for the TEAM group over implementation (T1, T2, and T3), for the control group over implementation, and for TEAM compared to control group means are reported in Table 3. For student-reported Total and Emotional Engagement, results generally indicated small to large declines in engagement over implementation for both the TEAM (ds range: 0.18–0.87) and the control group (ds range: 0.34–1.01). However, results indicated a small difference in favor of the TEAM group on student-reported Total (d = 0.16) and Emotional Engagement (d = 0.22) at the end of implementation. Specifically, from T1 to T3, the TEAM group evidenced a small decrease in student-reported Emotional Engagement (MT1 = 3.22, SDT1 = 0.48; MT3 = 2.96, SDT3 = 1.15, d = 0.30), and the control group evidenced a large decrease (MT1 = 3.26, SDT1 = 0.27; MT3 = 2.80, SDT3 = 0.78, d = 0.79), suggesting lesser decreases in this aspect of school engagement for the TEAM group over the control group. From T1 to T3, results indicated medium improvement in teacher-reported Total Engagement for the TEAM group (d = 0.39) and large improvement for the control group (d = 1.05). There was a medium difference in favor of the control group on teacher-reported Total Engagement (d = 0.58). Similarly, results indicated modest improvements in teacher-reported Behavioral Engagement for the TEAM group (d = 0.37) and large improvements for the control group (d = 0.85). There was a medium difference in favor of the control group on teacher-reported Behavioral Engagement (d = 0.49). In general, results indicated more favorable outcomes for the TEAM group on sense of school membership. No other outcomes suggested favorable effects for the intervention over the control group.

Table 2 Descriptive statistics for outcome variables for non-randomized pilot
Full size table
Table 3 Cohen’s d effect sizes for non-randomized pilot
Full size table

Lessons Learned

Results from Study Three suggest that previous revisions made to TEAM implementation and content resulted in relatively high levels of coach fidelity, acceptability, feasibility, and satisfaction. These results also suggest that it may be beneficial to revise TEAM implementation in several ways. First, given low coach attendance at skills and opportunities groups, it may be beneficial to consider whether coach attendance at groups is important, as initially conceived in TEAM design. In this way, TEAM could be revised to either not ask coaches to attend groups or consider scheduling several groups at different times (e.g., during the school day, after school) during the week they occur to allow for increased options for coaches to attend. Given evidence of few benefits for academic gains (e.g., GPA) than reported in studies of organization training interventions (e.g., Evans et al., 2016), TEAM should be revised to allow for a higher dose of intervention by either having a year-long implementation period or increasing the frequency of coaching sessions to two or more times per week. Finally, individual responses to TEAM were variable and may be due to differences in characteristics between participants, namely medication status. A close examination suggests that in future studies should measure and consider the effects of factors such as medication and ADHD symptom severity, as these could influence participants’ response to TEAM intervention.

Discussion

The purpose of the current study was to evaluate a graded implementation of TEAM to iteratively develop and refine the implementation procedures. This graded implementation included CDT meetings (Study One), a single-case pilot (Study Two), and a non-randomized pilot (Study Three) over one school year at one rural middle school. Overall, results suggest that TEAM is a feasible intervention for eighth-grade students to deliver and school staff to support, acceptable to school staff and students, and satisfactory for participants. In line with the DFM (Weisz et al., 2005), a secondary focus was to evaluate TEAM preliminary effects to determine if there is a signal that TEAM results in improvements in school engagement to inform future intervention development. Evaluation of outcomes suggests some promise for TEAM in enhancing aspects of school engagement.

Feasibility

Across the CDT meetings, single-case pilot, and non-randomized pilot, results lend support to the feasibility of TEAM for schools in terms of coach ability to deliver TEAM with fidelity, participant attendance, and time required for adult support of implementation. Results from this study suggest that providing TEAM in schools increases accessibility to services for families. Additionally, the use of peers to deliver coaching sessions reduces the demand on school staff to directly provide services to individual students. TEAM’s development was informed by student and staff in CDT meetings, who identified several considerations regarding the feasibility of TEAM. They emphasized the importance of a peer coach, student–adult, and student–student connections at school, suggesting that TEAM intervention activities generally aligned with school partners’ values. They also identified areas where TEAM could be changed to improve feasibility. For example, staff and students identified the importance of emphasizing the rewarding aspects of TEAM, the brevity of coaching sessions, students’ autonomy in choosing when to meet, and the opportunity for students to participate in engaging activities. School partners also described barriers, including a lack of time, lack of overlapping schedules for meetings, and a lack of interest, that could limit feasibility. Finally, partners identified characteristics of strong peer coaches. With these suggestions, it was possible to improve the feasibility of TEAM in the single-case and non-randomized pilots.

Results from the single-case and non-randomized pilots lend support to TEAM feasibility in that eighth-grade coaches evidenced relatively high levels of fidelity to delivering core intervention components. As a result of low fidelity for goal setting, coach supervision in Study One was revised such that brief supervision meetings occurred every week, rather than longer supervision meetings every other week. Additionally, in response to coach feedback after the first coaching session in the single-case pilot, a rapport-focused first coaching session was incorporated into TEAM for the non-randomized pilot. For the non-randomized pilot, TEAM supervision meetings were redesigned such that supervision included review and practice with feedback of core intervention components. These revisions likely supported the enhanced fidelity that coaches in the non-randomized pilot demonstrated. On average, coaches in the non-randomized pilot delivered every intervention component with at least 76% fidelity. This finding supports results from other studies that middle school students can deliver interventions on par with fidelity demonstrated by high school students (e.g., Flannery et al., 2020; Sibley et al., 2020). Across the project clinician and staff who participated in the trials, fidelity was 100%, suggesting that TEAM skills and opportunities groups can be delivered with high levels of fidelity.

Attendance at TEAM coaching sessions was high across the single-case pilot (100%) and non-randomized pilot (85%). This high level of attendance is greater than that in STRIPES trials (Sibley et al., 2020). There are several reasons that could explain the higher level of attendance. First, in TEAM, student–coach pairs were allowed to choose their meeting times. In STRIPES, school staff chose a uniform method for all pairs to have meetings: after school or during school (see Sibley et al., 2020). Second, a project clinician facilitated pairs meeting in TEAM so that they would not forget their meetings. Staff time required to facilitate attendance at coaching sessions was still relatively minor at approximately 10 to 15 min per week for the students involved in both trials, and total staff time required to facilitate TEAM Version Two was approximately two hours per week for all six participants. This is much less time required to deliver an intervention to six students than would be required to deliver outpatient intervention to six adolescents and less time than what is needed to implement the mentoring version of CHP (Evans et al., 2016) at about three hours per week.

Attendance at skills and opportunities groups was more variable than attendance at coaching sessions. In the single-case pilot, student and coach attendance at groups was 100%. In the non-randomized pilot, student attendance was 78%, but coach attendance was only 39%. The primary reasons for students missing group sessions were being absent from school or forgetting. The project clinician implemented reminders to limit students forgetting to stay after school. For coaches, the primary reason for missing groups was scheduling conflicts with other after school activities. Thus, holding skills and opportunities groups after school likely limits some of the feasibility of TEAM because ideal coaches are students who are highly involved in school. It may be that holding skills and opportunities groups during the school day improves attendance; however, it would complicate scheduling.

Acceptability and Satisfaction

Overall, coaches and students rated TEAM as acceptable. Usefulness and enjoyment ratings across coaches and students in the single-case pilot and the non-randomized pilot were relatively high. Coaches’ ratings of satisfaction with TEAM were also relatively high. Importantly, the peer-led component was indicated as a highly acceptable and satisfactory aspect of TEAM. The working alliance between students and their coaches was high, on average, across each trial. In general, students reported finding the coach-delivered component to be rewarding. Students commonly commented on finding coaches to be more credible and understanding of their concerns compared to adults. Several coaches reported finding it challenging to manage inattentive behavior in coaching sessions. In supervision sessions, the project clinician worked with coaches to manage these behaviors effectively and appropriately. However, greater attention to developmentally appropriate strategies to help coaches manage such behaviors would be beneficial. Even though efforts were made to ensure participant and coach comfort with providing responses to interview questions (e.g., they were told their responses were anonymous and all feedback was helpful for appropriately refining TEAM) and the project clinician had developed strong rapport with participants and coaches over the course of the intervention, it is possible that they may not have felt comfortable providing critical responses because the project clinician conducted interviews.

Preliminary Effects

Results from the non-randomized pilot suggest that there may be some benefits of TEAM, with evidence suggesting some favorable outcomes for the TEAM versus the control group for aspects of emotional school engagement (i.e., student-report EvsD Emotional Engagement scores, PSSM scores). However, outcomes for improved behavioral engagement favored the control group (e.g., teacher-report EvsD Behavioral Engagement scores, GPA). There was some evidence for improvements in skills (e.g., teacher-report COSS Total score) among students in the TEAM group. Students and coaches in each trial reported beneficial effects of TEAM in end of implementation interviews, specifically improvements in organization skills. Evaluation of preliminary effects of TEAM in the current study are limited for several reasons, notably the small sample size, and thus firm conclusions regarding TEAM effects cannot be determined.

According to the DFM, examination of intervention effects during implementation is an important step in informing future iterations of an intervention (Weisz et al., 2005). Thus, examining implications of these effects is useful for informing future iterations and trials of TEAM. Along with the limitations of this study already noted, the lack of strong benefits in favor of the TEAM group may be due to pre-existing differences between students in the intervention and control group combined with the small sample size that limit the firmness of conclusions about TEAM effects. Future studies with large samples are needed to identify characteristics of children whose response to TEAM may be unique and control for effects, such as medication.

TEAM was implemented over one semester, and it may be that a greater dose of TEAM is needed to evidence benefits. Future trials should increase the dose of TEAM by implementing TEAM over one school year and/or increasing the frequency of sessions. Studies of school-based organization training interventions suggest it may take longer to see the full benefits of interventions (e.g., DuPaul et al., 2021). Thus, follow-up assessment of TEAM effects is important for future studies.

DFM

The DFM (Weisz et al., 2005), guided intervention development and implementation in the current study. The use of this model allowed the research team to bring TEAM into our intended intervention context early in development and involve students and staff. With this model, we were able to obtain valuable partner feedback that shaped TEAM development early. Future evaluation of TEAM is still needed to draw firm conclusions about TEAM feasibility, acceptability, and effects. However, involving partners early in development may enhance the likelihood that the TEAM intervention will be an effective intervention that is also externally valid to help bridge the research-practice gap.

Limitations

In addition to limitations already noted, a limitation of the current study is that it may be difficult to identify one or multiple school-based clinicians/facilitators who have time available to devote to facilitating TEAM in other schools and trials. However, if the counselor has two hours per week, TEAM may be a very cost-effective option. Coding disagreements were settled by the first author who was also the project clinician, and this could have introduced bias into the data. Additionally, the project clinician was actively involved in implementation of TEAM and the research project. Even though this may introduce bias into study results, it also allows for firsthand experience with TEAM implementation that is a valuable step in refining TEAM for scaled up implementation in a more methodologically rigorous trial. All students in this study were White persons, so future studies of TEAM should evaluate the potential for variability due to race or ethnicity. For example, if student–coach pairs differ from each other by race and/or ethnicity, conscious or unconscious bias may be associated with unique student–coach relationship quality for some pairs that could affect results. Finally, the small sample size and the lack of random assignment further limits conclusions about TEAM effects. TEAM effects should be evaluated in a large randomized controlled trial.

Future Directions

Some future directions for research already noted above include adjusting skills and opportunities groups to allow more options for coaches to attend, evaluating whether coach attendance at groups is associated with enhanced school engagement, having a longer implementation period or more frequent sessions, and conducting a large randomized controlled trial with a diverse sample to evaluate TEAM effects. The intervention and implementation were specifically designed to consider our school intervention context (i.e., a rural middle school serving many students who are un- or under-insured with few available licensed psychologists and child psychiatrists). Future studies should evaluate the appropriateness of TEAM in under-resourced urban, suburban, and other rural schools. Finally, studies of TEAM should evaluate hypothesized causal mechanisms.

Conclusion

Overall, the results of this intervention development and implementation study suggest the feasibility, acceptability, and school partner satisfaction with TEAM. In particular, the peer-delivered aspect of TEAM appears to be particularly feasible, acceptable, and satisfactory for those involved. The promise of this peer-supported school-based intervention for enhancing school engagement for middle school students with attention problems is notable considering limited school resources. Future research is needed to understand TEAM’s promise and effects on school engagement and hypothesized mechanisms of change.