Inclusive Preschool Practitioners’ Implementation of Naturalistic Developmental Behavioral Intervention Using Telehealth Training

A Correction to this article is available

This article has been updated

Abstract

This single-case investigation was designed to evaluate the effects of telehealth training on practitioner implementation of a naturalistic developmental behavioral intervention (NDBI). Six general education preschool practitioners engaged in an intervention with six children with varying disabilities in inclusive classroom settings. The telehealth training package included a collaborative approach to intervention planning, online training module, video self-evaluation, and performance feedback via videoconferencing. Following telehealth training, practitioners reached criteria for implementation fidelity and increased communication opportunities. Additionally, child participants increased communication behaviors above baseline levels. All behaviors generalized to a different activity context and maintained over time. Social validity was measured and results suggest high levels of acceptability for the telehealth training package.

Introduction

The early childhood years are a critical time for the developing brain, and young children with developmental delays and disabilities are at greater risk of health concerns, wellbeing, and educational attainment than children without delays and disabilities (Shonkoff and Phillips 2000). Hence, the quality of services provided in each educational environment a young child with a disability participates in is of utmost importance. Preschool inclusive classrooms, which integrate children with and without disabilities, are widespread in the United States with two-thirds of children with disabilities, ages three through five, participating in inclusive settings for some portion of their school day (U.S. Department of Education 2018). Yet, inclusive settings only support children if practitioners are proficient in evidence-based practices for children with disabilities (Odom et al. 2011).

Naturalistic developmental behavioral interventions (NDBIs) are evidence-based approaches that integrate applied behavior and developmental sciences (Schreibman et al. 2015). NDBIs are well matched for inclusive preschool classrooms as they are designed for use in natural settings to teach developmentally appropriate skills with behavioral strategies (Schreibman et al. 2015). Two evidence-based NDBI approaches that provide support for communication skills of young children with disabilities are incidental teaching (IT) and pivotal response training (PRT; Schreibman et al. 2015; Wong et al. 2015). Within IT and PRT, practitioners arrange the environment in the context of naturally occurring classroom activities to create communication opportunities based on child interests (Wong et al. 2015). Both approaches involve creating clear opportunities for the child to communicate, utilize systematic prompting, and natural reinforcement (Koegel and Koegel 2006).

Although NDBI approaches are recommended for use in the child’s natural settings (Schreibman et al. 2015) and are recognized as evidence based (Wong et al. 2015), several factors exist that impact the implementation of NDBIs in inclusive preschool classrooms. First, research evidence highlights variability in preschool practitioner knowledge, skills, and competence to teach children with disabilities in inclusive settings (Dunst and Bruder 2014). Preschool practitioners report that they have insufficient knowledge to implement evidence-based practices in inclusive settings (Odom and Bailey 2001) and report concerns regarding their ability to work with children with more extensive communication needs and integrate individualized goals into the curriculum (Bruns and Mogharreban 2008). High quality training is required to increase the evidence-based practice knowledge and skills of preschool practitioners in inclusive classrooms (Odom et al. 2013). Yet, early childhood administrators (Barton and Smith 2015) and preschool practitioners (Muccio et al. 2014) cite numerous training barriers to support successful preschool inclusion. Practitioners note time, resources, cost, and implementation comfort as barriers to training (Wainer and Ingersoll 2013). Additionally, trainers report similar barriers to implementing effective and efficient training including large waiting lists, high costs, schedules, and time associated with travel (e.g., Wacker et al. 2013; Wainer and Ingersoll 2015). Training models that can address these barriers and effectively deliver NDBIs to preschool inclusive classrooms are needed.

Telehealth Training

Telehealth training utilizes communication and information technologies (e.g., online modules and videoconferencing) to provide training from a distance (Nickelson 1998). Telehealth training is well-established in psychiatric services (VandenBos and Williams 2000), and studies investigating the use of telehealth in education and applied behavior analysis (ABA) are beginning to emerge. Reviews focused on the use of telehealth and evidence-based ABA interventions (Ferguson et al. 2018; Neely et al. 2017; Tomlinson et al. 2018) have shown that this training platform is efficient (i.e., cost and time savings) and effective (i.e., increased implementation fidelity and socially significant child outcomes).

Videoconferencing with a trainer using real-time communication to provide coaching and feedback from a distance is the predominant method of telehealth training for ABA based interventions like NDBIs (Tomlinson et al. 2018). Initial training incorporated into telehealth packages has included providing a manual (e.g., Wacker et al. 2013), live training sessions via videoconference (e.g., Hay-Hansson and Eldevik 2013), and self-instruction through online modules or videos (e.g., Neely et al. 2016; Wainer and Ingersoll 2015). Following initial training, a telehealth delivery model can be used to provide ongoing coaching and delayed video feedback (e.g., Neely et al. 2016), essential components to support adult learners (Brock and Carter 2017; Rispoli et al. 2011). Self-evaluation (Dunst and Trivette 2009), a practice where the practitioner reviews their video performance and evaluates their implementation before meeting virtually with the trainer for performance feedback, has also been shown to be an effective addition to telehealth training (Neely et al. 2016, 2018).

In sum, telehealth training has the potential to address the training barriers cited by preschool practitioners and trainers (e.g., time, monetary cost, trainer availability) and has been found to be socially valid (Tomlinson et al. 2018). Telehealth approaches can also be implemented with notable cost (e.g., Lindgren et al. 2016) and time savings (e.g., Wacker et al. 2013). Additionally, when compared to in-person training methods, telehealth training methods have been shown to be equally effective (Hay-Hansson and Eldevik 2013; Pantermuehl and Lechago 2015; Vismara et al. 2009; Wacker et al. 2013), further supporting the use of telehealth to train practitioners to implement NDBIs.

Present Study

Initial evidence exists supporting telehealth training to prepare practitioners to implement NDBIs, yet research gaps exist. First, telehealth training has been used to prepare practitioners working with young children with autism spectrum disorder (ASD) to implement NDBIs including: IT (Neely et al. 2016, 2018), Project ImPACT (Wainer et al. 2017), and Early Start Denver Model (Vismara et al. 2009), yet no existing studies have implemented NDBIs in inclusive preschool classrooms with general education inclusive preschool practitioners (Ferguson et al. 2018; Neely et al. 2017; Tomlinson et al. 2018). Given the high inclusion rates for young children with disabilities (U.S. Department of Education 2018), research focused on the implementation of evidence-based practices with general education practitioners in inclusive preschool school settings is important. Additionally, researchers have focused on telehealth training to support practitioner implementation of evidence-based interventions primarily with children with ASD—only a small number of studies included participants with developmental delays and disabilities other than ASD (Tomlinson et al. 2018). Further, prior research has not included collaborative approaches to training (e.g., practitioner choice of intervention and child target behavior), which can create buy-in, and may impact both experimental and social validity outcomes (Hieneman et al. 2005). Finally, research in this area has infrequently included measures of generalization and maintenance (Neely et al. 2017; Tomlinson et al. 2018), and literature reviews in this area indicate low research quality (e.g., quality of baseline data, reported participant details, experimental control; Ferguson et al. 2018; Tomlinson et al. 2018).

Given the limitations in the existing literature, the purpose of this study is to further extend the literature and evaluate telehealth training for general education inclusive preschool practitioners to increase implementation fidelity of NDBIs for children with varied disabilities. We also include measures of generalization and maintenance while emphasizing adherence to quality indicators. Specifically, this study addressed the following research questions: (a) Is there a functional relation between the use of telehealth training and practitioner behaviors (i.e., frequency of target skill communication opportunities and NDBI implementation fidelity)?; (b) Is there a functional relation between telehealth training and frequency of independent child target communication behavior?; (c) Do practitioner and child behaviors generalize to a different activity context and maintain over time?; (d) Do practitioners report telehealth training as socially valid approach in the areas of goals, procedures, and outcomes?

Method

Participants

A total of six preschool practitioners from two Midwest school districts consented to participate in this study. Practitioners were recruited from separate classrooms and met the following criteria: (a) lead or assistant teacher in an inclusive preschool classroom where at least one child with a disability was enrolled and (b) no previous training in NDBIs. Each of the six practitioners described themselves as white, female, lead inclusive preschool teachers. See Table 1 for more information about practitioners.

Table 1 Practitioner participant information

Following practitioner consent, children within the classroom were recruited to participate with each practitioner. Child participants were eligible if they had: (a) a school disability label of early childhood developmental delay, speech and language impairment, autism spectrum disorder, other health impairment, and/or cognitive impairment; (b) Individualized Education Program (IEP) goal(s) in the area of social and/or language domains; and (c) consistent school attendance (i.e., at least 3 days per week). One child was selected per practitioner. Parental consent was obtained for each child. See Table 2 for information about child participants. Practitioner and child pairs are referred to as a dyad hereafter.

Table 2 Child participant information

Dyad A

Amy had 2 years of experience as a lead teacher of 4-year-old children in an inclusive preschool setting and was in her fifth year of teaching. Aaron was eligible for special education services under the eligibility label of ASD. His total language score on the Preschool Language Scale, Fifth edition (PLS-5; Zimmerman et al. 2012) was 62 (1st percentile) and the Autism Spectrum Rating Scale (Goldstein and Naglieri 2013) revealed elevated to very elevated scores in all categories. Aaron spent one school year in a segregated early childhood special education classroom for children with or at-risk for ASD prior to joining Amy’s inclusive preschool classroom. At the start of the study, Aaron verbally initiated a request for assistance using one word (i.e., help), and was working on independent verbal requests for desired or needed items during tasks.

Dyad B

Betty had 1 year experience as an inclusive preschool teacher of 4-year-old children before the start of the study and 15 years teaching experience. Bryan was eligible for special education services under the disability area of speech and language impairment. His expressive communication score on the PLS-5 was 77 (6th percentile) and based on results from the Clinical Assessment of Articulation and Phonology (Secord and Donohue 2014), he demonstrated a severe speech impairment. Bryan spent one school year in a segregated early childhood special education program prior to joining Betty’s inclusive preschool classroom. Bryan demonstrated delays in expressive communication and intelligible speech at the start of the study. Bryan used one to two words together, yet often used non-verbal expressions (i.e., gestures) to meet his needs. He had an IEP goal of verbally combining three or more words with different combinations of subjects, objects, and actions without adult modeling.

Dyad C

Carey was in her 12th year of teaching, eighth year as an inclusive preschool teacher. Chloe was beginning her second school year in an inclusive preschool. Chloe’s IEP and parent report noted a diagnosis of Down syndrome, yet her eligibility label for special education services was other health impairment due to a heart condition. Her scores on the REEL-3 were 75 receptive (5th percentile) and 72 expressive (3rd percentile). At the start of the study, she followed routine directions and accurately selected common objects in her environment when provided with a verbal prompt (e.g., “Show me the red ball” and she touches the red ball). She independently used one word to verbally label and make requests for objects but was not yet communicating in phrases or sentences. Parents and practitioners reported that Chloe displayed oppositional behavior.

Dyad D

At the start of the study, Danielle was beginning her first year as an inclusive preschool teacher of 4-year-old children and was in her fourth year of teaching. David was eligible for special education services under the disability area of speech and language impairment. His IEP did not include current standardized assessment scores, yet observation documentation from the Teaching Strategies GOLD® assessment system revealed a delay in the areas of literacy, social emotional, and language. David was beginning his second school year in an inclusive preschool classroom and was not yet using independent vocalizations to request. David had a behavior plan to address physical aggression for the purpose of gaining tangible items and/or attention.

Dyad E

Ethan received 5 months of special education support in an inclusive 3-year-old preschool setting prior to the start of the study, and Emily was beginning her seventeenth year as a teacher, second year in an inclusive preschool. Ethan was eligible for special education services under the disability area of early childhood developmental delay. His total language score on the PLS-5 was 70 (2nd percentile). He used his name to convey a variety of messages (e.g., stating “David” to convey, “I want that” or “My turn”) and did not independently combine words to functionally communicate with adults or peers.

Dyad F

Fae was beginning her second year as a general education inclusive preschool teacher to 3-year-old children and Fred was beginning his first year of preschool. Fred was determined eligible for special education services under the disability area of speech and language impairment. His expressive score on the Receptive-Expressive Emergent Language Test, Third edition (REEL-3; Bzoch et al. 2003) was 67 (1st percentile). Fred requested with language and gestures and imitated some words, yet he was not labeling items independently.

Coach

The coach (i.e., first author) provided all telehealth training to the practitioners. At the time of the study, the coach was a doctoral candidate in child development and held a master’s degree in special education. The coach also had 10 years of teaching experience and held early childhood and autism teaching endorsements as well as 3 years of previous experience in early childhood practitioner training and coaching. The coach was also working on coursework to obtain a certificate in applied behavior analysis and was completing supervised field work requirements for the Board Certified Behavior Analyst exam.

Settings and Materials

The study was implemented at the beginning of the school year in six separate inclusive preschool classrooms where children with and without disabilities were educated together for the whole school day. The inclusive classrooms enrolled 16–18 children and maintained a 1:8 adult/child ratio. Children with disabilities comprised an average of 25% of enrollment per classroom. Dyads were in programs that operated 4 days a week. Dyads A and B were half day programs (i.e., 3 h per day), and Dyads C, D, E, and F were full day programs (i.e., six and a half hours per day).

Each classroom utilized Creative Curriculum® for Preschool. Hence, each classroom’s daily schedule included the same activity contexts consisting of at least one small group and large group activity, a choice time (i.e., free play) lasting at least 1 h, and a meal or snack time. Based on Creative Curriculum®, the classroom was organized into 10 interest areas (e.g., blocks, dramatic play, toys and games, art, discovery) and during the 1 h choice time, children made choices about where and how to use materials throughout the classroom. Small group consisted of six to eight children and one adult participating in an adult-initiated, but not adult dominated activity lasting as long as it interested the children, which was 10–15 min. Small group was based on pre-planned activities addressing specific objectives for development and learning from the curriculum’s teaching guide, yet the teacher could add or modify learning materials to meet the needs of the children. Large group was also adult-initiated, but not adult dominated and included all of the children and adults in the classroom for an average of 15 min in activities like a read-aloud, shared writing, or music and movement. Primary activity contexts were chosen by individual practitioners for baseline, intervention, and maintenance probes, and a separate activity context for generalization probes (see Table 1).

A variety of toys and materials available in the child’s classroom were used throughout the study. Materials varied by dyad and session based on child preference. A tablet was provided to each practitioner to video record sessions (i.e., Amazon Fire HD 8®). A secure file sharing host (i.e., Dropbox®) was used with individual password protected accounts. Training was completed online using the provided tablet or other available device. Video conferencing with the practitioner was conducted via Zoom® using the provided tablet or other available device.

Experimental Design

A single-case multiple probe across participants design (Ledford and Gast 2018) was employed to investigate the effects of telehealth training on practitioner behaviors (i.e., frequency of communication opportunities and NDBI implementation fidelity) and, in turn, child target communication behavior. Visual analysis of level, trend, variability, immediacy, and overlap was conducted to determine a functional relation through systematic and sequential manipulation of telehealth training (Barton et al. 2018). Data are presented based on the intervention selected by practitioners. Three practitioners selected IT, and three selected PRT. Based on NDBI choice, participants were randomly assigned within the multiple probe design and telehealth was applied to one dyad at a time. Each dyad served as its own control. Demonstration of stability in level and trend of practitioner implementation fidelity during baseline was established before telehealth training was initiated for the first dyad. Baseline probe data collection continued for the remaining participants. Telehealth training was initiated sequentially for subsequent dyads once an increase was demonstrated for practitioner implementation fidelity of the previous dyad and a probe during baseline was administered to ensure changes in implementation fidelity occurred only when telehealth training was introduced.

Independent Variable

The independent variable in this study was the use of a telehealth training package in order to investigate practitioner implementation fidelity of an NDBI. The telehealth training package required practitioners to complete an online module through Autism Focused Intervention Resources and Modules (AFIRM) and participate in coaching sessions involving delayed video feedback via videoconferencing and video self-evaluation (see Procedures for more information). In order to increase the social validity of the procedures, the practitioners were given a choice between two online AFIRM modules focused on an NDBI approach (i.e., IT and PRT). Both modules were identically structured and included four lessons to learn basic knowledge and application of the NDBI in activity-based scenarios to promote real-world application (Amsbary and AFIRM Team 2017; Suhrheinrich et al. 2018). The independent variable of focus for this study was the telehealth training package which consisted of the online training module, delayed video feedback via videoconferencing, and video self-evaluation. Two variations of the independent variable were evaluated. Specifically, training and coaching on IT and PRT using the telehealth training package.

Dependent Variables and Measurement

Dependent variable data on practitioner and child behaviors were scored from the 10-min sessions recorded by the practitioner in each preschool inclusive classroom. Video length was standardized; coding began when the practitioner started recording and ended at 10-min.

Practitioner Behaviors

Data were collected on two practitioner behaviors for each 10-min recorded session. The first practitioner behavior was the frequency of target skill communication opportunities offered within each 10-min session as fidelity of NDBI implementation could not be collected if a target communication skill opportunity was not provided. A target skill communication opportunity occurred when the practitioner arranged the environment to illicit the target skill from the child (i.e., the first step of each NDBI procedure). A target skill communication opportunity was only counted when the practitioner displayed an action (e.g., moving pieces out of reach of the child) or vocalization (e.g., I see more colors over here) to encourage a child to display the target skill behavior and the child in turn initiated toward the item or activity physically (e.g., reaching, pointing) and/or verbally (e.g., requesting/labeling with or without the target skill requirement).

For each target skill communication opportunity provided, the fidelity of NDBI steps completed by the practitioner was collected. The percentage of correctly implemented NDBI steps (see Table 3) was calculated by taking the number of steps completed correctly divided by the total number of steps multiplied by 100. IT steps were based on Neely et al. (2018) and PRT steps were based on Suhrheinrich et al. (2018). Criterion mastery was set at 90% or greater fidelity across two consecutive sessions. Decisions related to movement between phases, were based solely on practitioner implementation fidelity. Since practitioners could offer multiple target skill communication opportunities within one session, an overall mean percentage was calculated for each session.

Table 3 NDBI procedures

Child Target Behavior

A socially significant target communication behavior was selected for each child participant to determine the impact of the telehealth training package on child target skill communication behavior (see Table 2). The child’s independent target skill communication behavior was measured with a frequency count during each session. An occurrence was defined as an independent display of the target skill communication behavior.

Interobserver Agreement (IOA)

The first author collected primary data for the accuracy of all dependent variables. To ensure reliable coding of data, a second researcher (i.e., third author- a board certified behavior analyst with a doctorate in education) conducted reliability for the three measures across 32% of baseline sessions, 40% of intervention sessions, 40% of maintenance sessions, and 33% of generalization sessions. Sessions were randomly selected and evenly distributed across all six dyads. Prior to reliability coding, the first author provided training specific to coding each dependent variable in the study. Training continued until the secondary coder demonstrated at least 90% agreement with the first author on two training videos. IOA was calculated using point-by-point agreement. Agreement was noted if both observers coded the same behavior within each trial. Disagreement was noted if coded behaviors did not match within a trial. Agreement was calculated by dividing the total agreements by the sum of agreements and disagreements and multiplying by 100 to produce a percentage.

IOA was 100% for Dyad A across all conditions and behaviors. Dyad B had 97.5% (range 83–100%) IOA overall with 100% IOA for all behaviors across baseline, maintenance, and generalization and 93% (range 83–100%) across intervention. For Dyad C, IOA was 98% (range 92–100%) overall with 100% IOA across baseline and generalization and 96% (range 92–100%) across intervention and 97% (range 93–100%) across maintenance sessions. For Dyad D, IOA was 99% (range 92–100%) overall with 100% IOA across baseline, intervention, and generalization for all behaviors and 96% (range 92–100%) across maintenance sessions. For Dyad E, IOA was 99% (range 94–100%) overall with 100% IOA across baseline, intervention, and maintenance, and 94% for the generalization probe. IOA for Dyad F was 99% (range 94–100%) overall with 100% IOA across baseline, maintenance, and generalization and 97% (range 94–100%) across intervention sessions.

Procedures

Prior to the start of data collection, the coach met with each practitioner at their school for collaborative planning. Child target behaviors in the area of social skills or language were collaboratively selected from the child’s IEP. The target behavior was operationally defined by the coach and practitioner (see Table 2). The practitioner was given a choice between two NDBI procedures: PRT or IT. The coach provided a video example of each NDBI from the AFIRM website (see Amsbary and AFIRM Team 2017; Suhrheinrich et al. 2018) to assist the practitioner in intervention selection. Then, the operational definition of the child’s target skill, chosen NDBI approach, child’s preferred toys or activities, and practitioner-chosen primary and generalization context for NDBI implementation were recorded by the practitioner for later reference. The practitioner and coach then reviewed and practiced the video recording and upload procedures and tested the videoconferencing program using the provided tablet. The meeting lasted approximately one and a half hours per practitioner.

Baseline

Probe sessions during the baseline phase of the study were conducted in the primary activity context of the practitioner’s classroom. Practitioners were directed to work on the target behavior with the assigned child as outlined on the collaborative planning worksheet and videotape a session with the child. The practitioners uploaded each video to Dropbox® and were instructed to not watch the videos. No further instructions or feedback were given during this phase. Each practitioner conducted at least five baseline probe sessions (maximum of one session per day/four per week). Practitioners remained in baseline while the first practitioner began intervention. Once the first practitioner displayed an increasing trend, the researcher administered probes to the second dyad to confirm stable behavior before introducing the independent variable. This process was repeated until all dyads entered intervention.

Intervention

Following completion of the baseline phase, each practitioner completed the self-paced AFIRM online module that corresponded with their selected intervention approach. The practitioners using IT completed the naturalistic instruction module (Ambsary and AFIRM Team 2017) and the practitioners using PRT completed the pivotal response training module (Suhrheinrich et al. 2018). The AFIRM module provided the practitioner with background knowledge on the NDBI approach and took approximately one and a half to two hours to complete. To confirm completion of the online module, the practitioner emailed the completion certificate to the coach. Upon receipt of the post-assessment certificate, the practitioner was given access to the NDBI procedure and self-evaluation checklist (see Table 3) and was instructed to videotape a 10-min session in which they implemented the selected NDBI approach to work on the child’s target communication behavior in the same setting as the baseline sessions. Therefore, the first video collected corresponds to the first data point in the intervention phase and took place after completion of the AFIRM module and access to NDBI procedure and self-evaluation checklist and prior to the first videoconference. Following the first session and each intervention session thereafter, the practitioner uploaded the video to Dropbox® and the coach and practitioner then both independently viewed the video and separately evaluated the video using the self-evaluation checklist. The self-evaluation checklist was identical to the procedures listed in Table 3 with an option for the practitioner to select their use of the skills on a scale including the options: all of the time, most of the time, rarely, or never.

Following video upload and self-evaluation completion, the coach and practitioner met at a mutually agreeable time via secure videoconference (i.e., Zoom®). Coaching sessions were recorded for later analysis using the built-in video recording capabilities. During the videoconference coaching session, the coach reviewed each step of the evaluation checklist with the practitioner and provided feedback according to a coaching integrity checklist (see Neely et al. 2016, 2018). The coach also asked open-ended questions (e.g., What are your thoughts about today’s session?). When requested or as needed, video segments were replayed by the coach. The mean length of a coaching sessions was 18.6 min (range 9.4–26.3 min). The intervention phase continued following the same procedures for each session (i.e., practitioner records session, uploads video, coach and practitioner separately watch video and complete self-evaluation checklist, and coach and practitioner meet via videoconference for feedback before the next session) until the practitioner implemented NDBI steps with 90% or greater fidelity for two consecutive sessions. The 10-min video recorded and uploaded by the practitioner was used by the coach to score practitioner implementation behavior.

Maintenance

Practitioners entered the maintenance phase once mastery criterion was met and stability in level or an increasing trend was demonstrated for practitioner implementation fidelity across at least five probes in the intervention phase. A minimum of five maintenance probes were collected once per week after the last intervention session. As in baseline, practitioners videotaped a session, uploaded the video, and were instructed not to watch the videos.

Generalization

Generalization was assessed through application of the intervention in a different activity context than the baseline and intervention sessions, which was chosen by the practitioner prior to the baseline phase. Each practitioner videotaped a 10-min session implementing the NDBI approach with the child in the generalization activity context and were instructed to upload the videos without watching them. During baseline, practitioners recorded a generalization session after the third baseline sessions. During intervention and maintenance, practitioners were asked to record one generalization session during each phase.

Procedural Fidelity

To ensure procedural fidelity during each treatment condition, procedural measures were recorded using researcher created checklists. First, a checklist was used during the collaborative meeting with each practitioner to document completion of the worksheet documenting the child’s target skill, chosen NDBI, and activity context, and the completion of video recording and uploading practice. To provide evidence of online module completion, all practitioners completed the post-assessment within the AFIRM module and emailed the certificate to the coach. Baseline, intervention, generalization, and maintenance procedural integrity was measured using a checklist that documented that the target child and practitioner were captured in the video during the chosen activity context for 10-min. Procedural reliability was scored by an independent observer for the collaborative meeting, baseline, generalization, and maintenance phases for each practitioner and was 100% for all sessions.

An independent observer scored treatment fidelity for coaching by viewing all coaching session videos for each practitioner (i.e., 30 sessions) and rating them using the coaching integrity checklist. The independent observer was trained on the coaching integrity checklist using coaching videos from a separate pilot study. Treatment integrity was calculated as the percentage of steps completed correctly divided by the total number of steps and multiplying the quotient by 100 to obtain a percentage. Treatment integrity had a mean of 98% across all sessions (range 90–100%). The most often missed step (i.e., three times) was: “Ask the practitioner if they have any questions”.

Social Validity

Four measures of social validity were collected to obtain consumer feedback regarding telehealth training in the areas of goals, procedures, and outcomes as suggested by Wolf (1978). Each measure was converted to an electronic format (i.e., Qualtrics®) and links were e-mailed to respondents from the coach (i.e., first author), but were completed without the coach present and were anonymously submitted. All four measures included a place where practitioners could leave comments. The Target Skill Prioritization questionnaire (Carter 2010) was completed prior to baseline in order to determine social validity of the child target skill. The eight-item questionnaire allowed the practitioner to rate agreement and disagreement regarding the target skill (e.g., “This is the best target skill that could be chosen”) using a six-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree) with a higher score signifying higher priority of the target skill. The questionnaire took approximately one min to complete.

The Teacher Efficacy for the Inclusion of Young Children with Disabilities scale (TEIYD; Walls 2007) was completed by participants before baseline and after the completion of maintenance. The TEIYD measured practitioner self-efficacy working with young children with disabilities in inclusive classrooms and contained items on efficacy in three areas: knowledge of young children with disabilities (5 items), teaching confidence (7 items), and perceived ability (3 items). Practitioners rated their responses on a scale ranging from 1 (no confidence) to 5 (very confident). The TIEYD took approximately five min to complete at each time point.

The Intervention Rating Profile-15 (IRP-15; Martens et al. 1985) was used to examine NDBI acceptability by practitioners and indirect consumers. Practitioners completed the IRP-15 before baseline and after maintenance. The IRP-15 was also emailed to an external respondent within the school community (i.e., administrator, speech and language pathologist, special education provider) who was blind to the purpose of the study and observed one implementation session during intervention in person or through video before responding to the questionnaire. The questionnaire included 15 items (e.g., “I find this intervention suitable for the child’s target skill development”), rated on a six-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree). Scores ranged from 15 to 90 with higher scores reflecting greater acceptability. The IRP-15 took approximately 3 min to complete.

A researcher developed questionnaire was also used to provide a rating of the training and coaching procedures and was emailed to practitioners when they entered the maintenance. The questionnaire included eight positively phrased statements (e.g., “I enjoyed the self-evaluation process”; “I found the video conferencing to be effective”) that targeted each component of training and coaching and used a six-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree). The questionnaire took approximately four min to complete. Scores ranged from 8 to 48 with higher scores reflecting greater acceptability.

Results

Practitioner Behavior

Results show a functional relationship between telehealth training and practitioner behaviors (see Figs. 1, 2). Amy demonstrated a stable level of zero during baseline. There was an immediate, absolute, and relative level change of IT implementation fidelity and frequency of target skill communication opportunities demonstrated between baseline and intervention with no overlapping data. The data revealed a consistent, increasing trend for practitioner implementation fidelity (range 86–100%) and stable, higher than baseline level of frequency of target skill communication opportunities (range 8–9) during the intervention condition. Amy’s implementation fidelity remained high and stable during maintenance (range 98–100%). The frequency of target skill communication opportunities provided during maintenance displayed a higher level than those of baseline and intervention with variability (range 9–19). Amy’s implementation fidelity generalized to a different activity context increasing from 0% in baseline to 100% implementation during intervention and maintenance. Amy’s frequency of target skill communication opportunities offered also generalized to a different activity context with zero opportunities at baseline, 10 in intervention, and 13 in maintenance.

Fig. 1
figure1

Practitioner IT implementation fidelity in closed triangles corresponding to left y-axis. Frequency of target communication opportunities provided in open circles and child target communication behavior in closed squares corresponding to the right y-axis. Generalization probes in open triangles for practitioner implementation fidelity, closed circles for communication opportunities, and open squares for child behavior

Fig. 2
figure2

Practitioner PRT implementation fidelity in closed triangles corresponding to left y-axis. Frequency of target communication opportunities provided in open circles and child target communication behavior in closed squares corresponding to the right y-axis. Generalization probes in open triangles for practitioner implementation fidelity, closed circles for communication opportunities, and open squares for child behavior

Betty demonstrated stable level of zero during baseline and an immediate, absolute, and relative level change upon intervention with no overlapping data. A stable, accelerating trend of IT implementation fidelity (range 98–100%) was demonstrated during intervention. Betty’s frequency of target skill communication opportunities provided demonstrated a stable then accelerating trend (range 6–18) during intervention with no overlapping data. Betty’s implementation fidelity maintained at a stable, high level (range 99–100%) during the maintenance condition and generalized to a different activity context from 0% in baseline to 100% implementation fidelity during intervention and maintenance. The frequency of target skill communication opportunities provided by Betty during maintenance displayed a higher level than those of baseline and intervention with variability (range 10–16). Betty’s frequency of target skill communication opportunities offered also generalized to a different activity context with zero opportunities at baseline, four in intervention, and five in maintenance.

Carey demonstrated a low level, decelerating trend of IT implementation fidelity (range 0–59%) and frequency of target skill communication opportunities (range 0–8) across baseline with zero-celebrating trend of responding across the last five probe sessions. There was an immediate, absolute, and relative level change from baseline to intervention for IT implementation fidelity and frequency of target skill communication opportunities provided by Carey. During the intervention condition, IT implementation fidelity data displayed a stable, accelerating trend with no overlapping data (range 98–100%). Carey demonstrated a higher than baseline level of frequency of target skill communication opportunities during intervention, yet high variability, a decelerating trend, and two overlapping data points (range 2–18). IT implementation fidelity remained stable and high during maintenance (range 98–100%) and target skill communication opportunities provided displayed a stable level and trend higher than during baseline (range 14–15). Carey’s IT implementation fidelity generalized to a different activity context with 0% during baseline, 95% during intervention, and 93% during maintenance. Carey’s frequency of communication opportunities provided also demonstrated generalization with zero opportunities during baseline, seven during intervention, and 10 during maintenance.

Danielle demonstrated a stable level of zero during baseline and an absolute and relative level change upon intervention for PRT implementation fidelity and frequency of target skill communication opportunities. There was an immediate, absolute, and relative level change of PRT implementation fidelity and frequency of target skill communication opportunities provided from baseline to intervention with no overlapping data. The data revealed a stable, accelerating trend for practitioner implementation fidelity (range 95–100%) during intervention. Data patterns for Danielle’s frequency of target skill communication opportunities provided during the intervention condition were stable and higher than baseline level (range 9–13). Danielle’s PRT implementation fidelity remained high and stable in maintenance (range 99–100%) and generalized to a different activity context from 0% during baseline to 100% during intervention and maintenance. The frequency of target skill communication opportunities provided during maintenance displayed a higher level than during baseline with variability (range 6–23). Danielle’s frequency of target skill communication opportunities offered also generalized to a different activity context with zero opportunities at baseline, 15 in intervention, and 10 in maintenance.

Emily demonstrated a stable level of zero during baseline and an immediate, absolute, and relative level change of PRT implementation fidelity and frequency of target skill communication opportunities provided from baseline to intervention with no overlapping data. The data patterns revealed a stable, accelerating trend for practitioner implementation fidelity (range 75–100%) and an increasing trend with variability of the frequency of target skill communication opportunities (range 5–21) provided during the intervention condition. Emily’s PRT implementation fidelity remained high and stable during maintenance (range 98–100%). The frequency of target skill communication opportunities provided by Emily during maintenance displayed a stable, higher level than baseline (range 13–18). Emily’s PRT implementation fidelity generalized to a different activity context increasing from 0% in baseline to 99% implementation fidelity during intervention and maintenance. Emily’s frequency of target skill communication opportunities offered also generalized to a different activity context with zero opportunities at baseline, 8 in intervention, and 15 in maintenance.

Fae demonstrated a low level of PRT implementation fidelity with some variability (range 0–37%) and a low, stable level of frequency of target skill communication opportunities provided (range 0–2) across baseline sessions with the last baseline session at 0% implementation fidelity and zero target skill communication opportunities provided. There was an immediate, absolute, and relative level change of implementation fidelity and frequency of target skill communication opportunities provided from baseline to intervention with no overlapping data. The data revealed a stable, accelerating trend for practitioner implementation fidelity (range 96–100%) and variable, increasing trend (range 9–13) of frequency of target skill communication opportunities provided during the intervention condition. Fae’s PRT implementation fidelity maintained a stable, higher than baseline level (range 98–100%) and generalized to a different activity context from 0% in baseline to 100% implementation fidelity during intervention and maintenance. The frequency of target skill communication opportunities provided by Fae during maintenance displayed a stable trend at a higher level than baseline (range 13–15). Fae’s frequency of target skill communication opportunities offered also generalized to a different activity context with zero opportunities at baseline, seven in intervention and maintenance.

Child Behavior

Results from the study also show a functional relation between telehealth and the frequency of child target communication behavior (see Fig. 1). During baseline, Aaron demonstrated a stable level of zero instances of his target communication behavior. There was an immediate, absolute, and relative level change from baseline to intervention and no overlapping data. Aaron’s data pattern displays an accelerating level during intervention and remains at higher than baseline level in maintenance with variability (range 9–16). Aaron’s target skill communication behavior generalized to a different activity context with 0 instances during baseline, 10 during intervention, and 13 during maintenance.

Bryan demonstrated a stable level of zero during baseline and an immediate, absolute, and relative level change upon intervention and no overlapping data. During intervention, Bryan demonstrated an accelerating trend (range 3–16) and frequency of his target communication behavior remained higher than baseline level in maintenance with variability (range 10–16). Bryan’s behavior generalized to a different activity context and increased from zero during baseline to five each during intervention and maintenance.

Chloe demonstrated her target communication behavior once during the second probe session then demonstrated a stable level of zero instances of target skill communication behavior during remaining baseline sessions. There was an immediate, absolute, and relative level change from baseline to intervention. Chloe’s data pattern displayed a higher than baseline level during intervention, yet high variability, a decelerating trend, and no overlapping data (range 5–24). Chloe’s frequency of target communication behavior was at a higher level than baseline with some variability (range 3–11). Chloe’s behavior generalized and increased from zero during baseline, two during intervention, and six during maintenance.

David demonstrated a stable level of zero instances of target communication behavior during baseline and there was an immediate, absolute, and relative level change from baseline to intervention and no overlapping data. During intervention, David’s target communication behavior was higher than baseline level and stable (range 7–13). David’s target behavior frequency remained higher than baseline level with variability (range 6–20). David’s target communication behavior generalized to a different activity context with 0 requests during baseline, 13 during intervention, and 10 during maintenance.

Ethan demonstrated a stable level of zero instances of his target communication behavior during baseline. There was an immediate, absolute, and relative level change from baseline to intervention with no overlapping data. During intervention, Ethan’s data pattern shows an accelerating trend (range 1–18). Ethan’s target communication behavior remained high and stable in maintenance (range 13–18) and generalized to a different activity context with zero instances during baseline, seven during intervention, and 14 during maintenance.

Fred demonstrated a low, stable level of his target communication behavior (range 0–1) during baseline with no instances during the last probe. There was an immediate, absolute, and relative level change from baseline to intervention and no overlapping data. During intervention, Fred’s behavior data demonstrated an accelerating trend (range 7–20). Fred’s target behavior level remained higher than baseline during maintenance with an accelerating trend (range 9–16). Fred’s target communication behavior generalized to a different activity context with zero instances during baseline, six during intervention, and seven during maintenance.

Social Validity

Goals

The target communication behavior was selected by the practitioner with assistance from the coach based on the child’s current IEP. Practitioner participants completed the social validity instrument independently after the target skill was chosen and prior to the beginning of the study. Results of the Treatment Goal Prioritization (Carter 2010) questionnaire revealed that all six practitioners agreed (rating of 5) or strongly agreed (rating of 6; M = 5.67) that the selected child target communication skill was of high priority.

Procedures

The treatment procedures assessed for social validity included NDBI procedures and telehealth. Prior to intervention, all practitioners rated their chosen NDBI as highly acceptable with a mean rating of 81.6 out of 90 on the IRP-15. After using the chosen NDBI, all practitioners rated the intervention higher with a mean of 87.48 out of 90. In open comments, one practitioner wrote, “I loved this intervention.” External raters were also asked to complete the IRP-15 and eight responses included an administrator, special education teacher, assistant teacher, and speech and language pathologist. External raters watched the practitioner implement the intervention in person or via video and reported that the chosen NDBI was highly acceptable with a mean rating of 81.42 out of 90. In open comments, external raters noted that they hoped the intervention would be used with all children in the classroom, and they saw great improvements in the target child.

Based on the researcher developed questionnaire, all practitioners found telehealth acceptable with a mean rating of 45.16 out of 48. Specifically, practitioners found video conferencing (M = 5.83) and self-evaluation (M = 5.5) to be enjoyable and effective and agreed that the online training module was an acceptable way to train practitioners on the NDBI (M = 5.67). Practitioners also agreed that the telehealth procedures were efficient (M = 5.5) and had an overall positive experience (M = 5.83). One practitioner commented: “I believe this type of training is a great strategy that uses the child’s natural environment to learn.”

Outcomes

Treatment outcomes of telehealth included consumer satisfaction of both practitioner and child outcomes. The TIEYD (Walls 2007) was completed by practitioner participants pre-study and post-intervention to assess the social validity of practitioner efficacy working with children with disabilities. Based on pre-study TIEYD results, practitioners held moderate confidence (M = 3.8 out of 5) for including young children with disabilities but increased post-study (M = 4.5) indicating a higher level of confidence. Pre-study, practitioners were confident in their knowledge of how disabilities can impact young children (M = 4.4) and showed a slight increase post-study (M = 4.6). Practitioners’ confidence in their ability to teach young children with disabilities was rated as moderate (M = 3.4) pre-study and increased to confident (M = 4.2) post-study. Pre-study perceptions of practitioner’s ability to implement both effective teaching strategies and modifications to meet the needs of young children with disabilities was rated as confident (M = 3.9) and increased to very confident (M = 4.7) post-study. Pre-study comments by one practitioner included: “I feel I have relied on the special education teacher to teach and maintain a lot of the IEP goals for the children unless they were more academic goals.” Post-study, one practitioner commented: “This experience has given me the confidence to dig into an IEP and see how to create activities to help a child with a disability.” Results also indicated positive social validity regarding child outcomes. Practitioners strongly agreed (M = 5.67) that the intervention was effective for child participants. One practitioner stated: “I am absolutely amazed at the results I have seen in the student I worked with! It has carried over to other parts of the day.”

Discussion

The primary aim of this study was to evaluate the effects of a telehealth training package on general education practitioners’ implementation fidelity of an NDBI in inclusive preschool classrooms. A functional relation was identified between the telehealth training and practitioner implementation fidelity for all practitioners, as implementation fidelity only increased when telehealth training was applied. All six practitioners reached the pre-set fidelity criteria (i.e., above 90% for two consecutive sessions) within the first three intervention sessions and maintained high implementation fidelity for the remainder of the intervention condition. This finding is noteworthy as the majority of participants trained to implement ABA based interventions via telehealth do not meet implementation fidelity criterion (Neely et al. 2017; Tomlinson et al. 2018). These findings provide initial evidence for the use of telehealth training as a model to increase NDBI implementation fidelity of general education practitioners in inclusive preschool classrooms.

It is important to note that practitioner fidelity was only measured when the practitioner provided an opportunity for the child to practice their target communication behavior. Practitioners provided near-zero target skill communication opportunities during baseline probes, hence implementation fidelity scores of zero are not a true zero but a measurement artifact. Even though Carey and Fae offered a few communication opportunities for their target child to practice the target communication behavior, they did not display the NDBI procedure behaviors with fidelity demonstrating that the procedure behaviors were not in the practitioner’s repertoire and telehealth training was effective in increasing NDBI implementation fidelity.

This study also demonstrates a functional relation between telehealth training and frequency of communication opportunities. With telehealth training, practitioners were able to maintain high fidelity of NDBI implementation while increasing the frequency of target skill communication opportunities provided, which is consistent with previous research (e.g., Neely et al. 2016, 2018). One practitioner (i.e., Carey) did not demonstrate an increasing trend of communication opportunities provided for the target skill during intervention. Carey’s implementation fidelity remained at a high level, frequency of target communication opportunities provided was variable and decreasing, yet the mean during intervention (M = 13) was higher than during baseline (M = 1.5) and a more consistent pattern (range 14–15) and higher than baseline level (M = 14.2) was established during the maintenance condition.

Results also suggest that increases in communication opportunities were directly related to increases in child behavior. For instance, the frequency of opportunities provided by Carey of Dyad C during intervention varies and shows a decreasing trend, which directly relates to Chloe’s target communication behavior while Betty of Dyad B demonstrated an increasing trend mirrored by Bryan. Overall, results demonstrate that independent child target communication behavior increased as the practitioners improved implementation fidelity and provided more communication opportunities. This finding aligns with research suggesting that providing multiple opportunities to young children with communication challenges is important, as these children may not respond consistently and may require more opportunities to demonstrate behavior change (Douglas et al. 2013).

Although improvement in practitioner implementation fidelity and frequency of target skill communication opportunities are important findings, child improvements are also significant measures of effectiveness. As such, the second research question focused on the effects of the telehealth training package on children with varying disabilities’ target communication behavior. Visual analysis of data shows a functional relation between the use of telehealth training and independent child target communication behavior. Child participants increased their frequency of target communication behavior above baseline level when telehealth training was applied and practitioners implemented the NDBI with increasing fidelity. Early intervention is key for young children identified with a disability or developmental delay. NDBIs can address socially significant target communication behaviors as they incorporate components of the fields of behavioral and developmental sciences to support the development of children within the context of the natural environment using behavioral strategies (Schreibman et al. 2015). The results of this study are encouraging as training general education practitioners via telehealth to implement NDBIs with high fidelity may be an ideal approach that is socially valid and can improve outcomes for children with disabilities in inclusive preschool classrooms.

The third research question focused on the effects of the telehealth training package on generalization of behaviors to a different activity context and maintenance of behavior change over time. Generalization of behavior to a different activity context increased from zero instances across all participants and behaviors during baseline to higher counts during intervention and maintenance conditions. Similarly, maintenance of behavior change was evident for all participants and measured behaviors. Of note, practitioner implementation maintenance probes were stable and remained at a high level for all six participants (range 98–100%).

The final research question of the study focused on its social validity. Multiple social validity measures were utilized to assess the social significance of the goals, acceptability of procedures, and the effectiveness of the outcomes (Wolf 1978). Although the primary respondents were the practitioner participants, which may provide biased results, direct participants are crucial in making social validity claims. Responses to social validity questionnaires indicated high acceptability of the telehealth training. External consumers—who were blind to study purpose and outcomes (i.e., special education teachers, speech and language pathologists, and administrators) also indicated high acceptability. These results corroborate the positive experimental outcomes of this study. Furthermore, the behaviors of practitioners and children in the study maintained over time, an important indicator of social validity (Kennedy 2002). Practitioner self-efficacy also increased in this study, an important finding given that attitudes and beliefs are a primary barrier to preschool inclusion (Barton and Smith 2015). Our results also align with research indicating that training and ongoing support help increase practitioner efficacy (Gebbie et al. 2012).

Implications for the Field

The current study extends the literature in many ways. First, our study participants are unique compared to past telehealth training studies. Specifically, telehealth training systematic reviews (Ferguson et al. 2018; Tomlinson et al. 2018; Neely et al. 2018) reveal that general education inclusive preschool practitioners are unique to this body of research and the inclusive preschool classroom is a rare setting for telehealth training with only two study settings identified as general education classrooms (i.e., Fischer et al. 2016; Gibson et al. 2010). Additionally, the majority of telehealth training studies are implemented with children with ASD while the current study includes children of varying disabilities and developmental delays (Tomlinson et al. 2018).

Also, the quality of this telehealth training study is notable. Ferguson et al. (2018) reviewed 28 telehealth training studies and none met sufficient quality indicators to be determined ‘high’ quality and only one study was determined as ‘adequate’ quality based on Reichow et al.’ quality indicators to determine evidence-based practices (Reichow 2011; Reichow et al. 2008). Tomlinson et al. (2018) also evaluated telehealth training studies for quality indicators based on Reichow et al. (2008) and Reichow (2011) and found that the most common ratings were ‘weak’ or ‘borderline adequate’ with only one single subject study rated as ‘strong’. This study attended to quality indicators for single subject research (i.e., participant characteristics, independent variables, dependent variables, baseline conditions, visual analysis, and experimental control) and includes evidence of four of the six secondary indicators (i.e., IOA, fidelity, generalization and maintenance, and social validity). In addition, the lack of reporting of generalization and maintenance of behavior is as a major weakness of the telehealth training literature (Neely et al. 2018; Tomlinson et al. 2018). This study provides evidence of generalization of child and practitioner behaviors to a different activity context. Results suggest sustained behavior change, up to 5 weeks post-intervention, but it is unclear if behaviors maintained beyond 5 weeks. Future research should explore the impact of this intervention on maintenance beyond 5 weeks.

Further, the implementation of the telehealth training package was practical and cost effective, which addresses barriers cited by education professionals. Specifically, training and coaching were conducted based on practitioner preference for time and location, which did not require travel as practitioner and coach met virtually using a low cost tablet (i.e., $80) that was provided to them. Further, duration of the online training module (range 1–1.5 h) and videoconference coaching (M = 1.83 h per practitioner) reflected a practical time commitment without the need to travel or schedule face to face sessions. Although training duration may not be an indicator of training effectiveness (Brock and Carter 2017), it can impact the social validity and intervention fidelity (Ingersoll and Berger 2015).

Limitations and Future Research Directions

Limitations exist within this study and provide important foci for future research. First, a multi-component telehealth training package (i.e., online module, video self-evaluation, and performance feedback via videoconferencing) was used in this study. Therefore, it is not possible to isolate the impact of individual components within this study. Practitioners implemented their first intervention session with high fidelity after just the AFIRM module and access to NDBI procedures and self-evaluation checklist. Although a systematic review of telehealth training for autism-focused interventions (Neely et al. 2017) highlighted performance feedback as an essential training component, future research should include a component analysis to assess the most essential components of the telehealth training package and help improve training efficiency and effectiveness.

A second limitation is the specific child target communication skill chosen for Fred of Dyad F (i.e., labeling) which was of a different response class compared to the other child participant’s target communication skills (i.e., requesting). In support of social validity of goal selection and the distal nature of child behavior in this study, developmentally appropriate communication targets were chosen and prioritized by practitioners based on the child’s IEP. Labeling was a valued priority to Fae and was a developmentally appropriate communication target for Fred. Future research should value preference and consider functional and appropriate skills when selecting a child’s goals.

Also, when considering the generalizability of this intervention package to other practitioners and settings, it is important to note that practitioners in the current study chose to teach in inclusive preschool programs. As such, practitioners may have been motivated to learn and implement the chosen NDBI with increasing fidelity. Additionally, the coach had a prior working relationship with practitioners, and this relationship may have contributed to the positive outcomes within this study (Taylor et al. 2018). The existing relationship with the coach may have also influenced social validity scores. Although electronic responses to social validity assessments were anonymous and voluntary, participants were aware that the coach would have access to the information. Future research should investigate factors that impact effectiveness of the telehealth training including the characteristics of the coach, practitioner, and school setting as well as the relations between these factors.

Summary

As the field of early childhood education continues to embrace inclusive preschool placements for young children with disabilities, the need to implement effective, efficient, and socially valid interventions becomes pressing. Current research surrounding the use of telehealth as a means to train practitioners, although still limited, is progressing. This study adds to the current literature demonstrating that telehealth training is a promising platform for the dissemination of NDBIs to general education inclusive preschool practitioners as results suggest that it reduces barriers like travel time and costs and increases implementation fidelity, opportunities provided, and independent child communication behaviors. When education professionals select professional development models to prepare inclusive practitioners to support young children with varying disabilities, telehealth training may be an attractive option.

Change history

  • 01 February 2021

    The original version of the article has incorrect labels for Figs. 1 and 2.

References

  1. Amsbary, J., & AFIRM Team. (2017). Naturalistic intervention. Chapel Hill, NC: National Professional Development Center on Autism Spectrum Disorder, FPG Child Development Center, University of North Carolina. Retrieved September 4, 2018 from http://afirm.fpg.unc.edu/Naturalistic-intervention.

  2. Barton, E., & Smith, B. (2015). Advancing high quality preschool inclusion: A discussion and recommendations for the field. Topics in Early Childhood Special Education, 35, 69–78. https://doi.org/10.1177/0271121415583048.

    Article  Google Scholar 

  3. Barton, E. E., Lloyd, B. P., Spriggs, A. D., & Gast, D. L. (2018). Visual analysis of graphic data. In Single case research methodology (pp. 179–214). Routledge.

  4. Brock, M., & Carter, E. (2017). A meta-analysis of educator training to improve implementation of interventions for students with disabilities. Remedial and Special Education, 38, 131–144. https://doi.org/10.1177/0741932516653477.

    Article  Google Scholar 

  5. Bruns, D. A., & Mogharreban, C. C. (2008). Working with young children with disabilities: Perceptions, skills, and training needs of Head Start teachers. NHSA DIALOG, 11, 54–66. https://doi.org/10.1080/15240750701831958.

    Article  Google Scholar 

  6. Bzoch, K. R., League, R., & Brown, V. L. (2003). Receptive-expressive emergent language test third edition (REEL-3). St. Antonio: Pearson.

    Google Scholar 

  7. Carter, S. (2010). The social validity manual: A guide to subjective evaluation of behavioral interventions. Boston, MA: Academic Press.

    Google Scholar 

  8. Douglas, S., McNaughton, D., & Light, J. (2013). Online training for paraeducators to support the communication of young children. Journal of Early Intervention, 35, 223–242. https://doi.org/10.1177/1053815114526782.

    Article  Google Scholar 

  9. Dunst, C. J., & Bruder, M. B. (2014). Preservice professional preparation and teachers’ self-efficacy appraisals of natural environment and inclusion practices. Teacher Education and Special Education, 37, 121–132.

    Article  Google Scholar 

  10. Dunst, C. J., & Trivette, C. M. (2009). Using research evidence to inform and evaluate early childhood intervention practices. Topics in Early Childhood Special Education, 29, 40–52. https://doi.org/10.1177/0271121408329227.

    Article  Google Scholar 

  11. Ferguson, J., Craig, E., & Dounavi, K. (2018). Telehealth as a model for providing behaviour analytic interventions to individuals with autism spectrum disorder: A systematic review. Journal of Autism and Developmental Disorders, 49, 1–35. https://doi.org/10.1007/s10803-018-3724-5.

    Article  Google Scholar 

  12. Fischer, A. J., Dart, E. H., Radley, K. C., Richardson, D., Clark, R., & Wimberly, J. (2016). An evaluation of the effectiveness and acceptability of teleconsultation. Journal of Educational and Psychological Consultation, 27, 1–22. https://doi.org/10.1080/10474412.2016.1235978.

    Article  Google Scholar 

  13. Gebbie, D., Ceglowski, D., Taylor, L., & Miels, J. (2012). The role of teacher efficacy in strengthening classroom support for preschool children with disabilities who exhibit challenging behaviors. Early Childhood Education Journal, 40, 35–46. https://doi.org/10.1007/s10643-011-0486-5.

    Article  Google Scholar 

  14. Gibson, J. L., Pennington, R. C., Stenhoff, D. M., & Hopper, J. S. (2010). Using desktop videoconferencing to deliver interventions to a preschool student with autism. Topics in Early Childhood Special Education, 29, 214–225.

    Article  Google Scholar 

  15. Goldstein, S., & Naglieri, J. A. (2013). Autism spectrum rating scales: ASRS. North Tonawanda (NY): MHS.

    Google Scholar 

  16. Hay-Hansson, A. W., & Eldevik, S. (2013). Training discrete trials teaching skills using videoconference. Research in Autism Spectrum Disorders, 7(11), 1300–1309.

    Article  Google Scholar 

  17. Hieneman, M., Dunlap, G., & Kincaid, D. (2005). Positive support strategies for students with behavioral disorders in general education settings. Psychology in the Schools, 42, 779–794. https://doi.org/10.1002/pits.20112.

    Article  Google Scholar 

  18. Ingersoll, B., & Berger, N. (2015). Parent engagement with a telehealth-based parent-mediated intervention program for children with autism spectrum disorders: Predictors of program use and parent outcomes. Journal of Medical Internet Research, 17, 227–245. https://doi.org/10.2196/jmir.4913.

    Article  Google Scholar 

  19. Kennedy, C. (2002). The maintenance of behavior change as an indicator of social validity. Behavior Modification, 26, 594–604. https://doi.org/10.1177/014544502236652.

    Article  PubMed  Google Scholar 

  20. Koegel, R. L., & Koegel, L. K. (2006). Pivotal response treatments for autism: Communication, social & academic development. Baltimore, MD: Paul H Brookes Publishing.

    Google Scholar 

  21. Ledford, J. R., & Gast, D. L. (2018). Single case research methodology: Applications in special education and behavioral sciences. New York: Routledge.

    Google Scholar 

  22. Lindgren, S., Wacker, D., Suess, A., Schieltz, K., Pelzel, K., Kopelman, T., et al. (2016). Telehealth and autism: Treating challenging behavior at lower cost. Pediatrics, 137, S167–S175. https://doi.org/10.1542/peds.2015-28510.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Martens, B., Witt, J., Elliott, S., & Darveaux, D. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16, 191. https://doi.org/10.1037/0735-7028.16.2.191.

    Article  Google Scholar 

  24. Muccio, L. S., Kidd, J. K., White, C. S., & Burns, M. S. (2014). Head start instructional professionals’ inclusion perceptions and practices. Topics in Early Childhood Special Education, 34, 40–48. https://doi.org/10.1177/0271121413502398.

    Article  Google Scholar 

  25. Neely, L., Rispoli, M., Boles, M., Morin, K., Gregori, E., Ninci, J., et al. (2018). Interventionist acquisition of incidental teaching using pyramidal training via telehealth. Behavior Modification. https://doi.org/10.1177/0145445518781770.

    Article  PubMed  Google Scholar 

  26. Neely, L., Rispoli, M., Gerow, S., & Hong, E. (2016). Preparing interventionists via telepractice in incidental teaching for children with autism. Journal of Behavioral Education, 25, 393–416. https://doi.org/10.1007/s10864-016-9250-7.

    Article  Google Scholar 

  27. Neely, L., Rispoli, M., Gerow, S., Hong, E., & Hagan-Burke, S. (2017). Fidelity outcomes for autism-focused interventionists coached via telepractice: A systematic literature review. Journal of Developmental and Physical Disabilities, 29, 849–874. https://doi.org/10.1007/s10882-017-9550-4.

    Article  Google Scholar 

  28. Nickelson, D. W. (1998). Telehealth and the evolving health care system: Strategic opportunities for professional psychology. Professional Psychology: Research and Practice, 29, 527–535. https://doi.org/10.1037/0735-7028.29.6.527.

    Article  Google Scholar 

  29. Odom, S. L., & Bailey, D. (2001). Inclusive preschool programs: Classroom ecology and child outcomes. In M. J. Guralnick (Ed.), Early childhood inclusion: Focus on change (pp. 253–276). Baltimore: Brookes.

    Google Scholar 

  30. Odom, S., Buysse, V., & Soukakou, E. (2011). Inclusion for young children with disabilities: A quarter century of research perspectives. Journal of Early Intervention, 33, 344–356. https://doi.org/10.1177/1053815111430094.

    Article  Google Scholar 

  31. Odom, S., Cox, A., Brock, M., & National Professional Development Center on ASD. (2013). Implementation science, professional development, and autism spectrum disorders. Exceptional Children, 79, 233–251. https://doi.org/10.1177/001440291307900207.

    Article  Google Scholar 

  32. Pantermuehl, R. M., & Lechago, S. A. (2015). A comparison of feedback provided in vivo versus an online platform on the treatment integrity of staff working with children with autism. Behavior Analysis in Practice, 8, 219–222.

    Article  Google Scholar 

  33. Reichow, B. (2011). Development, procedures, and application of the evaluative method for determining evidence-based practices in autism. In B. Reichow, P. Doehring, D. V. Cicchetti, & F. R. Volkmar (Eds.), Evidence-based practices and treatments for children with autism (pp. 25–39). New York: Springer.

    Google Scholar 

  34. Reichow, B., Volkmar, F. R., & Cicchetti, D. V. (2008). Development of the evaluative method for evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders, 38, 1311–1319. https://doi.org/10.1007/s10803-007-0517-7.

    Article  PubMed  Google Scholar 

  35. Rispoli, M., Neely, L., Lang, R., & Ganz, J. (2011). Training paraprofessional to implement interventions for people with autism spectrum disorder: A systematic review. Developmental Neurorehabilitation, 14, 378–388. https://doi.org/10.3109/17518423.2011.620577.

    Article  PubMed  Google Scholar 

  36. Schreibman, L., Dawson, G., Stahmer, A. C., Landa, R., Rogers, S., McGee, G., et al. (2015). Naturalistic developmental behavioral interventions: Empirically validated treatments for autism spectrum disorder. Journal of Autism and Developmental Disorders, 45, 2411–2428. https://doi.org/10.1007/s10803-015-2407-8.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Secord, W., & Donohue, J. S. (2014). CAAP-2: Clinical assessment of articulation and phonology-2. Greenville: Super Duper Publications.

    Google Scholar 

  38. Shonkoff, J. P., & Phillips, D. A. (2000). From neurons to neighbourhoods. The science of early childhood development. Washington DC: Academic Press.

    Google Scholar 

  39. Suhrheinrich, J., Chan, J., Melgarejo, M., Reith, S., Stahmer, A., & AFIRM Team. (2018). Pivotal response training. Chapel Hill, NC: National Professional Development Center on Autism Spectrum Disorder, FPG Child Development Center, University of North Carolina. Retrieved September 4, 2018 from http://afirm.fpg.unc.edu/Pivotal-response-training.

  40. Taylor, B. A., LeBlanc, L. A., & Nosik, M. R. (2018). Compassionate care in behavior analytic treatment: Can outcomes be enhanced by attending to relationships with caregivers? Behavior Analysis in Practice, 12(3), 654–666. https://doi.org/10.1007/s40617-018-00289-3.

    Article  PubMed  PubMed Central  Google Scholar 

  41. Tomlinson, S. R., Gore, N., & McGill, P. (2018). Training individuals to implement applied behavior analytic procedures via telehealth: A systematic review of the literature. Journal of Behavioral Education, 27, 172–222. https://doi.org/10.1007/s10864-018-9292-0.

    Article  Google Scholar 

  42. U.S. Department of Education. (2018). Annual report to congress on the implementation of the Individuals with Disabilities Act. Washington, DC: U.S. Department of Education.

  43. VandenBos, G. R., & Williams, S. (2000). The Internet versus the telephone: What is telehealth anyway? Professional Psychology: Research and Practice, 31(5), 490–492.

    Article  Google Scholar 

  44. Vismara, L., Young, G., Stahmer, A., Griffith, E., & Rogers, S. (2009). Dissemination of evidence-based practice: Can we train therapists from a distance? Journal of Autism and Developmental Disorders, 39, 1636–1651. https://doi.org/10.1007/s10803-009-0796-2.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Wacker, D. P., Lee, J. F., Dalmau, Y. C. P., Kopelman, T. G., Lindgren, S. D., Kuhle, J., et al. (2013). Conducting functional analyses of problem behavior via telehealth. Journal of Applied Behavior Analysis, 46, 31–46.

    Article  Google Scholar 

  46. Wainer, A., & Ingersoll, B. (2013). Disseminating ASD interventions: A pilot study of a distance learning program for parents and professionals. Journal of Autism and Developmental Disorders, 43, 11–24. https://doi.org/10.1007/s10803-012-1538-4.

    Article  PubMed  Google Scholar 

  47. Wainer, A. L., & Ingersoll, B. (2015). Increasing access to an ASD imitation intervention via a telehealth parent training program. Journal of Autism and Developmental Disorders, 45, 3877–3890. https://doi.org/10.1007/s10803-014-2186-7.

    Article  PubMed  Google Scholar 

  48. Wainer, A., Pickard, K., & Ingersoll, B. (2017). Using web-based instruction, brief workshops, and remote consultation to teach community-based providers a parent-mediated intervention. Journal of Child and Family Studies, 26, 1592–1602. https://doi.org/10.1007/s10826-017-0671-2.

    Article  Google Scholar 

  49. Walls, S. (2007). Early childhood preservice training and perceived teacher efficacy beliefs concerning the inclusion of young children with disabilities. Doctoral dissertation.

  50. Wolf, M. (1978). Social validity: The case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.

    Article  Google Scholar 

  51. Wong, C., Odom, S., Hume, K., Cox, A., Fettig, A., Kucharczyk, S., et al. (2015). Evidence-based practices for children, youth, and young adults with autism spectrum disorder: A comprehensive review. Journal of Autism and Developmental Disorders, 45, 1951–1966. https://doi.org/10.1080/10459881003785506.

    Article  PubMed  Google Scholar 

  52. Zimmerman, I. L., Steiner, V. G., & Pond, R. E. (2012). Preschool language scale (5th ed.). San Antonio, TX: Harcourt Assessment.

    Google Scholar 

Download references

Acknowledgments

The authors would like to acknowledge that this manuscript was prepared from the first author’s doctoral dissertation. Research enhancement funds from Michigan State University partially supported technology purchases for this dissertation research. The authors would also like to thank the participants and education professionals for their collaboration and support of this study.

Author Contributions

SRD conceived of the study, participated in the design and coordination, conducted all intervention sessions, analyzed and interpreted data, and drafted and revised the entire manuscript. SND participated in the design, assisted in data analyzation, and helped to draft and revise the manuscript. EH was the secondary coder of data and helped draft and revise the manuscript. All authors read and approved of the final manuscript.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Sophia D’Agostino.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the Michigan State University institutional research committee (STUDY00000919) and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained for all individual participants included in this study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

D’Agostino, S., Douglas, S.N. & Horton, E. Inclusive Preschool Practitioners’ Implementation of Naturalistic Developmental Behavioral Intervention Using Telehealth Training. J Autism Dev Disord 50, 864–880 (2020). https://doi.org/10.1007/s10803-019-04319-z

Download citation

Keywords

  • Telehealth
  • Preschool
  • Inclusion
  • Naturalistic developmental behavioral intervention
  • Single-case design