Introduction

One of the main characteristics of individuals with autism is limited social communication skills [1]. This means autistic individuals might have limited ability to naturally learn and develop communication skills to build and maintain relationships and interact with others. In addition, the limited social communication skills might be linked to engaging in challenging behaviors [2]. Caregivers (e.g., parents, other family members) are the first to encounter their child's developmental difficulties, and they have the most influence on their child’s development [3, 4]. Given the importance of the caregivers’ role in children’s development and the evidence supporting the effectiveness of their involvement, parent-implemented intervention is considered an empirically validated intervention for young children with autism [5•, 6, 7•].

Telepractice, also known as telehealth or telemedicine, is a distance service delivery model in which telecommunication and internet technology are used [8]. Utilizing telepractice to enhance service delivery models gained attention from researchers conducting parent-implemented interventions and was found to be an effective and efficient service delivery model [9]. Family members learn new skills and strategies via telepractice and implement them with fidelity with their own children [e.g., 10, 11]. Researchers who reviewed the literature on parent-implemented intervention via telepractice for children with autism, focusing on language and communication outcomes [e.g., 12•] or behavior analytic services [e.g., 13•] reported the intervention effectiveness. In addition, due to the COVID-19 pandemic, it is now common to use telepractice to get connected and communicate with others; therefore, it is essential to examine the current practices and trends in telepractice.

Although parent-implemented autism interventions are considered empirically validated interventions, it is important to ensure the participants or potential consumers perceive these interventions as feasible and acceptable, or in other words, find the intervention socially valid. This is especially important when adapting interventions to marginalized and minoritized populations. However, limited information related to social validity is reported in previous reviews [e.g., 9, 12•, 13•]. Social validity is a concept that was first developed by researchers who had concerns about the social meaningfulness of interventions in applied research [14, 15]; since then, capturing the social validity aspects in applied research has been highly recommended. For example, in single-case research, conducting social validity assessment is one of the gold standards for methodological rigor [16]. While the rationale for conducting social validity assessment might vary, it can fall under the following categories that Snodgrass and her colleagues explored and integrated [17•]: (a) helping to understand the full intervention effect; (b) including consumers’ and participants’ voice or opinion in the treatment; or (c) helping to inform or improve the intervention program (i.e., independent variable).

Wolf (1978) defined three different aspects of social validity including goals (whether the target/goal of the intervention is socially important), procedures (whether the participants and other consumers consider the intervention procedures acceptable and feasible), and outcomes (whether the outcomes of the intervention are considered effective for participants and other consumers). In general, there are three approaches to assessing social validity: (a) subjective evaluation (e.g., assessing participants’ opinions through questionnaire, interview, or masked rating); (b) normative comparison (e.g., comparing the goal/skill to others); and (c) maintenance (e.g., observing if/how well the intervention procedures/effects are sustained) [14, 15, 18]. See Table 1 for additional information on each approach.

Table 1 Additional information about each social validity approach
Full size table

Although guidelines for assessing social validity exist, researchers conducted it in many different ways with varying levels of depth and rigor. This scattered use of the social validity assessment might be due to the social validity concept containing various aspects/dimensions or based on determining social importance (e.g., feasibility, efficiency, effectiveness), which can be a subjective decision. As such, researchers have explored the trend of assessing social validity since it was first conceptualized [e.g., 19,20,21]. It was reported that most researchers use subjective assessment methods such as interviews or Likert-type questionnaires with the participants or other consumers [19, 20]. Also, researchers tend to capture the social validity of the intervention’s outcomes rather than the intervention’s goals or procedures. It was recommended to conduct and report social validity assessment in a more comprehensive way, such as utilizing objective measures in addition to subjective evaluation, obtaining data from multiple sources, and collecting and analyzing social validity data using rigorous scientific methods [20, 21].

Due to the COVID-19 pandemic, and the remarkable technological development, the telepractice approach for parent-implemented communication intervention for autistic children is continuously receiving attention in the field. However, little is known about the social validity assessments and outcomes from previous reviews on parent-implemented telepractice research. Thus, the purpose of this review is to explore the current practices of conducting social validity assessment and reporting its outcomes within parent-implemented telepractice communication intervention research for children with autism.

Method

Three databases were used in this review, including PsychArticles, PsycInfo, and ERIC. The search terms included (a) caregivers (parent-mediated, parent-implemented, caregiver-mediated, caregiver-implemented, family-mediated), (b) autism (autism, autistic, PDD-NOS, ASD), (c) social communication (communication, social communication), and (d) telepractice (tele-based, telepractice, remote, online, virtual, distance, telemedicine). For a study to be included, it had to (a) include caregivers as participants; (b) include telepractice intervention focusing on social communication outcomes; (c) of children aged 0 to 8 years with a diagnosis of autism; and (d) be published in a peer-reviewed journal in English within the past five years (2017–2022). Studies were excluded if they were (a) gray literature (e.g., dissertation, thesis) or (b) did not contain any experimental data.

We followed the PRISMA reporting guidelines [22] for a systematic search process. The search, conducted in June 2022, yielded 48 studies across the three databases, and one study was identified from a backward search. After removing duplicates (n = 8), the remaining 41 studies were screened based on their title and abstract, and 27 studies were excluded for the following reasons: (a) not an experimental study (n = 12); (b) published more than five years ago (n = 6); (c) gray literature (n = 4); (d) no participants with autism diagnosis (n = 3); (e) no telehealth component (n = 1); and (f) not published in English (n = 1). Then, the first two authors independently reviewed the remaining 14 studies with full-text to determine their eligibility, and three studies were excluded as they did not meet the inclusion criteria. The third author then reviewed five randomly selected studies and reached 100% of agreement. Therefore, this review includes data from a total of 11 studies. The first two authors coded the studies and extracted detailed information related to (a) demographic characteristics of participants, (b) interventions including telepractice components and fidelity, and (c) reported outcomes and social validity.

Results

Demographic Data

Overall, 257 caregivers were included in the identified studies. One study [23••] did not report the number of caregiver participants. Of the 257 caregivers, 113 were identified as mothers, 15 as fathers, and one as a grandmother. The role of 129 participants was not described. Of the 293 children who participated in the studies, 157 were identified as males and 44 as females. The gender of 92 child participants was not reported. The vast majority of children had an autism diagnosis, and only a few were diagnosed with other developmental disabilities, such as Rett syndrome, Down syndrome, and intellectual disability. Children were between 18 months and 13 years old; however, only a few were between 8 and 13 years old. The average age of the children across 11 studies ranged between 29 months and 6 years old. Notably, only a few studies provided information about the race and ethnicity of child and caregiver participants.

The majority of studies were completed only in the USA (n = 7; 64%). Two studies were conducted in Canada, one study in India and one study included participants from eight different countries (i.e., Costa Rica, Greece, Mexico, Russia, Saudi Arabia, Turkey, Ukraine, and the USA). Researchers also reported that two studies were conducted in rural and remote communities in the USA and Canada.

Interventions

Different types and components of the interventions were delivered via telepractice. Some researchers adapted manualized interventions (e.g., Early Start Denver Model, JASPER, Project ImPACT) to deliver it via telepractice [24••, 25••, 26••] while others delivered specific strategies (e.g., functional communication training) via telepractice [27••, 28••]. Researchers had different primary outcomes to measure the effectiveness of their interventions, including outcomes both at the caregiver and child levels. A few researchers reported outcomes at the caregiver’s level [e.g., 29••, 30••, 31••], which focused on the level of caregivers’ adherence to intervention protocol or latent variables such as parental stress. Outcomes at the child’s level included children’s social communication skills as a primary outcome [e.g., 24••, 27••]. Researchers also reported varying use of telepractice technology. While most researchers used synchronous telepractice options (e.g., live video conferencing applications) to deliver interventions, some used asynchronous options (e.g., providing self-paced online training, uploading recorded videos in a shared folder).

In terms of fidelity, there were two types of data that researchers reported, including fidelity of intervention (i.e., caregivers’ adherence to intervention protocol) and fidelity of implementation (i.e., researcher/therapists’ adherence to teaching caregivers)—these were often interchangeably used. All studies, except for Lindgren et al. [23••], reported some measurement of fidelity to the intervention. For example, Rooks-Ellis et al. [ 25••] reported using the caregiver fidelity rating system of the intervention they used, which measures caregivers’ adherence to the intervention protocol delivered to their own child. Additionally, researchers in some studies reported data related to the fidelity of implementation. For example, Gerow et al. [31••], measured adherence to telepractice parent training protocol using a fidelity task analysis for each session.

Outcomes and Social Validity

Overall, researchers of the identified studies reported positive outcomes of the parent-implemented telepractice intervention. Caregivers learned the targeted strategies or procedures (e.g., communication teaching strategies, functional assessment, functional communication training) via telepractice and implemented them with fidelity [e.g., 24••, 25••, 26••, 29••, 31••, 32••]. In addition, in most studies, caregiver implementation was related to an increase in children’s social communication behavior [e.g., 26••, 27••, 30••, 32••, 33••] or decrease in challenging behavior [e.g., 23••, 28••].

Of the 11 included studies, 9 studies assessed the social validity of their intervention at the end of the study, and two studies did not discuss social validity [24••, 29••]. Notably, the information related to social validity was limited in most studies. Researchers primarily used Likert-type questionnaires with 5-point [30••, 33••], 6-point [25••, 31••, 32••], or 7-point scales [23••, 27••, 28••] to assess caregivers’ acceptability, feasibility, and satisfaction. Four studies [23••, 27••, 28••, 31••] used a modified or revised version of the Treatment Acceptability Rating Form [34]. In one study [26••], researchers conducted a focus group with the caregivers that focused on participants’ experience with the technology and their perceived benefits and challenges. Finally, in a few studies, the researchers collected both quantitative (Likert questions) and qualitative (open-ended questions) data to explore the social validity of the intervention [25••, 30••, 32••, 33••].

To report the results of the social validity assessment, most researchers provided quantitative information. Researchers included the range, standard deviation, and/or average ratings of the Likert questionnaire [e.g., 23••, 27••,28••, 31••, 32••] or an overall satisfaction score [e.g., 30••] as the outcomes of the social validity assessment. Researchers who collected qualitative data through open-ended questions reported sample participants’ qualitative responses in the results section [e.g., 25••], an appendix [33••], or did not provide this information at all. Shire et al. [26••] who conducted a focus group coded qualitatively the transcript from the focus group and reported on themes that emerged from the data. Overall, researchers reported that their intervention was socially valid; however, there was a lack of transparency and details of social validity assessment and outcome in the reported information.

Discussion

Parent-implemented interventions are considered empirically validated interventions for autistic children [7•], and researchers found parent-implemented interventions that target the social communication skills of children with autism to be effective [3]. Furthermore, parent-implemented interventions for autistic children via telepractice are also reported to be effective and lead to positive outcomes for both caregivers and children [12•]. Although there are advantages to using telepractice to support caregivers to learn new skills and strategies and implement them in the natural environment with their children with autism (e.g., flexibility in scheduling, working in the home environment), there are also barriers and challenges to using it. Most reported challenges are related to (a) technology (e.g., connection issues, quality of video) and (b) logistics (e.g., access to knowledge and equipment, limited interaction with between service providers and children) [35]. These challenges need to be addressed when conducting parent-implemented intervention via telepractice.

With advancing technology and its increased use during the pandemic, it is expected that telepractice interventions will continue to be frequently used as a supplemental or alternative model for supporting children with autism and their families. As a field, there is a need to identify and evaluate ways to make parent-implemented telepractice interventions accessible and feasible for all. One way to understand the acceptability and feasibility of this type of intervention is to assess the social validity of its goals, procedures, and outcomes, and make iterations to the intervention based on data. An intervention can be reported to be effective with positive outcomes. However, if caregivers do not share the same sentiment, do not feel that they can implement it during their everyday routines, or do not believe the outcomes are meaningful or important, they will not continue to use the learned strategies with their children.

In this review, we sought to explore how social validity, within the context of parent-implemented telepractice interventions for autistic children, was assessed in recently published articles (i.e., articles published in the past five years). As reported in other studies [9, 12•, 13•], researchers in the reviewed articles reported positive outcomes for parent-implemented telepractice interventions. Notably, however, information about social validity assessments was limited. Two studies did not report on social validity data, and all studies, except one, focused primarily on a Likert questionnaire to evaluate the social validity of the intervention, which aligns with previous literature reviews on social validity [19–21]. Only one research team conducted a focus group to qualitatively explore in-depth participants’ experiences and perceptions of parent-implemented telepractice intervention [26••]. Researchers in all studies used subjective evaluation methods, and only at the end of the study. Furthermore, most researchers included limited information about the findings of the social validity assessment and rarely included in depth discussion of the social validity findings as they relate to the other findings in the study.

Implications and Limitations

This review of the recent literature reveals several important implications for both research and practice in the field of autism intervention. First, it should be noted that authors of some of the reviewed studies did not include sufficient information related to participant demographics (e.g., race, ethnicity, socioeconomic status), which has been recognized as a longstanding reporting issue in autism research [36]. Autism researchers should be careful not to omit this important information for advancing the intervention science in the field as well as to promote diversity within the field. Furthermore, even when researchers reported sufficient demographic information of participants, families of young autistic children who live in minoritized communities are disproportionately underrepresented. This calls for specific actions from researchers and practitioners, which may include (a) intentionally recruiting from community settings to test the effectiveness of a novel intervention, (b) rigorously adapting existing interventions to increase contextual fit, and (c) purposefully fostering community-academic partnership.

Second, as noted above, the majority of studies assessed the social validity of their interventions using Likert-type questionnaires. While Likert-type questionnaires are widely recognized as a valid method that could potentially reveal important information, this does not reflect the current advances in how intervention developers may measure social validity (see Table 1), including (a) qualitative inquiries, (b) assessing social validity with additional stakeholders, and (c) conducting masked ratings of changes in the target behaviors (e.g., having masked raters evaluated videos from the study). Diversified approaches and methods for assessing social validity may be necessary depending on the context or the type of intervention used, and such methods will also likely increase the credibility of the social validity data. It is also important to collect data on different dimensions of social validity, including goals, procedures, and outcomes, and across different times (e.g., before and after the intervention) when assessing social validity. Moreover, it is helpful to use the social validity data to understand other types of data and make adaptations to the intervention iteratively [37, 38].

There are some limitations of this review that warrant caution when interpreting the findings. First, despite the attempts to broaden the search by using different sets of keywords and databases, a small number of studies were included in this review. Second, we did not evaluate the methodological rigor of the included studies. These limitations warrant future researchers to conduct reviews with a larger body of literature. Third, although an independent coder reviewed studies for screening and inclusion to evaluate the reliability of these steps, agreement was not evaluated during data extraction. Finally, it is possible that the reported information in the published articles, specifically as it relates to the assessment of social validity, does not represent all the procedures and analyses conducted by the researchers, and therefore, does not accurately represent social validity assessments used by researchers. Due to page limitations when publishing in peer-reviewed journals, researchers might decide not to include information on what is sometimes considered secondary, social validity data. We believe that social validity data are equally important as other types of data and should be included in all reports.

Conclusion

In conclusion, the advancement and wider availability of technology catalyzed by the COVID-19 pandemic diversified how autism interventions are delivered to families, including the use of telepractice. The findings of this review suggest that while social validity is an important component of telepractice intervention, the reports of social validity data are considered limited in both quantity and quality. We call researchers to evaluate the social validity of parent-implemented telepractice interventions for autistic children using multiple data sources, approaches, and methods and report on their findings as they relate to other types of data. This will ensure interventions are more feasible, acceptable, and have a contextual fit for diverse populations.