An estimated 5.4 million US adults and one in 44 children have autism spectrum disorder (ASD; Centers for Disease Control & Prevention, 2022; Dietz et al., 2020). Data suggest that only 39.2% of people ages 21–64 with any disability in the US are employed compared to 80.7% of their peers (Erickson et al., 2022). Historically, young autistic adults experience the lowest employment rates of any disability category (Newman et al., 2011), and many autistic adults struggle to find and maintain competitive integrated employment. Despite increasing awareness of this discrepancy, autistic adults struggle to find and maintain competitive integrated employment. Employment is a key element of independent living, and access to sustained employment opportunities contributes to increased quality of life.

Social interaction issues in the workplace are among the most significant barriers to employment success for autistic individuals (Chen, 2015; Hendricks, 2010). These skills, hereafter referred to as vocational social skills (VSS), include interactions required for adequate job performance, such as asking for help with unclear tasks and everyday social interactions with customers or coworkers. A defining characteristic of autism is persistent difficulty with social communication (American Psychiatric Association, 2013), so it is unsurprising that VSS challenges affect autistic adults in the workplace. It follows that improved VSS are likely to lead to reduced employment barriers and increased long-term employment outcomes.

Carter et al. (2012) found that for youth with disabilities with greater support needs, those with little or no trouble communicating were three to four times more likely to be employed after high school than youth with communication difficulties. Therefore, it is likely that increasing VSS in autistic youth is likely to result in improved employment outcomes.

Vocational Social Skills Assessment

Assessment is key to transition interventions and transition-related research (Trainor et al., 2020). Few tools exist for determining instructional targets when designing interventions to support the development of common VSS. Lerman et al. (2017) developed a clinic-based assessment to allow for the identification of key VSS for behavioral intervention. The assessment involves placing participants in common, but potentially challenging workplace situations (referred to as evocative situations) that require problem-solving and interaction with supervisors to complete job tasks. The purpose of the assessment is to determine participants’ current ability to engage in expected social interactions during situations that are likely to occur frequently on the job. Evocative situations include giving vague instructions, missing or broken materials, asking participants to perform a task not in their repertoire, providing multi-step instructions, time pressure, and supervisor unavailability. Lerman and colleagues successfully used the assessment to identify important skills that were missing from the participants’ repertoire, thus providing an objective approach to identifying behavioral targets for intervention.

Grob et al. (2019) later used this assessment to inform an intervention involving behavioral skills training (BST) plus stimulus prompts to teach identified VSS to three autistic young adults during role-play scenarios. After conducting the assessment, the researchers identified making confirming statements (e.g., “Got it. I will do it the way you showed me”), asking for a task model (e.g., “Can you show me how?”), and apologizing in response to corrective feedback as behavioral targets, because these behaviors are considered expected workplace behavior but were observed infrequently during the assessment. Researcher-implemented BST with added prompting was effective at increasing the use of targeted skills for two of three participants (with one participant requiring additional reinforcement-based intervention) and generalized to a new supervisor and a new setting for all three participants.

Grob and colleagues (2019) emphasized the importance of using prompts to facilitate generalization and the need for prompts to be easily accessible and socially valid. While these authors provided compelling evidence for using the VSS assessment to inform the selection of VSS behavioral targets for transition-age autistic youth, effective implementation of the interventions utilized in the study (e.g., BST, multiple exemplar training, and tangible reinforcement) requires qualified professionals. The cost of hiring these professionals can be high, resulting in the need for research on more efficient intervention techniques, such as video modeling (VM), that can be easily implemented by practitioners (Pollard et al., 2014).

Video Modeling

One approach for teaching VSS to transition-aged autistic youth is VM. VM is based on social learning theory (Bandura, 1969), which suggests that one-way individuals learn is through observing others. Because VM relies on visual, rather than auditory presentation, it may be a better fit for autistic individuals due to the differences in neurological processing for this population (Williams, 2007). Researchers have long used VM to teach autistic children and adults a variety of skills, such as social behavior, academic skills, and tasks of daily living (Green et al., 2013; O’Connor, 1969; Rausa et al., 2016), and VM has also been classified as an evidence-based practice to teach some transition-related skills (Rowe et al., 2021). In relation to employment skills, VM has most often been used to teach job skills associated with successful completion of workplace tasks; however, some researchers have also used VM to teach VSS (Bross et al., 2021; Whittenburg, 2020).

Van Laarhoven et al. (2014) used a multiple treatments with reversal design to compare the effects of VM and video feedback to teach job skills, including VSS, to four transition-age autistic youth. In the VM condition, students watched models of the skills before the opportunity to perform them. In the video feedback condition, they watched recordings of themselves performing the tasks and were asked to review their performance. The authors found that only one student benefited more from VM than video feedback; however, performance improved for three out of four students when the video feedback condition immediately followed the VM condition. Although carryover effects are a potential confound, the findings suggest that feedback is an important element of VSS instruction and that using VM to teach students how to perform a task followed by feedback on their performance may result in robust improvement.

Stauch and Plavnick (2020) used VM to teach vocational skills and VSS in tandem to two transition-age autistic youth. The authors chose high-frequency social behaviors based on common skills taught or assessed in the literature, including accepting compliments and criticism and making small talk as targets for intervention. They observed increased use of both sets of skills; however, participants demonstrated higher levels of proficiency with the job skills and lower proficiency with the VSS.

Bross et al. (2020) used VM to teach customer service skills to five autistic adults in a community employment setting. Participants watched a brief video before their shift and were asked about appropriate phrases they would use during the workday, with verbal praise given for correct responses. Following intervention, all participants demonstrated increased use of appropriate greetings, service, and closing phrases, although total accuracy varied across participants. The participants continued to show improved use of learned skills when a supervisor, job coach, or coworker implemented the VMs following training. However, target behaviors for this study were job specific. This study provides evidence of VM’s effectiveness in teaching VSS in the natural environment and that social skills can generalize to other intervention agents.

The use of VM has consistently been shown to be an effective practice for teaching autistic individuals various skills (Delano, 2007; McCoy & Hermansen, 2007); however, while promising, the evidence is somewhat mixed related to its effectiveness in teaching VSS. Researchers have often taught VSS in combination with other job skills (Stauch & Plavnick, 2020; Van Laarhoven et al., 2014) or job-specific skills (Bross et al., 2020). Common VSS may require isolated intervention to increase the likelihood of skill acquisition. Additionally, considering Van Laarhoven et al. (2014) found that video feedback improved responding compared to VM alone, the effects of VM are likely improved by adding performance feedback. More evidence is needed to determine the effects of VM in conjunction with other intervention components, such as feedback, on the acquisition and generalization of common VSS.

Feedback

Performance feedback is the presentation of a stimulus (usually verbal or written) following a person’s behavior that varies as a function of that behavior and allows an individual to change their performance (Mangiapanello & Hemmes, 2015; Sleiman et al., 2020). Wisniewski et al. (2020) conducted a meta-analysis on educational feedback research and found that feedback is most effective when it is corrective and contains clear information on what mistakes a student made, why they made these mistakes, and what they can change in the future to improve. Feedback is an intervention component that is regularly used in job settings and tends to be effective, including when used in concert with other interventions (Sleiman et al., 2020). It is likely that including feedback as a component of a VSS intervention will result in increased acquisition of targeted skills.

Purpose and Research Questions

The purpose of this work is to extend the literature on vocational assessment and training by using the assessment first described by Lerman et al. (2017) and later used by Grob et al. (2019) to inform selecting behavioral targets for an intervention package involving the use of VM plus feedback to teach common VSS to transition-aged autistic youth in a simulated work environment. Additionally, we sought to understand the effects of using practitioner-implemented VM and feedback on autistic youths’ performance of common VSS in a simulated work environment and if VSS mastered via VM and feedback in a simulated work environment generalized across supervisors and settings. We pose the following research questions:

  1. 1.

    What are the effects of using practitioner-implemented VM and feedback on autistic youths’ performance of common VSS in a simulated work environment?

  2. 2.

    To what extent do VSS mastered via VM and feedback in a simulated work environment generalize across supervisors and settings?

Method

Participants and Setting

Participants were recruited through a local service provider that offers day school and outpatient behavioral services for autistic children and adults. To be eligible to participate in the study, participants had to be between the ages of 14–30 years and either (1) be enrolled in pre-vocational or vocational services at the school or (2) be autistic and employed in the community or have interest in employment and a high potential for independent employment (i.e., could communicate verbally and have no dangerous problem behavior). Transition services in public schools are typically provided between the ages of 14–22. We extended the age range to 30 to account for the adult services provided by the agency. We recruited three participants for this study based on recommendations from agency staff and word of mouth.

Dwayne was a white male with ASD aged 17. Dwayne communicated verbally using complex language and could read and write independently. Dwayne was placed in an alternative school specializing in ASD to better meet his learning needs and had never been employed. Nihil was a white autistic male aged 19 who self-identified as non-binary. Nihil was a student at a large, public state university, taking core undergraduate courses. Nihil communicated verbally using complex language and had never been employed. Troy was a Chinese male with ASD aged 23 and recently graduated from his local high school’s post-high vocational program. Troy communicated verbally using shorter, less complex sentences with more stereotyped language and an atypical tone of voice. Troy was employed as a line chef at a restaurant and had previous work experience stocking shelves at an electronics store. All participants could read and write independently.

The instructor was a Registered Behavior Technician® (RBT) working at the local service agency. She was a 24-year-old white female with 3 years of experience working in the field. She had no experience working with Nihil and Troy prior to the study and had little experience working with Dwayne, although they occasionally interacted while at the school.

The study took place at the local service agency. Assessments were performed in a room set up to simulate a coffee shop, and the intervention took place in the same room for Nihil and Troy and in a classroom within the same building for Dwayne. The simulated coffee shop in the day school was used as a space for the school to provide vocational training and was built to closely resemble a small coffee shop. The room included a service counter with a register, a small table with two chairs, and two large countertop spaces that held coffee brewing supplies and merchandise. The classroom where training for Dwayne occurred was a large, mostly open space, with several tables that served as the workspace for Dwayne and a countertop with built-in cabinets and drawers along one wall. Dwayne attended the day school and his training occurred during the school day which required us to use the classroom for training rather than the café due to scheduling purposes. The café was equipped with a camera with audio and video recording, which allowed for remote observations, so each participant was alone in the workspace when working. We chose the café as the setting for assessment and training because it most closely resembled a workplace; however, we also included tasks and materials not commonly associated with work in a café (e.g., clerical tasks or hanging clothes) to increase the likelihood of generalization of trained skills to a variety of workplace settings in which the participants may seek employment.

Response Measurement, Evocative Situations, and Reliability

Targeted behaviors were based on the VSS assessment (Lerman et al., 2017), with priority given to behaviors with the lowest score. Behaviors identified for intervention include making confirming statements when given a task, asking for help with missing or broken materials, responding to an interrupted task, returning to work when the supervisor is unavailable, and notifying the supervisor of task completion. We broke down each behavior into its component skills and collected data on each step in the task analysis (see Table 1). If a participant consistently failed to demonstrate a given step in the task analysis during the assessment that step was identified as a potential target for intervention (noted with a * in Table 1). For example, a participant might accurately perform all steps when asking for help with missing materials except for searching the area for more materials. If so, searching for missing materials would be a potential target for intervention. All materials used during data collection as well as raw data for the study are openly available at https://osf.io/5kdf8/?view_only=575fecb7ff5a418ba9ee1ba0b6520a84.

Table 1 Operational definitions of behavioral targets
Full size table

Evocative Situations

During baseline, probe, and intervention trials, we created an opportunity for the participant to demonstrate each targeted skill by contriving situations likely to serve as establishing operations (EOs) that would evoke the targeted behavior. Behaviors targeted across the three participants included making confirming statements, searching for missing materials, returning to work when the supervisor was unavailable, asking about returning to work on a prior task when interrupted, and notifying the supervisor when a task was completed. We used the following methods for creating evocative situations for each of these targets.

To observe the participant making confirming statements, the instructor gave the participant a direction to complete a task (e.g., “I need you to do X.”) and waited 2–3 s for a response. A correct response consisted of the participant confirming they heard the instruction and repeating part of the instruction back to the instructor. For example, “Ok, I will fold the shirts.” If no response occurred, the instructor said, “I’ll be in my office,” and left the workspace. All other responses were scored as incorrect.

To contrive an EO for searching for missing materials, we intentionally withheld sufficient supplies necessary for completion of the required task. For example, the instructor would ask the participant to hang shirts and would not give them enough hangers. A correct response involved the participant opening the drawer where additional supplies were located and searching the drawer for the materials. If the participant did not search their area for additional materials, this was scored as incorrect. Sometimes materials were available in the drawer allowing the participant to complete the task independently and sometimes they were not, resulting in the need to leave the workspace and ask the instructor for additional materials.

To observe the participant returning to work when the supervisor is unavailable, the instructor would leave her office and wait in a different part of the building after giving the participant their instructions. Once the participant completed the task, they would go to the instructor’s office and find that the instructor was not there. Correct responding included returning to the workspace and finding a task to occupy their time. An incorrect response occurred if the participant returned to the workspace and waited for the instructor without doing any additional work. During these trials, we ensured that there was an incomplete task available in the workspace by leaving an interrupted task in the environment or by having cleaning supplies available as the participant always had the option to “tidy their workspace.”

For asking about returning to a prior task when interrupted, the supervisor would present a task to the participant and leave the workspace. While the participant was still working on the task, the instructor would return and say, “I need you to stop working on X, and instead do Y.” She then provided instructions on how to do the new task and waited 2–3 s to allow the participant to ask whether they should return to the first task after completing the new one. The instructor would then say either yes or no. An incorrect response occurred when the participant failed to ask about returning to the prior task.

Finally, a participant had the opportunity to notify the instructor of a completed task following the full completion of any task. This required the participant to leave the workspace, knock on the instructor’s office door, wait for the instructor to answer, and indicate to the instructor that they finished the task. The instructor would then provide a new task for the participant. An incorrect response occurred when the participant finished a task and waited for the instructor to return without leaving the area to notify the instructor that they finished the task.

Interobserver Agreement

A second independent observer was trained by the lead author to collect interobserver agreement (IOA) data. Training occurred during a 1-h session before the beginning of the study. The lead researcher explained the study’s procedures, reviewed the data collection materials, and answered any questions. The observer was given the opportunity to practice during the assessment phase for the first participant, and IOA data were reviewed regularly.

The second independent observer collected data during intervention probe trials for all three participants as well as post-assessment data for Nihil and Troy. We calculated trial-by-trial IOA by comparing the two data collectors’ scores for the targeted behavior, adding the total number of trials with agreement, then dividing by the total number of trials with agreement or disagreement and multiplying by 100. The data were binary (i.e., “yes” or “no”) so agreements were coded as one, and disagreements were coded as zero. IOA data were collected for 42.3% of trials for Dwayne with an average agreement of 100%, 21.3% of trials for Nihil with an average agreement of 92.3%, and 22% of trials for Troy with an average agreement of 66.7%. Trial-by-trial IOA across all three participants was 90.9%.

Procedural Fidelity

The lead researcher trained the instructor on the intervention and probe procedures during a 1-h session prior to beginning the study. The lead researcher explained the study goals and procedures and answered the instructor’s questions. The instructor then role-played the scenarios with the researcher until the instructor completed all steps correctly.

Before each session, the researcher briefly met with the instructor to discuss the activities for the day. If the instructor committed procedural fidelity errors, the researcher provided immediate feedback privately to the instructor before beginning the next trial. Instructor fidelity was assessed using an implementation checklist containing the following steps: (1) plays video model, (2) presents task to participant, (3) includes evocative situation, (4) responds to client, and (5) delivers appropriate feedback. We scored steps as either correct, incorrect, or not applicable (e.g., VMs were not used during probe trials). Implementation data were collected on the percentage of steps the instructor completed correctly during each trial. The average instructor fidelity across all trials was 95% correct (range: 67–100%). A second independent observer collected treatment fidelity data during 40% of trials. We calculated trial-by-trial IOA for the procedural fidelity measure by adding the total number of agreements in each trial and dividing this by the total number of agreements plus disagreements, then multiplying by 100. The average trial-by-trial IOA for procedural fidelity was 90.5% (range: 33–100%).

Materials

During assessment, baseline and probe trials, and intervention sessions, researchers followed a prescribed day plan that detailed tasks and task materials. Researchers provided the participants with materials to complete a variety of tasks. Researchers placed some additional materials in a supply cabinet that were accessible to participants for situations that occasioned the participant to look for materials. Other materials were not accessible and required the participant to ask for them if needed. Materials varied across tasks, which included general office items (e.g., mail, envelopes, file folders, and scissors), food service items (e.g., salt and pepper shakers, silverware, and napkins), and other retail-related items (e.g., clothes, hangers, and books). Day plans and a full list of materials required can be found on the OSF website linked above.

Researchers produced a set of five VMs prior to initial intervention sessions. Each VM depicted the supervisor introducing a task, followed by the employee encountering a problem and correctly resolving it through communication with the supervisor or independent problem-solving. The lead researcher and a confederate served as actors in each of the VMs and followed scripts. The lead researcher filmed all VMs using a cell phone, edited using iMovie on a MacBook computer, and uploaded to a cell phone for viewing during intervention sessions. Each VM began with a brief rationale for the importance of the skill and included on-screen written steps for each targeted step in the skill synchronized with a voiceover explanation. All VMs were less than 2 min long. Each video took approximately 1 h and 30 min to film and edit.

Experimental Design and Procedures

We used a multiple probe design across behaviors (Ledford & Gast, 2018) to examine the effects of VM plus feedback to teach VSS. The multiple probe design is well suited to evaluate the acquisition and maintenance of multiple skills within participants, with a staggered introduction of intervention across skills (Horner et al., 2005). Two participants had three behavioral targets each, and one had two targets, based on the assessment results. Observers collected baseline data until we observed a stable, non-improving trend with at least three data points per behavior before intervening on the first behavior. We randomized the order of behaviors for intervention for most target behaviors; however, we taught notifying your supervisor when you complete a task prior to other skills that require leaving the workspace to find the supervisor for sequencing purposes and to avoid carryover effects of training. The criterion for mastery was three consecutive trials of accurate responding. Once responding met criteria, we conducted an additional probe for all behavioral targets.

Assessment

We began by assessing VSS as described by Lerman et al. (2017). This involved setting up evocative situations in a simulated workplace to observe the extent to which participants used various VSS. The assessment took place over 2 or 3 days to provide an opportunity to establish a stable baseline. The first and second authors had experience conducting this assessment and took turns acting as the participants’ “supervisors” during the assessment. Each day began with the supervisor welcoming the participant and telling them that the goal of the assessment was to see how they would behave in a workplace environment and that they should treat the assessment the same way they would a real job. The supervisor then showed the participant the workspace, pointed out the drawer where additional materials were located, and showed the participant the location of the supervisor’s office which was located outside of the room where the participant worked.

The supervisor presented the participant with a variety of tasks, providing opportunities to demonstrate several common VSS. Tasks included clerical tasks, food service tasks, retail tasks, and basic processing tasks. During each trial, the supervisor began by presenting a task to the participant (e.g., “I need you to fold these shirts”), waited 2–3 s to allow the participant to respond, then said, “I will be in my office,” and left the room. We collected data on participant responses to the evocative situations to identify behavioral targets for intervention. The assessment took place during 1-h sessions, and each trial consisted of the presentation of the task, the evocative situation, and the participant’s response. Each assessment included either 11 or 12 trials across 2–3 days, and the typical length of a trial was approximately 10 min.

We targeted the three skills with the lowest accuracy for intervention. No participants had more than three skills that warranted intervention. All skills related to commonly occurring workplace interactions necessary for high-quality job performance, and, therefore, all were appropriately suitable intervention targets.

Baseline and Probe Trials

The assessment results served as baseline data. Following completion of the assessment, the instructor served as supervisor during probe and intervention trials. Prior to beginning the intervention, we conducted a probe trial for each skill to ensure that responding remained consistent. Probe and intervention data were collected during 1-h sessions. Sessions occurred either in a classroom or in the café used for the assessment.

First, the instructor informed participants that they were going to practice job skills and that they should treat the practice like a real work experience. The instructor presented a task that included an evocative situation to occasion the target behavior. Tasks were relatively short in duration, such that they would take the participant 5–15 min to complete. Actual length of time varied, however, based on the participants’ pace of work. For participants who worked slowly or methodically, we decreased the amount of work required to ensure tasks did not take more than 15–20 min. Each probe trial consisted of an opportunity to demonstrate one of three target behaviors and was conducted identically to assessment trials. There were no programmed consequences following either a correct or incorrect response during probe trials. We selected job tasks present within each of the participant’s repertoire.

Intervention

The intervention phase started with the participant watching the VM corresponding with the current behavior target. Each VM depicted an employee performing the steps in the corresponding task analysis (see Table 1). Following the VM, the instructor presented the task. During each trial, the participant encountered situations (e.g., missing materials) to demonstrate the target behavior (e.g., searches for materials). If the participant responded accurately, the instructor delivered behavior specific praise (e.g., “Great job asking me for the materials you need.”). If the participant did not respond correctly, the instructor provided immediate corrective feedback (e.g., “Good job telling me you finished, next time make sure to search for the materials you need first”) and asked the participant to re-watch the VM before presenting the next task. Intervention sessions lasted between 45 min and 1 h and typically included at least three trials.

The instructor conducted intervention trials until the participants’ responding met the mastery criterion, defined as three correct responses across consecutive trials. Following mastery, we conducted an additional probe trial for each behavior. If a participant responded incorrectly across three consecutive trials, we introduced a prompt-delay procedure in which the supervisor stated the participant’s name, waited for 3 s, then verbally prompted the participant to demonstrate the target skill. For one participant (Dwayne), the prompt-delay procedure resulted in variable responding. We introduced a visual prompt on trial 26. The instructor placed a small paper tent with written instructions directly in front of the participant that read “Ask About Prior Task” and gave explicit verbal instructions on when and how to use the skill. Once Dwayne successfully demonstrated the skill three times consecutively using the visual prompt, we removed the visual prompt and conducted an additional trial without prompting.

Post-Assessment and Community Setting Probes

Within 1 week of meeting mastery criteria for all target behaviors, the researchers administered a post-assessment, providing an opportunity to demonstrate learning, skill maintenance, and generalization with supervisors who were not involved in the intervention. Procedures for the post-assessment sessions were the same as assessment sessions. To observe the participants’ use of learned skills in a novel, naturalistic setting, two participants (Nihil and Troy) participated in a 1.5 h work experience at a community business. Nihil worked at a large grocery store chain 10 days after completing the post-assessment, and Troy worked at a thrift store 6 weeks after completing the post-assessment. A local job coach helped set up the work experience and was on-site to offer support as needed. Each participant worked alongside an employee at the job site who served as their supervisor during the session. These employees had some experience working with the day school to provide short-term work experiences for transition-age students but otherwise received no special training. The researcher met with the job coach and the on-site employee 15 min before beginning the session to explain the goals of the session including that the researcher might occasionally ask for help setting up a scenario.

During the session, the researcher observed and communicated with the job coach or supervisor as needed to ensure opportunities for the participant to engage in each targeted behavior. The on-site supervisor presented tasks (e.g., stocking shelves and unpacking merchandise), which provided the participants the opportunity to respond to similar situations encountered during training. We asked each supervisor to present typical tasks for the setting and gave them additional instructions as necessary. For example, “Can you please interrupt them and ask them to work on a new task.” Following the community setting probe, the researchers discussed the participants’ performance with the on-site supervisor to gather information on any skills the researchers were unable to observe directly. Due to issues with scheduling and coordinating a job site, Dwayne was unable to complete a community setting probe prior to the end of the study.

Social Validity

The participants and instructor were asked to complete a brief researcher-developed questionnaire providing feedback on the intervention. Following completion of the study, the participants and instructor were sent an email containing the survey and completed the survey either individually (Nihil and instructor) or with a parent (Troy) or teacher (Dwayne). After completing the survey, they sent their responses back to the researcher via email. We asked participants whether they enjoyed the experience, found it helpful, would want to continue with the intervention to learn other skills, would think others would find it useful, and for any feedback, they had on the intervention. Responses were either binary (yes or no) or Likert-type ratings, along with a single open-ended question. Questions for the instructor related to enjoyment of the intervention, perceived usefulness, likelihood of recommending to others, and ease of use. The survey questions are included as supplementary material.

Results

Dwayne

Data for Dwayne are presented in Fig. 1. We graphed the results using a “yes”/“no” binary format indicating whether the participant successfully used the targeted key, independent step in each skill with one indicating “yes” and zero indicating “no.” During assessment, Dwayne consistently made confirming statements to the supervisor, responded well when given corrective feedback on a task, including correcting his mistakes, and sought help when instructions were vague or a task unfamiliar. He consistently searched for the supervisor when he finished a task and when he encountered missing materials. Dwayne did not return to work when the supervisor was not available and instead sat at the workplace and waited for further instructions. He never searched for materials within the environment when encountering missing materials and, when interrupted, did not ask the supervisor whether he should return to complete the prior task. We identified searching for missing materials, returning to work when the supervisor was unavailable, and asking about previous tasks when interrupted as behavioral targets for Dwayne.

Fig. 1
figure 1

Dwayne Results. Note. Black squares represent assessment and post-assessment data. Black circles represent training data with white triangles indicating where we introduced a visual prompt during training

Full size image

Following intervention, Dwayne immediately and consistently began searching for missing materials in the appropriate location and returned to working when the supervisor was unavailable. However, when interrupted, Dwayne struggled to consistently ask about returning to a prior task and required additional prompting to learn this behavior. He immediately demonstrated the behavior after the first instance of feedback but failed to consistently demonstrate the behavior at the appropriate time for the following 12 trials. The instructor implemented a prompt-delay procedure before providing feedback. She would present the EO, wait for 3 s, say the participant’s name, wait an additional 2–3 s, then provide feedback. The participant quickly began responding to the instructor saying his name, then began responding to waiting only. However, during some trials, an unfinished task was present in the environment because he did not ask about returning to this task during a previous trial. As a result, he would regularly ask about returning to this task when presented with a new task at the start of a trial, rather than when the instructor presented the EO by interrupting his current task. It is likely that this behavior came under faulty stimulus control due to the presence of an unfinished task in the environment immediately following feedback, as these stimuli were like those in the targeted scenario. To address this, we began removing all tasks from the environment before beginning a new trial, and the instructor continued using the prompt-delay procedure. Dwayne’s responding remained variable, so we introduced a visual prompt (see triangle markers in Fig. 1). This resulted in consistent performance improvement which maintained for a single trial following removal of all prompts. During the final training probe trial, Dwayne continued to search for missing materials and asked about returning to an interrupted task but failed to return to work when the supervisor was unavailable, instead returning to the work station to wait for further instructions.

Generalization of skills was inconsistent during Dwayne’s post-assessment. He searched for missing materials at each opportunity (n = 2); an improvement from the pre-intervention assessment, during which he searched during only one of three opportunities. However, Dwayne never asked about returning to a prior task when interrupted, nor did he return to work when the supervisor was unavailable. Additionally, after the first instance of the supervisor being unavailable, Dwayne stopped knocking on the office door when coming to notify the supervisor of task completion and instead stood outside the door without knocking and then returned to the workspace without interacting with the supervisor. After the post-intervention assessment, he indicated that he did not see the supervisor in the office (although he was present) and did not want to bother the other person working there, so he refrained from knocking. Additional graphs comparing pre- and post-assessment data for all participants are included as Supplemental Figures.

Nihil

Data for Nihil are presented in Fig. 2. During the assessment, Nihil responded well to corrective feedback and consistently asked for help when given a vague instruction, an unfamiliar task, or when encountering missing materials. They also searched for missing materials and switched between tasks easily when interrupted. Nihil never sought out the supervisor to inform them that they completed a task and instead stopped working and waited until the supervisor returned and asked them directly how they were doing. We were unable to assess whether they worked when the supervisor was unavailable because they never searched out the supervisor, but they did not find additional work after completing a task. Nihil was quiet throughout the assessment and only gave partial confirming statements (e.g., “OK”) that were almost inaudible. Nihil also did not ask about returning to an interrupted task or continue working on the previous task after interruption. We identified making confirming statements, notifying your supervisor when you complete a task, and asking about returning to the previous task when interrupted as behavioral targets for Nihil.

Fig. 2
figure 2

Nihil Results. Note. Black squares represent assessment and post-assessment data. Black circles represent training data, and white circles represent novel setting probe data

Full size image

During the intervention for each skill, Nihil immediately began performing the skills correctly. This improvement maintained during additional training probes. Nihil demonstrated all targeted behaviors during the post-assessment. However, they did not provide a confirmation statement during one opportunity and provided a partial statement of “Alright, can do!” during another opportunity (scored as incorrect for not being task specific). Nihil notified the supervisor when they completed each task and asked about returning to work on interrupted tasks. All targeted behaviors showed a marked improvement compared to the pre-assessment. Additionally, while not directly targeted, Nihil did return to work when the supervisor was unavailable during one of two opportunities.

Nihil completed their work experience at a large grocery store chain and worked with an employee from the store to stock shelves. Nihil reverted to making quiet, non-specific confirming statements (i.e., “OK”) during this session, but they notified the supervisor when they completed the first task assigned to them. Partway through the session, the supervisor asked Nihil to stop stocking in one aisle and stock in a different aisle instead. Nihil did not ask whether they should return to the first aisle to continue stocking when finished. They did, however, find work to do when they completed a task, and the supervisor was unavailable. Overall, Nihil’s performance of the trained skills in the novel, naturalistic setting was inconsistent.

Troy

Data for Troy are presented in Fig. 3. Troy was the only participant with previous work experience and remained employed during the study. During the assessment, he made frequent confirming statements that varied in complexity but clearly demonstrated that he was paying attention to the supervisor’s instructions. He responded well to feedback, easily switched to new tasks when interrupted, and always returned to complete the prior task. He inconsistently asked for help when given vague instructions or an unfamiliar task. With the first supervisor, he did not ask for help with an unfamiliar task and instead worked to complete it incorrectly. However, with the second supervisor, he consistently asked for help with both vague instructions and unfamiliar tasks. Troy never sought out the supervisor when finished with a task and did not search for missing materials in the environment. Instead, when the supervisor returned, he either indicated that he had completed the task or indicated that something was wrong. For example, after running out of staples he told the supervisor, “It’s out of staples.” Because he did not search for the supervisor, we could not assess how he responded when the supervisor was unavailable; however, he consistently returned to work on previous tasks when interrupted. We identified notifying the supervisor of task completion and searching for missing materials as behavioral targets for Troy. We chose only two targets for Troy because he demonstrated other behaviors with enough consistency that they did not warrant intervention.

Fig. 3
figure 3

Troy Results. Note. Black squares represent assessment and post-assessment data. Black circles represent training data, and white circles represent novel setting probe data

Full size image

During intervention, Troy never engaged in the target skills after watching the VM alone, but immediately began responding appropriately after the first instance of corrective feedback following intervention. He maintained responding during all additional training probes. Troy demonstrated generalization of all taught skills during the post-assessment. He regularly notified the supervisor when he completed a task and searched for missing materials in two out of three opportunities. During the final trial of the post-assessment, Troy was asked to cut shapes and glue them to paper cards; however, the glue was missing. He did not search for the glue and instead notified the supervisor that the task was complete when all shapes had been cut out. When the supervisor pointed out this mistake, Troy immediately opened the drawer containing materials to check for the missing glue.

Troy completed his community work experience at a large thrift store. He worked alongside an employee with some additional support from a job coach who assisted with setting up the experience. Troy was tasked with unpacking boxes of donated items and setting them on a counter for the associate to price out. To gauge Troy’s use of skills, we set up a scenario in which he finished unloading a box and had an opportunity to notify his supervisor that he was finished. We also asked him to break down the boxes with a box cutter or scissors but failed to provide him with any tools. Troy encountered two instances where he finished a task and had an opportunity to inform his supervisor and/or ask for more work. On the first opportunity, he stood and looked at the supervisor and waited without saying anything until the supervisor came over and opened another box for Troy to unload. He thanked the supervisor and continued working. During the second opportunity, he threw away the box and said, “Box number three is down,” and the supervisor then provided him with a new box. This statement was not clearly directed at the supervisor (i.e., no eye contact, did not get the supervisor’s attention first) but was functionally appropriate and clearly communicated the intended purpose to the supervisor. Troy had three opportunities to ask for scissors to tear down boxes. On the first opportunity, he made several non-directed statements indicating he needed the materials (i.e., “I should have asked for a box cutter”) but did not directly search for or ask for the missing materials. On the next opportunity, he searched for the scissors, saw that the job coach was holding them, and asked her for them. Finally, he searched for the scissors, found them on the table near the supervisor, waited for an appropriate time, and asked, “Mind if I borrow those scissors?”

Social Validity

Social validity data indicated that all participants in the study had a positive experience (see Table 2 for detailed results). Participants indicated that they enjoyed the study (µ = 4.67), found it to be useful (µ = 4.67), would be interested in learning other skills using this method, and would recommend this intervention to others.

Table 2 Social validity data
Full size table

The instructor indicated that she enjoyed participating in the study, found this intervention to be useful, would be interested in continuing to use this intervention, and would recommend this intervention to others. She also indicated that she found this intervention much easier to implement than other interventions.

Discussion

In this study, we evaluated the effects of an assessment-informed VSS intervention using VM and feedback implemented by a naturalistic intervention agent (i.e., an RBT®) in a simulated work setting. Introduction of the intervention resulted in immediate performance improvement for all three participants, except for one behavior (responding to an interrupted task) for Dwayne and Troy, which required a single instance of verbal feedback for each behavior. Generalization and maintenance of skills varied across participants, with most skills generalizing to new supervisors during the post-assessment. However, only some of the target behaviors generalized during a community setting probe involving a realistic work situation. This study provides additional evidence of the effectiveness of practitioner-implemented VM to teach common VSS to autistic youth in a simulated work environment, especially when combined with other interventions.

Findings from this study contribute to this area of research in several ways. First, we found that VM plus feedback was an effective method for teaching high-frequency VSS to autistic youth. VM is clearly an effective intervention for teaching people with autism a variety of skills across age ranges (Green et al., 2013; Qi et al., 2018; Rausa et al., 2016), and our findings provide further evidence of its effectiveness when teaching job-related social skills and problem-solving that may require social interaction. Social interactions are generally complex, and this intervention allowed participants to learn expected behaviors in a variety of circumstances and use them consistently. The addition of feedback and/or prompting during instruction is likely to increase the effectiveness of VSS interventions for some learners. This is consistent with findings from other studies (Grob et al., 2019; Van Laarhoven et al., 2014) and should be considered in practice to increase the efficiency of learning. As practitioners work with students, they may consider including immediate feedback when appropriate. For students with higher support needs, teachers may also include additional supports such as environmental prompts and arbitrary reinforcement (Lerman, 2023).

Each skill targeted in this study is complex, often requiring several steps to accurately complete in full. However, we only presented binary data concerning a key step in the task analysis as some of the skills lack independence as they contained similar steps (e.g., leave to locate supervisor). The VMs used in this study depicted all the steps associated with each skill, and participants generally were able to perform the singular key step accurately after watching the video. While the VMs in this study were brief (i.e., less than 2 min), some individuals may need VMs that isolate the targeted skill to assist with acquisition. For example, Dwayne may have benefited from a more targeted VM that focused solely on asking about returning to the prior task.

An additional contribution of this study is the use of an RBT® as the intervention agent during training. The previous studies have primarily used researchers to conduct the training (Bross et al., 2020; Stauch & Plavnick, 2020; Van Laarhoven et al., 2014). We demonstrated that a novel, naturalistic agent can effectively implement this type of intervention with oversight from the researchers. The trainer in this study indicated that the intervention was easy to use and effective, which is similar to positive experiences with VM in employment settings reported in other studies (Bross et al., 2020). To provide additional evidence for use in practice, future research could examine the effects of this type of intervention when conducted in a workplace setting by supervisors or job coaches or in a public school setting when implemented by teachers or teaching assistants in the classroom. Additionally, this study provides further evidence for the usefulness of the VSS assessment (Lerman et al., 2017) for reliably identifying participants’ use of high-frequency VSS. We established a stable baseline for intervention targets across all three participants and used the assessment findings to inform our intervention. This assessment is a reliable tool for informing interventions in practice to support the acquisition of valuable VSS for autistic youth.

Most participant behavior generalized to two new supervisors during the post-assessment. Nihil and Troy consistently demonstrated all learned behaviors during the post-assessment with new supervisors, and Dwayne demonstrated one out of three. Dwayne’s inability to demonstrate multiple skills (i.e., asking about an interrupted task and returning to work when the supervisor is unavailable) during post-assessment is likely an issue of maintenance rather than generalization, as he did not show consistent use of these two skills during training. With additional training or the addition of prompts such as re-watching the VMs or environmental prompts (such as in Grob et al., 2019), it is possible that the behaviors would have generalized to the post-assessment considering his generally quick acquisition of behavior during instruction. However, due to time constraints, we were not able to test this. Future research could examine the extent to which re-watching VMs prior to applying skills in novel settings (e.g., a natural work environment) during generalization sessions results in sustained mastery of skills.

Limitations and Future Research

The current results should be considered in light of the following limitations. Training occurred in a simulated work environment and in a clinical setting. Simulated work environments are necessary for individuals who are not yet employed but still need to acquire these skills; however, the use of simulated settings likely limited generalization of some skills across supervisors or to the novel, realistic environment. We programmed for generalization of skills by including tasks and materials similar to those found in many common jobs including the jobs chosen for community probe settings. This probably increased the likelihood that participants responded in the presence of relevant stimuli following the intervention. We did not program for generalization to new supervisors, which may be necessary given the inconsistent generalization results in the community setting probe. Practitioners and future researchers should consider programming for generalization across all relevant stimuli. Further, we did not conduct a community setting probe during baseline, so we cannot make strong claims as to whether the intervention contributed to behavior change in the novel setting. It is possible the participants would have used these skills regardless of training. Nevertheless, observing participants engage in learned skills in naturalistic settings was promising.

Work experience predicts increased post-secondary outcomes (Rowe et al., 2021) and genuine work opportunities (as opposed to simulated ones) should be included as part of a transition plan if possible. Future research could examine which additional components, such as prompts or aligning stimuli in the training environment more closely with the expected work environment, are required to increase the maintenance and generalization of trained skills. Additionally, future research could examine the effects in other training settings, such as a public school, or when implemented by other intervention agents such as teachers, teaching assistants, or job coaches.

This study examined the effects of VM in conjunction with feedback. Therefore, it is not possible to make substantive claims about the effects of VM or feedback in isolation when teaching VSS. Two out of three participants immediately demonstrated use of skills after watching the VM, providing preliminary evidence that VM can be used for initial acquisition of VSS for some individuals. However, participant responding was always followed by either positive or corrective feedback. Similarly, other studies included checks for understanding combined with verbal praise following introduction of the VM (Bross et al., 2020) or feedback following incorrect responding (Stauch & Plavnick, 2020), and Van Laarhoven et al. (2014) found VM in isolation to be less effective than VM in combination with video feedback. It remains unclear whether VM alone can result in acquisition and maintenance of VSS.

We observed variance in skill acquisition, maintenance, and generalization across participants, which may be due in part to their varied backgrounds in education, work, and life experiences. Work history and level of support needs likely play a significant role in an individual’s response to intervention, and different skills may require different forms of intervention. It is possible that a less intensive intervention could have been sufficient for some participants or skills. Similar to the procedures used in Grob et al. (2019), future research could examine using a response-to-intervention framework (Fox et al., 2010) to better understand the process for selecting the best and least intensive intervention necessary to teach a given skill, and how intervention intensity aligns with learner profiles. For example, some learners may learn certain skills (i.e., notifying the supervisor of task completion) following verbal or written instructions, but may need additional supports to acquire more complex skills such as responding to feedback.

While participants in this study had similar language use and skill levels to those in other VM studies with strong results (i.e., Bross et al., 2020), individuals with less advanced skills may require more frequent or complex interventions to acquire VSS. Practitioners may need to implement additional procedures such as video feedback (Van Laarhoven et al., 2014), BST, additional on-sight antecedent or stimulus prompts, multiple exemplar training, or use of reinforcement (Grob et al., 2019). Work experience, language use, and other skill levels likely have substantial influence over the rate of acquisition and maintenance of VSS using this intervention. More research is necessary to determine the generality of our findings to populations of individuals with developmental disabilities with more diverse skill levels and experiences.

Some of our response criteria in this study were less than ideal, as the definitions for correct responding were narrow. For example, the response “Alright, can do!” may be an appropriate confirming statement in some contexts, but did not meet our specific criteria for this study. Practitioners and future researchers should consider including broader response classes to further improve the social validity of the target behaviors. There is potential that the social validity survey resulted in biased responding because the responses were not anonymous. Finally, IOA for Troy was relatively low (66.7%) compared to other participants as he had the lowest number of trials with IOA data (n = 9) as well as the highest number of disagreements (n = 3). Due to scheduling issues, there was a 3–4-week gap between sessions for which the second observer collected IOA data which could have resulted in additional researcher error. Although we maintain that our data are reliable due to high IOA across all three participants, this discrepancy should be noted when interpreting Troy’s data.

Conclusion

Autistic youth often require additional support to experience success in the workplace due to difficulties with social interactions (Hendricks, 2010). Effective interventions that can be implemented in applied settings are important for teaching key social skills applicable across work environments and supporting young people with autism in relating to their supervisors and fellow employees. VM plus feedback was an effective approach to teaching these skills in a simulated work environment, although some skills did not generalize to a realistic work setting. More work is needed to determine the best methods for providing instruction and training that allows autistic youth to engage in expected social behavior in the workplace, resulting in increased employment outcomes for this population.