A Comparison of Secondary Target Location in Instructive Feedback Procedures

  • Christopher A. Tullis
  • Ashley R. Gibbs
  • Madeline Butzer
  • Sarah G. Hansen


Instructive feedback (IF) is an effective strategy for increasing the efficiency of targeted instruction. Although effective, the mechanisms underlying the acquisition of secondary targets via IF are unknown. In the current investigation, two forms of IF were compared to determine if indiscriminable contingencies were responsible, in part, for the acquisition of secondary targets during IF procedures. During teaching, IF stimuli were presented either before or after the praise statement for mastered tacts to two learners with autism spectrum disorders. Across both participants, IF before the praise statement resulted in faster acquisition of secondary targets that were maintained for 16–18 weeks post intervention. These results extend the IF literature by providing evidence that acquisition of secondary targets via IF may at least partially attributed to the occurrence of indiscriminable contingencies.


Instructive feedback Autism spectrum disorder Indiscriminable contingencies 

Instructive feedback (IF) is a teaching procedure where non-targeted information (i.e., secondary targets) is introduced into the consequent events of teaching trials for other explicitly targeted instruction (primary targets; Werts et al. 1995). For example, if an instructor is teaching labeling a picture stimulus of a tiger (primary target), they may introduce a feature of the target into the consequent event of those teaching trials as a secondary target (e.g., tigers are cats). IF has been effective in teaching language (Tullis et al. 2017), academic (Werts et al. 2011), play (Grow et al. 2017), and social skills to learners with intellectual and developmental disabilities (IDD; Albarran and Sandbank 2018; Werts et al. 1995).

Although IF procedures have been effective in enhancing targeted instruction (Albarran and Sandbank 2018; Nottingham et al. 2014), the mechanisms that underlie the emergence of secondary targets without explicit instruction is not fully apparent. Previous commentary has suggested three possible mechanisms that may underlie the acquisition of secondary targets during IF procedures (Nottingham et al. 2015; Wolery et al. 1993). First, observational or incidental learning may be responsible for acquisition of secondary target responses. Observational learning is defined as acquiring responses by watching the responding of others and the consequences of those responses (Catania 2013). Recent research has demonstrated that explicit programming might be required for learners with IDD and ASD to acquire skills observationally or incidentally (McGhan and Lerman 2013; Taylor et al. 2012). Although previous research supports the use of explicit programming of monitoring responses for some learners to acquire skills without direct programming, it is unclear if this is a foundational skill for acquisition of secondary targets via IF.

Another hypothesis is that characteristics of the demand context may impact the acquisition of secondary targets (Nottingham et al. 2015; Wolery et al. 1993). For example, Tullis et al. (2017) presented both primary and secondary targets to participants while in similar locations in their typical intervention setting with a familiar interventionist. All participants had a history of interacting with the interventionist in an educational environment, attending to relevant stimuli, and responding to prompts for targeted stimuli in structured teaching sessions. These educational interactions may have produced a history of reinforcement that resulted in learners being more likely to respond to secondary targets because of the increased probability of responses towards those stimuli resulting in reinforcement in that setting.

Finally, it is possible that learners with a generalized repertoire of imitation may acquire secondary targets more readily than those that do not engage in generalized imitation. Previous work has noted the emergence of echoic behavior during IF procedures, which may result in the acquisition of secondary targets and maintained by indiscriminable contingencies (Nottingham et al. 2017; Vladescu and Kodak 2013), which are environmental events where antecedents or consequences for responses are less salient (Stokes and Osnes 1989). During typical IF procedures, delivery of secondary targets follows the learner engaging in a response to the primary target that is followed by reinforcement. For example, Loughrey et al. (2014) provided reinforcement for correct responding on primary targets after the presentation of IF. This type of presentation of IF stimuli may establish a behavior chain that results in the increase of both primary and secondary targets.

Previous research has established several viable explanations for the effectiveness of IF procedures for the acquisition of secondary targets. To date, however, these hypotheses have not been fully investigated (Nottingham et al. 2015). Further investigation into the mechanisms underlying the acquisition of secondary targets via IF may provide information that results in more precise introduction of IF stimuli such that secondary targets are acquired more readily. Additionally, previous investigations of IF procedures have included unmastered primary and secondary targets, which may partially obscure identifying the mechanisms underlying acquisition of secondary targets. The purpose of the current investigation was to extend the IF literature to support initial findings by Loughrey et al. (2014) by comparing IF presentation locations in relation to reinforcement for primary targets to determine if acquisition could be explained, in part, by the occurrence of indiscriminable contingencies.



Participants were two children who had been diagnosed with autism spectrum disorder (ASD) by an independent physician. Both participants received occupational, speech, and applied behavior analysis (ABA) services per week at a private, outpatient clinic. Both participants communicated vocally, were able to engage in vocal imitation, could remain seated and participate in an adult-directed activity for at least 10 min without problem behavior, and did not emit vocal stereotypy that had previously interfered with learning. Participants were referred for inclusion in the study by their treating therapist.

Sydney was an 8-year-old girl who communicated vocally with sentences and phrases of approximately five words. She had occasional errors with articulation and prosody secondary to an additional diagnosis of childhood apraxia of speech but was easily understood by novel listeners. Based on the Vineland Adaptive Behavior Scales, Second Edition (VABS-II; Sparrow et al. 2005), Sydney’s adaptive behavior composite was moderately low. As measured by the Clinical Evaluation of Language Fundamentals-Fifth Edition (CELF-5; Wiig et al. 2013), Sydney’s core, receptive, and expressive language skills placed her below the 0.1st percentile, in the 0.5th percentile, and in the 0.1st percentile, respectively. Sydney was also assessed using the Verbal Behavior Milestones Assessment and Placement Program (VB-MAPP; Sundberg 2008) and was categorized as a level-two learner with established level three skills in visual performance/match to sample, math, reading, and writing, and emerging level-three skills in tacting, listener responding, and listener responding by function, feature, and class. She was able to utilize full sentences to express her wants and needs, engage in reciprocal conversation with scripted questions (e.g., “what is your favorite ____?”, “what school do you go to?”, “how old are you?”), and answer function-based “who,” “what,” and “where” questions. At the time of the study, Sydney received speech and occupational therapy once per week, as well as 5.5 h of ABA. Since the age of 3, Sydney had received anywhere between 4 and 10 h per week of intensive behavioral intervention.

Alexander was an 8-year-old boy who communicated vocally. His speech production was evaluated using the Goldman-Fristoe Test of Articulation-2 (GFTA-2; Goldman and Goldman 2000), and was a 55 (2nd percentile), which is characterized as a severe difficulty in speech production. Alexander’s general conceptual ability as measured by the Differential Ability Scales-Second Edition (DAS-II; Elliot 2007) was a 52, and his adaptive behavior composite on the VABS-II (Sparrow et al. 2005) was low, categorized as a mild deficit. His scores on the CELF-5 (Wiig et al. 2013) placed his core, receptive, and expressive language skills below the 0.1st percentile, in the 0.2nd percentile, and below the 0.1st percentile, respectively. Upon evaluation with the VB-MAPP (Sundberg 2008), Alexander was classified as a level-two learner, with established level-two visual performance/match to sample, echoic, play, and group responding skills, and emerging level-two skills in listener responding, imitation, and intraverbals. He was able to independently communicate his wants and needs using one to three-word phrases but could speak in full sentences given a prompt. He was able to answer basic “what,” “who,” “where,” and “when” questions. At the time of the study, Alexander was receiving occupational and speech therapy once per week and 6 h of ABA per week. He had received anywhere between 3 and 6 h of behavioral intervention per week since the age of 5.


The study included four conditions: baseline, intervention, best treatment, and maintenance. During baseline, intervention, and best treatment conditions, each session had both tact trials and receptive by feature function or class (RFFC) probes, with 30 min elapsing between session components. Only RFFC probes occurred during the maintenance condition. One session was conducted with each participant per day, with sessions taking place 3 days per week. The experimenter utilized a token board during all sessions with Alexander, which was an established intervention prior to the initiation of experimental procedures. Tokens were awarded on a VR3 schedule for compliance with mastered demands interspersed between trials and probes, and were able to be exchanged at the conclusion of the session.

Setting and Materials

Experimental sessions were conducted by one of two board-certified behavior analysts, one of which was the second author, who were both participants’ regular treating therapist. All sessions were conducted in private treatment rooms within the outpatient clinic where the participants received direct therapy. Rooms were approximately 4.6 m × 3.4 m and were equipped with a table(s) and chairs, cabinets of therapeutic materials, and bookcases and/or shelves with therapeutic materials occluded by a cloth cover. During experimental sessions, the experimenter and participant sat at an individual table, though there were occasionally other children working with therapists at other tables in the room.

Experimental materials included two sets of 12.7 cm × 10.16 cm stimulus cards. Target stimuli sets were selected in consultation with each participant’s behavior analyst. Each set was comprised of three stimuli that shared a common feature or were members of the same class, and each stimulus had previously been trained as a tact, but not as an RFFC target, within each participant’s individual therapy sessions (see Table 1). Each stimulus card was comprised of a color stimulus mounted on a white background, and multiple exemplars were used for each stimulus. Data collection materials included a paper data sheet, a pencil, and an iPad mini® for video recording.
Table 1

Stimuli and RFFC designation for Sydney and Alexander




Set one

IF Before


IF After





[Class – toiletries]

IF After


[Class – utensils]

IF Before

Set two






[Class – mammals]


[Feature – handle]


Tact and RFFC sessions were conducted separately, with six trials occurring per session. For tacts, the stimulus card was shown to the participant and the experimenter provided an instruction for the participant to respond (e.g., “What is this?”). No IF of any kind was given during baseline tact sessions and these were conducted to insure the chosen tacts were in the participants’ repertoires. During each RFFC probe, an array of five stimulus cards was placed in front of the participant, and the experimenter provided an instruction for the participant to indicate which card was part of a specific class (e.g., “Which one is a mammal?”) or had a specific feature (e.g., “Touch the one that has a handle”). Stimulus cards consisted of one target stimulus and four distractor stimuli that were not included in other target sets. Social praise for participation (e.g., “Thanks for working so hard!”) was provided following each trial regardless of accuracy, and mastered demands were interspersed following 1–4 probes on a VR3 schedule. Social praise was delivered as a preferred consequence for participation and on all subsequent teaching trials because it had been established previously as reinforcer.

IF Before

Tact trials were conducted using the same procedure that was used during baseline sessions; however, contingent on correct responding, IF with corresponding RFFC information was given, followed by a praise statement (e.g., “What is this?” “Soap” “Soap is a toiletry, great job!”). If the participant emitted an incorrect response, or did not respond, the experimenter delivered the primary target information, then represented the trial (e.g., Experimenter presented a picture of a fork and asked “What is this?”, the participant responded with an incorrect answer “knife,” the experimenter then provided the correct answer, and then represented the trial “A fork. What is this?”) and contingent on the correct response, delivered IF followed by a praise statement. Three trials were conducted per stimulus for a total of nine tact + IF trials.

IF After

Sessions were identical to those during the IF before condition, with the exception of IF delivery. Contingent on a correct response to a tact trial, a praise statement was given, followed by IF with corresponding RFFC information (e.g., “What is this?” “A bag” “Nicely done! A bag has a handle”). If no response, or an incorrect response was emitted, a correction trial was initiated, and IF was delivered following error correction.

Best treatment phase

Following steady responding indicative of one IF variation resulting in greater skill acquisition than the other, a best treatment phase was implemented for tact + IF trials with both sets of stimuli utilizing the procedures associated with the more effective form of IF.

Secondary Target Probes

At the conclusion of the tact trials, a timer was set for 30 min, and during that time, each participant’s typical therapeutic skill acquisition programs were run. After 30 min had elapsed, RFFC probes were conducted using the same procedure that was used during baseline. Three probes were conducted per stimulus, for a total of nine RFFC probes.

Maintenance Probes

Once a participant reached mastery criteria for a set of stimuli, that set entered maintenance, with each maintenance session consisting of three RFFC probes utilizing the same procedure as used during baseline sessions. Each participant had a total of ten maintenance probes per set spanning approximately 16 weeks with the length of time increasing between each probe according to a set schedule (i.e., three probes/week, two probes/week, one probe/week, biweekly probe, probe every 3 weeks, probe once per month).

Social Validity

After the intervention was completed, the parents of each participant were given a description of the intervention, viewed labeled graphs, and were shown video clips of their child during baseline and intervention after which they filled out a treatment evaluation inventory (TEI; Kelly et al. 1989). The questionnaire consisted of a set of statements which participants’ parents responded to using a Likert scale, with 1 equaling strongly disagree and 5 equaling strongly agree. From the completed surveys, the experimenter calculated the mean acceptability of the intervention. When calculating mean acceptability, the score given by each participant’s parents to item six was adjusted to address the reverse scoring of this item (i.e., if the parents rated this item a 1, strongly disagree, this was reversed to a 5 for calculation of the mean acceptability).


Correct primary responses (tacts) were vocal responses that had been directly taught to participants in the presence of their corresponding stimuli (e.g., saying “toothpaste” in the presence of a toothpaste stimulus card). Correct secondary responses (receptive function, feature, or class; RFFC) were either physical card touches or vocal responses to receptive instructions (e.g., touching the bag stimulus card when directed to “touch the one that has a handle”, saying “soap” when directed to “find the one that is a toiletry”). For a secondary response to be considered correct, it had to match the instructive feedback provided during primary response trials. Experimenters measured the percent of correct independent responses by dividing the number of correct responses by the total number of trials within the session.

Interobserver Agreement and Procedural Fidelity

A second, trained observer scored data for a 38% of sessions in all conditions for each participant from video recordings. Interobserver agreement (IOA) was calculated by totaling the number of agreements (i.e., trials in which both observers scored the same response), then dividing the number of agreements by the total number of trials and multiplying the quotient by 100. Agreement for both participants across both primary and secondary responses was 100%.

Procedural fidelity data were collected for 38% of the sessions in each condition for each participant from video recordings. Fidelity was measured using a structured data sheet to document the occurrence of each implementation step in baseline, intervention (including best treatment), and maintenance conditions. During RFFC baseline, correct implementation consisted of (a) presenting the appropriate array of stimuli, (b) delivering the SD (e.g., “find the one with wheels?”), (c) providing praise after each trial for participation, (d) presenting six trials in the session, and (e) interspersal of mastered skills. During the tact baseline, correct implementation consisted of (a) presenting the appropriate stimulus, (b) delivering the SD (e.g., “what is this?”), (c) providing praise after each trial for participation, (d) presenting six trials in the session, and (e) interspersal of mastered skills. In intervention, correct implementation consisted of (a) presenting the primary target first with the appropriate SD (e.g., “What is it?”), (b) delivering IF, (c) implementing error correction as outlined, (d) delivering maintenance trials between IF trials, and (e) delivering praise either before or after the IF statement depending on the condition. In secondary target probes, correct implementation included (a) presenting an array of five stimuli with the appropriate SD, (b) providing praise for correct responses, (c) advancing to the next trial if an incorrect was observed, (d) delivering maintenance trials between probe trials, and (e) initiating the correct probe (i.e., probing the appropriate set). To calculate the percentage of steps implemented with fidelity, the number of correct steps was divided by the total number of steps possible, and then multiplied by 100 to yield a percentage. For Sydney, procedural fidelity was 100% across all phases. For Alexander, fidelity was 100% for all phases, except IF before. During this condition, fidelity was 94.5%.

Data Analysis

Mastery Criterion

Mastery criteria were three consecutive sessions with 80 to 100% correct responding for tact trials, and one session with 80 to 100% correct responding for secondary target trials.

Experimental Design

An adapted alternating treatments design (Gast & Ledford 2014) was used to compare the effects of delivering IF before praise to delivering IF after praise on the acquisition of secondary targets. Logical analysis (Gast & Ledford 2014) was used to insure equal difficulty of responses across conditions. Each secondary target utterance was 2–3 syllables and were responses that each participant had the repertoire to utter vocally. For each participant, a best treatment phase was introduced after differentiation was observed between each data path, indicating that one variation resulted in greater skill acquisition.


Figure 1 depicts Sydney’s percentage of correct secondary responses during both experimental sessions and maintenance probes. Prior to intervention, Sydney responded incorrectly across all trials during baseline probe sessions. Upon entering intervention, correct responding for set one in the IF before condition immediately increased, with 100% correct responding achieved and maintained across all intervention sessions. In comparison, correct responding for set two in the IF after condition was initially low (11%), and despite increasing in subsequent sessions (M = 89%), never achieved ceiling levels. After initiating a best treatment condition and implementing IF before with set two, correct responding immediately reached 100% and remained stable for three consecutive sessions. Sydney maintained correct responding at 100% during each of the ten maintenance probes conducted per set.
Fig. 1

Secondary probes for Sydney

Figure 2 displays Alexander’s percentage of correct secondary responses during both experimental sessions and maintenance probes. During baseline probe sessions, Alexander demonstrated stable responding with low accuracy (M = 7.2%, range, 0–16.7%). When intervention was initiated, incorrect responding was observed across all trials for set one in the IF after condition. In comparison, correct responding for set two in the IF before condition increased and reached mastery criteria within three sessions (M = 85.3%, range, 67–100%). Once set two was exposed to IF before in the best treatment condition, correct responding had an immediate change in level, and despite some initial variability, achieved mastery criteria within four sessions, with stable responding observed for an additional two sessions (M = 83.3%, range, 33–100%), before the intervention was concluded. Alexander also maintained correct responding at 100% during the ten maintenance probes conducted per set.
Fig. 2

Secondary probes for Alexander

Table 2 depicts the questions and ratings from both participants’ parents on the TEI (Kelly et al. 1989) for IF before, as this was the IF variation that resulted in greater skill acquisition for both children. Overall, parents responded favorably to the effects of the intervention with their child. Both parents inquired about further applications of IF within their child’s therapeutic skill acquisition programming and at home to support skill generalization.
Table 2

Treatment Evaluation Inventory (TEI) questions and average parent ratings for IF Before procedures


IF before



1. I find this intervention to be an acceptable way of addressing my child’s skill acquisition



2. I would be willing to use this intervention at home to address my child’s skill acquisition



3. I believe that it would be acceptable to use this intervention without my child’s consent



4. I like the procedures used in this intervention



5. I believe this intervention is effective for my child



6. I believe that my child experiences discomfort during this intervention



7. I believe this intervention is likely to result in permanent improvement in my child’s learning



8. I believe that it would be acceptable to use this intervention with children who cannot choose interventions for themselves



9. Overall, I have a positive reaction to this intervention



Items were rated on a Likert-type scale of 1–5 with 1 indicating strongly disagree, 3 indicating neutral, and 5 indicating strongly agree. Item 6 was reverse scored


In the current study, IF delivery before and after the praise statement for a primary tact target was investigated. Overall, both participants demonstrated faster acquisition of secondary targets when IF was delivered prior to the praise statement. Visually, this effect was demonstrated to a lesser degree for Sydney, but mastery was met within three sessions in the IF before condition. The occurrence of secondary target acquisition in the IF before condition supports the observation of Loughrey et al. (2014) that one mechanism underlying the effectiveness of IF procedures, in part, is the occurrence of indiscriminable contingencies (i.e., the specific response that is being reinforced is less salient). Overall, these findings are consistent with previous IF research in that participants acquired secondary targets via IF (e.g., Loughrey et al. 2014). Although consistent with previous findings, these data extend the IF literature in three meaningful ways.

First, the current study demonstrated that the acquisition of secondary targets during IF procedures maybe due to the occurrence of indiscriminable contingencies, at least in part (Nottingham et al. 2015). For both participants, secondary targets were acquired faster when the IF stimulus was introduced prior to the consequence event of an already established contingency (i.e., mastered tacts). Previous research has been supportive of this conclusion, but the inclusion of unmastered primary targets may be limiting. For example, Loughrey et al. (2014) presented IF stimuli in both correct and incorrect primary target trials, but both IF and primary target stimuli were unmastered, which may hinder the conclusion that secondary targets were acquired via indiscriminable contingencies. Vladescu and Kodak (2013) and Nottingham et al. (2017) have taught multiple targets successfully using IF methodology, but these demonstrations only support the conclusion that people with disabilities can learn with multiple sources of IF, not the mechanisms that may underlie acquisition.

Second, in the current study, secondary targets maintained for up to 16 weeks for one participant and 18 weeks for the other. Many IF investigations have included brief follow-up phases, but have not demonstrated maintained performance over an extended period of time (e.g., Tullis et al. 2018). In one exception, Reichow and Wolery (2011) assessed maintenance for at 2 weeks post-teaching and then again after 2 months. Although secondary targets were maintained at a high level at the 2-week probe, the same level of responding was not observed at the 2-month probe. The current study gradually increased the amount of time between maintenance probes without a decrement in learner performance. Gradually fading the presentation of secondary target probes may have facilitated maintenance of observed gains by limiting structured exposure to trained targets in a manner that may be analogous to intermittent reinforcement (Stokes and Osnes 1989; Swan et al. 2016). Although this finding is promising, further replication is necessary to validate this type of maintenance probe presentation.

Last, the results of the TEI (Kelly et al. 1989) indicate that the participants’ parents viewed the current procedures as highly effective and socially acceptable. Across studies focusing on IF procedures, none include a measure of social validity, which prevents evaluation of the social acceptability of the results or procedures (Wolf 1978). In the overall behavior analytic literature, social validity remains a sparsely studied concept (Snodgrass et al. 2018). The inclusion of the TEI in the current study procedures provides meaningful preliminary information on the social acceptability of IF procedures that may be helpful in an overall determination of not only the effectiveness of IF, but also in how the outcomes and procedures are viewed by stakeholders. Future studies can benefit from not only the inclusion of social validity measures from parents or teachers, but also from participants.

The current data may be supportive of indiscriminable contingencies being one of many underlying mechanisms for IF procedures (Nottingham et al. 2015; Werts et al. 1995), but two additional mechanisms cannot be ruled out. First, Sydney and Alexander presented with strong tact and echoic repertoires, which may facilitate the acquisition of secondary targets if these are generalized repertoires (Nottingham et al. 2015; Tullis et al. 2017). Nottingham et al. (2017) as well as others (e.g., Haq et al. 2017) have reported the occurrence of consistent echoic behavior during IF delivery (e.g., restating the IF stimulus), which was not observed in the current investigation. The absence of overt echoic behavior during IF may not sufficiently exclude the occurrence of covert echoic responding (Schlinger 2008), and the participant (listener) may covertly echo the utterance of the speaker, making future similar utterances more probable. Future research may benefit from further analysis of the role that overt or covert echoic repertoires have on secondary target acquisition after IF procedures.

Topographically, overt and covert echoic may seem similar, but they differ in terms of controlling contingencies (Esch et al. 2010). First, covert echoics are under the stimulus control of utterances from the same person, and not the utterances of a separate speaker. Second, unlike overt echoics, covert echoics are reinforced by the learner’s own behavior, and may occur to increase the probability that a learner may perform more effectively (e.g. recalling a building name or type of food). Last, motivating operations (Laraway et al. 2003) for overt echoics are established by the learner’s verbal community. In comparison, the conditions under which covert echoics are strengthened are when the conditions to engage in the overt response are absent (e.g., being asked a question on a test). The presence of covert echoic behavior has been implicated as playing a role in the emergence of novel responses and stimulus control (e.g., naming; Horne and Lowe 1996).

Procedures to assess overt echoic repertoires were employed in the current study as part of the VB-MAPP (Early Echoic Skills Assessment; Esch 2008), but an assessment for covert or self-echoic behavior may be beneficial in an effort to determine if its presence may result in acquisition of secondary targets during IF procedures. Esch (2008) compared the efficacy of a test for self-echoic behavior with young children with and without an ASD diagnosis. Sessions were 20 trials in length and consisted of 10 trials where overt echoic responding was tested and 10 where self-echoic responding was tested. Across conditions, the experimenter recited a set of three predetermined numerals, and asked the participant to repeat them. Correct responses for overt echoics were scored if the participant engaged in a correct response immediately after the experimenter. For self-echoics, correct responses were scored only after the experimenter emitted a “lead” (e.g., “what?”) that had previously evoked a response in pre-assessment procedures. In comparison to typically developing participants, those with ASD engaged in a lesser level of responding when self-echoic behavior was potentially required. These data demonstrate one viable way of assessing for the presence of covert or self-echoics, which may be one future avenue to assess covert verbal repertoires in future research.

Second, the current results may be accounted for by the characteristics of the instructional context (Nottingham et al. 2015; Wolery et al. 1993). All sessions occurred in the typical intervention context with familiar interventionists. Sydney and Alexander had a history of intensive intervention in this context where attending and responding resulted in high levels of reinforcement. The experience in this context may have resulted in another instance of indiscriminable contingencies for generalized responding to instructor behavior (Nottingham et al. 2015). Future research should attempt to isolate each of these potential confounding factors.


The results of the current investigation are promising, but further investigation may be fruitful in three main areas. First, the mechanisms responsible for secondary target acquisition after IF procedures are not fully investigated, and the participant sample size was small. These limitations not withstanding, the current study provides preliminary support for indiscriminable contingencies being one plausible explanation for the success of IF procedures, but additional sources may also play an integral role in skill acquisition. Future research may benefit from testing the effects of proposed mechanisms individually and in combination to determine the most influential environmental variables that result in the success of IF methodology across larger numbers of participants. For example, future work may focus on determining the role that demand context has either singly or in combination with presenting IF stimuli prior to the praise statements in established contingencies. Further replications of this type of investigation may provide more substantial information related to environmental characteristics or arrangements that may result in the acquisition of secondary targets.

Second, the current investigation focused on presenting expansion IF, and not parallel or novel IF (Werts et al. 1995). If secondary targets are acquired in part via indiscriminable contingencies, the primary target and IF stimuli may enter into a behavior chain that results in the two stimuli becoming related. This process may be different for stimuli that are not as explicitly related, especially for more skilled vocal learners. Future research may benefit from comparing the acquisition of secondary targets when delivering novel or parallel IF either before or after the praise statement of a learning trial.

Third, although maintenance of observed gains was assessed, generalization of these gains to novel stimulus arrangements was not tested. It may be argued that the acquisition of secondary targets via IF procedures are a form of response generalization, but this is only one type of generalized outcome that may be desirable during skill acquisition (Stokes and Osnes 1989). An additional method of testing for generalization may be to test stimulus relations that may emerge as a function of mastering the tact, and acquiring a stimulus as a secondary target. Previous IF research has demonstrated that learners may acquire multiple secondary targets via IF (e.g., Nottingham et al. 2017), but demonstrations of the emergence of untrained yet related skills has not been demonstrated (e.g., Groskreutz et al. 2010). Future research may benefit from arranging primary and secondary stimuli in a manner that may result in the emergence of new untaught relations. For example, teaching a learner to tact a stimulus (e.g., car), then delivering IF related to the function of a car (e.g., you ride in it), and testing for both secondary stimulus acquisition, as well as emergent skills (e.g., an intraverbal fill-in).

Last, the participants in the current study did not engage in vocal stereotypy, which may hinder the acquisition of functional vocal responses (Chebli et al. 2016). Previous research has indicated that vocal stereotypy can also hinder acquisition secondary targets in IF procedures (Tullis et al. 2017). For example, Tullis et al. reported that participants who scored higher on the self-stimulation subsection of the VB-MAPP Barriers Assessment (Sundberg 2006) required more intensive teaching to acquire secondary target skills. Future research should investigate the role that both vocal and motor stereotypy may have on the acquisition of secondary targets during IF procedures.

Overall, IF procedures have been demonstrated to be an effective procedure for people with ASD and IDD (Werts et al. 1995; Nottingham et al. 2015). Although effective, little research has attempted to isolate the variables that may underlie the acquisition of secondary targets, which may lead to more precise skill acquisition programming. The current study is supportive of the occurrence of indiscriminable contingencies being one of potentially many processes (e.g., observational learning) that results in secondary target mastery via IF procedures. Future research is necessary to fully elucidate these mechanisms to create a more inclusive account of the mechanisms underlying and prerequisite skills necessary to IF procedures to be maximally effective.


Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflicts of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained for all individuals in the study.


  1. Albarran, S. A., & Sandbank, M. P. (2018). Teaching non-target information to children with disabilities: an examination of instructive feedback literature. Journal of Behavioral Education
  2. Catania, A. C. (2013). Learning (5th ed.) Cornwall-on-Hudson, NY: Sloan.Google Scholar
  3. Chebli, S. S., Martin, V., & Lanovaz, M. J. (2016). Prevalence of stereotypy in individuals with developmental disabilities: a systematic review. Review Journal of Autism and Developmental Disorders, 3, 107–118.CrossRefGoogle Scholar
  4. Elliot, C. D. (2007). Differential ability scales: second edition (DAS-II). San Antonio: Pearson.Google Scholar
  5. Esch, B. E. (2008). Early echoic skills assessment. The verbal behavior milestones assessment and placement program: The VB-MAPP. Concord: AVB Press.Google Scholar
  6. Esch, J. W., Esch, B. E., McCart, J. D., & Petursdottir, A. I. (2010). An assessment of self-echoic behavior in young children. The Analysis of Verbal Behavior, 26, 3–13.CrossRefPubMedGoogle Scholar
  7. Goldman, R., & Goldman, F. M. (2000). Fristoe test of articulation 2. Circle Pines: American Guidance Service.Google Scholar
  8. Groskreutz, N. C., Karsina, A., Miguel, C. F., & Groskreutz, M. P. (2010). Using complex auditory—visual samples to produce emergent relations in children with autism. Journal of Applied Behavior Analysis, 43, 131–136.CrossRefPubMedGoogle Scholar
  9. Grow, L. L., Kodak, T., & Clements, A. (2017). An evaluation of instructive feedback to teach play behavior to a child with autism spectrum disorder. Behavior Analysis in Practice, 10, 313–317.CrossRefPubMedGoogle Scholar
  10. Haq, S. S., Zemantic, P. K., Kodak, T., LeBlanc, B., & Ruppert, T. E. (2017). Examination of variables that affect the efficacy of instructive feedback. Behavioral Interventions, 32, 206–216.CrossRefGoogle Scholar
  11. Horne, P. J., & Lowe, C. F. (1996). On the origins of naming and other symbolic behavior. Journal of the Experimental Analysis of Behavior, 65, 185–241.CrossRefPubMedGoogle Scholar
  12. Kelly, M. L., Heffer, R. W., Gresham, F. M., & Elliot, S. N. (1989). Development of a modified treatment evaluation inventory. Journal of Psychopathology and Behavioral Assessment, 11, 235–247.CrossRefGoogle Scholar
  13. Laraway, S., Snycerski, S., Michael, J., & Poling, A. (2003). Motivating operations and terms to describe them: some further refinements. Journal of Applied Behavior Analysis, 36, 407–414.CrossRefPubMedGoogle Scholar
  14. Loughrey, T. O., Betz, A. M., Majdalany, L. M., & Nicholson, K. (2014). Using instructive feedback to teach category names to children with autism. Journal of Applied Behavior Analysis, 47, 425–430.CrossRefPubMedGoogle Scholar
  15. McGhan, A. C., & Lerman, D. C. (2013). An assessment of error correction procedures for learners with autism. Journal of Applied Behavior Analysis, 46, 626–639.CrossRefPubMedGoogle Scholar
  16. Nottingham, C. L., Vladescu, J. C., & Kodak, T. M. (2015). Incorporating additional targets into learning trials for individuals with autism spectrum disorder. Journal of Applied Behavior Analysis, 48, 227–232.CrossRefPubMedGoogle Scholar
  17. Nottingham, C. L., Vladescu, J. C., Kodak, T., & Kisamore, A. N. (2017). Incorporating multiple secondary targets into learning trials for individuals with autism spectrum disorder. Journal of Applied Behavior Analysis, 50, 653–661.CrossRefPubMedGoogle Scholar
  18. Reichow, B., & Wolery, M. (2011). Comparison of progressive prompt delay with and without instructive feedback. Journal of Applied Behavior Analysis, 44, 327–340.CrossRefPubMedGoogle Scholar
  19. Snodgrass, M. R., Chung, M. Y., Meadan, H., & Halle, J. W. (2018). Social validity in single-case research: a systematic literature review of prevalence and application. Research in Developmental Disabilities, 74, 160–173.CrossRefPubMedGoogle Scholar
  20. Sparrow, S. S., Cicchetti, D. V., & Balla, D. A. (2005). Vineland adaptive behavior scales: (Vineland II), survey interview form/caregiver rating form. Livonia: Pearson.Google Scholar
  21. Stokes, T. F., & Osnes, P. G. (1989). An operant pursuit of generalization. Behavior Therapy, 20, 337–355.CrossRefGoogle Scholar
  22. Sundberg, M. L. (2008). Verbal behavior milestones assessment and placement program: The VB-MAPP. Concord: AVB Press.Google Scholar
  23. Swan, A. J., Carper, M. M., & Kendall, P. C. (2016). In pursuit of generalization: an updated review. Behavior Therapy, 47, 733–746.CrossRefPubMedGoogle Scholar
  24. Taylor, B. A., DeQuinzio, J. A., & Stine, J. (2012). Increasing observational learning of children with autism: a preliminary analysis. Journal of Applied Behavior Analysis, 45, 815–820.PubMedGoogle Scholar
  25. Tullis, C. A., Frampton, S. E., Delfs, C. H., & Shillingsburg, M. A. (2017). Teaching problem explanations using instructive feedback. The Analysis of Verbal Behavior, 33, 64–79.CrossRefGoogle Scholar
  26. Tullis, C. A., Marya, V., & Shillingsburg, M. A. (2018). Enhancing the efficiency of instruction via instructive feedback for a child with autism using a speech-generating device. The Analysis of Verbal Behavior
  27. Vladescu, J. C., & Kodak, T. M. (2013). Increasing instructional efficiency by presenting additional stimuli in learning trials for children with autism spectrum disorders. Journal of Applied Behavior Analysis, 46, 805–816.CrossRefPubMedGoogle Scholar
  28. Werts, M. G., Hoffman, E. M., & Darcy, C. (2011). Acquisition of instructive feedback: relation to target stimulus. Education and Training in Autism and Developmental Disabilities, 46, 134–149.Google Scholar
  29. Werts, M. G., Wolery, M., Holcombe, A., & Gast, D. L. (1995). Instructive feedback: review of parameters and effects. Journal of Behavioral Education, 5, 55–75.CrossRefGoogle Scholar
  30. Wiig, E. H., Semel, E., & Secord, W. A. (2013). Clinical evaluation of language fundamentals: fifth edition (CELF-5). Bloomington: NCS Pearson.Google Scholar
  31. Wolery, M., Holcombe, A., Werts, M. G., & Cipolloni, R. M. (1993). Effects of simultaneous prompting and instructive feedback. Early Education and Development, 4, 20–31.CrossRefGoogle Scholar
  32. Wolf, M. M. (1978). Social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203–214.CrossRefPubMedGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Christopher A. Tullis
    • 1
  • Ashley R. Gibbs
    • 1
    • 2
  • Madeline Butzer
    • 1
  • Sarah G. Hansen
    • 1
  1. 1.Department of Learning SciencesGeorgia State UniversityAtlantaUSA
  2. 2.Kiddos’ ClubhouseAlpharettaUSA

Personalised recommendations