Schedules of reinforcement can be complex contingencies with many potential sources of influence on responding operating to determine rates of response (Ferster & Skinner, 1957). For example, both “molecular” aspects of the schedules, such as the reinforcement of inter-response times (IRTs; e.g., Morse, 1966; Peele, Casey, & Silberberg, 1984), and “molar” aspects of the schedule, such as the feedback function relating rate of response to rate of reinforcement (e.g., Baum, 1981; McDowell & Wixted, 1986), could control performance (see Peele et al., 1984; McDowell & Wixted, 1986; Reed, 2011). For example, either of these two factors could explain the often obtained higher rates of response on a random ratio (RR) schedule relative to a random interval (RI) schedule (Ferster & Skinner, 1957; Zuriff, 1970). The former view suggests that longer IRTs are differentially reinforced on an RI compared to an RR schedule, producing lower rates of response on the RI schedule (see Morse, 1966; Peele et al., 1984). In contrast, the molar view suggests that as rate of reinforcement is directly related to rate of responding on the RR but not on the RI schedule, this serves to reinforce higher response rates on the RR schedules (Baum, 1981; McDowell & Wixted, 1986). A long-standing theoretical debate has centered around which of these aspects of the contingency might be more dominant in controlling responding (cf. Baum, 1981; Peele et al. 1984; McDowell & Wixted, 1986; Morse, 1966), and under which circumstances might each factor be more influential (see e.g., Reed, 2007a, 2011, 2015).

The random-interval-plus-linear-feedback (RI+) schedule (McDowell & Wixted, 1986) has been employed in order to tease apart the influence of the molar and molecular aspects of such free-operant contingencies (e.g., Cole, 1999; McDowell & Wixted, 1986; Reed, 2006; Reed, 2007a, 2007b, 2015; Reed, Soh, Hildebrandt, DeJongh, & Shek, 2000). The RI+ schedule is suited to this task as it has the molar properties of a RR schedule (i.e., a linear function relating response rate to reinforcement rate), but the molecular properties of a RI schedule (i.e., it differentially reinforces long IRTs). Given this, it has been suggested that, when the rates of response on the RI+ schedule are similar to those obtained on an RI schedule, this indicates that the molecular aspects of the schedule are dominant in controlling behavior. However, when the rate of responding on the RI+ schedule is higher than that on an RI schedule matched for rates of reinforcement, this suggests that the molar aspects of the contingency are influential (see Cole, 1999; McDowell & Wixted, 1986; Reed, 2007a; Reed, 2015).

The response-reinforcer feedback function of an RI+ schedule is given by the following equation: a = (i / n) * b; where: i = the time from the last reinforcement to the present moment; n = the number of responses made during the period i; b = the equivalent RR value; and a = the interval currently scheduled to give reinforcement. An example might illustrate how this contingency works. If the RI+ schedule was to have the molar characteristics of a RR-30 schedule, and the time from the last reinforcement was 60 s, with 60 responses having been made during that time, the interval to the next reinforcement would equal (60/60)*30 = 30 s. During this 30 s, at 60 responses per minute, 30 responses would be emitted. If 30 responses had been made during that 60-s period, then the interval would be: (60/30)*30 = 60 s. During this 60 s, with a response rate of 30 responses per minute, 30 responses would be emitted prior to the next reinforcement. Alternatively, if 120 responses had been made during the 60-s period, the interval becomes (60/120)*30 = 15 s. During this 15 s, at 120 responses per minute, 30 responses would be emitted prior to the reinforcement. Thus, the interval to the next reinforcement varies inversely with the rate of responding, and reinforcement is delivered for 30 responses.

McDowell and Wixted (1986) found that similar response rates for humans were produced on RR and RI+ schedules when they had the same response-reinforcement feedback functions as one another. This result suggested that behavior on these free-operant schedules was controlled by the molar aspects of the contingencies. Similarly, human sensitivity to the molar aspects of free-operant schedules was noted by Reed (2007b), who found differences between the response rates maintained by RI+ and RI schedules that delivered the same rates of reinforcement to one another. These results both suggest that humans can display sensitivity to the molar aspects of the schedule such as the response-reinforcement feedback function.

In contrast to these results from human subjects, several studies using rats as subjects have not found evidence of any impact of the molar contingency on response rates. For example, Reed et al. (2000; see also Cole, 1999; Reed, Hildebrandt, DeJongh, & Soh, 2003) found that response rates on an RI+ schedule were similar to those on an RI schedule even when the two schedules were matched for rate of reinforcement. In these studies, the RI and the RI+ schedules had similar reinforced IRTs to one another (see also Cole, 1999; Reed et al., 2003). These data suggest that it is the molecular properties of the schedule that are primary in driving free-operant response rates (see Peele et al., 1984).

It should be noted that there are many procedural differences between the studies that have and have not obtained molar sensitivity to the RI+ schedule (cf. Cole, 1999; McDowell & Wixted, 1986; Reed, 2007b; Reed et al., 2003). These differences have been investigated in the light of the question concerning when molar and molecular aspects of the contingencies might influence behavior (see Cole, 1994; Reed, 2006, 2007a). In the studies showing molar sensitivity, the subjects had a high response rate (McDowell & Wixted, 1986; Reed, 2007b), which might have allowed the subjects’ behavior to contact the contingency in such a way that the subjects actually experienced the differential reinforcement of high rates of responding on the RI+ schedule. In the limiting case, if subjects only displayed low rates of responding, then high rates would not be reinforced. Reed (2006, 2007a) investigated this factor using rats, and found some evidence of a molar sensitivity to an RI+ schedule when procedures were put in place that maintained the rats’ high rates of response.

The suggestion that when high rates of responding are maintained subjects can show sensitivity to the molar aspects of an RI+ schedule is consistent with a finding noted by Cole (1999; see also Reed et al. 2003, Reed 2015). Cole (1999) found that the RI+ schedule is incapable of supporting behavior in some rats, and severe response strain is found on RI+ schedules (see also Reed, 2015), sometimes to the point of extinction (see Reed et al. 2003, Reed 2006, 2007a, 2007b). Ratio strain/extinction is likely to occur on an RI+ schedule, as pauses from responding can extend the required interval prior to the delivery of reinforcement by substantial amounts (see Cole, 1999, for a discussion). Given this property of an RI+ schedule, it might be suggested that, if relatively high rates of responding are not maintained, then subjects cannot contact the molar contingencies of an RI+ schedule – only when manipulations are put in place to maintain responding do response rates on an RI+ and RR schedule become similar to one another (see Cole, 1999; Reed 2007a, 2007b, 2015).

In fact, Cole (1999; see also Reed, 2015) explored whether reducing the potential impact of the RI+ contingency on the inter-reinforcement interval would alleviate such extinction/ratio strain effects, and allow response rates to become higher on the RI+ schedule compared to an RI schedule. The procedural alteration adopted by Cole (1999) was to allow the timer responsible for scheduling the RI+ reinforcement to shorten if response rates increased above 60 responses per minute, but, once the time to reinforcement had been shortened, it did not increase again if the response rate fell. It was hoped that this would allow behavior to be related to outcomes directly, as on an RR schedule, but would not differentially make the RI+ schedule produce extinction. Both Cole (1999) and Reed (2015) presented data from rats responding on the modified RI+ schedule that showed signs of molar sensitivity to the RI+ contingency, in that responding on a modified RI+ schedule was higher than that on a yoked RI schedule.

The current experiment explored another possible modification to the RI+ schedule designed to overcome the possibility of response/extinction strain. It has been noted that, when a response requirement is added to a schedule such that it significantly impacts the rate of reinforcement, then ratio strain is more likely to be obtained (Rider & D’Angelo, 1990). In a review of the literature, Anger and Anger (1976) noted that the chances of extinction also increase with repeated extinction/conditioning episodes (see also Davenport, 1969), making the response/extinction problem especially likely to occur on an RI+ schedule. On such an RI+ schedule, response rates can vary within a session (see Cole, 1999; Reed et al. 2003) to produce periods of higher rates of responding during which reinforcement is obtained, and periods of lower rate responding where reinforcement is not obtained. This feature of RI+ schedules would produce apparent conditioning/apparent extinction cycle within a session, making extinction more likely due to the repetitive nature of these cycles (see Anger & Anger, 1976). However, Franks and Lattal (1976) noted that extinguished behavior could be reinstated with the delivery of response-independent reinforcement. While the delivery of such response-independent reinforcement usually decreases ongoing rates of responding (see Burgess & Wearden, 1981), on schedules with lean rates of reinforcement the addition of response-independent reinforcement can sometimes increase rates of response (see Lattal & Bryan, 1976).

Given these considerations, the current experiment employed a novel modification to the RI+ schedule in which, when rates of response fell to a low level, and no responding had been emitted for a period of time (indicating a period of extinction), response-independent food would be delivered in an attempt to reinstate ongoing responding. It is suggested that, under standard conditions with no schedule modification, response rates will be similar for rats on an RI and an RI+ schedule (see Cole, 1999; Reed, 2011; Reed et al., 2003). It is assumed that in part this is due to problems with extinction (Cole, 1999; Reed, 2015). However, with the above modification, response rates may increase on the RI+ schedule, as the reinstatement of responding this manipulation produces (see Franks & Lattal, 1976) would maintain rates of responding and allow them to contact the molar aspects of the RI+ contingency. This effect of response-independent reinforcement would not be expected on an RI schedule with the same modification, as increased responding would not allow contact with a linear response-reinforcement feedback function on the latter schedule. Such an effect would suggest that, under some conditions, rats are capable of showing sensitivity to the molar aspects of the RI+ schedule.

Method

Subjects

Thirty-two experimentally-naïve male Lister rats were employed in the present experiment. The subjects were 3 months old at the start of training, had a free-feeding body-weight range of 300–345 g, and were maintained at 85 % of this weight throughout the experiment. The subjects were experimentally naïve, and were housed in groups of four, with water constantly available in the home cage.

Apparatus

Four identical operant conditioning chambers (Campden Instruments Ltd., Lafayette, IN, USA) were used, and experimental events were controlled using Paul Frey Spider software. Each chamber was housed in a light- and sound- attenuating case, ventilated by a fan that provided background masking noise (65-db[A] above background). Each chamber had two levers, both of which were permanently inserted into the chamber. Only the left lever was operative during the experiment, and this required a response force of .40N in order to depress. Reinforcement consisted of one 45-mg food pellet, and this was delivered to a centrally located food hopper that was covered by a clear Perspex, hinged flap.

Procedure

The subjects received two 30-min sessions of magazine training on a random time (RT) 60-s schedule. On this schedule each successive second was assigned the same probability (i.e., p = 1/60) of delivering reinforcement. They then received two 20-min sessions of lever-press training with a continuous reinforcement (CRF) schedule. On these sessions responses to the levers were reinforced, but there was no shaping of responding. All subjects then responded for four, 30-min sessions on an RI 30-s schedule. On this schedule the first response following a criterion time elapsing received reinforcement. Each successive second was assigned the same probability (i.e., p = 1/30) of satisfying the time criterion and allowing subsequent responses to be reinforced. Once the time criterion was satisfied, reinforcement remained available until the next response was emitted. Once the reinforcement had been delivered, the process started again.

The subjects were then randomly assigned to four equally-sized groups (n = 8). Two groups were presented with an RI+ schedule, such that the feedback function relating the rate of response to the rate of reinforcement was the same as would be experienced on an RR-30 schedule. The inter-reinforcement interval on the RI+ schedule was determined by the function: (i / n) * b; where: i = the interval between the last reinforcement to the present time; n = the number of responses made during the period i; and b = the equivalent RR value (i.e. 30). For one group (Group RI+), these were the only contingencies in operation. For the other group (Group RI+(free)), response-independent food was delivered if there had been no response for 60 s, and response-independent food continued to be delivered on a RT-60 schedule until a response was made.

The other two groups were exposed to RI schedules that were matched to the above RI+ schedules: that is, rats in Group RI-y were yoked to rats in Group RI+; and rats in Group RI-y(free) were yoked to rats in Group RI+(free). One rat in the yoked group was yoked to one rat in the master group, and this master-yoked pair remained the same throughout the experiment. The yoked rat in the RI group received the same rate of response-dependent reinforcement as the master rat in the appropriate RI+ group. Training for the rats in master-yoked pairs was conducted simultaneously. When the master (RI+) rat had received a response-dependent reinforcement, reinforcement became available for the yoked (RI) rat; thus, rats in the Group RI-y(free) were each yoked to a rat in Group RI+(free) in terms of response-dependent reinforcement. In addition to the yoking of the response-dependent reinforcement, rats in Group RI-y(free) also received the same reinstatement contingency as Group RI+(free). This response-independent food was scheduled by the program running their contingency (i.e., this was determined by the performance of the rats in Group RI-y(free), and not by a yoking procedure).

All subjects received 60 30-min sessions of training, which was thought sufficient to generate stable responding.

Results

Figure 1 presents the group-mean rates of response for the four groups over each four-session block of training. Inspection of these data reveals that responding increased for all groups across training until around block 12 of training, after which responding stabilized. Rates became highest in Group RI+(free), were similar to one another in Groups RI+ and RI-y, and were lowest in Group RI-y(free). A three-factor mixed model (ANOVA) with schedule type (RI+ versus RI) and response-independence (present vs. absent) as between-subject factors, and block as a within-subject factor, was conducted on these data. This analysis revealed statistically significant main effects of schedule type, F(1,28) = 13.52, p < .001, partial eta 2 = .326, and block, F(14,392) = 391.39, p < .001, partial eta 2 = .933, but no main effect of response independence, F < 1, partial eta 2 = .019. There were significant interactions between schedule type and response independence, F(1,28) = 9.69, p < .001, partial eta 2 = .257, schedule type and block, F(14,392) = 5.15, p < .01, partial eta 2 = .155, response independence and block, F(14,392) = 2.12, p < .05, partial eta 2 = .070, and between all three factors, F(14,392) = 6.95, p < .0001, partial eta 2 = .199. To further explore these interactions, and to assess responding during steady-state responding, the final block of four sessions of training was analyzed.

Fig. 1
figure 1

Group-mean response rates for each four-session block of training. RI+ = random interval with an equivalent feedback function to a random ratio (RR) schedule; RI+(free) = random interval with an equivalent feedback function to an RR schedule with response-independent reinforcement modification; RI-y = random interval yoked to the RI+ schedule; RI-y(free) = random interval yoked to the modified RI+ schedule with free-reinforcement modification

Full size image

Figure 2 displays the group-mean response rates, averaged over the last four sessions of training where responding was taken to be stable, for all of the four groups. Inspection of these data shows that there was little difference between the response rates for Group RI+ and Group RI-y. The response rate for Group RI+(free), in which response-independent food was delivered if there was no response for 60 s, was higher than that in the unmodified RI+ schedule (Group RI+). In contrast, the response rates for Group RI-y(free), yoked to Group RI+(free) in terms of rates of earned reinforcement, and with the same response-independent contingency, were lower than those for Group RI+(free), and also were lower than those for Group RI-y.

Fig. 2
figure 2

Group-mean response rates (error bars = 95 % confidence intervals) on the last four-session block of training. RI+ = random interval with an equivalent feedback function to a random ratio (RR) schedule; RI+(free) = random interval with an equivalent feedback function to an RR schedule with response-independent reinforcement modification; RI-y = random interval yoked to the RI+ schedule; RI-y(free) = random interval yoked to the modified RI+ schedule with free-reinforcement modification

Full size image

A two-factor between-subject ANOVA (schedule type vs. response-independence) was conducted on these data and revealed a significant main effect of schedule type, F(1,28) = 13.96, p < .001, partial eta 2 = .333, no main effect of response independence, F < 1, partial eta 2 = .016, but a significant interaction between the two factors, F(1,28) = 10.11, p < .005, partial eta 2 = .265. Tukey’s Honestly Significant Difference (HSD) tests revealed that Group RI+(free) had a higher rate than each of the other three schedule conditions, Group RI+ and Group RI-y did not differ from each other, p > .05, but each differed from Group RI-y(free) schedule, ps < .05.

The group mean (standard deviation) reinforced IRTs (s), averaged across the last four sessions, were: Group RI+ = 2.47 (± .88); Group RI-y = 2.52 (± .80); Group RI+(free) = 1.99 (± .52); and Group RI-y(free) = 3.60 (± .53). A two-factor between-subject ANOVA (schedule type × response independence) revealed a significant main effect of schedule type, F(1,28) = 11.20, p < .005, partial eta 2 = .286, no significant main effect of response independence, F(1,28) = 1.41, p > .20, partial eta 2 = .048, but a significant interaction between the two factors, F(1,28) = 9.90, p < .005, partial eta 2 = .261. Tukey’s HSD tests revealed that the only pairwise difference that was significant indicated that Group RI+(free) had a shorter reinforced IRT than Group RI-y(free), p < .05.

Figure 3 shows the group mean rates of response-dependent reinforcement and the rates of response-independent food delivered to the four groups on the first and last four-session block of training. Inspection of the rates of response-dependent reinforcement reveals that these increased from the first to last block of training for all groups, and that the yoking procedure was effective in that Groups RI+ and RI-y, and Groups RI+(free) and RI-y(free), had similar rates to one another. A three-factor mixed-model ANOVA (schedule type × response independence × block) revealed a significant main effect of block, F(1,28) = 588.31, p < .001, partial eta 2 = .955, but no main effects of schedule type, F < 1, partial eta 2 = .002, or response independence, F(1,28) = 2.33, p > .10, partial eta 2 = .002. There was a significant interaction between response-independence and block, F(1,28) = 13.66, p < .001, partial eta 2 = .328, but no interactions between schedule type and response independence, F < 1, partial eta 2 = .001, schedule type and block, F < 1, partial eta 2 = .002, or between all three factors, F < 1, partial eta 2 = .001.

Fig. 3
figure 3

Group-mean rates of response-dependent reinforcement and response-independent food (error bars = 95 % confidence intervals) on the first and last four-session block of training. RI+ = random interval with an equivalent feedback function to a random ratio (RR) schedule; RI+(free) = random interval with an equivalent feedback function to an RR schedule with response-independent reinforcement modification; RI-y = random interval yoked to the RI+ schedule; RI-y(free) = random interval yoked to the modified RI+ schedule with free-reinforcement modification

Full size image

The data for response-independent food (obviously only for Groups RI+(free) and RI-y(free)) show that these values decreased from the first to the last sessions. A two-factor mixed-model ANOVA (schedule × block) revealed significant main effects of schedule, F(1,14) = 5.57, p < .05, partial eta 2 = .284, and block, F(1,14) = 31.52, p < .001, partial eta 2 = .692, and no interaction between the two factors, F(1< 1, partial eta 2 = .028.

Discussion

The current findings replicated previously documented failures to note a RI+ versus RI schedule difference, in terms of response rate in rats, when the two schedules were matched in terms of reinforcement rate (see Cole, 1999; Reed, 2007a; Reed et al., 2000). However, when a manipulation was put into place to deliver response-independent food should rates of responding become low, rates of response were found to be higher on the RI+ schedule than on an RI schedule matched for reinforcement rate. A similar response-independent manipulation on an RI schedule failed to promote higher response rates.

This finding corroborates others that suggest there are circumstances in which rats can show sensitivity to an RI+ schedule in terms of its response-reinforcement feedback function (see Cole, 1999; Reed, 2007a). The current data suggest that one of these factors involves a prevention of response/extinction strain developing (see also Cole, 1999; Reed 2015).

It has been shown previously that the delivery of response-independent reinforcement, although typically associated with a decline in responding (Burgess & Wearden, 1986), can enhance responding when superimposed over schedules of response-dependent reinforcement that result in low rates of reinforcement and periods of extinction (see Lattal & Bryan, 1976). Using this finding to prevent ratio strain and/or extinction for occurring on an RI+ schedule appears to allow response rates to remain high, and to contact the molar properties of the RI+ schedule. Similar findings have been obtained using other manipulations to prevent response/extinction strain on an RI+ schedule (see also Cole, 1999; Reed, 2007a, 2015). Together these findings suggest that, although the RI+ schedule does not typically sustain higher rates of responding than a matched RI schedule, when the RI+ contingency is modified to reduce the chances of extinction, then responding may become sensitive to the response-reinforcement feedback function on the RI+ schedule.

It should be noted that the higher rates of response maintained by the RI+(free) contingency were not associated with higher rates of response-dependent reinforcement than on the equivalent RI schedule. Neither was responding on the RI+ schedule associated with the reinforcement of shorter IRTs than the equivalent RI+ schedule lacking the response-independent food manipulation. That neither rate of reinforcement nor reinforced IRTs were different on the RI+(free) and yoked RI schedules implies that it was the response-reinforcement feedback function that was driving rates of response. The reduction in response rate on the RI schedule when yoked response-independent food was delivered suggests that this manipulation per se will not lead to higher rates of response (see also Burgess & Wearden, 1986), but the response-independent food will interact with the pattern of responding generated by the schedule and with the molar contingencies programmed by that schedule.

That responding on the RI+ schedule was not higher than that on the RI schedule without the response-independent modification (see also Cole, 1999; Reed et al., 2000) implies that the positive linear molar feedback function on the RI+ schedule, which is not present on the RI schedule (see McDowell & Wixted, 1986), was not primarily responsible for driving response rates. That the reinforced IRTs on the RI and RI+ schedules were similar to one another suggests that this molecular factor may play a stronger role in performance under these conditions (see also Peele et al., 1984; Tanno & Silberberg, 2012).

In summary, putting in place a procedure to maintain responding on RI+ schedules supports higher rates of responding on an RI+ schedule than on an equivalent RI schedule lacking a positive molar feedback function between response rates and reinforcement rates. This finding suggests that if responding can be maintained at high enough levels on an RI+ schedule, then it may contact the molar aspects of the contingency. Indeed, those studies which have seen some evidence of molar sensitivity have used subjects and apparatus which tend to maintain high rates of response (McDowell & Wixted, 1986; Reed, 2006), or have produced manipulations that have supported such rates of responding (Cole, 1999; Reed, 2007a). Thus, rats performance can be sensitive to the molar aspects of the RI+ schedule under some conditions.