Motivated to time: Effects of reinforcer devaluation and opportunity cost on interval timing

Abstract

Prior research suggests that interval timing performance is sensitive to reinforcer devaluation effects and to the rate of competing sources of reinforcement. The present study sought to replicate and account for these findings in rats. A self-paced concurrent fixed-interval (FI) random-ratio (RR) schedule of reinforcement was implemented in which the FI requirement varied across training conditions (12, 24, 48 s). The RR requirement—which imposed an opportunity cost to responding on the FI component—was adjusted so that it took about twice the FI requirement, on average, to complete it. Probe reinforcer devaluation (prefeeding) sessions were conducted at the end of each condition. To assess the effect of contextual reinforcement on timing performance, the RR requirement was removed before the end of the experiment. Consistent with prior findings, performance on the FI component tracked schedule requirement and displayed scalar invariance; the removal of the RR component yielded more premature FI responses. For some rats, prefeeding reduced the number of trials initiated without affecting timing performance; for other rats, prefeeding delayed responding on the FI component but had a weaker effect on trial initiation. These results support the notion that timing and motivational processes are separable, suggesting novel explanations for ostensible motivational effects on timing performance.

Interval timing—the capacity to track the passage of time in the seconds-to-minutes range—is often assessed using the peak-interval procedure (Barrón et al., 2020; Buriticá & Alcalá, 2019; Church et al., 1994; Gibbon, 1977; Roberts, 1981; Sanabria & Killeen, 2007). In this procedure, subjects are trained on a fixed-interval (FI) schedule of reinforcement, in which reinforcement is contingent on the first response after an interval elapses. Temporal control is demonstrated in longer unsignaled extinction probe trials, where response rate typically rises abruptly before the criterial interval (at the start time) and declines abruptly after the criterial interval (at the stop time; Cheng & Westwood, 1993; Church et al., 1994). The midpoint time—the average of start and stop times—typically falls near the criterial interval, indicating the accuracy of behavior in tracking that interval. The width of the period of high response rate—the difference between start and stop times—indicates the precision of behavior in tracking the criterial interval. Alternative measures of temporal precision may include the dispersion of start times, stop times, midpoints, and widths (Daniels et al., 2015; Gupta et al., 2019).

An examination of temporal precision in the peak-interval procedure reveals the loose control that interval schedules of reinforcement exert over responses: It is not so tight that animals only respond at the time of reinforcement (i.e., width > zero), but also not so loose that animals respond at a constant rate through the interval (i.e., width < FI requirement). Sanabria et al. (2009) suggest that such loose control reflects both a limit in the precision of a timing mechanism (which keeps width > zero) and the competition between scheduled and contextual reinforcement over the control of behavior (which keeps width < FI requirement).

Consistent with their hypothesis, Sanabria et al. (2009) demonstrated in pigeons that all measures of temporal precision improve when a concurrent non-timing schedule of reinforcement is programmed. They called this preparation timing with opportunity cost because, with each timed (FI) response, the subject incurs the cost of potentially missing reinforcers from the concurrent schedule. The present experiment sought two objectives. The first objective was to replicate Sanabria et al.’ (2009) main findings in another common laboratory species: rats. The second objective was to examine the effect of reinforcer devaluation on motivation and performance in the timing-with-opportunity-cost procedure.

Sanabria et al.’ (2009) findings suggest that changes in motivation and other non-timing factors may interfere with temporally entrained behavior (Daniels & Sanabria, 2017; Galtress et al., 2012; Sanabria & Killeen, 2008). Even when behavior is under the control of multiple rich schedules of reinforcement, adjunctive behaviors and contextual sources of reinforcement (grooming, resting, etc.) are invariably present in operant responding (Killeen & Fetterman, 1988), and may indeed compete with temporal control (Killeen & Pellón, 2013). Target responses in timing tasks are also sensitive to changes in incentive value (Plowright et al., 2000; Sanabria & Killeen, 2008; Ward et al., 2016; Ward & Odum, 2006). Increased reward magnitude yields earlier responding in fixed-interval schedules, whereas reward devaluation defers peak response times and reduces overall response rate (Galtress et al., 2012; Plowright et al., 2000). Thus, shifts in the central tendency and variability of ostensibly timed responses may reflect shifts in motivation to engage in a timing task, rather than changes in the timing mechanism.

Motivation may be procedurally dissociated from timing-related behaviors through the implementation of response-initiated trials, in which the initiating response is distinct from the timed responses. Daniels and Sanabria (2019) showed, for instance, that presession feeding lengthened the latency to initiate switch-timing trials without significantly affecting timing measures, but only if the initiating and timing responses were distinct. Moreover, even if motivational manipulations affect performance in a timing-with-opportunity-cost procedure, they are expected to affect the FI schedule and the concurrent schedule similarly. Consequently, timing measures derived from the alternation between concurrent schedules are expected to be robust to motivational manipulations. Additionally, responses on the concurrent schedule compete with adjunctive behaviors, and thus may serve to bring a greater proportion of operant responding under measured procedural control. When the timing-with-opportunity-cost procedure is implemented with response-initiated trials, all devaluation-induced changes in timing indices (start times, stop times, width, midpoint) may be, therefore, more reliably interpreted as changes in the mechanism governing interval timing.

Methods

Subjects

Six male Sprague-Dawley rats (Charles River Laboratories, Hollister, CA) served as subjects, and were pair-housed upon arrival on approximately PND 120. Rats were housed in a 12:12-h light cycle, with lights on at 1900 h. Behavioral training and testing was always conducted in the dark phase of the light cycle. Following 1 week of acclimation to their housing, access to food was reduced daily from 24, to 18, 12, and finally 1 h/day. Food was placed in the hopper of rat home cages during the dark phase of the light cycle. During behavioral training, food was provided 30 min after the end of each training session (except during prefeeding; see Procedure section), such that at the beginning of the next session weights were, on average, 75% of the mean ad libitum weighs estimated from growth charts provided by the breeder. Water was always available in home cages. All animal handling procedures in the proposed studies follow National Institutes of Health guidelines and were approved by the Arizona State University Institutional Animal Care and Use Committee.

Apparatus

All testing was conducted in 6 MED Associates (St. Albans, VT, USA) modular test chambers (305-mm long, 241-mm wide, 210-mm high). Each chamber was enclosed in a sound- and light-attenuating cabinet equipped with a ventilation fan that provides approximately 60 dB of masking noise. The front and back walls and the ceiling of the chambers were made of plexiglass, and the front wall was hinged and served as the door to the chamber. The two side panels were made of aluminum. The floor consisted of thin metal bars positioned above a catch pan. On the right-side panel, the reinforcement port was a square opening (51-mm sides) located 15 mm above the floor and centered on the test panel. The port provided access to a dipper (MED Associates, ENV-202M-S) fitted with a cup (MED Associates, ENV 202-C) that can hold 0.01 cc of a liquid reinforcer (33% sweetened condensed milk diluted in tap water; Kroger, Cincinnati, OH). The port was equipped with a head entry detector (MED Associates, ENV-254-CB). A multiple tone generator (MED Associates, ENV-223) was used to produce 1–20 kHz tones at approximately 75 dB through a speaker (MED Associates, ENV-224 AM) centered on the top of the left side panel and 240 mm above the floor of the chamber. Two retractable levers (MED Associates, ENV-112CM) flanked the reinforcement port. Lever presses were recorded when a force of approximately 0.2 N is applied at the end of the lever. The opposite side panel was equipped with an illuminated nose-poke device (MED Associate, ENV-114BM) at the bottom panel. A house light on this side panel could dimly illuminate test chambers. Experimental events were arranged via a MED PC^® interface connected to a PC controlled by MED-PC IV^® software.

Procedure

Training sessions were conducted 7 days/week in 2-h sessions. Each session began with a 3-min acclimation period during which no manipulanda or stimuli were activated.

Preexperimental training

Reinforcer consumption training

Prior to training on the timing task, all rats were trained to consume the reinforcer (sweetened condensed milk) from the liquid dipper in the reinforcement port. Following the 3-min acclimation period, a reinforcer was made available at the liquid dipper. All subsequent reinforcers were made available at variable intervals, with a mean intertrial interval (ITI) of 45 s. During the ITI, no stimuli or manipulanda were activated. When a reinforcer was delivered, the liquid dipper was activated. Head entries into the reinforcement port activated a 15-kHz tone. The dipper and tone were deactivated 2.5 s after the head entry. Reinforcer consumption training continued until rats received 100 reinforcers per session and the median time to retrieve a reinforcer was 4 s or less, which took two sessions.

Manipulandum shaping

Following 2 days of reinforcer consumption training, lever-pressing and nose-poking were shaped, using a Pavlovian conditioned approach (auto-shaping) procedure. After the 3-min acclimation period, a single reinforcer was delivered followed by a 7.5-s ITI. At the end of each ITI thereafter, the houselight was activated, and a single manipulandum (left lever extended, right lever extended, or nose-poke device illuminated) was pseudorandomly selected from a list, such that no manipulandum could be selected consecutively in more than six trials. After a response was made, or 8 s elapsed, the houselight was turned off, reinforcement was delivered (as described in reinforcer consumption training), and the manipulandum was deactivated. This phase continued until all rats completed at least 100 trials per session, which took five sessions.

Lever and nose-poke training

The shaping procedure was modified such that reinforcement was only delivered following a single response on the active manipulandum. This continued until all rats were reliably lever pressing and responding on the nose-poke device, completing at least 100 trials per session, which took 3 sessions.

Timing-with-opportunity-cost procedure

Once rats were consistently responding to the active manipulandum, the timing-with-opportunity-cost procedure was implemented. This procedure consists of a dependent concurrent random-ratio (RR) fixed-interval (FI) schedule of reinforcement (Fig. 1). Each component of the schedule—RR, FI—was assigned to a different lever. Following trial initiation by nose poke, the houselight turned on and both levers were extended. Either an FI-active or RR-active trial was pseudorandomly sampled from a 16-item list such that neither trial type occurred consecutively more than eight times; the active schedule was not signaled.

Fig. 1

Diagram of response-initiated timing-with-opportunity cost trials. This procedure consists of a dependent concurrent random-ratio (RR) fixed-interval (FI) schedule of reinforcement. The opportunity cost of timing was manipulated by changing x, the RR schedule requirement. Following response initiation via nose poke (NP), RR-active or FI-active trials are assigned with equal probability. Trials end when the active schedule is reinforced (RFT), followed by a 7.5-s intertrial interval (ITI). Diamonds indicate selection between consequences with probabilities p and q = 1 – p

On RR-active trials, each press on the RR lever was reinforced with probability 1/x, where x is the RR requirement; pressing the FI lever was extinguished. On FI-active trials, the first press on the FI lever after an interval t elapsed was reinforced; pressing the RR lever was extinguished.

Experimental training conditions

Random-ratio (RR) adjustment

Figure 2 depicts the sequence of training conditions that all rats underwent, starting with preexperimental training. Across experimental training conditions t was increased from 12 to 24 to 48 s. Across the initial sessions of each condition, x was adjusted individually to each rat, increasing it progressively until it took about 2t on average to obtain a reinforcer on RR-active trials. The value of x was adjusted for each subject as follows: for the first session of the first experimental condition (t = 12 s), x was set to 20 for all subjects. Following each subsequent RR-adjustment session, the median RR-active trial lengths for each subject were assessed, and x was adjusted so as to increase the median RR-active trial length by 25% for each subject, based on their individual response rates. The number of RR-adjustment sessions is detailed in Table 1.

Fig. 2

Order of training conditions. Following preexperimental training, fixed-interval (FI) requirement t was increased to 12, 24, and 48 s. Within each FI requirement, the random-ratio (RR) requirement x was adjusted. Baseline (BL) training was conducted under a stable x, with 1-h extra-session feeding 30 min after each session. Prefeeding training (PF) training was also conducted under a stable x, with 1-h extra-session feeding immediately before each session. Training ended with the removal of the RR component

Table 1 Details of experimental conditions

Baseline

Once x was stable for each subject, assessment of baseline stability and data analyses were restricted to presses on the FI lever during long RR-active trials, in which the RR schedule was active and lasted at least 2t, which accounted for approximately 25% of all trials. In each condition, assessment of performance stability started after 15 baseline sessions. For each session, all long RR-active trial were pooled within each session to construct a rate curve (response rate in 1-s bins) and a cumulative record. Peak times (times when response rate was maximal) were obtained from each rate curve. Interquartile ranges (IQRs; the difference between the time of 75% and 25% of cumulative FI responses) were obtained from cumulative records. For each rat, baseline stability was defined as a non-significant linear regression of peak times and IQRs over the last five baseline sessions.

In the timing-with-opportunity-cost procedure, rate of reinforcement is maximized by continually responding on the RR lever, pressing the FI lever just once at time t. Such strategy, however, assumes a very precise tracking of time. To the extent that time tracking is imprecise, start times (when rats switch from RR to FI) indicate the perceived proximity of t, whereas stop times (when rats switch from FI back to RR) indicate the perceived likelihood that t elapsed (Sanabria et al., 2009).

Prefeeding

Upon completing five sessions of stable performance with a criterion t, postsession feeding was discontinued and substituted with 1 h of ad libitum access to food immediately before each of three consecutive sessions.

FI 48-s only

The timing-with-opportunity cost procedure was modified for 10 consecutive sessions after testing on t = 48 s such that all trials were FI-active trials. The procedure was otherwise the same, and the RR lever was still inserted after each trial initiation, but was never active. This constituted a “no opportunity cost” condition, allowing an assessment of the effect of opportunity cost on FI 48-s start times.

Data analysis

Parameter estimation

The analysis of performance in the timing-with-opportunity-cost procedure assumed that temporal control is expressed in each trial as a sequence of three response states, in which rats transition from an initial low rate of responding to a high rate of responding at a start time, then maintain this higher rate of responding for a period (width) that typically envelops the criterial interval t, and then transitions back to a low rate of responding at a stop time (Cheng & Westwood, 1993; Church et al., 1994; Sanabria et al., 2009). Further, trial-to-trial variability in each of these measures of temporal control suggests differential sensitivity of timing subprocesses (clock speed, response threshold, memory) relative to other variables such as motivational state, reward value, and rate of reinforcement (Cheng & Westwood, 1993; Church et al., 1994).

Start and stop times were estimated from FI responses during long RR-active trials—those in which the RR schedule was active and lasted at least 2t. Estimates of start and stop times were obtained using the method described by Church et al. (1994). Briefly, in each long RR-active trial, every possible combination of two responses on the FI schedule was tested as potentially occurring at the start time (s₁, the earlier of the two responses) and at the stop time (s₂, the later of the two responses). Times s₁ and s₂ were estimated as those that maximized the expression

$${s}_1\left(r-{r}_1\right)+\left({s}_2-{s}_1\right)\left({r}_2-r\right)+\left(v-{s}_2\right)\left(r-{r}_3\right)$$

(1)

where r is the response rate on the FI lever computed over the whole trial, r₁ is the (low) FI response rate computed over the interval between trial onset and s₁, r₂ is the (high) FI response rate computed over the interval between s₁ and s₂, and r₃ is the (low) FI response rate computed over the interval between s₂ and the end of the trial (v). This estimation procedure was implemented in Wolfram Systems Mathematica (Version 12.0.0). Estimates of widths and midpoints were derived from the estimates of s₁ and s₂—namely, width = s₂ – s₁ and midpoint = (s₁ + s₂) /2.

Stable estimates

Analyses of start, stop, midpoint times, and widths were restricted to long RR-active trials from stable sessions at the end of each baseline condition. For each parameter, stable trials were identified for each rat in each baseline condition by an iterative process conducted in Wolfram Systems Mathematica v12. This process started by building a dataset containing the estimates from every trial, arranged in chronological order. This dataset was then split evenly into two blocks, such that one block contained the first half of the trials and another block contained the second half of the trials. A t-test for independent means was then conducted comparing these blocks. If the t-test revealed a significant difference between blocks (p < .05), trials were combined again into a single data set, about 1% of estimates were removed from the earlier part of the dataset, and the remaining dataset was then split and blocks compared as described. This split-and-compare process was repeated until the t-test failed to detect a significant difference between blocks. Estimates of central tendency and dispersion of each parameter were obtained from the remaining dataset.

Latency to initiate trials (LTIs)

The latency to initiate a trial (LTI) was defined as the interval between the illumination of the nose-poke device and the subsequent trial-initiating nose poke. When the end of the session truncated this interval, the LTI was coded as greater than the truncated interval and included in the computation of median LTI. LTIs from all trials in stable sessions, including FI-active trials and RR-active trials shorter than 2t were calculated and included in analyses. Each session could have at most one more LTI than completed trials. For example, if the subject initiated the 10th trial, but the session ended prior to its completion, that trial would have a measured LTI, but no other response measures.

Effects tested

Effects of FI requirement (t)

The first analysis verified temporal control of behavior in baseline performance. It tested whether (a) median midpoints tracked t; (b) median s₁, s₂, and widths covaried with t; (c) the coefficient of quartile variation (CQV) of midpoints, s₁, and s₂ covaried with t. CQVs were calculated as (Q3 – Q1)/(Q3 + Q1), where Q1 and Q3 represent Quartiles 1 and 3 of the parameter (Q2 is the median). The effect of t was also assessed on the median LTI. Because LTIs were expected to lengthen with lower rates of rate of reinforcement, and longer t implied lower rates of reinforcement, a positive correlation between median LTIs and t was expected.

Effect of prefeeding on LTIs

Prefeeding was expected to lengthen median LTIs and reduce the number of trials initiated per session. These effects were assessed using 2 × 3 (prefed vs. not prefed × t = 12 vs. 24 vs. 48 s) Bayesian repeated-measures analyses of variance (ANOVAs). Bayesian analyses were conducted because they reduce the impact of outliers and allow for support for hypotheses to be quantified for both null and alternative hypotheses (Dienes & Mclatchie, 2018; Keysers et al., 2020). This allowed for a model selection approach, wherein each Bayesian analysis was used to calculate a Bayes Factor, a measure of the strength of evidence for a model relative to the null model. The log of the Bayes factor (LogBF₁₀) was used, where a positive LogBF₁₀ value indicates support for the model (alternative hypothesis or H₁) and a negative LogBF₁₀ value indicates substantial evidence for the null hypothesis (H₀). LogBF₁₀ ≥ 1.098 were considered as having substantial evidence support for H₁ (Kass & Raftery, 1995; Kruschke, 2014)_. Selected models with evidence for H₁ (most positive LogBF₁₀) were probed using Bayesian one-way ANOVAs and Bayesian t tests.

Effects of prefeeding on temporal responding

The reduction in number of trials initiated implied a reduction in the size of the sample from which to estimate central tendency and dispersion measures of timing parameters (see Table 1). Because this reduction could inflate dispersion measures (i.e., CQVs) and cause temporal measures to be otherwise unreliable, prefeeding effects on timing parameters were assessed using a bootstrapping analysis conducted on Wolfram Systems Mathematica v12. For each parameter, t requirement, and rat, 10,000 samples were taken from the stable baseline dataset, where the size of each sample was the number of long RR-active trials that rat completed in the following prefeeding condition. For example, if a rat completed 17 long RR-active trials in the t = 12-s condition under prefeeding, 10,000 samples of 17 trials were drawn from the stable baseline t = 12-s datasets; parameter estimates were then drawn from those trials (i.e., 17 s₁, 17 s₂, etc., per sample). The median and CQV of these estimates defined one bootstrap sample. The 10,000 bootstrap samples yielded the bootstrap distribution of medians and CQV. In cases where subjects initiated only one long RR-active trial under prefeeding, temporal estimates from that trial were considered the median, but no CQV could be calculated. The size of the prefeeding effect on a median or CQV of a parameter estimate was computed as the difference between the prefeeding estimate and the corresponding mean of the bootstrap distribution, expressed in standard deviations (σ) of the bootstrap distribution (see Appendix 1 for a more detailed description of this procedure). Differences greater than 2σ were deemed statistically significant.

Effects of opportunity cost

The removal of an opportunity cost to timing in the FI 48-s-only condition was expected to yield worse measures of temporal precision. Because reinforcement at 48 s truncated trials in this condition, the measures of temporal precision available were only the median and CQV of s₁ obtained from FI-active trials. Stable estimates of these parameters were obtained using the split-and-compare process (see Stable Estimates section above) from performance in the FI 48-s-only condition. These parameters were compared with those obtained from the preceding baseline t = 48-s condition using Bayesian t tests. In the FI 48-s-only condition, median s₁ estimates were expected to be shorter and their CQV larger than in the comparable baseline condition.

Results

Figure 3 shows a representative set of raster plots obtained from one subject (Fig. 3a–c) and mean response rate curves across subjects (Fig. 3d–f). It shows that, in the timing-with-opportunity-cost procedure, FI lever presses typically clustered around FI requirement t, that the widths of these clusters were roughly proportional to t, and that continual streams of RR lever presses flanked these clusters. Additionally, on average, the response rate on the FI lever was maximal at time t. Average response rates on the RR lever were maximal at times that flanked the FI peaks.

Fig. 3

a–c Illustrative raster plots of FI (blue) and RR (gray) lever press times in long RR-active trials with FI requirement t = 12 s, 24 s, and 48 s from a representative subject (Subject 4). Vertical dashed lines denote t. d–f Mean stable response rates on FI (blue) and RR (gray) levers across subjects in long RR-active trials with FI requirement t = 12 s, 24 s, and 48 s, normalized by the respective maximum response rate of each schedule. (Color figure online)

Effects of fixed interval (FI) requirement (t)

Bayesian repeated-measures ANOVA revealed strong evidence for an effect of FI-requirement (LogBF₁₀ = 27.275) on median midpoints (Fig. 4a). Post hoc testing confirmed that midpoints increased as t increased (smallest LogBF₁₀ = 4.414), indicating that they tracked t. Start and stop times (s₁ and s₂) also covaried with t (Fig. 4b, c; smallest LogBF₁₀ = 18.731), where s₁ and s₂ increased as t increased (smallest post hoc LogBF₁₀ = 2.800).

Fig. 4

Effects of FI requirement on central tendency (top row) and dispersion (bottom) of FI lever responding during stable long RR-active trials. Individual median midpoints (a), start times (b), stop times (c), and widths (d) and their respective coefficients of quartile variation (CQVs; e, f, g, h) for t =12 s, t = 24 s, and t = 48 s (x-axis)

There was insufficient evidence that the CQV of midpoints (Fig. 4e) and s₂ (Fig. 4g) covaried with t (largest LogBF₁₀ = 0.344), but there was substantial evidence that the dispersion of s₁ covaried with t (LogBF₁₀ = 1.661; Fig. 4f). Post hoc testing indicated that relative dispersion of s₁ increased when t was raised from 12 to 24 s (LogBF₁₀ = 1.153), but not when t was raised from 24 to 48 s (LogBF₁₀ = −0.621). Similarly, latencies to initiate (LTIs) covaried with t (LogBF₁₀ = 8.501; Fig. 5), increasing when t was raised from 12 to 24 s (LogBF₁₀ = 4.311), but less so when t was raised from 24 to 48 s (LogBF₁₀ = 0.399).

Fig. 5

Effect of FI requirement on median latency to initiate trials (LTI) for t =12 s, t = 24 s, and t = 48 s

Effect of prefeeding on LTIs

Although median LTIs covaried with t, there was insufficient evidence for a main effect of deprivation level on LTIs (LogBF₁₀ = 0.640; Fig. 6a). However, there was evidence for an interaction effect of deprivation level and FI requirement t on median LTIs (LogBF₁₀ = 9.899), where prefeeding increased LTIs when t = 12 s (LogBF₁₀ = 1.536), but not when t = 24 s (LogBF₁₀ = 0.124) and only moderately when t = 48 s (LogBF₁₀ = 1.084). In contrast, there was strong evidence for an effect of prefeeding on the number of trials initiated per session across FI requirements (LogBF₁₀ = 4.337; Fig. 6b; smallest LogBF₁₀ = 2.030). Overall, these results suggest that prefed rats initiated fewer trials than in baseline, an effect that median LTIs, perhaps due to their distribution, did not fully capture (e.g., note increasing dispersion of LTIs among subjects as t increases).

Fig. 6

Individual median LTIs (left) and trials initiated per session (right) during prefeeding as differences relative to baseline for t =12 s, t = 24 s, and t = 48 s. Dashed lines at y = 0 indicate no change from baseline

Effects of prefeeding on temporal responding

Effects of deprivation level on measures of temporal precision were assessed using a bootstrapping analysis (described in Data Analysis section and Appendix 1). Table 2 lists d(θ), the weighted average of the difference between prefeeding and baseline values for the median and CQV of each timing parameter; d(θ) is expressed in units of standard deviation and is weighted by the number of trials completed by each subject in each prefeeding condition. Figure 7 depicts individual and mean unweighted d(θ) values.

Table 2 Weighted mean d(θ)

Fig. 7

Individual d(θ) values for median midpoints, start times, stop times, and widths (a, b, c, d) and their respective CQVs (e, f, g, h). Dashed lines at y = 2 and y = −2 denote, respectively, the threshold for a significant increase and decrease for that parameter under prefeeding

Although the analysis of weighted d(θ) suggests that prefeeding right-shifts the start time and midpoint of the high response state when t = 24 s, individual (unweighted) d(θ) values suggest a more complex effect (Fig. 7). Individual unweighted d(θ) values indicate that, although prefeeding significantly affected central tendency and dispersion measures of FI lever responding for some subjects, these changes are not systematic across subjects or measures of temporal responding. Further, significant weighted mean d(θ) values appear to be driven by individual d(θ) values from subjects that completed a relatively high number of trials under prefeeding (Fig. 8).

Fig. 8

Correlations (with simple linear regression lines) between the number of trials completed in prefeeding (x-axis) and individual d(θ) values (y-axis; labeled with subject number; see Table 1) for median start times (a, b, c) and stop times (d, e, f). For brevity, midpoints and widths are excluded, as they are derived from start and stop times. A Bayes factor is reported for each correlation; those with evidence for H₁ are boldface

Effects of opportunity cost

When the RR component was removed following the t = 48 s condition, analysis revealed strong evidence that this loss of opportunity cost induced earlier median s₁ (LogBF₁₀ = 9.14), but did not affect their CQV (LogBF₁₀ = −0.660; Fig. 9). Responding on the RR lever decreased under extinction: the mean response rate during the last session of t = 48 s with the RR component active was 1.31 responses/s, which decreased to 0.07 responses/sec in the last session where the RR lever responses were in extinction.

Fig. 9

Individual median start times (a) and CQV of start times (b) in the FI 48-s-only condition, reported as differences relative to baseline. Horizontal solid lines indicate the mean difference from baseline. Dashed lines at y = 0 indicate no change from baseline

Discussion

A timing-with-opportunity-cost task (Sanabria et al., 2009) was implemented in rats. In its present implementation, this task involved a response-initiated dependent concurrent schedule of reinforcement with two components, RR and FI. Timing performance was assessed primarily in the FI component across three requirements (t = 12, 24, 48 s) and only when it was inactive (long RR-active trials). This task was thus like the peak-interval procedure (Balci et al., 2009; Matell et al., 2006; Roberts, 1981), but trials were self-paced and timing involved an opportunity cost—the cost of missing potential RR reinforcers.

On each long RR-active trial, rats generally responded first on the RR component, then switched over to the FI component before t elapsed (start time, or s₁), and finally switched back to the RR component after t elapsed (stop time, or s₂; Fig. 3). This pattern tracked t, with the midpoint of the pattern [(s₁ + s₂)/2] falling close to t (Fig. 4a), dwelling times in the FI component (widths) proportional to t (Fig. 4d), and the relative dispersion (CQVs) of most performance parameters remaining approximately constant over t (Fig. 4e–h). These results are consistent with typical findings in the peak-interval procedure, showing the scalar invariance of timing precision (Gibbon, 1977; Oprisan & Buhusi, 2014; Simen et al., 2013).

It is, nonetheless, somewhat surprising that, despite the large changes in RR requirement over t (from RR 65 to RR 235; Table 1), the largest deviation from scalar invariance was a slight increase in the CQV of s₁ when t was raised from 12 to 24 s (Fig. 4f). This robustness of scalar invariance was confirmed in the FI 48-s-only condition. The removal of opportunity cost (i.e., of the RR component) in this condition had no substantial impact on the CQV of s₁ (Fig. 9b). This is inconsistent with previous findings in pigeons, in which the removal of opportunity cost yielded less precise timing (Sanabria et al., 2009). The provenance of this discrepancy is unclear; we can only point out three differences between the study by Sanabria and colleagues and the present study that may account for it: the species (pigeons vs. rats), the pacing of trials (externally paced vs. self-paced), and the FI requirement t (15 vs. 12–48 s).

Nonetheless, the removal of opportunity cost influenced when rats sought food in the FI component (s₁)—it did not increase its relative variability, but it prompted rats to seek food earlier (Fig. 9). This effect is consistent with previous finding in pigeons, and with the notion that schedule performance reflects the competition between scheduled and contextual reinforcement over the control of behavior (Hernnstein, 1970; Sanabria et al., 2009). Interpretation of the effects of removing opportunity cost are somewhat limited due to the lack of peak trials in this condition, such that only s₁ could be measured. Earlier initiation of timed responding suggests that, when opportunity cost was removed, the entire response function would have widened, had peak trials been included.

Prior research suggests that the efficacy of reinforcement of a pattern of responding is reflected in the rate at which the pattern is initiated (Brackney et al., 2011; Brackney et al., 2017; Daniels & Sanabria, 2017, 2019; Mazur et al., 2014; Rojas-Leguizamón et al., 2018). Two findings from the present study support this hypothesis. First, as t increased (and rate of reinforcement declined) the latency to initiate trials (LTIs) also increased (Fig. 5). Second, prefeeding reduced the number of trials initiated regardless of t (Fig. 6b).

Although prefeeding was effective at reducing the number of trials initiated, the only motivational effect expected from prefeeding on within-trial performance was a reduction in response rate in both FI and RR components. The competition between these components over control of behavior was thus expected to be robust to prefeeding. Prefeeding-induced changes in performance, therefore, may be interpreted as reflecting changes in timing mechanisms, or as a prefeeding-induced increase in the relative efficacy of adjunctive behaviors. Unexpectedly, results suggest individual-subject variability in the sensitivity of timing mechanisms to deprivation state.

For most subjects, prefeeding significantly reduced the number of trials completed (Fig. 6b), but left performance within each trial relatively intact (Fig. 7). This effect is consistent with the notion, drawn from behavioral systems theory (Timberlake, 2000) that trial self-pacing protects timing performance from motivational fluctuations (Daniels & Sanabria, 2019). However, some subjects showed the opposite pattern: prefeeding delayed and widened the intervals spent on the FI component (see data points above +2 significance threshold in Figs. 7 and 8) but only slightly reduced the number of trials completed. This latter effect is unlikely an artifact of the small number of trials in the prefeeding condition, as Type I errors increase only slightly in bootstrap tests with smaller samples (Dwivedi et al., 2017). It is, however, inconsistent with the expected protective effects of trial self-pacing; it is, instead, consistent with the notion that reduced arousal slows down the internal clock (Galtress et al., 2012; Galtress & Kirkpatrick, 2010; Killeen & Fetterman, 1988; Ludvig et al., 2007).

Given the small number of subjects tested, it would be premature to draw strong conclusions regarding individual differences in the sensitivity of interval timing on motivational variables. Further, signaled response-initiated FI schedules, like those utilized in the present task, have been shown to reduce timing precision (Fox & Kyonka, 2016), which suggests that the presence of the response-initiation requirement may limit comparison between temporal responding in the present task and that in unsignaled, externally-initiated FI schedules. However, it is not unreasonable to speculate that, for some subjects, trial initiation may fall under habitual action control and display robustness to changes in motivation, akin to the resistance of habitual sign-tracking to reinforcer devaluation (Keefer et al., 2020; Morrison et al., 2015). In these subjects, reinforcer devaluation may instead influence the temporally entrained responses that are more proximal to reinforcement (Smedley & Smith, 2018).

In summary, the implementation of the timing-with-opportunity-cost procedure in rats replicated (a) typical findings in the peak-interval procedure and (b) key findings from implementing the timing-with-opportunity-cost procedure in pigeons (Sanabria et al., 2009). These replications support the notion that timing performance is not only a function of the operation of a timing mechanism, but also reflects a competition between scheduled and contextual reinforcement for such control. Isolating these sources of variance in timing performance requires controlling contextual rates of reinforcement, which is rarely done. Such exercise, implemented in the present study, suggests that reducing motivation slows down the internal clock in some subjects, but selectively affects the frequency of trial initiation in others. The mechanisms that govern the interaction between motivational and timing processes thus appear to vary at the individual subject level. Further research is needed to characterize biological and procedural factors associated with these differences.

Appendix 1: Bootstrap analysis of prefeeding effects

The procedure described in the Data Analysis section yields bootstrap distributions of medians and CQVs of stable parameter estimates (s₁, s₂, midpoint, width) prior to prefeeding. Let θ_B represent any of these statistics (e.g., the sampled CQVs of s₁), so μ(θ_B) represents its mean and σ(θ_B) its standard deviation. Let θ_P represent the corresponding prefeeding statistic θ. The size of the prefeeding effect on statistic θ was computed as

$$d\left(\uptheta \right)=\left[{\uptheta}_{\textrm{p}}-\upmu \left({\uptheta}_{\textrm{B}}\right)\right]/\sigma \left({\uptheta}_{\textrm{B}}\right)$$

(2)

d(θ) > 2 was indicative of a significant prefeeding-induced increase in parameter θ; d(θ) < −2 was indicative of a significant prefeeding-induced reduction in parameter θ.