Abstract
The distribution of latencies and interresponse times (IRTs) of rats was compared between two fixed-interval (FI) schedules of food reinforcement (FI 30 s and FI 90 s), and between two levels of food deprivation. Computational modeling revealed that latencies and IRTs were well described by mixture probability distributions embodying two-state Markov chains. Analysis of these models revealed that only a subset of latencies is sensitive to the periodicity of reinforcement, and prefeeding only reduces the size of this subset. The distribution of IRTs suggests that behavior in FI schedules is organized in bouts that lengthen and ramp up in frequency with proximity to reinforcement. Prefeeding slowed down the lengthening of bouts and increased the time between bouts. When concatenated, latency and IRT models adequately reproduced sigmoidal FI response functions. These findings suggest that behavior in FI schedules fluctuates in and out of schedule control; an account of such fluctuation suggests that timing and motivation are dissociable components of FI performance. These mixture-distribution models also provide novel insights on the motivational, associative, and timing processes expressed in FI performance. These processes may be obscured, however, when performance in timing tasks is analyzed in terms of mean response rates.
Similar content being viewed by others
Interval timing is the entrainment of an animal’s behavior to a target periodicity in the environment, on the basis of an endogenous time-keeping mechanism (Buhusi & Meck, 2005; Oprisan & Buhusi, 2011). This process is often studied using fixed interval (FI) schedules of reinforcement. In FI schedules, the first response after some interval has elapsed is reinforced. The behavior engendered by FI schedules is highly regular: Following an initial pause, or latency, response rate increases and plateaus near the end of the interval. This organization of behavior has been observed in a wide range of species (Lejeune & Wearden, 1991) and also in Pavlovian conditioning. Indeed, Pavlov observed that, in the presence of a temporally extended conditioned stimulus, well-trained animals initially pause before responding continually until the expected time of reinforcement (Pavlov, 1927; Rescorla, 1967; Vogel, Brandon, & Wagner, 2003). The resemblance between FI performance and performance in Pavlovian analogues has stimulated much theory development (Harris, 2015; Kalafut, Freestone, MacInnis, & Church, 2014; Molet & Miller, 2014).
The foundation of many theories of timing is a basic algorithm first formalized by Treisman (1963). Briefly, onset of a stimulus, signaling the to-be-timed interval, empties an accumulator and initiates the emission of pulses from a pacemaker to the accumulator (together, these comprise the internal clock). Once the pulse count in the accumulator becomes similar to a pulse count criterion sampled from memory, a target response (e.g., reporting that 10 s have elapsed) is emitted. Following reinforcement, pulses in the accumulator update memory. This simple structure yields a temporal distribution of the target response centered on a proportion of the to-be-timed interval (Gibbon, 1977).
Theories of timing typically differ in how they instantiate the components of the pacemaker-accumulator algorithm. For instance, the behavioral theory of timing suggests that transitions between behavioral states constitute pulses (Killeen & Fetterman, 1988; Machado 1997), whereas the multiple timescales theory of timing suggests that the clock is a form of memory decay (Staddon, 2005; Staddon, Chelaru, & Higa, 2002; Staddon & Higa, 1999). More recently, theorists have attempted to ground this basic mechanism into biologically plausible neural networks (Karmarkar & Buonomano, 2007; Oprisan & Buhusi, 2011) and in drift diffusion models that appear to approximate neuronal dynamics (Simen, Balci, Desouza, Cohen, & Holmes, 2011; Simen, Rivest, Ludvig, Balcı, & Killeen, 2013). The challenge for these theories is to account for a variety of classic properties of interval timing. In the context of FI schedules, this includes the sigmoidal response function and the scalar property (Guilhardi & Church, 2005; Machado, Malheiro, & Erlhagen, 2009). However, these aggregate properties of behavior provide a relatively weak criterion for selecting among models of timing, because most of these models, despite their different assumptions, provide a reasonable account of such properties. Other properties of timing performance, such as its responsiveness to motivational manipulations, may provide more informative criteria for selecting between models of timing.
Accounting for the effect of motivational manipulations on timing performance involves some substantial challenges. Recent studies suggest that timing and motivation processes are intimately related (Avlar et al., 2015; Balcı, 2014; Kirkpatrick, 2014). Consistent with this notion, neural circuits implicated in interval timing also regulate incentive motivation processes (Berridge & Kringelbach, 2013; Coull, Hwang, Leyton, & Dagher, 2012; Kirkpatrick, 2014). However, such conclusions are often drawn from theoretical frameworks that assume that, while in a timing task, subjects are always engaged in timing. Such an assumption is inconsistent with a substantial amount of data that suggest that responsiveness to the temporal regularities of periodic stimuli fluctuates between trials (Daniels, Fox, Kyonka, & Sanabria, 2015; Daniels, Watterson, et al., 2015; Freestone, Balcı, Simen, & Church 2015; Lejeune & Wearden, 1991; Mazur, Wood-Isenberg, Watterson, & Sanabria, 2014; Mika et al., 2012; Sanabria & Killeen, 2008). More specifically, these data suggest that, at the beginning of each trial, subjects enter either a timing state, emitting target responses generated by a timing mechanism, or they enter a nontiming state, emitting target responses randomly. Alternation between these two states may reflect a more general fluctuation in behavioral control by operant schedules (Brackney, Cheung, Neisewander, & Sanabria, 2011; Cheung, Neisewander, & Sanabria, 2012; Shull, 2004; Shull, Grimes, & Bennet, 2004; Shull, Gaynor, & Grimes, 2001, 2002; Gibbon, 1995; Myerson & Miezen, 1980).
Performance in FI schedules is thus likely an expression of cycling into and out of multiple behavioral states until reinforcement is obtained. Although previous studies have attempted to implement models embodying this notion (Guilhardi & Church, 2005; Harris, 2015; Kirkpatrick, 2002), their implementation in FI schedules remains poorly characterized and has yet to be fully appreciated. Validation and implementation of fluctuation models in the analysis of FI performance would provide clarity on whether or not timing and motivation are related processes (Balcı, 2014; Balcı et al., 2010; Belke & Christie-Fougere, 2006; Galtress & Kirkpatrick, 2010; Galtress, Marshall, & Kirkpatrick, 2012; Kirkpatrick, 2014; Ludvig, Balcı, & Spetch, 2011; Ludvig, Conover, & Shizgal, 2007; Plowright, Church, Behnke, & Silverman, 2000; Sanabria, Thrailkill, & Killeen, 2009). Specifically, if motivation and timing are not dissociable, manipulations of motivation should influence performance in the timing state; if motivation and timing are dissociable, manipulations of motivation should influence performance only in the nontiming state. The latter finding would reinforce the notion that timing is robust to changes in the motivational state of the animal (Gibbon, 1977, 1995).
Thus, the purpose of this study was two-fold: (1) to validate a generative model of FI performance by isolating the potential stochastic processes underlying individual components of FI performance, and (2) to determine which of those processes are affected by schedule and motivational manipulations. Using strategies formulated by Killeen, Hall, Reilly, and Kettle (2002), this study pursued these goals by conducting a microstructural analysis of performance (Cheung et al., 2012) in two FI schedules and two levels of food deprivation.
Figure 1 shows a schematic of how behavior is organized in an FI trial. In a well-trained animal, performance in an FI trial begins with a latency, followed by responses of various durations (RDs) that alternate with interresponse times (IRTs). In this study, a theoretically motivated generative model of latencies and IRTs was derived from a general model of operant performance (Brackney et al., 2011). For RDs, the empirical data were leveraged to recover aggregate performance. According to this general model, underlying each component is a two-state process wherein the animal either enters a schedule-controlled state (latencies are timed, IRTs are short) or it enters an alternative state (latencies of random length, IRTs are long). The analysis also determined the extent to which the parameters that generate IRTs change as a function of time in the FI.
Illustrative diagram of performance in a fixed-interval (FI) trial. Time moves from left to right. Performance begins with a latency and is followed by a series of responses (black bars) of variable duration (RD) separated by interresponse times (IRTs). RDs and IRTs repeat until the reinforcer is delivered following a response
For two groups of food-deprived rats, head entries into a food receptacle were reinforced according to an FI 30-s (group FI30) or an FI 90-s (group FI90) schedule, until performance stabilized according to a predefined criterion. FI90 rats were then exposed to a prefeeding manipulation for five sessions. Following data collection, a model-space investigation was conducted to determine whether the full complexity of a two-state static or dynamic model was justified to describe each component of performance. Then, the effects of schedule and prefeeding manipulations on parameter estimates were assessed. Finally, a simulation of FI performance based on parameter estimates was conducted to determine the likelihood that the selected model and parameter estimates generates the observed performance.
Method
Subjects
Sixteen male Wistar rats (Charles River Laboratories, Hollister, CA), divided into two cohorts (groups FI30 and FI90), served as subjects. Rats arrived on postnatal day 60 and were pair-housed immediately upon arrival. Rats were housed on a 12:12 h light cycle, with dawn at 1900 h; all behavioral training was conducted during the dark phase of the light cycle. Behavioral training and food restriction protocols were implemented shortly after arrival. Access to food was reduced daily from 24, to 18, 12, and finally 1 h/day. During behavioral training, food was provided 30 min after the end of each training session, such that at the beginning of the next session weights were, on average, 85 % of mean ad libitum weights estimated from growth charts provided by the breeder. Water was always available in home cages. All animal handling procedures followed National Institutes for Health guidelines and were approved by the Arizona State University Institutional Animal Care and Use Committee.
Apparatus
Experiments were conducted in eight MED associates (St. Albans, VT, USA) modular test chambers (305 mm long, 241 mm wide, and 210 mm high), each enclosed in a sound- and light-attenuating box equipped with a ventilation fan that provided masking noise of approximately 60 dB. The front and back walls and the ceiling of test chambers were made of Plexiglas; the front wall was hinged and served as a door to the chamber. The floor consisted of thin metal bars positioned above a catch pan. One of the two aluminum side panels served as a test panel. The reinforcer receptacle was a square opening (51-mm sides) located 15 mm above the floor and centered on the test panel. The receptacle provided access to a dipper (MED Associates, ENV-202 M-S) fitted with a cup (ENV-202C) that could hold 0.01 cc of a liquid reinforcer (33 % sweetened condensed milk diluted in tap water; Great Value brand, Walmart, Bentonville, AK). The receptacle was furnished with a head-entry detector (ENV-254-CB) with a temporal resolution of 10 ms. A house light located behind the wall opposite to the test panel could dimly illuminate the test chamber. Experimental events were arranged via a MED PC interface connected to a PC controlled by MED-PC IV software.
Procedure
All training sessions for each group were conducted once daily, 7 days a week.
Phase 1: Chamber and reinforcer acclimation
Each session began with a 3-min warm-up period during which the house light was on. After completion of this warm-up period, each trial began by turning off the house light. Reinforcement was programmed according to a fixed-time (FT) 30-s schedule of reinforcement for group FI30 and a FT 90-s schedule of reinforcement for group FI90. The end of the interval was followed by a 5-s reinforcer and onset of the house light, which was on until the completion of the intertrial interval (ITI). The ITI was in effect until the rat removed its head from the reinforcement receptacle for 2 s. For FI30 and FI90 rats, each session lasted 60 and 180 min, respectively, or until 60 reinforcers were earned. Although the duration of sessions for each group was different, it ensured a similar number of trials per session. The number of head entries into the reinforcement receptacle during every FT interval was measured; all rats had to make 30 or more head entries in a session to progress to the next phase. This phase lasted five sessions.
Phase 2: FI training
FI-training sessions were similar to those in Phase 1, but reinforcement was delivered only if (a) the rat’s head was detected inside the reinforcer receptacle, and (b) the FT had elapsed. Reinforcement was therefore programmed on a fixed-interval (FI) 30-s schedule for group FI30 and a FI 90-s schedule for group FI90. Training continued for a minimum of 20 sessions and until all rats demonstrated stable temporal control. Stability was determined by a nonsignificant (p > .05) regression of the median and interquartile range (IQR) of latencies (the interval between trial onset and first head entry) over five consecutive sessions. FI30 rats were trained for 23 sessions; FI90 rats were trained for 25 sessions.
Phase 3: Prefeeding
Sessions were similar to those in Phase 2, except that the daily 1 h of free access to food in the home cage was provided immediately before the session instead of 30 min after the session. This phase was implemented for five sessions. Only rats in group FI90 were exposed to this phase.
Data analysis
During Phases 2 and 3, the time of occurrence and duration of every head entry into the reinforcement receptacle was measured in every trial, except for the first trial in each session, which was considered a warm-up trial. Data from the last five sessions of Phase 2 served as steady-state baseline performance for groups FI30 and FI90; data from the five sessions of Phase 3 served as prefeeding performance for group FI90. For each baseline and prefeeding trial, each component of FI responding—latency, interresponse times (IRTs), and response durations (RDs)—was obtained.
Latencies and IRTs were modeled and quantitatively analyzed on the basis of well-specified theories of timing and operant behavior. Because no such theory is available for RDs, these were not modeled and were only qualitatively examined. Latencies and IRTs were fit to variations of a shifted dynamic gamma-exponential mixture distribution. Each model is described conceptually in its corresponding results section. Appendix A details the mathematical rationale for the latency and IRT models, and the relation between these models.
The model of each component of FI performance contains nested models that correspond to different hypotheses about the distribution underlying each component. These models were fit and validated using maximum likelihood estimates (MLE; Myung, 2003) and the corrected Akaike information criterion (AICc; Burnham & Anderson, 2002). AICc is a model selection criterion that favors models that balance high likelihood with low complexity. AICc is asymptotically equivalent to the leave-one-out cross-validation method (Fang, 2011; Stone, 1977). See Appendix B for further details concerning the model selection process.
Selected models were fit to the baseline data of each FI30 and FI90 rat and to the prefeeding data of each FI90 rat. The selected models were intended to be general-process models, in that they provided the best account of the processes underlying FI performance in all rats, in two FI durations, and in two levels of food deprivation. Individual differences are thus reflected in the variability of parameter estimates. Estimates of all parameters were analyzed, along with selected statistics derived from these parameters. Selected estimates and derived statistics were also rescaled as proportions of the FI. Parameter estimates and derived statistics, whether rescaled or not, were log transformed or, in the case of mixture weights, log-odds transformed for statistical analysis (Cheung et al., 2012). A two-tailed Grubbs’ test was performed on log-transformed estimates before each statistical analysis, to detect potential outliers, because the MLE method sometimes yields extreme estimates when applied to mixture-distribution models (Cheung et al., 2012). Outlier data were removed when they were detected with significance criterion α = .05; in most instances, outliers were detected with α = .01 and removal of the outlier did not change the outcome of significance tests. Statistical analysis consisted of independent t tests or Welch’s t test (in the case of unequal n because of removal of an outlier) of the effect of FI schedule (baseline FI30 vs. FI90), and dependent t tests of the effect of prefeeding (baseline FI90 vs. prefeeding FI90), with α = .05. All parameter estimates are reported back-transformed ±SEM.
Results
The top panel of Figure 2 shows the mean probability of a response as a function of time in the FI 30-s and FI 90-s schedules for groups FI30 and FI90, respectively. Baseline behavior appears to be under temporal control of the schedule for FI30 and FI90 rats, as suggested by the progressive increase in response probability,Footnote 1 plateauing close to the end of interval. The mean probability of responding at the end of the interval was slightly higher in FI30 (.80) than in FI90 (.70).
Top panel: Mean response function for group FI30 (squares) and FI90 (circles) as a function of time in the FI. The response function of each rat is the probability of detecting a head entry as a function of time (in 1-s bins) in the FI. Middle panel: Mean normalized response functions. Normalization was conducted for each rat by representing time as a proportion of the FI, and representing response probability as a proportion of the maximum response probability. Bottom panel: Mean response function of FI90 baseline (circles) and prefeeding (triangles) performance
The middle panel of Figure 2 shows the data from the top panel on a normalized x-axis (time divided by FI duration) and y-axis (proportion of maximum probability of responding). During the first 10th of the interval, response probability rose more rapidly toward its maximum for FI90 than for FI30 rats. In the remainder of the interval, however, the relative steepness of these slopes was reversed, mostly because of changes in the slope of response probability in FI90 rats. This suggests that strict adherence of FI performance to Weber’s law was not observed, a phenomenon reported in past research (e.g., Bizo, Chu, Sanabria, & Killeen, 2006).
The bottom panel of Figure 2 shows the response probability function of baseline and prefeeding FI 90-s performance. Prefeeding appears to flatten the slope of the response probability function.
Latencies
Model-space investigation
The top panel of Figure 3 shows the mean empirical cumulative distribution of latencies for groups FI30 and FI90; the middle panel shows the same data normalized against the FI duration; the bottom panel shows group FI90 performance during baseline and prefeeding conditions. Following previous research (Daniels, Fox, et al., 2015; Daniels, Watterson, et al., 2015; Sanabria & Killeen, 2008), it was assumed that, at the beginning of a trial, subjects often—but not always—enter a timing state, where responding is sensitive to the periodicity of reinforcement. Thus, the distribution of latencies was modeled as a weighted mixture of timed (gamma distributed) and nontimed (exponentially distributed) latencies. The parameters of the mixture distribution of latencies are listed in Table 1 (for details, see Appendix A).
Top panel: Mean empirical cumulative distribution of latencies for groups FI30 (squares) and FI90 (circles). Middle panel: Mean distribution of normalized latencies. Normalization was conducted for each rat by representing time as a proportion of the FI. Bottom panel: Mean distribution of latencies for FI90 baseline (circles) and prefeeding (triangles) performance. Solid lines are mean traces of the selected gamma + exponential model. Latencies are organized in 40 bins of equal number of latencies
The results of the latency model-space investigation are reported in Appendix C, Table 5. The nested gamma and exponential models provided substantially poorer fits to mean latencies (see Appendix C; Fig. 11). Of the three models tested, allowing q L to vary freely between 0 and 1 provided the best balance between fit and parsimony. The selected model was e 110 times more likely than the next-best model, in which q L = 1. Figure 3 shows that mean fits of the selected mixture model provided an adequate description of the mean data (for individual fits, see Supplemental Material, Figures S1–S3).
Estimates of the parameters of the selected latency model are shown in Table 2. According to these estimates, FI30 and FI90 rats entered a timing state, on average, on 80 % of trials. When timing, rats produced gamma-distributed latencies of about 12 s and 45 s for FI30 and FI90, respectively. When not timing, rats produced exponentially distributed latencies of about 6 s and 17 s for FI30 and FI90, respectively. Figure 4 illustrates the expression of changes in q L , c L , and k L on the cumulative distribution of latencies.
Effects of changes in parameters q L , c L , and k L (Equation A5) on illustrative cumulative distributions of latencies. Left panel: A reduction in q L flattens the distribution of latencies and shifts it leftward if, as was generally observed, the mean timed latency [c L (1 + ε L )] is longer than the mean nontimed latency (k L ). Center panel: An increase in c L flattens the distribution of latencies and shifts it rightwards. Right panel: A reduction in k L reduces the mean nontimed latency, which is expressed as a steeper “bump” on the left end of the distribution
Schedule and prefeeding effects on latencies
Grubbs’ test revealed a single outlier, Rat 2 in group FI30, for the estimation of ε L and the derived coefficient of variation (CV), p < .050; these measures of Rat 2 were removed from statistical analysis. There were significant differences between FI30 and FI90 performance in estimates of c L , t(14) = 5.549, p < .001, and k L , t(14) = 2.227, p = .043, the mean, t(14) = 8.819, p < .001, and standard deviation of the gamma distribution, t(14) = 10.131, p < .001, and the mean latency predicted by the model, t(14) = 7.809, p < .001. These parameters and derived statistics appear to increase as the duration of the FI schedule increases; all other tests were nonsignificant (all ps > .050).
Values in parentheses in Table 2 are parameter estimates and derived statistics rescaled relative to FI duration; the same outliers were detected as before. Independent t tests revealed no significant differences between FI30 and FI90 performance in rescaled parameter estimates and derived statistics (all ps > .050). The absence of significant schedule effects suggests that all mean parameter estimates and derived statistics are approximately proportional to the duration of the FI; such a finding is consistent with the observation of similar CVs in FI30 and FI90.Footnote 2 Surprisingly, it appears that both timed and nontimed latencies are scalar invariant. This is because timed pulses (which accumulate to a response threshold) and nontimed pulses (which do not) appear to be emitted at intervals (c L and k L , respectively) proportional to those between trial onset and reinforcement.
Table 2 also shows the mean parameter estimates and derived statistics for FI90 prefeeding performance. For all potential comparisons, Grubbs’ test revealed a single outlier for the estimates of ε L , c L , q L , and the derived mean and CV of the gamma distribution: Rat 6 under prefeeding (all ps < .050); it was removed from statistical analysis for those parameters. Prefeeding significantly reduced q L , t(6) = 3.498, p = .010, and increased k L , t(7) = 2.675, p = .032, and the mean latency predicted by the model, t(7) = 3.501, p = .009; all other tests were nonsignificant (all ps > .050). These results suggest that prefeeding increases the prevalence and length of nontimed latencies.
Interresponse times
Model-space investigation
The top panel of Figure 5 shows the mean IRTs for groups FI30 and FI90 as a function of time in the interval; the middle panel shows the same data normalized against the FI duration; the bottom panel shows the mean IRTs for group FI90 as a function of time in the interval during baseline and prefeeding conditions. Following previous research (Brackney et al., 2011; Shull 2004), it was assumed that subjects cycle in and out of states of schedule engagement, expressed as bouts of short exponentially distributed IRTs separated by long exponentially distributed IRTs. Thus, the distribution of IRTs at time t into each trial was modeled as a weighted mixture of exponentially distributed within- and between-bout IRTs, where the former are shorter than the latter.
Top panel: Mean IRT as a function of time (1-s bins; IRTs were assigned to the bin in which they started) in the interval for groups FI30 (squares) and FI90 (circles). Middle panel: Mean normalized IRTs (divided by interval duration) as a function of normalized interval. Normalization was conducted for each rat by representing time as a proportion of the FI. Bottom panel: Mean IRT as a function of time in the interval for FI90 baseline (circles) and prefeeding (triangles) performance. Solid lines are mean traces of the selected IRT model. Note that the y-axis is log base 10 and the x-axis is linear
Mean IRTs of FI30 and FI90 decreased as a function of time in the interval, suggesting that the parameters of the mixture distribution of IRTs do not remain constant over the interval. To account for these changes, it was assumed that the means of the exponential distributions and the odds against within-bout IRTs (a) could decay exponentially over the interval, (b) such change does not begin until some interval has elapsed, and (c) mean IRTs reach an asymptote greater than zero. The parameters of the mixture distribution of IRTs are listed in Table 3 (for details, see Appendix A). The decay parameters are expressed and analyzed in terms of their half-lives, denoted as J 1/2, k 1/2, and c 1/2.
The results of the IRT model-space investigation are reported in Appendix C (Table 6). Of the nested models tested, the dynamic mixture of two exponential distributions provided the best balance between parsimony and fit to the data when the mean within-bout IRT, c It ,, remained constant (i.e., α I = 0 and thus c 1/2 is indeterminate) for every rat in both FI30 and FI90 (see Appendix C, Figure 12). For these conditions, and after correcting for free parameters, this model was at least nearly as likely as the most complex model, and at least e 3.3 = 27 times more likely than the next-best simpler model. According to this model, J It and k It decay exponentially beginning at time τ I , and k It decays to Ω I . This suggests that, as the FI elapses, the mean bout length (1/J It + 1) increases, and the time between bouts decreases to an asymptote. Figure 5 shows that mean fits of the selected mixture model provided an adequate description of the mean data (for individual fits, see Supplemental Material, Figures S4–S6).
Estimates of the parameters of the selected IRT model are shown in Table 4. According to these estimates, the probability of a within-bout IRT—and, thus, bout length—increased progressively throughout the interval. Within bouts, rats responded 2–4 times per second. Bouts were emitted every 7–33 s at the beginning of each interval; this time was reduced by less than half by the end of the interval. Figure 6 illustrates the expression of changes in γ I , β I , Ω I , and τ I on mean IRTs as a function of time in the FI.
Effects of changes in parameters Ω I , τ I , γ I , and β I on illustrative mean IRTs over the course of the FI. The standard IRT function (solid lines) is based on FI30 estimates. It consists of three parts: an initial constant mean IRT, a very rapid decline, and a subsequent slower decline. The rapid decline is primarily due to the decay of between-bout IRTs followed by a slow decline due to the progressive lengthening on bouts. Top-left panel: A reduction in Ω I yields a deeper rapid decline in IRTs. Bottom-left panel: Increasing τ I delays the time of decay onset. Top-right panel: A reduction in γ I further slows down the slower decline in IRTs. Bottom-right panel: A reduction in β I slows down the rapid decline in IRTs
Schedule and prefeeding effects on IRTs
Grubbs’ test revealed the estimate of J 1/2 and τ I for rat 1 in FI30 in FI90, respectively, as outliers (ps < .050), and were removed from analysis. Significant differences between groups FI30 and FI90 were detected in estimates of k I0, t(14) = 8.85, p < .001, c I , t(14) = 2.24, p = .042, τ I , t(13) = 13.39, p < .001, J 1/2, t(13) = 5.26, p < .001, and k 1/2, t(14) = 2.50, p = .025; at every time point in the FI, estimates of mean IRTs were shorter for group FI30 than for FI90, t(14) > 2.56, p < .050; all other tests were nonsignificant (all ps > .050). These results suggest that, compared to FI30, FI90 performance was characterized by longer IRTs, with between-bout IRTs and bout lengths declining later and more slowly.
Values in parentheses in Table 4 are rescaled parameter estimates. The same outliers as with nonrescaled parameters were detected and removed from analysis. Significant differences between groups FI30 and FI90 were detected in rescaled estimates of τ I , t(13) = 3.96, p = .001, and k 1/2, t(14) = 2.21, p = .044. At t = FI/2, rescaled estimates of mean IRT were shorter for group FI30 than for FI90, t(14) = 2.28, p = .038; all other tests were nonsignificant (all ps > .050). These results suggest that parameter estimates were largely scalar invariant, except for between-bout IRTs and the time of decay onset, both of which were significantly longer for group FI90 compared to FI30, even when rescaled.
To determine the relative contribution of each decay parameter to the within-interval decline of IRTs, γ I and β I were each separately set to zero while keeping all other parameters at the values reported for each condition in Table 4. Figure 7 shows the impact of each parameter manipulation on the mean trace of the selected IRT model, compared against the mean IRTs of FI30 and FI90. Setting γ I = 0 and β I = 0 increased AICc, respectively, for FI30 by 6,055 and 2,718, and for FI90 by 50,640 and 1,231. This analysis suggests that mean IRTs declined over the course of the FI mainly because response bouts increased in length over the FI.
Mean IRT as a function of time (1-s bins; IRTs were assigned to the bin in which they started) in the FI for FI30 (left panel, squares) and FI90 (right panel, circles). Lines are mean traces of the selected IRT model (solid), changed so γ I = 0 (dashed) and β I = 0 (dotted). These curves suggest that the decline of IRTs over the FI is mainly driven by the decay of J I , that is, by the increase in bout length. Note that the y-axis is log base 10 and the x-axis is linear
Table 4 also shows the mean parameter estimates and derived statistics for baseline and prefeeding FI90 performance. Grubbs’ test revealed the following estimates as outliers in the FI90 prefeeding probe: c I and k 1/2 for Rat 6, k I0 and τ I for Rat 2, and 1/J I0 + 1 for Rat 5; all ps < .050. A significant difference between baseline and prefeeding was detected in the estimates of k I0, t(6) > 3.35, p = .020, and J 1/2, t(7) = 2.44, p = .044; at t = 0 and t = FI, estimates of mean IRT were longer under prefeeding compared to baseline, t(7) > 2.64, p < .008; all other tests were nonsignificant (p > .050). Longer prefeeding IRTs at the beginning of the interval probably reflect longer between-bout IRTs. Longer prefeeding IRTs at the end of the interval appear to reflect shorter bouts: at t = FI, mean estimates of k It did not vary significantly between conditions, baseline = 8.76 s, prefeeding = 11.85 s, t(7) = 1.41, p = .200, whereas mean estimates of bout length (1/J It + 1) did vary, baseline = 39, prefeeding = 14, t(7) = 3.41, p = .009. Taken together, these results suggest that the effects of prefeeding are largely due to longer between-bout IRTs and slower rates of bout lengthening, leaving the rate of within-bout responding relatively intact.
Response durations
The top panel of Figure 8 shows the mean response duration (RD) of groups FI30 and FI90 as a function of time in the FI; the middle panel shows the mean normalized RD as a function of normalized time; the bottom panel shows the mean RD of group FI90 under baseline and prefeeding conditions. Interestingly, RDs were generally constant across the duration of the FI (censored by the FI duration). It is unclear whether RDs are proportional and thus scale with the FI duration. RDs appear to be robust to the effects of prefeeding.
Top panel: Mean response duration (RD) as a function of time (1-s bins; RDs were assigned to the bin in which they started) in the FI for groups FI30 (squares) and FI90 (circles). Middle panel: Mean normalized RDs (divided by interval duration) as a function of normalized interval. Normalization was conducted for each rat by representing time as a proportion of the FI. Bottom panel: Mean RD as a function of time in the interval for FI90 baseline (circles) and prefeeding (triangles) performance. Note that the y-axis is log base 10 and the x-axis is linear. Plotting only uncensored RDs yielded similar graphs
Monte Carlo simulation of response functions
The previous sections outlined and evaluated two stochastic models, one for latencies and one for IRTs. To verify that these models account for FI performance, a Monte Carlo simulation was conducted using each rat’s estimated parameters in each FI and condition in order to reproduce the response functions in Figure 2.
Figure 9 shows a schematic of the simulator. In each trial, the simulator first sampled either a timed latency from a gamma distribution (1 + ε L , c L ) with probability q L , or a nontimed latency from an exponential distribution (k L ) with probability 1 – q L . The sampled latency was added to the clock t. Then an RD was sampled from a dynamic empirical distribution (for each rat in each condition, the FI was divided into 1 s bins; each bin was then populated with all the RDs that began in that bin). The sampled RD was added to t. If t ≥ FI (30 s or 90 s), then the trial finished. If t < FI, then either a long IRT was sampled from one exponential distribution (k It ) with probability 1 – q It , or a short IRT was sampled from another exponential distribution (c It ) with probability q It . The sampled IRT was added to t; then another RD was sampled, and so on until t ≥ FI.
A generative model of FI performance, constructed from the concatenation of latency, RD, and IRT subroutines. The latency and IRT subroutines are selected models detailed in Appendix A; the RD subroutine samples from a dynamic empirical distribution of RDs, here represented by the letter E. Some of the parameters of the IRT distribution (q It and k It ) change exponentially as the interval progresses
The simulation was conducted in MATLAB (MathWorks, Natick, MA). Each run of the simulator consisted of 295 trials (the approximate number of trials analyzed for each rat under each condition); 5,000 runs were conducted per rat. Mean response functions ±2 standard deviations were drawn from these simulations for each rat.
Figure 10 shows the results of these simulations (see Supplemental Material, Figures S7–S9, for simulations of individual rats). The top row of Figure 10 shows the mean observed and simulated response functions in each FI and condition. The second and third rows show the mean observed and simulated response functions of two representative rats from each group and condition. To assess goodness-of-fit, both R 2 and norm of residuals (n.o.r. = square root of the sum of square residuals) were calculated and are reported in Figure 10. These measures of goodness of fit suggest that the model provides an adequate account of the data. Moreover, almost all observed means are within 2 standard deviations of simulated means, indicating that the FI algorithm was likely to generate the observed data.
Top panels: Mean empirical response functions (calculated as in Fig. 2) for each FI and condition plotted against the mean simulated response function (solid line) ±2 standard deviations (dashed lines). From left to right, the response functions correspond to FI30 (squares), FI90 Baseline (circles), and FI90 prefeeding (triangles). Middle and bottom panels: Response functions for representative rats plotted against the output of the simulator. Note that representative rats are the same for FI90 baseline and prefeeding. Each plot also shows mean (top row) and individual (middle and bottom rows) R 2 and norm of residuals (n.o.r)
Discussion
Each of three components of FI performance—latency to first response, interresponse times (IRTs), and response durations (RDs)—was analyzed separately, testing various models nested within a general framework (see Appendix A). Taken together, the selected models configure the algorithm depicted in Figure 9. This algorithm is a generative model of FI performance that adequately reproduces the observed aggregate response functions (Figure 10) and their underlying components (Figures 3 and 5). Ultimately, this generative model is a general-process model: It reasonably reproduces the performance of any rat, in any FI, and under any level of food deprivation.
Latencies
The analysis of latencies suggests that they are mostly, but not always, timed (i.e., gamma distributed, centered on a proportion of the FI). On average, about 80 % of latencies were timed, with a mean equal to about half the FI. Timed latencies were roughly scalar invariant and robust to changes in reinforcer efficacy. Interestingly, such invariance appears to be driven by changes in the scale parameter of the gamma distribution, providing support to the notion that the speed of the clock decreases as the duration of the FI increases (i.e., as the rate of reinforcement decreases; Beam, Killeen, Bizo, & Fetterman, 1998; Bizo & White, 1994, 1997). In contrast, the shape parameter of the gamma distribution was not significantly affected by the schedule or by a decrease in motivation. Furthermore, the notion that latencies are generated by a timing mechanism is consistent with previous research showing that latencies account for the curvature of FI cumulative records (Gentry, Weiss, & Laties, 1983), are roughly proportional to the duration of the FI (Lowe, Harzem, & Spencer, 1979; Lowe & Wearden, 1981; Shull, 1971; Wearden, 1985; Zeiler & Powell, 1994), and track rapid within-session changes in the duration of the FI (Higa, 1997; Ludvig & Staddon, 2004; Sanabria & Oldenburg, 2014; Wynne, Staddon, & Delius, 1996).
About 20 % of latencies were not timed and had a mean of about one fifth of the FI. These latencies also appear to be scalar invariant, suggesting that, similar to timed latencies, nontimed latencies are sensitive to the passage of time. Although timing implies sensitivity to the passage of time and (to some extent) scalar invariance, the opposite is not true: neither sensitivity to time nor scalar invariance implies timing, at least not in the sense of expressing the operation of a pacemaker-accumulator mechanism. Exponentially distributed intervals suggest a constant probability of responding, not the accumulation of pulses leading to a response, which is the defining feature of counting and timing. Scalar invariance may simply reflect a positive correlation between rate of reinforcement and the constant probability of responding.
Unlike timed latencies, nontimed latencies are sensitive to changes in reinforcer efficacy. Specifically, prefeeding increases both the prevalence and mean of nontimed latencies. Taken together, these findings suggest, contrary to previous research (Plowright et al., 2000), that parameters governing the distribution of timed responses are robust to changes in motivation, and that the effects of motivation on timing performance reflect changes in its nontimed component.
The presence of rapid responding at the beginning of a schedule of reinforcement is often referred to as burst responding (e.g., Richards, Sabol, & Seiden, 1993) and has been observed in other schedules of reinforcement, such as the differential reinforcement of low rates (Richards et al. 1993; Sanabria & Killeen, 2008), fixed minimum interval (Mazur et al., 2014; Mika et al., 2012; Watterson, Mazur & Sanabria, 2015), switch task (Daniels, Fox, et al., 2015; Daniels, Watterson, et al., 2015), and lever holding (Sanabria & Killeen, 2008). Similar to nontimed FI latencies, burst-generated intervals appear to be exponentially distributed (e.g., Sanabria & Killeen, 2008) and, thus, likely not generated by a timing mechanism. Although it is unclear what processes modulate nontimed latencies, in contrast to timed latencies, they appear to be elicited by the to-be-timed stimulus without control by a timing mechanism.Footnote 3 It has been proposed that a random-response component improves the fit of timing models to performance in FI (Lejeune & Wearden, 1991), temporal bisection, and temporal generalization procedures (Droit-Volet & Izaute, 2005; Droit-Volet & Wearden, 2001). The reported model-space investigation of latencies (see Table 5) also supports a two-state model of timing.
Interresponse times
The analysis of IRTs revealed that their distribution, both in FI 30 s and FI 90 s, was well characterized by a mixture of two exponential distributions with parameters changing exponentially as a function of time into the FI. This characterization is consistent with those of IRTs in nontiming paradigms, such as the variable-interval (VI) schedule of reinforcement (Brackney et al. 2011; Cheung et al., 2012; Conover, Fulton, & Shizgal, 2001; Shull 2004; Shull et al., 2001; Shull et al., 2002). Also, mean IRTs in FI 30-s trials and in the last 30 s of FI 90-s trials were remarkably similar, suggesting that IRTs are sensitive to the expected time of food. Taken together, the analysis of IRTs suggests that responses are organized in bouts separated by relatively long pauses, which are sensitive to reinforcement expectancy.
The parameters of the dynamic exponential mixture model may be intuitively mapped onto features of FI performance. The reciprocal of the within-bout IRT is the rate at which the operant is emitted when rats are seeking food, often expressed as “peak responding” around the time of reinforcement (Roberts, 1981). This IRT scaled up with a longer FI schedule, but was robust to the proximity of reinforcement and to deprivation level. Between-bout IRTs may reflect fluctuating levels of motivation that, combined with reinforcement expectancy, are low early in the interval (when the subject is engaged in other, interim behaviors), and increase as the time to reinforcement approaches (cf., Staddon & Simmelhag, 1971). Consistent with this interpretation, between-bout IRTs were longer under prefeeding, at least at the beginning of each interval, when they were most visible.
Bout length appears to be particularly sensitive to the temporal proximity of reinforcement. Past research suggests that bout length expresses a response–reinforcer association that is robust against changes in motivation (Brackney et al., 2011; Brackney & Sanabria, 2015). In support of this interpretation, evidence suggests that food deprivation does not affect VI bout length, and adding a tandem ratio requirement to a VI schedule yields longer response bouts, even though rate of reinforcement is mostly unaffected (Brackney et al., 2011; Brackney & Sanabria, 2015; Shull & Grimes, 2003; Shull et al. 2004, 2001). The tandem-requirement effect appears to stem from a selective reinforcement of longer bouts (Brackney et al., 2011; Brackney & Sanabria, 2015; Killeen, 1969). Changes in bout length during the FI, however, are inconsistent with this explanation. Bouts initiated early in the FI are reinforced only if they are long; such selective reinforcement should yield shorter bouts as the FI elapses, which is the opposite of what was observed. Moreover, bouts proximal to reinforcement appear to be sensitive to changes in reinforcer efficacy in the FI 90-s schedule.
A potential solution to these inconsistencies posits that the response–reinforcer association reflected in the length of bouts is directly related to the underlying associative structure supported by the schedule and is modulated by reinforcer efficacy. In VI schedules, the distribution of intervals between trial onset and reinforcement is typically exponential. This distribution yields a constant hazard function of reinforcement, supporting a constant subjective expectation that reinforcement is available now. In contrast, the hazard function of reinforcement supported by FI schedules is positive, increasing as the FI increases (Dragoi, Staddon, Palmer, & Buhusi, 2003; Kirkpatrick, 2002; Machado, 1997; Staddon et al., 2002). It is possible that bout length is shaped by reinforcement (thus explaining tandem-requirement effects in VI schedules), but also reflects the subjective expectation of reinforcement (thus explaining the within-trial pattern of growth in FI schedules).
There are two plausible explanations for the prefeeding-induced shortening of bout lengths at the time of reinforcement. The first explanation is that this is a mathematical artifact. Reinforcement terminates the ongoing response bout, so its uncensored length is estimated from the pattern of change of response bouts over the FI. It is possible that this estimation is flawed in such a way as to predict shorter (censored) bouts when bouts are infrequent, such as under prefeeding. The second explanation is that the interaction between reinforcer expectation and motivation directly affects bout length. It is possible that when the expectation of reinforcement is low (as in VI schedules and early in FI trials), changes in motivation do not affect bout length, but when the expectation is high, they do. Future research may test these hypotheses.
It seems reasonable to propose, consistent with the two-state description of FI performance suggested by Schneider (1969), that the time of decay onset (τ I ) indicates a stepwise change in expectancy to reinforcement (Gibbon, 1977). However, timed latencies may also index a stepwise change in reinforcer expectancy. Despite the similarity in sensitivity of both parameters to the length of the FI and their robustness to changes in reinforcer efficacy, estimates of τ I were neither scalar invariant nor positively related to timed latencies: estimates of time of decay onset were either negatively correlated (in FI 30-s, r = -.18) or nearly zero-correlated (in baseline FI 90-s, -.01 < r < 0) with mean timed latencies. Further research appears to be necessary to establish the distinct psychological significance of time of decay onset.
Response durations
Unlike latencies and IRTs, RDs were not modeled and their empirical distribution was only used to recover aggregate behavior. Although not reported, RDs were initially modeled similarly to latencies and IRTs, using even more complex and flexible models, and yet simulations based on RD parameter estimates diverged substantially from the data. The failure of the computational model is likely due to the absence of guidance from a quantitative theory of RDs.
The RD data reported here merits a few qualitative observations. First, the duration of FI-reinforced head entries appears to be relatively constant across the FI (see Figure 8). This observation is in contrast to previous reports suggesting the duration of a lever press decreases as the time to reinforcement approaches (Roberts & Gharib, 2006). Second, RDs appear to be systematically longer in FI 90 than in FI 30-s, even when rescaled proportionally to the FI, suggesting that they do not follow Weber’s law. Third, RDs do not appear to be sensitive to changes in reinforcer efficacy. These observations are potentially informative and useful for the development of theoretical models of operant response duration (e.g., Gharib, Derby, & Roberts, 2001; Hurwitz, 1954).
Implications for timing research
The concatenation of the stochastic processes underlying latencies and IRTs resulted in a comprehensive generative model of FI performance. This model adequately reproduced response functions of individual animals as well as mean response functions. The success of the simulator suggests that the proposed generative model provides a viable route by which to understand FI behavior.
Rescaled FI 30-s and FI 90-s response functions partially overlapped, diverging primarily around the middle of the interval between reinforcers. This divergence suggests an exception to Weber’s law. Although generally observed in the temporal dimension (Gibbon, 1977), Weber’s law does not appear to be universal (Bizo et al., 2006; Ivry & Hazeltine, 1995). In this study, it appears that differences in time to decay onset (and to a lesser extent decay of between-bout IRTs) across schedules is primarily responsible for the observed deviations from scalar invariance: even when expressed as a proportion of the FI, the time of decay onset was longer in FI 90-s than in FI 30-s, yielding a steeper rescaled response function (see Figure 2, middle panel). The model of FI performance proposed in this study effectively isolated this variable from other variables that did follow Weber’s law.
The proposed generative model of FI performance also suggests that temporal control in FI schedules is confined to a subset of latencies and, therefore, estimates of such control may be isolated from potentially confounding processes. Such dissociation provides an alternative characterization of performance in a common timing paradigm, the peak-interval procedure. Previous research suggests that the mean and dispersion of peak-interval gradients isolate key features of the control that periodic reinforcement exerts over behavior (Buhusi & Meck, 2005; Roberts, 1981). This would imply that all responses observed in the peak procedure are generated by a timing mechanism. However, recent research suggests that these features may be best characterized by the distribution of latencies, or start times (Saulsgiver, McClure, & Wynne, 2006; Taylor, Horvitz, & Balsam, 2007; but see Balci, Ludvig, & Brunner, 2010, for a discussion of stop times as a potential measure of timing). This revised interpretation of peak performance is consistent with present findings that suggest that (a) the response run is characterized by random bouts whose length grows as a function of time in the trial (potentially driven by a timing process, though it was affected by prefeeding), and (b) only a subset of latencies is sensitive to the periodicity of reinforcement. Typical inferences drawn from the peak procedure thus appear to conflate motivation and timing (Sanabria et al., 2009).
The dissociation of processes underlying latencies and response runs also sheds light on previous research that suggests that peak-interval performance is sensitive to changes in reinforcer efficacy. Such sensitivity is demonstrated by horizontal shifts in the peak-interval gradient and in the distribution of latencies (e.g., Belke & Christie-Fougere, 2006; Galtress & Kirkpatrick, 2009; Galtress et al., 2012; Ludvig et al., 2007; Plowright et al., 2000). Based on these effects, it has been theorized that motivation and timing processes are intimately connected and thus interact to produce overt behavior (Kirkpatrick, 2014). This study suggests an alternative explanation: The horizontal shift in the distribution of latencies and in the peak-interval gradient are likely due to a decrease in the prevalence of timed latencies, an increase in the mean of nontimed latencies, an increase in between-bout IRTs, and a shortening of response bouts around the time when reinforcement is anticipated. These changes do not imply a change in the operation of the pacemaker-accumulator mechanism; instead, they suggest a reduction in motivation for the reinforcer, which results in reduced schedule control and response rate. Furthermore, this suggestion is consistent with the notion that reduction in motivation for the reinforcer alters preference for engaging the FI schedule over alternative behaviors (Gibbon, 1995; Killeen & Pellón, 2013, Sanabria et al., 2009).
The selective expression of the pacemaker-accumulator mechanism in one component of the distribution of latencies yields other important predictions. It predicts, for instance, that changes in reinforcement contingencies, as long as they do not significantly change the periodicity of reinforcement, should have a more substantial impact on run rates and IRTs (more specifically, on the length of response bouts) than on latencies and postreinforcement pauses. This prediction is consistent with reported effects of response requirement on FI and FT performance (Morgan, 1970; Pinkston & Branch, 2004; Shull & Brownstein, 1975; Shull, Guilkey, & Witty, 1972). The model also predicts similar selective effects of changes in response effort (e.g., minimum force to press a lever); the scant empirical evidence on this effect is also consistent with qualitative model predictions (Gollub & Lee, 1966).
The proposed model is consistent with previous attempts to characterize schedule-controlled behavior in terms of two-state Markov models (Brackney et al., 2011; Gibbon, 1995; Harris, 2015; Myerson & Miezin 1980; Shull et al., 2001). For example, Harris (2015) compared two models of aggregate fixed-time (FT) performance, a two-state and a continuous change model. Within both models was a nested mixture model in which, according to some probability, animals would be in an engaged state (emitting a response) and, with complementary probability, in a disengaged state (not emitting a response). The two-state model assumed that, as the interval elapsed, the probability of entering an engaged state increased; in contrast, the continuous-change model assumed that this probability was constant, but that the rate of responding in the engaged stated increased as the interval elapsed. Harris found that the two-state model provided the best description of aggregate FT performance. This finding is consistent with the notion, supported by the present results, that what drives the decay of IRTs between periodic reinforcers is a progressive lengthening of bouts (i.e., a progressive increase in the probability of a within-bout IRT). However, our analysis revealed that, albeit to a smaller degree than bout length, between-bout IRTs also change as the interval elapse. This suggests a complexity to FI performance otherwise missed if analyses are restricted to aggregate data (Hanson & Killeen, 1981).
The present generative performance model is also consistent, to some extent, with the Packet theory of timing (Church & Guilhardi, 2005; Guilhardi & Church, 2005; Guilhardi, Keen, MacInnis, & Church, 2005; Guilhardi, Yi, & Church, 2007; Kirkpatrick, 2002; Kirkpatrick & Church, 2003 also see Dragoi et al., 2003, for an alternative to Packet theory). Packet theory is the only theory of timing that operates on the assumption of bout-like behavior; it stipulates that responding in interval schedules of reinforcement is a composite of the temporal structure of bouts and the rate at which bouts are generated. According to Packet theory, what drives the increasing response rate across the FI is the increased expectation for the reinforcer, which yields an increased probability of a bout. Implementations of the theory produce response bouts such that, if in the middle of a bout another bout is generated, then that new bout is concatenated with the previous bout (Kirkpatrick, 2002); the concatenation of bouts therefore increases as the FI elapses. This process may be reflected in the proposed model as an increase in bout length over the course of the FI, and is consistent with our suggestion that bout length reflects the subjective hazard function of reinforcement supported by the schedule.
There are, however, a few important differences between the present model and Packet theory. Packet theory assumes that latencies are not differentiated from IRTs, both of which are described by a Wald distribution, suggesting that both arise from a timing process. These assumptions are in conflict with the present data and with previous research suggesting that (a) behavior fluctuates into and out of states of schedule control (e.g., Brackney et al., 2011; Shull et al., 2004), and (b) latencies and run rates characterize two different processes (e.g., Gentry et al. 1983; Wearden & Lejeune, 2006). From the present data, Packet theory would incorrectly infer that prefeeding affects the timing process.
Limitations and future directions
Levels of food deprivation were only examined in the FI 90-s schedule and not in the FI 30-s schedule. Although this design precludes the analysis of any potential interaction between FI duration and deprivation level, inferences drawn from this study do not hinge upon the presence or absence of such an effect. Furthermore, the observation that prefeeding reduced schedule control without affecting parameters governing timing or within-bout responding is consistent with previous research on the microstructure of VI performance (Brackney et al., 2011). Future research may aim to generalize the present findings to a wider range of FI schedules, to examine potential schedule × motivation interaction effects.
Another potential limitation of the proposed model is that it is agnostic regarding correlations within and between components of FI performance (e.g., consecutive latencies, consecutive IRTs and RDs), and it was evaluated without taking those potential sequential dependencies into consideration. Prior research has shown interesting patterns of sequential dependency between latencies (Shull, 1971) and between IRTs (Gentry et al., 1983). Furthermore, recent research has revealed that even after animals are well trained, both latencies and run rates may change as a function of time in the session (Balci et al., 2010). Future developments of the proposed generative model may incorporate the relation between components of FI performance and their changes within each session.
Conclusion
This study provides evidence that timing processes may be isolated from other confounding processes using established models of operant performance. Timing processes appear to be expressed in a subset of latencies to the first response, whose distribution is scale-invariant and robust against changes in reinforcer efficacy. Response runs appear to be organized in bouts whose length increases as the time to reinforcement approaches and is sensitive to both the duration of the FI and reinforcer efficacy. The dissociation of these components of FI performance, and the observation that prefeeding does not directly affect timed latencies, provides useful insights into the relation between timing and motivation, and for the development of analytical tools for testing hypotheses regarding timing and its sensitivity to reward value (Galtress & Kirkpatrick, 2009; Ludvig et al., 2007; Plowright et al., 2000). Specifically, timing and motivation appear to be dissociable components of interval timing. Although these tools were applied to a limited range of data in this study, they are derived from a single general equation (see Appendix A), have already been tested in other schedules of reinforcement, and are capable of reproducing FI performance at various levels of aggregation. Moreover, the present generative model of FI performance identifies opportunities for progress in models of timing, highlighting the need to refine our understanding of the microstructure of FI performance.
Notes
The representation of response functions as probability of a response within 1-s bins has an important limitation: whether a head entry within a bin is 10- or 1,000-ms long, it is counted as one response. Nonetheless, binned response probability functions are only used here to represent performance, to evaluate goodness of fit, and as a criterion for successful simulation. The performance parameters that served as the basis for analysis were estimated from individual latencies and IRTs.
A Bayesian dependent t test conducted in JASP (Love et al., 2015) on the CV revealed a small amount of evidence for the null hypothesis but no evidence for the alternative hypothesis (the log of the ratio of evidence for the alternative hypothesis = -0.986, where substantial evidence for the null hypothesis is typically below -1.098; Rouder, Morey, Speckman, & Province, 2012; Kass & Raftery, 1995). This finding was robust to the width of the prior distribution. This outcome supports the claim that timed latencies adhere to Weber’s law.
Alternatively, it might be that nontimed latencies are actually timed latencies produced by an internal clock that fails to reset between trials (Roberts, 1981; Church, Meck, & Gibbon, 1994; Matell & Meck, 1999), despite seemingly salient cues of trial termination (reinforcement, ITI; Sanabria & Killeen, 2007). In particular, a gamma-exponential mixture of latencies may result from single pulses carried over from one trial to the next.
This assumes an implementation in which interpulse intervals are exponentially distributed (i.e., generated by a Poisson process). Alternative implementations include normally distributed interpulse intervals (e.g., de Carvalho, Machado & Vasconcelos, 2016), which yield normally distributed temporal judgments. However, when the accumulation of several pulses is required for responding, Erlang and normally distributed temporal judgments are essentially indistinguishable.
References
Avlar, B., Kahn, J. B., Jensen, G., Kandel, E. R., Simpson, E. H., & Balsam, P. D. (2015). Improving temporal cognition by enhancing motivation. Behavioral Neuroscience, 129(5), 576.
Balcı, F. (2014). Interval timing, dopamine, and motivation. Timing & Time Perception, 2(3), 379–410.
Balcı, F., Ludvig, E. A., Abner, R., Zhuang, X., Poon, P., & Brunner, D. (2010). Motivational effects on interval timing in dopamine transporter (DAT) knockdown mice. Brain Research, 1325, 89–99.
Balci, F., Ludvig, E. A., & Brunner, D. (2010). Within-session modulation of timed anticipatory responding: When to start responding. Behavioural Processes, 85(2), 204–206.
Beam, J. J., Killeen, P. R., Bizo, L. A., & Fetterman, J. G. (1998). How reinforcement context affects temporal production and categorization. Animal Learning & Behavior, 26, 388–396.
Belke, T. W., & Christie-Fougere, M. M. (2006). Investigations of timing during the schedule and reinforcement intervals with wheel-running reinforcement. Behavioural Processes, 73(3), 240–247.
Berridge, K. C., & Kringelbach, M. L. (2013). Neuroscience of affect: Brain mechanisms of pleasure and displeasure. Current Opinion in Neurobiology, 23(3), 294–303.
Bizo, L. A., Chu, J. Y., Sanabria, F., & Killeen, P. R. (2006). The failure of Weber’s law in time perception and production. Behavioural Processes, 71(2), 201–210.
Bizo, L. A., & White, K. G. (1994). The behavioral theory of timing: Reinforcer rate determines pacemaker rate. Journal of the Experimental Analysis of Behavior, 61, 19–33.
Bizo, L. A., & White, K. G. (1997). Training with controlled reinforcer density: Implications for models of timing. Journal of Experimental Psychology: Animal Behavior Processes, 23(1), 44.
Brackney, R. J., Cheung, T. H., Neisewander, J. L., & Sanabria, F. (2011). The isolation of motivational, motoric, and schedule effects on operant performance: A modeling approach. Journal of the Experimental Analysis of Behavior, 96(1), 17–38.
Brackney, R. J., & Sanabria, F. (2015). The distribution of response bout lengths and its sensitivity to differential reinforcement. Journal of the Experimental Analysis of Behavior, 104(2), 167–185.
Buhusi, C. V., & Meck, W. H. (2005). What makes us tick? Functional and neural mechanisms of interval timing. Nature Reviews Neuroscience, 6(10), 755–765.
Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach. New York, NY: Springer Science & Business Media.
Cabrera, F., Sanabria, F., Jiménez, Á. A., & Covarrubias, P. (2013). An affordance analysis of unconditioned lever pressing in rats and hamsters. Behavioural Processes, 92, 36–46.
Cheung, T. H., Neisewander, J. L., & Sanabria, F. (2012). Extinction under a behavioral microscope: Isolating the sources of decline in operant response rate. Behavioural Processes, 90(1), 111–123.
Church, R. M., & Guilhardi, P. (2005). A Turing test of a timing theory. Behavioural Processes, 69(1), 45–58.
Church, R. M., Meck, W. H., & Gibbon, J. (1994). Application of scalar timing theory to individual trials. Journal of Experimental Psychology: Animal Behavior Processes, 20(2), 135.
Conover, K. L., Fulton, S., & Shizgal, P. (2001). Operant tempo varies with reinforcement rate: Implications for measurement of reward efficacy. Behavioural Processes, 56(2), 85–101.
Coull, J. T., Hwang, H. J., Leyton, M., & Dagher, A. (2012). Dopamine precursor depletion impairs timing in healthy volunteers by attenuating activity in putamen and supplementary motor area. The Journal of Neuroscience, 32(47), 16704–16715.
Daniels, C. W., Fox, A. E., Kyonka, E. G. E., & Sanabria, F. (2015). Biasing temporal judgments in rats, pigeons, and humans. International Journal of Comparative Psychology. Retrieved from http://escholarship.org/uc/item/50n6389s.
Daniels, C. W., Watterson, E., Garcia, R., Mazur, G. J., Brackney, R. J., & Sanabria, F. (2015). Revisiting the effect of nicotine on interval timing. Behavioural Brain Research, 283, 238–250.
Dragoi, V., Staddon, J. E. R., Palmer, R. G., & Buhusi, C. V. (2003). Interval timing as an emergent learning property. Psychological Review, 110(1), 126.
de Carvalho, M. P., Machado, A., & Vasconcelos, M. (2016). Animal timing: a synthetic approach. Animal cognition, 19, 707–732.
Droit-Volet, S., & Izaute, M. (2005). The effect of feedback on timing in children and adults: The temporal generalization task. The Quarterly Journal of Experimental Psychology Section A, 58(3), 507–520.
Droit-Volet, S., & Wearden, J. H. (2001). Temporal bisection in children. Journal of Experimental Child Psychology, 80(2), 142–159.
Fang, Y. (2011). Asymptotic equivalence between cross-validations and Akaike information criteria in mixed-effects models. Journal of Data Science, 9, 15–21.
Freestone, D. M., Balcı, F., Simen, P., & Church, R. M. (2015). Optimal response rates in humans and rats. Journal of Experimental Psychology: Animal Learning and Cognition, 41(1), 39.
Galtress, T., & Kirkpatrick, K. (2009). Reward value effects on timing in the peak procedure. Learning and Motivation, 40, 109–131.
Galtress, T., & Kirkpatrick, K. (2010). The role of the nucleus accumbens core in impulsive choice, timing, and reward processing. Behavioral Neuroscience, 124(1), 26.
Galtress, T., Marshall, A. T., & Kirkpatrick, K. (2012). Motivation and timing: Clues for modeling the reward system. Behavioural Processes, 90(1), 142–153.
Gentry, G. D., Weiss, B., & Laties, V. G. (1983). The microanalysis of fixed-interval responding. Journal of the Experimental Analysis of Behavior, 39(2), 327–343.
Gharib, A., Derby, S., & Roberts, S. (2001). Timing and the control of variation. Journal of Experimental Psychology: Animal Behavior Processes, 27(2), 165.
Gibbon, J. (1977). Scalar expectancy theory and Weber's law in animal timing. Psychological review, 84(3), 279.
Gibbon, J. (1995). Dynamics of time matching: Arousal makes better seem worse. Psychonomic Bulletin & Review, 2(2), 208–215.
Gollub, L. R., & Lee, R. M. (1966). Response force under fixed-interval reinforcement. Psychonomic Science, 4, 9–10.
Guilhardi, P., & Church, R. M. (2005). Dynamics of temporal discrimination. Learning & Behavior, 33(4), 399–416.
Guilhardi, P., Keen, R., MacInnis, M. L., & Church, R. M. (2005). How rats combine temporal cues. Behavioural Processes, 69(2), 189–205.
Guilhardi, P., Yi, L., & Church, R. M. (2007). A modular theory of learning and performance. Psychonomic Bulletin & Review, 14(4), 543–559.
Hanson, S. J., & Killeen, P. R. (1981). Measurement and modeling of behavior under fixed-interval schedules of reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 7(2), 129.
Harris, J. A. (2015). Changes in the distribution of response rates across the CS-US interval: Evidence that responding switches between two distinct states. Journal of Experimental Psychology: Animal Learning and Cognition, 41(3), 217.
Higa, J. J. (1997). Rapid timing of a single transition in interfood interval duration by rats. Animal Learning & Behavior, 25(2), 177–184.
Hill, J. C., Herbst, K., & Sanabria, F. (2012). Characterizing operant hyperactivity in the spontaneously hypertensive rat. Behavioral and Brain Function, 8(5), 1–15.
Hurwitz, H. M. B. (1954). Response-duration of lever pressing in the rat. Quarterly Journal of Experimental Psychology, 6(2), 62–71.
Íbias, J., Pellón, R., & Sanabria, F. (2015). A microstructural analysis of schedule-induced polydipsia reveals incentive-induced hyperactivity in an animal model of ADHD. Behavioural Brain Research, 278, 417–423.
Ivry, R. B., & Hazeltine, R. E. (1995). Perception and production of temporal intervals across a range of durations: Evidence for a common timing mechanism. Journal of Experimental Psychology: Human Perception and Performance, 21(1), 3.
Kalafut, K. L., Freestone, D. M., MacInnis, M. L., & Church, R. M. (2014). Integrating timing and conditioning approaches to study behavior. Journal of Experimental Psychology: Animal Learning and Cognition, 40(4), 431.
Karmarkar, U. R., & Buonomano, D. V. (2007). Timing in the absence of clocks: Encoding time in neural network states. Neuron, 53(3), 427–438.
Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90(430), 773–795.
Killeen, P. (1969). Reinforcement frequency and contingency as factors in fixed-ratio behavior. Journal of the Experimental Analysis of Behavior, 12(3), 391–395.
Killeen, P. R. (1991). Behavior’s time. In G. H. Bower (Ed.), The psychology of learning and motivation (pp. 295–334). New York, NY: Academic Press.
Killeen, P. R., & Fetterman, J. G. (1988). A behavioral theory of timing. Psychological Review, 95(2), 274.
Killeen, P. R., & Pellón, R. (2013). Adjunctive behaviors are operants. Learning & Behavior, 41, 1–24.
Killeen, P. R., Hall, S. S., Reilly, M. P., & Kettle, L. C. (2002). Molecular analyses of the principal components of response strength. Journal of the Experimental Analysis of Behavior, 78(2), 127–160.
Kirkpatrick, K. (2002). Packet theory of conditioning and timing. Behavioural Processes, 57(2), 89–106.
Kirkpatrick, K. (2014). Interactions of timing and prediction error learning. Behavioural Processes, 101, 135–145.
Kirkpatrick, K., & Church, R. M. (2003). Tracking of the expected time to reinforcement in temporal conditioning procedures. Learning & Behavior, 31(1), 3–21.
Lejeune, H., & Wearden, J. H. (1991). The comparative psychology of fixed-interval responding: Some quantitative analyses. Learning and Motivation, 22(1), 84–111.
Love, J., Selker, R., Marsman, M., Jamil, T., Dropmann, D., Verhagen, A. J., . . . Wagenmakers, E.-J. (2015). JASP (Version 0.7) [Computer software]. Retrieved from https://jasp-stats.org/faq/.
Lowe, C. F., Harzem, P., & Spencer, P. T. (1979). Temporal control of behavior and the power law. Journal of the Experimental Analysis of Behavior, 31(3), 333–343.
Lowe, C. F., & Wearden, J. H. (1981). Weber law and the fixed-interval post-reinforcement pause. Behaviour Analysis Letters, 1(1), 27–32.
Ludvig, E. A., Balcı, F., & Spetch, M. L. (2011). Reward magnitude and timing in pigeons. Behavioural Processes, 86(3), 359–363.
Ludvig, E. A., Conover, K., & Shizgal, P. (2007). The effects of reinforcer magnitude on timing in rats. Journal of the Experimental Analysis of Behavior, 87(2), 201–218.
Ludvig, E. A., & Staddon, J. E. (2004). The conditions for temporal tracking under interval schedules of reinforcement. Journal of Experimental Psychology: Animal Behavior Processes, 30(4), 299.
Machado, A. (1997). Learning the temporal dynamics of behavior. Psychological Review, 104(2), 241.
Machado, A., Malheiro, M. T., & Erlhagen, W. (2009). Learning to time: A perspective. Journal of the Experimental Analysis of Behavior, 92(3), 423–458.
Matell, M. S., & Meck, W. H. (1999). Reinforcement-induced within-trial resetting of an internal clock. Behavioural Processes, 45(1), 159–171.
Mazur, G. J., Wood-Isenberg, G., Watterson, E., & Sanabria, F. (2014). Detrimental effects of acute nicotine on the response-withholding performance of spontaneously hypertensive and Wistar Kyoto rats. Psychopharmacology, 231(12), 2471–2482.
Mika, A., Mazur, G. J., Hoffman, A. N., Talboom, J. S., Bimonte-Nelson, H. A., Sanabria, F., & Conrad, C. D. (2012). Chronic stress impairs prefrontal cortex-dependent response inhibition and spatial working memory. Behavioral Neuroscience, 126(5), 605.
Molet, M., & Miller, R. R. (2014). Timing: An attribute of associative learning. Behavioural Processes, 101, 4–14.
Morgan, M. J. (1970). Fixed interval schedules and delay of reinforcement. Quarterly Journal of Experimental Psychology, 22, 663–673.
Myerson, J., & Miezen, F. M. (1980). The kinetics of choice: An operant systems analysis. Psychological Review, 87(2), 160.
Myung, I. J. (2003). Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology, 47(1), 90–100.
Oprisan, S. A., & Buhusi, C. V. (2011). Modeling pharmacological clock and memory patterns of interval timing in a striatal beat-frequency model with realistic, noisy neurons. Frontiers in Integrative Neuroscience. doi:10.3389/fnint.2011.00052.
Pavlov, I. P. (1927). Conditioned reflexes (G. V. Anrep, Trans.). London, UK: Oxford University Press.
Pinkston, J. W., & Branch, M. N. (2004). Effects of cocaine on performance under fixed-interval schedules with a small tandem ratio requirement. Journal of the Experimental Analysis of Behavior, 82, 293–310.
Plowright, C. M. S., Church, D., Behnke, P., & Silverman, A. (2000). Time estimation by pigeons on a fixed interval: The effect of pre-feeding. Behavioural Processes, 52(1), 43–48.
Rescorla, R. A. (1967). Inhibition of delay in Pavlovian fear conditioning. Journal of Comparative and Physiological Psychology, 64(1), 114.
Richards, J. B., Sabol, K. E., & Seiden, L. S. (1993). DRL interresponse‐time distributions: Quantification by peak deviation analysis. Journal of the Experimental Analysis of Behavior, 60(2), 361–385.
Roberts, S. (1981). Isolation of an internal clock. Journal of Experimental Psychology: Animal Behavior Processes, 7(3), 242.
Roberts, S., & Gharib, A. (2006). Variation of bar-press duration: Where do new responses come from? Behavioural Processes, 72(3), 215–223.
Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012). Default Bayes factors for ANOVA designs. Journal of Mathematical Psychology, 56(5), 356–374.
Sanabria, F., & Killeen, P. R. (2007). Temporal generalization accounts for response resurgence in the peak procedure. Behavioural Processes, 74, 126–141.
Sanabria, F., & Killeen, P. R. (2008). Evidence for impulsivity in the spontaneously hypertensive rat drawn from complementary response-withholding tasks. Behavioral Brain Function. doi:10.1186/1744-9081-4-7.
Sanabria, F., & Oldenburg, L. (2014). Adaptation of timing behavior to a regular change in criterion. Behavioural Processes, 101, 58–71.
Sanabria, F., Thrailkill, E. A., & Killeen, P. R. (2009). Timing with opportunity cost: Concurrent schedules of reinforcement improve peak timing. Learning & Behavior, 37(3), 217–229.
Saulsgiver, K. A., McClure, E. A., & Wynne, C. D. L. (2006). Effects of d-amphetamine on the behavior of pigeons exposed to the peak procedure. Behavioural Processes, 71(2), 268–285.
Schneider, B. A. (1969). A two-state analysis of fixed-interval responding in the pigeon. Journal of the Experimental Analysis of Behavior, 12, 677–687.
Shull, R. L. (1971). Sequential patterns in post-reinforcement pauses on fixed-interval schedules of food. Journal of the Experimental Analysis of Behavior, 15(2), 221–231.
Shull, R. L. (2004). Bouts of responding on variable-interval schedules: Effects of deprivation level. Journal of the Experimental Analysis of Behavior, 81(2), 155–167.
Shull, R. L., & Brownstein, A. J. (1975). The relative proximity principle and the postreinforcement pause. Bulletin of the Psychonomic Society, 5, 129–131.
Shull, R. L., & Grimes, J. A. (2003). Bouts of responding from variable-interval reinforcement of lever pressing by rats. Journal of the Experimental Analysis of Behavior, 80, 159–171.
Shull, R. L., Gaynor, S. T., & Grimes, J. A. (2001). Response rate viewed as engagement bouts: Effects of relative reinforcement and schedule type. Journal of the Experimental Analysis of Behavior, 75(3), 247–274.
Shull, R. L., Gaynor, S. T., & Grimes, J. A. (2002). Response rate viewed as engagement bouts: Resistance to extinction. Journal of the Experimental Analysis of Behavior, 77(3), 211–231.
Shull, R. L., Grimes, J. A., & Bennett, J. A. (2004). Bouts of responding: The relation between bout rate and the rate of variable-interval reinforcement. Journal of the Experimental Analysis of Behavior, 81(1), 65–83.
Shull, R. L., Guilkey, M., & Witty, W. (1972). Changing the response unit from a single peck to a fixed number of pecks in fixed-interval schedules. Journal of the Experimental Analysis of Behavior, 17, 193–200.
Simen, P., Balci, F., Desouza, L., Cohen, J. D., & Holmes, P. (2011). Interval timing by long-range temporal integration. Frontiers in Integrative Neuroscience, 5, 28.
Simen, P., Rivest, F., Ludvig, E. A., Balcı, F., & Killeen, P. (2013). Timescale invariance in the pacemaker-accumulator family of timing models. Timing & Time Perception, 1(2), 159–188.
Staddon, J. E. R. (2005). Interval timing: Memory, not a clock. Trends in Cognitive Sciences, 9(7), 312–314.
Staddon, J. E. R., Chelaru, I. M., & Higa, J. J. (2002). Habituation, memory and the brain: The dynamics of interval timing. Behavioural Processes, 57(2), 71–88.
Staddon, J. E. R., & Higa, J. J. (1999). Time and memory: Towards a pacemaker-free theory of interval timing. Journal of the Experimental Analysis of Behavior, 71(2), 215–251.
Staddon, J. E. R., & Simmelhag, V. L. (1971). The “superstition” experiment”: A reexamination of its implications for the principles of adaptive behavior. Psychological Review, 78(1), 3–43.
Stone, M. (1977). An asymptotic equivalence of choice of model by cross-validation and Akaike's criterion. Journal of the Royal Statistical Society. Series B (Methodological), 39, 44–47.
Taylor, K. M., Horvitz, J. C., & Balsam, P. D. (2007). Amphetamine affects the start of responding in the peak interval timing task. Behavioural Processes, 74(2), 168–175.
Treisman, M. (1963). Temporal discrimination and the indifference interval: Implications for a model of the “internal clock”. Psychological Monographs: General and Applied, 77(13), 1.
Vogel, E. H., Brandon, S. E., & Wagner, A. R. (2003). Stimulus representation in SOP: II. An application to inhibition of delay. Behavioural Processes, 62(1), 27–48.
Watterson, E., Mazur, G. J., & Sanabria, F. (2015). Validation of a method to assess ADHD-related impulsivity in animal models. Journal of Neuroscience Methods, 252, 36–47.
Wearden, J. H. (1985). The power law and Weber’s law in fixed-interval postreinforcement pausing: A scalar timing model. The Quarterly Journal of Experimental Psychology, 37(3), 191–211.
Wearden, J. H., & Lejeune, H. (2006). “The stone which the builders rejected …”: Delay of reinforcement and response rate on fixed-interval and related schedules. Behavioural Processes, 71(2), 77–87.
Wynne, C. D., Staddon, J. E., & Delius, J. D. (1996). Dynamics of waiting in pigeons. Journal of the Experimental Analysis of Behavior, 65(3), 603–618.
Zeiler, M. D., & Powell, D. G. (1994). Temporal control in fixed-interval schedules. Journal of the Experimental Analysis of Behavior, 61(1), 1–9.
Acknowledgments
This research was supported by the National Institutes of Health (MH094562), a seed grant from the College of Liberal Arts and Sciences, Arizona State University, and a Grant-in-Aid of Research awarded to the first author from the National Academy of Sciences, administered by Sigma Xi, The Scientific Research Society. The results of this study comprise portions of the first author’s master’s thesis. We thank Raul Garcia, Paula Overby, Christine Herrera, Jesse St. Amand, Sanjana Khana, Natasha Sinchuk, and Jake Gilmour for helping with data collection. Briana Martinez, Andrew Nye, and Cavan Winikates also provided important support to data collection. Elizabeth Watterson, Gabriel Mazur, and Ryan Brackney provided helpful discussions. Portions of these data were presented at the 2014 meeting of the Society for the Quantitative Analysis of Behavior, Chicago, Illinois; the 2014 meeting of the International Society of Comparative Psychology, Bogotá, Colombia; the 2015 meeting of the Association for Behavioral Analysis International, San Antonio, Texas; and the 2015 Fall meeting of the Comparative Cognition Society, Chicago, Illinois. We thank Peter Killeen for his helpful discussions, advice, and feedback on earlier drafts of this document. Feedback from an anonymous reviewer and a reviewer identified as Armando Machado was invaluable. Finally, the first author would like to dedicate this article to his grandfather, Robert Lillion Carter, who always provided helpful guidance and support.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(DOCX 34998 kb)
Appendices
Appendix A: Derivation of latency and IRT models
General framework
The model of FI latencies and the model of IRTs are variations of a single general model of interval production. Latencies are sampled from a mixture of gamma-distributed (timed) latencies and exponentially distributed (nontimed) latencies. IRTs are sampled from a mixture of two exponential distributions (between- and within-bout IRTs), whose parameters change over the course of the FI. Because setting its scale parameter to 1 reduces a gamma to an exponential distribution, mixture models of latencies and IRTs are special cases of a single, gamma-exponential mixture model, described in this section.
Let X be a random variable that is exponentially distributed with scale parameter k > 0 and shift parameter δ > 0 [i.e., X ~ exp(k,δ)]. The probability density of X = D, for D > 0, is
Let Y be a random variable that is gamma distributed with shape parameter 1 + ε, where ε ≥ 0, scale parameter c > 0, and shift parameter δ > 0 [i.e., Y ~ Γ(1 + ε, c, δ)]. The probability density of Y = D, for D > 0, is
where T(1 + ε) is the gamma function evaluated at 1 + ε. Note that if ε = 0, then Equation A2 reduces to Equation A1, where c would be the scale parameter of an exponential distribution.
If Z is a mixture of X and Y with weights of 1 – q and q, respectively, where 0 ≤ q ≤ 1, then the probability density of Z = D, for D > 0, is
Substituting Equations A1 and A2 for F Y (D) and F X (D) in Equation A3,
Latency
Let Z L be a latency emitted at time t = 0, where the L subscript stands for latency. Following previous research (Daniels, Fox, et al., 2015; Daniels, Watterson, et al., 2015; Mazur et al., 2014; Sanabria & Killeen, 2008), it is assumed that Z L is the sum of two random variables, one gamma distributed (the pacemaker model of timing yields an Erlang distribution of temporal judgments;Footnote 4 the gamma distribution results from generalizing this model to allow for fractional pulses; see Killeen, 1991) and the other exponentially distributed, with mixture weights q L and 1 – q L , respectively,
The meaning of each parameter can be found in Table 1 in the body of the article; the constraints on parameters are as previously listed. Additionally, δ L = 0 because, otherwise, it would imply that the actual interval being timed is equal to FI – δ L instead of the FI.
Interresponse times (IRTs)
Let Z It be an IRT emitted at time t, where the I subscript stands for IRT and t stands for time at which the IRT began (0 ≤ t ≤ FI). Following previous research (Brackney et al., 2011; Cabrera, Sanabria, Jiménez, & Covarrubias, 2013; Conover, Fulton, & Shizgal, 2001; Gibbon, 1995; Hill, Herbst, & Sanabria 2012; Íbias, Pellón, &, Sanabria, 2015), it is assumed that Z It is the sum of two shifted exponentially distributed random variables, with scale parameters c It and k It , where k It ≥ c It , shift parameter δ I > 0, and mixture weights 1 – q It and q It ,
The meaning of each parameter can be found in Table 2 in the body of the article; the constraints on parameters are as previously listed. Note that δ I is included in the model of IRTs; it is the minimum amount of time necessary emit a response, typically estimated as the shortest IRT observed (Brackney et al., 2011; Cheung et al., 2012).
To account for the change in IRTs over the course of the FI, let J It = (1 – q It ) / q It (i.e., the odds against the shorter IRTs). Assume that J It , c It , and k It are constant between the onset of the FI and a time τ I, where 0 ≤ τ I ≤ FI. Beginning at τ I , parameters J It , c It , and k It decay exponentially as a function of t. Parameters c It and k It decay to an asymptote of Ω I , where k It ≥ c It ≥ Ω I .
J I0, c I0, and k I0 are the parameters of Eq. A6 at the onset of the FI; γ, α, and β are decay rates greater than zero, where α ≥ β (longer IRTs do not decay faster than shorter IRTs, to prevent their decay functions from crossing over).
Appendix B: Model selection with AICc
A model-space investigation was conducted using data from each group of rats under baseline conditions to determine which free parameters in Equation A5 (the model of latencies) and Equations A6 and A7 (the model of IRTs) were justified for each component of FI performance. All potential nested models of each component were fit to the data of each rat from group FI30 and FI90 using the method of maximum likelihood (Myung, 2003). For each model, the maximum likelihood estimate (MLE) was used to compute the corrected Akaike information criterion (AICc; Burnham & Anderson, 2002). Briefly, AICc favors nested models that balance goodness of fit (higher MLE) against parsimony (fewer free parameters); lower AICc are indicative of better balance. The analysis yielded a selection criterion ΔAICc for each nested model; ΔAICc of nested model i is the AICc of nested model i minus the lowest AICc across all models compared. Thus, the nested model with the lowest AICc has a ΔAICc = 0, and all other nested models have a ΔAICc > 0. The simplest constrained nested models with ΔAICc < 4 were selected (this threshold is recommended by other researchers; see Brackney et al., 2011; Burnham & Anderson, 2002) as the model providing the best balance between fit and parsimony.
Appendix C: Model selection outcomes
Mean empirical cumulative distribution of latencies (squares) for FI30 (top panel) and FI90 (bottom panel), and mean fit of each model in Table 5: gamma + exponential (solid line), gamma (dashed line), exponential (dotted line). Latencies are organized in 40 bins of equal number of latencies
Mean IRT of FI30 (top panel) and FI90 (bottom panel) rats as a function of time (1-s bins) in the FI. The continuous curve is the mean fit of the selected IRT model; the dashed curves are mean between-bout IRTs (k It ), and the dotted curves are the mean within-bout IRTs (c It ), estimated from the selected IRT model. Note that y-axes are log base 10 and x-axes are linear
Rights and permissions
About this article
Cite this article
Daniels, C.W., Sanabria, F. Interval timing under a behavioral microscope: Dissociating motivational and timing processes in fixed-interval performance. Learn Behav 45, 29–48 (2017). https://doi.org/10.3758/s13420-016-0234-1
Published:
Issue Date:
DOI: https://doi.org/10.3758/s13420-016-0234-1