Delay Gradients for Spout-Licking and Magazine-Entering Induced by a Periodic Food Schedule

Abstract

The present experiments studied impulsivity by manipulating the delay between target responses and presentation of a reinforcer. Food-deprived SHR, WKY, and Wistar rats were exposed to a fixed-time 30-s schedule of food pellet presentation until they developed stable patterns of water spout-licking and magazine-entering. In successive phases of the study, a resetting delay contingency postponed food delivery if target responses (licks or entries) occurred within the last 1, 2, 5, 10, 20, 25, or 28 s of the inter-food interval. Response-food delays were applied independently for the two behaviors during separate experimental phases, and order of presentation and the behavior that was punished first were counterbalanced. Licking was induced in the order of Wistar > SHR > WKY, and magazine entries were in the order of SHR > WKY > Wistar. Magazine entries showed steeper delay gradients than licking in SHR and Wistar rats but were of similar great inclination in the WKY rats. The different responses were differentially sensitive to delays. This suggests a different ordering of them in the interval between reinforcers. It also has implications for attempts to change impulsive behavior, both in terms of the nature of the response and its removal from reinforcing consequences.

Appendix

Appendix 1 The gradients

Power functions were used to describe the delay of reinforcement gradients in this paper because of their flexibility and because when exponential or other such gradients with large variability in parameters (across animals, or conditions) are averaged, power functions are the asymptotic form (Murre & Chessa, 2011). The Mazur hyperbola did a worse job of fitting these data. A disadvantage of such power functions is that, as the delay approaches 0, the curve goes to infinity. No reinforcers are actually obtained with 0-s delay between response and receipt, therefore the nominal value of 75 ms, the time required to drop the pellet, was added to all delays, as that maximized the goodness of fit overall for the curves in Fig. 3.

Appendix 2 The indices

The indices a and AUC′ measure persistence in the immediately reinforced response—here, schedule-induced drinking or magazine entries—when punished by a delay to the pellets. The parameter a is the coefficient of the power functions describing the delay of reinforcement gradients. It directly gives the height of the curve at 1 s. It has the advantage of being independent of the range of delays studied and the steepness of the gradient. It is informed by all of the data but most heavily by the data around 1 s delay. Thus, it is not so efficient in the use of data as the AUC′, which gives more balanced weight to all of the data. It has the advantage of being independent of the shape parameter (b) of the delay gradient, as a value of 1 is 1 no matter what exponent it is raised to.

AUC′, giving a more balanced weight to all the data, is a more efficient statistic. The range of delays studied, however, affects its value. A very short range will give a larger value, while the inclusion of delays of great length will give it a lower value, as it is an average. The curve itself does not change because its range is extended or truncated. Therefore AUC′ may be standardized across studies by using a standard upper limit for it. It does not matter if that limit exceeds or falls short of the range used in any particular study, as long as all of the data are used to fix the curve. Here, that standard value was 30 s. It is gratifying that both indices of persistence tell very similar stories about these data (see Figs. 5 and 6). An additional advantage of AUC′ over a is that it is less sensitive to the somewhat arbitrary minimal delay (75 ms) added to a nominal 0-s delay.

AUC′ measures the proportional area under the curve, not the area under the data, as do empirical AUCs (e.g., Myerson, Green, & Warusawitharana, 2001, strictly an AUData). AUC′ has the disadvantage of being model-dependent. It has the advantage that, by imposing certain reasonable priors (smoothness and monotonicity), it may give a more generally accurate measure of the impact delays on conditioned responding, and on delay discounting.

A third index of potential general utility is the half-life of the delay gradient. As long as a theoretical delay gradient is close to the data around the half-decay point, it does not matter which model of the gradient is used. If a power function is used, that half-life is d_0.5 = (2a)^1/b. The median half-life across strains for licking in this study was 16 s, whereas for entries it was 3 s. This statistic, again, indicates much shallower gradients for licking than for magazine entering: It required five times as long a delay to reduce licking to the same level as entering.

A problem with half-life is that for shallow gradients there will be large estimation error associated with it. To see this noted that the same error in preferences structures around p = 0.8 will be reflected to a very narrow range on the x-axis, but around p = 0.2 it will be reflected to a very broad range on the x-axis. Because the estimation error gets more severe with shallower gradients, a measure such as d_0.75, the time to fall to ¾ of the (nominal) value of 1 at minimal delay might be generally preferable. Here, that value is d_0.75 = (4a/3)^1/b. For proportional power functions in general, it is d _p  = (a/p)^1/b _.

Appendix 3

Table 2 Obtained mean inter-food intervals