Skip to main content
SpringerLink
Log in

A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology

  • Review
  • Published:
Psychopharmacology Aims and scope Submit manuscript

Abstract

Rationale

Mathematical principles of reinforcement (MPR) provide the theoretical basis for a family of models of schedule-controlled behaviour. A model of fixed-ratio schedule performance that was applied to behaviour on progressive ratio (PR) schedules showed systematic departures from the data.

Objective

This study aims to derive a new model from MPR that will account for overall and running response rates in the component ratios of PR schedules, and their decline toward 0, the breakpoint.

Method

The role of pausing is represented in a real-time model containing four parameters: T 0 and k are the intercept and slope of the linear relation between post-reinforcement pause duration and the prior inter-reinforcer interval; a (specific activation) measures the incentive value of the reinforcer; δ (response time) sets biomechanical limits on response rate. Running rate is predicted to decrease with negative acceleration as ratio size increments, overall rate to increase and then decrease. Differences due to type of progression are explained as hysteresis in the control by reinforcement rates. Re-analysis of extant data focuses on the effects of acute treatment with antipsychotic drugs, lesions of the nucleus accumbens core, and destruction of orexinergic neurones of the lateral hypothalamus.

Results

The new model resolves some anomalies evident in earlier analyses, and provides new insights to the results of these interventions.

Conclusions

Because they can render biologically relevant parameters, mathematical models can provide greater power in interpreting the effects of interventions on the processes underlying schedule-controlled behaviour than is possible for first-order data such as the breakpoint.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Aberman JE, Ward SJ, Salamone JD (1998) Effects of dopamine antagonists and accumbens dopamine depletions on time-constrained progressive-ratio performance. Pharmacol Biochem Behav 61:341–348

    Article  PubMed  CAS  Google Scholar 

  • Arnold JM, Roberts DCS (1997) A critique of fixed and progressive ratio schedules used to examine the neural substrates of drug reinforcement. Pharmacol Biochem Behav 57:441–447

    Article  PubMed  CAS  Google Scholar 

  • Aston-Jones G, Smith RJ, Moorman DE, Richardson KA (2009) Role of lateral hypothalamic orexin neurons in reward processing and addiction. Neuropharmacology 56:112–121

    Article  PubMed  CAS  Google Scholar 

  • Baldo B, Kelley AE (2007) Discrete neurochemical coding of distinguishable motivational processes: insights from nucleus accumbens control of feeding. Psychopharmacology 91:439–459

    Article  Google Scholar 

  • Baron A, Mikorski J, Schlund M (1992) Reinforcement magnitude and pausing on progressive-ratio schedules. J Exp Anal Behav 58:377–388

    Article  PubMed  CAS  Google Scholar 

  • Barr AM, Phillips AG (1999) Withdrawal following repeated exposure to d-amphetamine decreases responding for a sucrose solution as measured by a progressive ratio schedule of reinforcement. Psychopharmacology 141:99–106

    Article  PubMed  CAS  Google Scholar 

  • Berridge CW, Espana RA, Vittoz NM (2010) Hypocretin/orexin in arousal and stress. Brain Res 1314:91–102

    Article  PubMed  CAS  Google Scholar 

  • Bezzina G, Cheung THC, Asgari K, Hampson CL, Body S, Bradshaw CM, Szabadi E, Deakin JFW, Anderson IM (2007) Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis. Psychopharmacology 195:71–84

    Article  PubMed  CAS  Google Scholar 

  • Bezzina G, Body S, Cheung THC, Hampson CL, Deakin JFW, Anderson IM, Szabadi E, Bradshaw CM (2008) Effect of quinolinic acid-induced lesions of the nucleus accumbens core on performance on a progressive ratio schedule of reinforcement: implications for inter-temporal choice. Psychopharmacology 197:339–350

    Article  PubMed  CAS  Google Scholar 

  • Bittar EG, Del-Claro K, Bittar LG, da Silva MCP (2012) Towards a mathematical model of within-session operant responding. J Exp Psychol: Anim Behav Proc. doi:10.1037/a0029086

  • Bizo LA, Killeen PR (1997) Models of ratio schedule performance. J Exp Psychol: Anim Behav Process 23:351–367

    Article  CAS  Google Scholar 

  • Bowman EM, Brown VJ (1998) Effects of excitotoxic lesions of the rat ventral striatum on the perception of reward cost. Exp Brain Res 123:439–448

    Article  PubMed  CAS  Google Scholar 

  • Carlezon WA, Thomas MJ (2009) Biological substrates of reward and aversion: a nucleus accumbens activity hypothesis. Neuropharmacology 56(1):122–132

    Article  PubMed  CAS  Google Scholar 

  • Cheeta S, Brooks S, Willner P (1995) Effects of reinforcer sweetness and the D2/D3 antagonist raclopride on progressive ratio operant performance. Behav Pharmacol 6:127–132

    Article  PubMed  CAS  Google Scholar 

  • Corrigan PW, Reinke RR, Landsberger SA, Charate A, Toombs GA (2003) The effects of atypical antipsychotic medications on psychosocial outcomes. Schizophrenia Res 63:97–101

    Article  Google Scholar 

  • Covarrubias P, Aparicio CF (2008) Effects of reinforcer quality and step size on rats' performance under progressive ratio schedules. Behav Processes 78:246–252

    Article  PubMed  Google Scholar 

  • Cunningham Owens DG (1999) A guide to the extrapyramidal side-effects of antipsychotic drugs. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Czachowski CL, Samson HH (1999) Breakpoint determination and ethanol self-administration using an across-session progressive ratio procedure in the rat. Alcoholism: Clin Exp Res 23:1580

    Article  CAS  Google Scholar 

  • Date Y, Ueta Y, Yamashita H, Yamaguchi H, Matsukura S, Kangawa K, Sakurai T, Yanagisawa M, Nakazato M (1999) Orexins, orexigenic hypothalamic peptides, interact with autonomic, neuroendocrine and neuroregulatory systems. Proc Natl Acad Sci USA 96:748–753

    Article  PubMed  CAS  Google Scholar 

  • de Lecea L, Kilduff TS, Peyron C, Gao X, Foye PE, Danielson PE, Fukuhara C, Battenberg EL, Gautvik VT, Bartlett FS, Frankel WN, van den Pol AN, Bloom FE, Gautvik KM, Sutcliffe JG (1998) The hypocretins: hypothalamus-specific peptides with neuroexcitatory activity. Proc Natl Acad Sci USA 95:322–327

    Article  PubMed  Google Scholar 

  • den Boon FS, Body S., Hampson CL, Bradshaw CM, Szabadi E, de Bruin N (2012) Effects of amisulpride and aripiprazole on progressive-ratio schedule performance: comparison with clozapine and haloperidol. J Psychopharmac. doi:10.1177/0269881111421974

  • Ferguson SA, Paule MG (1997) Progressive ratio performance varies with body weight in rats. Behav Processes 40:177–182

    Article  Google Scholar 

  • Ferster CB, Skinner BF (1957) Schedules of reinforcement. B. F. Skinner Foundation, Cambridge

    Book  Google Scholar 

  • Gancarz AM, Kausch MA, Lloyd DR, Richards JB (2012) Between-session progressive ratio performance in rats responding for cocaine and water reinforcers. Psychopharmacology. doi:10.1007/s00213-012-2637-9

  • Goudie AJ, Cooper GD, Cole JC, Sumnall HR (2007) Cyproheptadine resembels clozapine in vivo folowing both acute and chronic administration in rats. J Psychopharmac 21:179–190

    Article  CAS  Google Scholar 

  • Graham SJ, Langley RW, Bradshaw CM, Szabadi E (2001) Effects of haloperidol and clozapine on prepulse inhibition of the acoustic startle response and the N1/P2 auditory evoked potential in man. J Psychopharmac 15:243–250

    Article  CAS  Google Scholar 

  • Harris GC, Aston-Jones G (2006) Arousal and reward: a dichotomy in orexin function. Trends Neurosci 29:571–577

    Article  PubMed  CAS  Google Scholar 

  • Hartfield A, Moore N, Clifton P (2003) Serotonergic and histaminergic mechanisms involved in intralipid drinking? Pharmacol Biochem Behav 76:251–258

    Article  PubMed  CAS  Google Scholar 

  • Herrnstein RJ (1970) On the law of effect. J Exp Anal Behav 13:243–266

    Article  PubMed  CAS  Google Scholar 

  • Herrnstein RJ (1974) Formal properties of the matching law. J Exp Anal Behav 21:159–164

    Article  PubMed  CAS  Google Scholar 

  • Herrnstein RJ, Rachlin H, Laibson DI (1997) The matching law. Harvard University Press, Cambridge

    Google Scholar 

  • Ho MY, Body S, Kheramin S, Bradshaw CM, Szabadi E (2003) Effects of 8-OH-DPAT and WAY-100635 on performance on a time-constrained progressive ratio schedule. Psychopharmacology (Berl) 167:137–144

    CAS  Google Scholar 

  • Hodos W (1961) Progressive ratio as a measure of reward strength. Science 134:943–944

    Article  PubMed  CAS  Google Scholar 

  • Hodos W, Kalman G (1963) Effects of increment size and reinforcer volume on progressive ratio performance. J Exp Anal Behav 6:387

    Article  PubMed  CAS  Google Scholar 

  • Kheramin S, Body S, Herrera FM, Bradshaw CM, Szabadi E, Deakin JF, Anderson IM (2005) The effect of orbital prefrontal cortex lesions on performance on a progressive ratio schedule: implications for models of inter-temporal choice. Behav Brain Res 156:145–152

    Article  PubMed  CAS  Google Scholar 

  • Killeen PR (1978) Superstition: a matter of bias, not detectability. Science 199:88–90

    Article  PubMed  CAS  Google Scholar 

  • Killeen PR (1979) Arousal: Its genesis, modulation, and extinction. In: Zeiler MD, Harzem P (eds) Advances in analysis of behavior, vol 1, Reinforcement and the organization of behavior. Wiley, Chichester, pp 31–78

    Google Scholar 

  • Killeen PR (1994) Mathematical principles of reinforcement. Behav Brain Sci 17:105–172

    Article  Google Scholar 

  • Killeen PR (1998) The first principle of reinforcement. In: Wynne CDL, Staddon JER (eds) Models of action: mechanisms for adaptive behavior. Erlbaum, Mahwah, pp 127–156

    Google Scholar 

  • Killeen PR, Sitomer MT (2003) MPR. Behav Processes 62:49–64

    Article  PubMed  Google Scholar 

  • Killeen PR, Hanson SJ, Osborne SR (1978) Arousal: its genesis and manifestation as response rate. Psychol Rev 85:571–581

    Article  PubMed  CAS  Google Scholar 

  • Killeen PR, Posadas-Sanchez D, Johansen EB, Thrailkill EA (2009) Progressive ratio schedules of reinforcement. J Exp Psychol: Anim Behav Proc 35:35–50

    Article  Google Scholar 

  • Li N, He S, Parrish C, Delich J, Grasing K (2003) Differences in morphine and cocaine reinforcement under fixed and progressive ratio schedules; effects of extinction, reacquisition and schedule design. Behav Pharmac 14:619–630

    Article  CAS  Google Scholar 

  • Markou A, Weiss F, Gold LH, Caine SB, Schulteis G, Koob GF (1993) Animal models of drug craving. Psychopharmacology (Berl) 112:163–182

    Article  CAS  Google Scholar 

  • Meltzer H, Perry E, Jayathilake K (2003) Clozapine-induced weight gain predicts improvement in psychopathology. Schizophrenia Res 59:19–27

    Article  Google Scholar 

  • Mobini S, Chiang T-J, Ho M-Y, Bradshaw CM, Szabadi E (2000) Comparison of the effects of clozapine, haloperidol, chlorpromazine and d-amphetamine on performance on a time-constrained progressive ratio schedule and on locomotor behaviour in the rat. Psychopharmacology 152:47–54

    Article  PubMed  CAS  Google Scholar 

  • Müller-Spahn F (2002) Current use of atypical antipsychotics. Eur Psychiat 17(suppl 4):377–384

    Article  Google Scholar 

  • Olarte Sánchez CM, Valencia Torres L, Body S, Cassaday HJ, Bradshaw CM, Szabadi E, Goudie AJ (2012a) A clozapine-like effect of cyproheptadine on progressive-ratio schedule performance. J Psychopharm 26:857–870

  • Olarte Sánchez CM, Valencia Torres L, Body S, Cassaday HJ, Bradshaw CM, Szabadi E (2012b) Effectof orexin-B-saporin induced lesions of the lateral hypothalamus on a progressive-ratio schedule. J Psychopharm 26:871–886

  • Perone M, Courtney K (1992) Fixed-ratio pausing: joint effects of past reinforcer magnitude and stimuli correlated with upcoming magnitude. J Exp Anal Behav 57:33–46

    Article  PubMed  CAS  Google Scholar 

  • Peyron C, Tighe DK, van den Pol AN, de Lecea L, Heller HC, Sutcliffe JG, Kilduff TS (1998) Neurons containing hypocretin (orexin) project to multiple neuronal systems. J Neurosci 18:9996–10015

    PubMed  CAS  Google Scholar 

  • Ping-Teng C, Lee ES, Konz SA, Richardson NR, Roberts DCS (1996) Progressive ratio schedules in drug self-administration studies in rats: a method to evaluate reinforcing efficacy. J Neurosci Meth 66:1–11

    Article  Google Scholar 

  • Reilly MP (2003) Extending mathematical principles of reinforcement into the domain of behavioral pharmacology. Behav Processes 62:75–88

    Article  PubMed  Google Scholar 

  • Richardson NR, Roberts DCS (1996) Progressive ratio schedules in drug self-administration studies in rats—a method to evaluate reinforcing efficacy. J Neurosci Methods 66:1–11

    Article  PubMed  CAS  Google Scholar 

  • Rickard JF, Body S, Zhang Z, Bradshaw CM, Szabadi E (2009) Effect of reinforcer magnitude on performance maintained by progressive-ratio schedules. J Exp Anal Behav 91:75–87

    Article  PubMed  CAS  Google Scholar 

  • Roberts DCS, Richardson NR (1992) Self-administration of psychostimulants using progressive ratio schedules of reinforcement. In: Boulton A, Baker G, Wu PH (eds) Neuromethods vol 24: animal models of drug addiction. Humana, New York, pp 233–269

    Chapter  Google Scholar 

  • Sakurai T, Amemiya A, Ishii M, Matsuzaki I, Chemelli RM, Tanaka H, Williams SC, Richarson JA, Kozlowski GP, Wilson S, Arch JR, Buckingham RE, Haynes AC, Carr SA, Annan RS, McNulty DE, Liu WS, Terrett JA, Elshourbagy NA, Bergsma DJ, Yanagisawa M (1998) Orexins and orexin receptors: a family of hypothalamic neuropeptides and G protein-coupled receptors that regulate feeding behavior. Cell 92:573–585, and page following p. 696

    Article  PubMed  CAS  Google Scholar 

  • Salamone JD, Correa M, Farrar A, Mingote SM (2007) Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits. Psychopharmacology 191:461–482

    Article  PubMed  CAS  Google Scholar 

  • Schmelzeis MC, Mittleman G (1996) The hippocampus and reward: effects of hippocampal lesions on progressive-ratio responding. Behav Neurosci 110:1049–1066

    Article  PubMed  CAS  Google Scholar 

  • Schneider BA (1969) A two-state analysis of fixed-interval responding in the pigeon. J Exp Anal Behav 12:677–687

    Article  PubMed  CAS  Google Scholar 

  • Siegel JM (2004) Hypocretin (orexin): role in normal behavior and neuropathology. Ann Rev Psychol 55:125–148

    Article  Google Scholar 

  • Siegel JM (2005) Hypocretin/orexin and motor function. In: Nishino S, Sakurai T (eds) The orexin/hypocretin system: physiology and pathophysiology. Humana Press, New York

    Google Scholar 

  • Skinner BF (1948) Superstition in the pigeon. J Exp Psychol 38:168–172

    Article  PubMed  CAS  Google Scholar 

  • Skjoldager P, Pierre PJ, Mittleman G (1993) Reinforcer magnitude and progressive ratio responding in the rat: effects of increased effort, prefeeding, and extinction. Learn Motiv 24:303–343

    Article  Google Scholar 

  • Stafford D, Branch MN (1998) Effects of step size and break-point criterion on progressive-ratio performance. J Exp Anal Behav 70:123–138

    Article  PubMed  CAS  Google Scholar 

  • Stafford D, LeSage MG, Glowa JR (1998) Progressive-ratio schedules of drug delivery in the analysis of drug self-administration: a review. Psychopharmacology (Berl) 139:169–184

    Article  CAS  Google Scholar 

  • Sutton RS, Barto AG (1990) Time-derivative models of Pavlovian reinforcement. In: Gabriel M, Moore J (eds) Learning and computational neuroscience: foundations of adaptive networks. MIT Press, Cambridge, pp 497–537

    Google Scholar 

  • Timberlake W, Lucas GA (1985) The basis of superstitious behavior: chance contingency, stimulus substitution, or appetitive behavior? J Exp Anal Behav 44:279–299

    Article  PubMed  CAS  Google Scholar 

  • Timberlake W, Lucas GA (1989) Behavior systems and learning: from misbehavior to general principles. In: Klein SB, Mowrer RR (eds) Contemporary learning theories: instrumental conditioning theory and the impact of constraints on learning. Erlbaum, Hillsdale, pp 237–275

    Google Scholar 

  • Trivedi P, Yu H, MacNeil DJ, Van der Ploeg LH, Guan XM (1998) Distribution of orexin receptor mRNA in the rat brain. FEBS Lett 438:71–75

    Article  PubMed  CAS  Google Scholar 

  • Valencia-Torres L, Olarte-Sánchez CM, da Costa AS, Body S, Bradshaw CM, Szabadi E (2012) Nucleus accumbens and delay discounting in rats: evidence from a new quantitative protocol for analysing inter-temporal choice. Psychopharmacology 219:271–283

    Article  PubMed  CAS  Google Scholar 

  • Wise RA (1982) Neuroleptics and operant behavior: the anhedonia hypothesis. Brain Behav Sci 5:39–87

    Article  Google Scholar 

  • Wise RA (2006) Role of brain dopamine in food reward and reinforcement. Phil Trans Roy Soc Lond B: Biol Sci 361:1149–1158

    Article  CAS  Google Scholar 

  • Wynne CDL, Staddon JER, Delius JD (1996) Dynamics of waiting in pigeons. J Exp Anal Behav 65:603–618

    Article  PubMed  CAS  Google Scholar 

  • Zhang Z, Rickard JF, Asgari K, Body S, Bradshaw CM, Szabadi E (2005a) Quantitative analysis of the effects of some "atypical" and "conventional" antipsychotics on progressive ratio schedule performance. Psychopharmacology (Berl) 179:489–497

    Article  CAS  Google Scholar 

  • Zhang Z, Rickard JF, Body S, Asgari K, Bradshaw CM, Szabadi E (2005b) Comparison of the effects of clozapine and 8-hydroxy-2-(di-n-propylamino) tetralin (8-OH-DPAT) on progressive ratio schedule performance: evidence against the involvement of 5-HT 1A receptors in the behavioural effects of clozapine. Psychopharmacology (Berl) 181:381–391

    Article  CAS  Google Scholar 

Download references

Conflicts of interest

None.

Author information

Authors and Affiliations

  1. Psychopharmacology Section, Division of Psychiatry University of Nottingham B109 Medical School, University of Nottingham, Nottingham, NG7 2UH, UK

    C. M. Bradshaw

  2. Department of Psychology, Arizona State University, Tempe, AZ, 85287-1104, USA

    P. R. Killeen

Authors
  1. C. M. Bradshaw
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. P. R. Killeen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to C. M. Bradshaw or P. R. Killeen.

Additional information

The order of authorship is alphabetical. Correspondence may be addressed to either author.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(XLS 0.99 MB)

Appendix

Appendix

A1. Coupling

Not all responding that is excited by incentives results in proper, on-target responding (Killeen 1978; Skinner 1948; Timberlake and Lucas 1985, 1989). The coupling coefficient tells us what proportion of stimulated behaviour will be measured on the lever. Because the reinforcer itself interrupts the stream of target response, it truncates the reach of those delay of reinforcement gradients. On PR schedules, however, animals have experience with long sequences of the target response on the longest ratios. It can be assumed that these drive coupling arbitrarily high. In the current model, it is set 1, to conserve that parameter: C PR ≈ 1.

A2. Reduction to FR model and prediction of de jure breakpoint

Rewrite Eq. 6c for the case where successive values of N and T TOT are equal:

$$ {T_{\text{TOT}}} = {T_0} + k{T_{\text{TOT}}} + N\delta \left( {1 + {T_{\text{TOT}}}/a} \right) $$

Simplify by setting T 0 to 0, and solve:

$$ {T_{\text{TOT}}} = \frac{{N\delta }}{{1 - k - N\delta /a}} \cdot $$

Compute overall response rate:

$$ \begin{array}{*{20}{c}} {\frac{N}{{{{T}_{{{\text{TOT}}}}}}} = \frac{{N\left( {1 - k - N\delta /a} \right)}}{{N\delta }}} \\ { = \frac{{1 - k}}{\delta } - \frac{N}{a}} \\ \end{array} $$

Thus, the FR model.

Set to 0 to compute the de jure break point:

$$ {N_{\text{BP}}} = \frac{{\left( {1 - k} \right)a}}{\delta } $$

A3. Computer pseudocode*

figure a

*The standard values are initiated to provide some ‘burn-in’, with the assumption of a nominal 2 s T TOT under an FR1. SSs is an unweighted sum of SS RUN and SSOVERALL.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bradshaw, C.M., Killeen, P.R. A theory of behaviour on progressive ratio schedules, with applications in behavioural pharmacology. Psychopharmacology 222, 549–564 (2012). https://doi.org/10.1007/s00213-012-2771-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00213-012-2771-4

Keywords

Search

Navigation