, Volume 112, Issue 1, pp 13–20 | Cite as

5HT drugs in animal models of anxiety

  • S. L. Handley
  • J. W. McBlane


It has been widely accepted that 5HT neurones promote anxiety, in humans as well as in animal models. This could be termed the “classic” hypothesis and it has led to a determined search for drugs which reduce 5HT function, especially agents which have selective actions at 5HT receptor subtypes. However, these novel agents tend to have weak and/or variable effects in animal models and more detailed examination of their actions suggests that not all findings are accounted for by the classic hypothesis. There appear to be circumstances in which increased 5HT activity can reduce anxious behaviour. There is increasing evidence for multiple anxiety mechanisms, which may be able to explain differential patterns of drug effects within and between models. Animal models of anxiety may also detect non-anxiety factors: effects on cognition or on impulsivity could be reflected in some models. This could be important in the light of recent evidence that 5HT-selective reuptake inhibitors are effective in impulsivity disorders. The classic hypothesis of 5HT function in anxiety may be only one part of an increasingly complex story. Unravelling the rest of this story is likely to lead to new insights in our understanding of anxiety and related disorders.

Key words

5HT Anxiety Animal model 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Almeida SS, De Oliviera LM, Graeff FG (1990) Decreased reactivity to anxiolytics caused by early protein malnutrition in rats. Pharmacol Biochem Behav 36:997–1000Google Scholar
  2. Audi EA, de Olveira RMW, Graeff FG (1991) Microinjection of propranolol into the dorsal periaqueductal gray causes an anxiolytic effect in the elevated plus maze antagonised by ritanserin. Psychopharmacology 105:553–557Google Scholar
  3. Beckett SRG, Lawrence AJ, Marsden CA, Marshall PW (1992) Attenuation of a chemically induced defence response by 5HT1 receptor agonists administered into the periaqueductal grey Psychopharmacology 118:110–115Google Scholar
  4. Blier P, de Monitgny C (1990) Electrophysiological investigation of the adaptive response of the 5-HT system to the administration of 5-HT1A receptor agoinists. J Cardiovasc Pharmacol 15:S42–48Google Scholar
  5. Bockeart J, Dumuis A, Bouhelal R, Sebbe M, Cory RN (1987) Piperazine derivatives including the putative anxiolytic drugs, buspirone and ipsapirone, are agonists at 5-HT1A receptors negatively coupled with adenylate cyclase in hippocampal neurons. Naunyn-Schmiedeberg's Arch Pharmacol 335:588–592Google Scholar
  6. Bradley PB, Smounya L, Fenuik W. Fozard JJ, Humphrey PPA, Middlemiss DN, Mylechrane EJ, Richardson BP, Saxena PR (1986) Proposals for the classification and nomenclature of functional receptors for 5-hydroxytryptamine. Neuropharmacol 25:563–576Google Scholar
  7. Cadogan AK, Wright IK, Coombs I, Marsden CA, Kendall DA, Tulloch I (1992) Repeated paroxetine administration in the rat produces an anxiolytic profile in the elevated X-maze and a decreased3H-ketanserin binding. Neurosci Lett Suppl 42:S8Google Scholar
  8. Ceulemans DLS, Hoppenbrowers MLKA, Gelders YG, Reyntjens AJM (1985) The influence of ritanserin, a serotonin receptor antagonist, in anxiety disorders: a double blind placebo-controlled study versus lorazepam. Pharmacol Biochem Behav 18:303–305Google Scholar
  9. Charney DS, Heninger GR, Renhard JF, Sternberg DE, Hafstead KM (1982) Effect of intravenousl-tryptophan on prolactin and growth hormone and mood in healthy subjects. Psychopharmacology 77:217–222Google Scholar
  10. Charney DS, Woods SW, Goodman WK, Heninger GR (1987) Serotonin functions in anxiety II. Effects of the serotonin agonist MCPP in panic disorder patients and healthy subjects. Psychopharmacology 92:14–24Google Scholar
  11. Chopin P, Briley M (1987) Animal models of anxiety: the effect of compounds that modify serotonergic function. TIPS 8:383–388Google Scholar
  12. Costall B, Kelly ME, Tomkins DM, Tyers MB (1989a) Profile of aciton of diazepam and 5-HT3 receptor antagonists in the elevated X-maze. J Psychopharmacol 3:10PGoogle Scholar
  13. Costall B, Kelly ME, Naylor RJ, Onaivi ES, Tyers MB (1989b) Neuroanatomical sites of action of 5-HT3 receptor agonists and antagonists for alteration of aversive behaviour in the mouse. Br J Pharmacol 96:325–332Google Scholar
  14. Critchley MAE, Handley SL (1987) Effects in the X-maze anxiety model of agents acting at 5-HT1 and 5-HT2 receptors. Psychopharmacology 93:502–506Google Scholar
  15. Critchley MAE, Njung'e K, Handley SL (1987) 5-HT ligand effects in the social interaction test of anxiety. Br J Pharmacol 92:659PGoogle Scholar
  16. Critchley MAE, Njung'e K, Handley SL (1992) Actions and some interactions of some 5-HT1A ligand in the elevated X-maze and effects of dorsal raphe lesions. Psychopharmacology 106:484–490Google Scholar
  17. Deakin JFW (1983) Roles of serotonergic systems in escape, avoidance and other behaviours. In: Cooper SJ (ed) Theory in psychopharmacology, vol. 2. Academic Press, New York, pp 149–193Google Scholar
  18. Deakin JFW, Graeff FG (1991) 5-HT and mechanisms of defence. J Psychopharmacol 5:305–315Google Scholar
  19. de Montigny C, Blier P (1992) Effects of antidepressant treatments on 5-HT neurotransmission. In: Marsden CA, Heal DJ (eds) Central serotonin receptors and psychotropic drugs. Blackwell, London, pp 223–238Google Scholar
  20. Den Boer JA, Westenberg HGM (1990) Serotonin function in panic disorder: a double blind placebo controlled study with fluvoxamine and ritanserin. Psychopharmacology 102:85–94Google Scholar
  21. Dourish CT, Hutson PH, Curzon G (1986) Putative anxiolytics 8-OHDPAT buspirone and TVXQ 7821 are agonists at 5-HT1A autoreceptors in the raphe nuclei. TIPS 7:212–214Google Scholar
  22. Eison AS, Eison MS, Stanley M, Riblet LA (1986) Serotonergic mechanisms in the behavioural effects of buspirone and gepirone. Pharmacol Biochem Behav 24:701–707Google Scholar
  23. Engel JA, Hjorth S, Svensson K, Carlsson A, Liljequist B (1984) Anticonflict effects of the putative serotonin receptor agonist 8-hydroxy-2-(di-n-propylamino) tetralin (8-OH-DPAT). Eur J Pharmacol 105:365–368Google Scholar
  24. Engelman K, Lovenberg W, Sjoerdsma A (1967) Inhibition of serotonin synthesis byp-chlorophenylalanine in carcinoid syndrome. New Eng J Med 227:1103–1108Google Scholar
  25. File SE, Hyde JRG (1979) Can social interaction be used to measure anxiety? Br J Pharmacol 62:19–24Google Scholar
  26. File SE, Johnston AJ (1989) Lack of effects of 5-HT3 receptor antagonists in the social interaction and elevated plus-maze tests of anxiety in the rat. Psychopharmacology 99:248–251Google Scholar
  27. Gardner CR (1985) Recent developments in 5-HT-related pharmacology of animal models of anxiety. Pharmacol Biochem Behav 24:1479–1485Google Scholar
  28. Gardner CR, Guy AP (1985) Pharmacological characterization of a modified social interactin model of anxiety in the rat. Neuropsychobiology 13:194–200Google Scholar
  29. Glaser T, Traber J (1983) Buspirone: action on serotonin receptors in calf hippocampus. Eur J Pharmacol 88:137–138Google Scholar
  30. Graeff FG (1988) Animal models of aversion. In: Simon P, Soubrié P, Widlocher D (eds) Animal models of psychiatric disorders. Karger, Basel, pp 115–141Google Scholar
  31. Graeff FG (1992) Amygdala and periaqueductal gray in anxiety and panic. Behav Brain Res (in press)Google Scholar
  32. Graeff FG, Schoenfeld RI (1970) Tryptaminergic mechanisms in punished and nonpunished behaviour. J Pharmacol Exp Ther 173:277–283Google Scholar
  33. Graeff FG, Zuardi AW, Gjimiglio JS, Filho EC, Karniol IG (1985) Effect of metergoline on human anxiety. Psychopharmacology 86:334–338Google Scholar
  34. Grantham P (1987) Benzodiazepine abuse. Br J Hosp Med 37:292–300Google Scholar
  35. Gray JA (1982) The neuropsychology of anxiety. Oxford University Press, OxfordGoogle Scholar
  36. Green S (1991) Benzodiazepines putative anxiolyitics and animal models of anxiety. TINS 14:101–104Google Scholar
  37. Guimaraes FS (1992) 5-HT receptors and consolidation of stressful memories. Behav Brian Res (in press)Google Scholar
  38. Guimaraes FS, Wang M, Deakin JFW (1990a) Ritanserin reduces the expression of a conditioned fear response in neurotic patients. J Psychopharmacol 4:259PGoogle Scholar
  39. Guimaraes FS, Cosme RA, Zuardi AW, Graeff FG (1990b) Ipsapirone reduces increases in anxiety during a simulated public speaking test. J Psychopharmacol 4:296PGoogle Scholar
  40. Handley SL (1991) Serotonin in animal models of anxiety: the importance of stimulus and response. In: Cowen P, Idzikowski C (eds) Serotonin, sleep and mental disorder. Wrightson, Petersfield, pp 89–115Google Scholar
  41. Handley SL, McBlane JW (1991) 5-HT — the disengaging transmitter? J Psychopharmacol 5:322–326Google Scholar
  42. Handley SL, McBlane JW (1993) An assessment of the elevated X-maze for studying anxiety and anxiety modulating drugs. J Pharmacol Methods (in press)Google Scholar
  43. Handley SL, Mithani S (1984) Effects of α2-adrenoceptor agonists and antagonists in a maze exploration model of fear-motivated behaviour. Naunyn-Schmiedeberg's Arch Pharmacol 327:1–5Google Scholar
  44. Handley SL, McBlane JW, Critchley MAE, Njung'e K (1992) Multiple serotonin mechanisms in animal models of anxiety: the importance of stimulus and response. Behav Brain Res (in press)Google Scholar
  45. Hensman R, Guimaraes FS, Wang M, Deakin JFW (1991) The effects of ritanserin on aversive classical conditioning in humans. Psychopharmacology 104:220–224Google Scholar
  46. Higgins GA, Bradbury AJ, Jones BJ, Oakley NR (1988) Behavioural and biochemical consequences following activation of 5-HT1-like and GABA receptors in the dorsal raphe nucleus of the rat. Neuropharmacology 27:993–1001Google Scholar
  47. Higgins GA, Jones BJ, Oakley NR, Tyers MB (1991) Evidence that the amygdala is involved in the disinhibitory effects of 5HT3-receptors antagonists. Psychopharmacology 104:545–551Google Scholar
  48. Hodges H, Green S, Glenn B (1987) Evidence that the amygdala is involved in benzodiazepine and serotonergic effects on punished responding but not on discrimination. Psychopharmacology 92:491–504Google Scholar
  49. Hoyer D, Schoeffter P (1991) 5-HT receptors: subtypes and second messengers. J Rec Res 11:197–214Google Scholar
  50. Iversen SD (1984) 5-HT and anxiety. Neuropharmacology 23:1553–1560Google Scholar
  51. Jenck F, Broekkamp CLR, van Delft AML (1989) Opposite control mediated by central 5-HT1A and non-5-HT1A(5-HT1B or 5-HT1C) receptors on periaqueductal gray aversion. Eur J Pharmacol 161:219–221Google Scholar
  52. Jenck F, Broekkamp CLE, van Delft AML (1990) 5-HT1C receptors in the serotonergic control of periaqueductal gray induced aversion in rats. Psychopharmacology 100:373–376Google Scholar
  53. Kennett GA, Dourish CT, Curzon G (1987) Anti-depressant-like actions of 5-HT1A agonists and conventional antidepressants in an animal model of depression. Eur J Pharmacol 134:265–274Google Scholar
  54. Kennett GA, Whitton P, Shah K, Curzon G (1989) Anxiogenic-like effects ofmCPP and TFMPP in animal models are opposed by 5-HT1C receptor antagonists. Eur J Pharmacol 164:445–454Google Scholar
  55. Kiser RS, Lebowitz RM (1975) Monoaminergic mechanisms in aversive brain stimulation. Physiol Behav 15:47–53Google Scholar
  56. Klint T (1991) Effects of 8-OH-DPAT and buspirone in a passive avoidance test and in the elevated X-maze test in rats. Behav Pharmacol 2:481–489Google Scholar
  57. Kostowski W, Platznik A, Stefanski R (1989) Intra-hippocampal buspirone in animal models of anxiety. Eur J Pharmacol 168:393–396Google Scholar
  58. Kostowski W, Dyr W, Krzscik P (1990) The effects of 5-HT1A receptor agonists in animal models of anxiety and depression. Psychopharmacology 101:S31Google Scholar
  59. Kshama D, Hrishikeshaven HJ, Shanbogue R, Munonyedi US (1990) Modulation of baseline behaviour in rats by putative serotonergic agents in three ethoexperimental paradigms. Behav Neural Biol 54:234–253Google Scholar
  60. Lader MH (1991) Ondansetron in the treatment of anxiety. Presented at the 5th World Congress of Biological Psychiatry, Satellite symposium, The role of ondansetron, a novel 5-HT3 antagonist in the treatment of psychiatric disorders, Florence, pp 17–19Google Scholar
  61. LeDoux JE (1993) Emotional memory systems in the brain. Behav Brain Res (in press)Google Scholar
  62. Lightowler S, Williams IJR, Hegarty J, Kennett GA, Fears RB, Tulloch IF (1992) Anxiolytic effect of paroxetine in the rat social interaction model of anxiety. Br J Pharmacol 106:44PGoogle Scholar
  63. Lister RG (1990) Ethologically-based animal models of anxiety disorders. Pharmacol Ther 46:321–340Google Scholar
  64. Lopez-Ibor JJ (1988) The involvement of serotonin in psychiatric disorders and behaviour. Br J Psychiat 153 [Suppl 3]:26–39Google Scholar
  65. Luscombe GP, Mazurkiewicz SE, Heal DJ (1992) The 5-HT1A ligand BP 554 mimics the anxiolytic activity of buspirone gepirone and ipsapirone in the elevated plus-maze in rats. Br J Pharmacol 106:220–130PGoogle Scholar
  66. Mansbach RS, Geyer MA (1988) Blockade of potentiated startle responding in rats by 5-hydroxytryptamine1A receptor ligands. Eur J Pharmacol 156:375–383Google Scholar
  67. Meller E, Goldstein M, Bohmaker K (1990) Receptor reserve for 5-hydroxytryptamine1A-mediated inhibition of serotonin synthesis: possible relationship to anxiolytic properties of 5-hydroxytryptamine1A agonists. Mol Pharamcol 37:231–237Google Scholar
  68. Moser PC (1989) An evaluation of the elevated plus-maze test using a novel anxiolytic buspirone. Psychopharmacology 99:48–53Google Scholar
  69. Moser PC, Tricklebank MD, Middlemiss DN, Mir AK, Hibert MF, Fozard JR (1990) Characterisation of MDL 7300EF as a 5-HT1A selective ligand and its effects in animal models of anxiety; comparison with buspirone 8-OH-DPAT and diazepam. Br J Pharmacol 99:343–349Google Scholar
  70. Motta V, Maisonnette S, Morato S, Castrechini P, Brandao ML (1992) Effects of blockade of 5-HT2 receptors and activation of 5-HT1A receptors on the exploratory activity of rats in the elevated plus-maze. Psychopharmacology 107:135–139Google Scholar
  71. Moulton B, Morinan A (1990) The effect of RS-30199 on anxiety and hippocampal monoamine oxidase activity in the rat. Br J Pharmacol 101:516PGoogle Scholar
  72. Moyer JA, Lucki I (1985) The piperazine serotonin (5-HT) agonistsm-chlorophenylpiperazine (mCPP) and 6-chloro-2(1-piperzinyl)pyrazine (MK 212) impair avoidance responding in a discrete trial conditioned avoidance procedure. Soc Neurosci Abstr 11:1187Google Scholar
  73. Newton RE, Marunycz JD, Alderdice MT, Napoliello MJ (1986) Review of the side effect profile of buspirone. Am J Med 89[Suppl 3]:17–21Google Scholar
  74. Njung'e K (1990) Effects of some 5-hydroxytryptamine and related ligands in anxiety models. PhD thesis, Aston University, UKGoogle Scholar
  75. Nutt DJ, Glue P (1989) Clinical pharmacology of anxiolytics and antidepressants: a psychopharmacological perspective. Pharmacol Ther 44:309–334Google Scholar
  76. Pellow S, Chopin P, File SE, Briley M (1985) Validation of open: closed arm entries in an elevated plus-maze as a measure of anxiety in the rat. J Neurosci Methods 14:149–167Google Scholar
  77. Pellow S, Johnston A, File SE (1987) Selective agonists and antagonists for 5-hydroxytryptamine receptor subtypes and interactions with yohimbine and FG7142 using the elevated plus-maze test in the rat. J Pharm Pharmacol 39:917–928Google Scholar
  78. Peroutka SJ (1985) Selective interaction of novel anxiolytics with-5-hydroxytryptamine 1A receptors. Biol Psychiatry 20:971–979Google Scholar
  79. Redfern WS, Williams A (1989) Acute effects of centrally acting drugs on the behaviour of rats in an elevated X-maze and a partially shaded holeboard. Br J Pharmacol 97:9PGoogle Scholar
  80. Robichaud RC, Sledge KL (1969) The effects ofp-chlorophenylalanine on experimentally induced conflict in the rat. Life Sci 8:965–969Google Scholar
  81. Schweitzer E, Rickels K (1991) Serotonergic anxiolytics: a review of their clinical efficacy. In: Rodgers RJ, Cooper SJ (eds) 5-HT1A agonists and 5-HT3 antagonists and benzodiazepines: their comparative behavioural pharmacology. Wiley, NY, pp 366–376Google Scholar
  82. Sheehan DV, Raj AB, Sheehan K, Soto S (1988) The relative efficacy of buspirone, imipramine and placebo in panic disorder: a preliminary report. Pharmacol Biochem Behav 29:815–817Google Scholar
  83. Söderpalm B, Hjorth S, Engel JA (1989) Effects of 5-HT1A receptor agonists and L-5-HTP in Montgomery's conflict test. Pharmacol Biochem Behav 32:259–265Google Scholar
  84. Soubrié P (1988) Serotonin and behaviour, with special regard to animal models of anxiety, depression and waiting ability. In: Osborne NN, Hamon M (eds) Neuronal serotonin. Wiley, NY, pp 255–270Google Scholar
  85. Stein L, Wise CD, Berger BD (1973) Antianxiety actions of benzodiazepines: decrease in activity of serotonin neurones in the punishment system. In: Garattini S, Mussini E, Randall IO (eds) The benzodiazepines. Raven Press, NY, pp 299–326Google Scholar
  86. Tenen SS (1967) The effects ofp-chlorophenylalanine, a serotonin depletor on avoidance aquisition, pain sensitivity and related behaviour in the rat. Psychopharmacology 10:204–219Google Scholar
  87. Thiebot M-H (1986) Are serotonergic neurones involved in the control of anxiety and in the anxiolytic activity of benzodiazepines. Pharmacol Biochem Behav 24:1471–1477Google Scholar
  88. Thiébot M-H, Hamon M, Soubrié P (1983) The involvement of nigral serotonin innervation in the control of punishment-indiced behavioural inhibition in rats. Pharmacol Biochem Behav 19:225–229Google Scholar
  89. Thiébot M-H, Le Bihan C, Soubrié P, Simon P (1985) Benzodiazepines reduce the tolerance to reward delay in rats. Psychopharmacology 86:147–152Google Scholar
  90. Tomkins DM, Costall B, Kelly ME (1990a) Actions of ritanserin and DOI on the elevated X-maze. Psychopharmacology 101:S57Google Scholar
  91. Tomkins TM, Costall B, Kelly ME (1990b) Release of suppressed behaviour of rat on the elevated X-maze by 5-HT3 antagonists injected into the basolateral amygdala. J Psychopharmacol 4:262Google Scholar
  92. Traber J, Glaser T (1987) 5-HT1A receptor-related anxiolytics. TIPS 8:432–437Google Scholar
  93. Tye NC, Iversen SD, Green AR (1979) The effects of benzodiazepine and serotonergic manipulations on punished responding. Neuropharmacology 18:689–695Google Scholar
  94. Upton N, Blackburn TP (1991) Anxiolytic-like activity of the selective 5-HT3 receptor antagonnist, BRL 46470A, in the rat elevated X-maze. Br J Pharmacol 102:253PGoogle Scholar
  95. VanderMaelen CP, Matheson GK, Wilderman RC, Patterson LA (1986) Inhibition of serotonergic dorsal raphe neurones by systemic and iontophoretic administration of buspirone, a non-benzodiazepine anxiolytic drug. Eur J Pharmacol 129:123–130Google Scholar
  96. Van Wijngaarden IK, Thulp MThM, Soudin W (1990) The concept of selectivity in 5-HT receptor research. Eur J Pharmacol 188:301–312Google Scholar
  97. Viana MB, Tomaz CAB, Graeff FG (1992) Effect of diazepam in rats placed on the elevated T-maze, a possible experimental model of anxiety and memory. In: VII Annual Meeting of FESBE (Caxambu, Brazil, Abstr, p 49)Google Scholar
  98. Whitton P, Curzon G (1990) Anxiogenic-like effect of infusing 1-(3-chlorophenyl) piperazine (mCPP) into the hippocampus. Psychopharmacology 100:138–140Google Scholar
  99. Winslow JT, Insel TR (1991) Serotonergic modulation of the rat pup ultrasonic isolation call test; studies with 5-HT1 and 5-HT2 subtype selective agonists and antagonists. Psychopharamcology 105:513–520Google Scholar
  100. Wright IK, Heaton M, Upton N, Marsden CA (1992) Comparison of acute and chronic treatment of various serotonergic agents with those of diazepam and idazoxan in the rat elevated X-maze. Psychopharamcology 107:405–414Google Scholar

Copyright information

© Springer-Verlag 1993

Authors and Affiliations

  • S. L. Handley
    • 1
  • J. W. McBlane
    • 1
  1. 1.Pharmaceutical Sciences InstituteAston UniversityGosta GreenUK

Personalised recommendations