Introduction

Throughout his scholarly career, Mike Domjan has championed the idea that learning mechanisms are adaptations to ecological conditions. His contributions to the understanding of conditioned taste aversion and neophobia in rats (Domjan, 1977) and subsequently to sexual conditioning in quail (Domjan, 1990) illustrate his evolutionary and comparative approach to research on associative learning (see Domjan et al., 2012). Domjan (1994) also took this approach to a more theoretical level with the development of a behavior system for sexual conditioning that summarized years of research into an adaptive framework (see Domjan & Gutiérrez, 2019). The research on frustrative nonreward reviewed in this article has historically followed a different path, mainly derived from Amsel’s (1992) frustration theory. In this case, the emphasis has been on the “how question” (mechanism), including developmental and comparative research (Amsel & Stanton, 1980; Papini, 2002), without much reference to the “why question” (adaptive function). The adaptive-function hypothesis advanced here aims to specifically address the “why question,” and rests on the notion that frustrative nonreward activates an aversive emotional state. Thus, our first goal is to review the evidence supporting the hypothesis that surprising reward omissions and devaluations trigger an emotional response of frustration, while emphasizing mechanistic aspects of frustrative nonreward. Then, we address the issue of the potential adaptive significance of such an emotional response to surprising reward omissions. We suggest that the adaptive function of frustrative nonreward is linked to the ability of this aversive emotion to break an attachment to a rewarding site or stimulus, an idea we call the incentive-disengagement hypothesis, following Klinger’s (1975) suggestion.

In this article, frustrative nonreward refers to the aversive emotional state generated by significant and surprising reward downshifts. Reward refers to an incentive, a stimulus that animals are willing to work to obtain, whether in qualitative (type of reward) or quantitative terms (magnitude of reward). Nonreward refers to the omission of a reward or to the presentation of a less preferred (qualitative) or reduced (quantitative) reward relative to what was presented before the downshift. Animals might detect a reward omission or devaluation without experiencing frustrative nonreward when the difference between obtained and expected rewards is below a theoretical threshold. Reward omissions or devaluation must also be surprising or unexpected, that is, they should occur in the presence of signals or situational cues previously associated with the expected reward.

Reward expectancies and behavior

An adult mammal foraging for food is likely to use prior experience to find its way around. To the extent that experience plays a role in foraging decisions, behavior would be guided by expectancies about the locations that are likely to yield rewards and by expectancies about the type of food that should be found there. Several lines of research provide evidence for reward expectancies, but here we concentrate on the successive negative contrast (SNC) effect (Flaherty, 1996; see Table 1). SNC was first reported in the instrumental behavior (iSNC) of rats running in runways and complex mazes (Crespi, 1942; Elliott, 1928), as well as in the consummatory behavior (cSNC) of monkeys that showed rejection of a downshifted reward in a free-choice task (Tinklepaugh, 1928). The SNC phenomenon is triggered by a negative discrepancy between the small reward actually found and the large reward signaled by current stimuli or contextual cues. Thus, a rat can learn to find sunflower seeds in the goal box of a complex maze, but this reward would disrupt the appetitive behavior of another rat that had already learned to locate a much-preferred wet cereal in the goal box of the same maze (Elliott, 1928). Experiments confirm that instrumental behavior of the kind involved in foraging for food is guided by specific and detailed expectancies of the rewards to be found (e.g., Trapold, 1970; Urcuioli, 2005).

Table 1 Some of the behavioral effects mentioned in the article
Full size table

SNC demonstrates that a rat foraging for food can encode information about the specific properties of the reward found in a given location, such that changes in the quality, quantity, value, or even stimulus properties of the reward can affect behavior and decision making. What, then, are the consequences of violating such reward expectancies?

Adjustments to expectancy violations

The encoding of a reward expectancy is necessary to experience a negative discrepancy between obtained and anticipated rewards. Detecting a negative discrepancy sets in motion one of two possible mechanisms for adjustment to the change in reward conditions. One possibility is to update behavioral strength to match the value of the new reward, a cognitive mechanism usually referred to as prediction error (e.g., Yau & McNally, 2018). For example, rats trained to run down a runway to collect a large sucrose reward developed a higher running speed than rats rewarded with a small sucrose reward, which shows that the two magnitudes had differential control over behavior. Subsequently, a downshift in reward magnitude led to a gradual adjustment of runway performance to the level supported by the small reward (Sastre et al., 2005; see also Pellegrini & Papini, 2007). Similar gradual adjustments are observed in rats treated with chlordiazepoxide, a benzodiazepine anxiolytic, and trained in a runway situation (Rosen & Tessel, 1970) and in rats with lesions in the insular cortex trained in a consummatory situation (Lin et al., 2009). This pattern of responding characterized by differential performance before the reward downshift and by a gradual behavioral change after reward downshift is referred to as a reversed SNC effect (Papini, 2014). Therefore, a detected prediction error leads to a behavioral adjustment to the absolute value of the new reward that reduces the predicted error (Papini, 2022).

The detection of a negative discrepancy can also activate an emotional response that influences the behavioral adjustment to the change in reward conditions. This emotional mechanism reveals that the new reward is appraised in relative terms, not for what it is worth in absolute terms, but in relation to the value of the expected reward (Flaherty, 1996). For example, in the same experiment with rats described previously (Sastre et al., 2005), the same animals that exhibited a reversed iSNC effect in the running response also displayed a different pattern of behavior in terms of licking responses to the sucrose reward in the goal box – a cSNC effect. A few seconds after starting to lick the devalued sucrose solution, rats stopped licking and moved around the goal box, leading to a lower licking frequency than that of an unshifted sucrose control. This cSNC effect is observed in the consummatory behavior of animals exposed to sucrose solutions or solid food rewards (Pellegrini & Mustaca, 2000; Vogel et al., 1968). The disruption of licking after a sucrose downshift is observed even if the rat is restrained and prevented from moving about the environment (Lopez Seal et al., 2013). Therefore, rats can respond to an event involving reward downshift in terms of either the absolute or the relative value of the downshifted reward (Papini, 2022). Rats respond in terms of absolute value when their behavior gradually adjusts to the absolute value of the new reward (e.g., reversed SNC), while rats respond in terms of relative value when their behavior exhibits an exaggerated response governed by the relative value between the old and new rewards (e.g., SNC). These two outcomes are illustrated in Fig. 1 with data from rats and toads.

Fig. 1
figure 1

a In rats, a downshift from a large (L; wet cereal) to a small (S; sunflower seeds) reward, marked by the vertical dashed line, leads to a deterioration of behavior detected in terms of an increase in the number of errors in a complex maze. The difference in performance between the groups after the L-to-S downshift, when all animals are receiving the same reward, is known as successive negative contrast (SNC). Data from Elliott (1928; public domain). b In toads (Rhinella arenarum), an analogous downshift from a large (L; access to distilled water promoting extensive rehydration) to a small reward (S; access to a hypertonic solution promoting limited rehydration) leads to a gradual adjustment of runway behavior. This reversed SNC effect is characterized by a preshift difference in performance to the two rewards combined with a gradual adjustment of behavior after an L-to-S downshift. Data from Muzio et al. (2011; reproduced with permission from PLoS ONE). Photos courtesy of S. Guarino (rat) and R. Muzio (toad)

Full size image

Reward downshift and frustration

Behavioral evidence

Behavioral evidence points to a connection between SNC and an aversive emotional response (for an extended review of behavioral evidence, see Papini et al., 2015). For example, reward downshift promotes behaviors that facilitate escape from the context in which the downshift occurred – the escape-from-frustration effect (Daly, 1974; Norris et al., 2009), and impairs aggressive (Mustaca, Martínez, & Papini, 2000b) and sexual responses (Freidín & Mustaca, 2004). An event involving an unexpected reward reduction also leads to reduced sensitivity to pain (Jiménez-García et al., 2016; Mustaca & Papini, 2005). In turn, both peripheral pain (Ortega et al., 2011) and restraint stress (Ortega et al., 2013) enhance the cSNC effect.

Endocrine evidence

There is extensive evidence showing that SNC activates the hypothalamic-pituitary-adrenal axis. For example, rats exposed to a 32%-to-4% sucrose devaluation showed increased plasma levels of corticosterone and adrenocorticotropic hormone compared to unshifted controls (Flaherty et al., 1985; Mitchell & Flaherty, 1998; Pecoraro et al., 2009). Likewise, the increased response variability observed after reward downshift seems to be dependent on the levels of circulating corticosterone; such increased variability was absent in adrenalectomized rats (Pecoraro et al., 2005), as was the usual increase in response strength after reward omission (Thomas & Papini, 2001). Similar stress-related hormonal responses have been found in consummatory and instrumental situations involving reward omission (e.g., Kawasaki & Iwasaki, 1997; Romero et al., 1995).

In addition, the administration of the stress hormone corticosterone immediately after the first experience with a 32%-to-4% sucrose downshift retarded the subsequent behavioral adjustment to the 4% sucrose reward prolonging consummatory suppression (Bentosela et al., 2006; Ruetti et al., 2009). This effect was not present in animals receiving an 8%-to-4% reward devaluation, suggesting that the magnitude of the reward disparity determines whether animals experience negative emotion, as indexed in terms of the differential effects of corticosterone administration (Papini et al., 2015). Thus, the spontaneous release of stress hormones during SNC (see previous paragraph) may strengthen the aversive memory of the emotions triggered by reward failures.

cSNC is also influenced by testosterone treatment. Testosterone has complex effects on behavior, including attenuating anxiety under some test conditions (Domonkos et al., 2018). Pretraining administration of this hormone (either before the start of the consummatory task or 30 min before preshift and postshift sessions) attenuated the cSNC effect in terms of absolute and relative (preshift vs. postshift) sucrose consumption (Justel et al., 2011; Justel et al., 2012). The fact that testosterone also increased exploratory behavior in the central area of an open field, along with evidence indicating that the effects of incentive downshift in male rats are attenuated by a pretrial opportunity to ejaculate (Freidin et al., 2005), suggest an anxiolytic-like effect of testosterone in the adjustment to incentive downshifts.

Pharmacological evidence

The pharmacology of SNC reveals its predictive validity to assess the anxiolytic (GABAergic-dependent) and analgesic (opioid- and cannabinoid-dependent) effects of drugs (for an extended review of the pharmacology of SNC, see Ortega et al. (2017), Papini et al. (2015), and Torres and Papini (in press)). Particularly relevant are studies involving the administration of drugs with mechanisms of action related to GABAergic neurotransmission. Barbiturates, for example, reduce SNC when administered before either the first or the second downshift session (Flaherty et al., 1982). By contrast, benzodiazepine anxiolytics and alcohol reduce consummatory suppression induced by reward downshift when administered before the second postshift session, but not when injected before the first downshift session (e.g., Becker & Flaherty, 1982; Flaherty et al., 1990; Flaherty et al., 1986; Ortega et al., 2014), as illustrated in Fig. 2. Exposing animals to repeated reward devaluation cycles or increasing the time of access to the initial experience with the downshifted reward elicits an anxiolytic effect from these drugs during the initial downshift event (Flaherty, 1996; Flaherty et al., 1986; Flaherty et al., 1996). This suggests that benzodiazepines act only after the animal has had some experience with the downshifted reward, a fact consistent with the development of an approach-avoidance conflict after initial exposure to the downshifted reward: drink the downshifted solution versus reject and look elsewhere for the preshift solution (Flaherty, 1996).

Fig. 2
figure 2

a The benzodiazepine chlordiazepoxide (CDP) does not interfere with the cSNC effect on the first downshift trial (trial 11), but it eliminates the cSNC effect in administered before the second downshift trial (trial 12), relative to saline (Sal) controls. Data from Ortega et al. (2014), reproduced with permission from Elsevier. b Activation of the central amygdala with clozapine N-oxide (CNO, after infusion of inhibitory designer receptors exclusively activated by designer drugs, DREADDs) during reward downshift (postshift) eliminates the cSNC effect relative to controls also infused with DREADDs, but treated with the vehicle (Veh) rather than with the DREADD activator, CNO. Data from Guarino et al. (2020b), reproduced with permission from Elsevier. The arrows point to the key comparison in each experiment

Full size image

Further evidence for an emotional component of the SNC derives from studies providing access to anxiolytic drugs for voluntary consumption after having experienced an unexpected reward downshift event. In one experiment, Wistar rats were exposed to 32% sucrose for ten sessions, each one immediately followed by a 2-h test with access to alcohol (2%), chlordiazepoxide (1 mg/kg), or water in a free-choice preference test. Animals increased alcohol and chlordiazepoxide intake after 32%-to-4% sucrose downshift sessions, an effect that was not observed in unshifted groups (always receiving access to 4% sucrose) and in downshifted and unshifted groups exposed only to water during the preference test (Manzo et al., 2015a). Additional studies have replicated this phenomenon with a variety of alcohol doses (from 2% to 32%), animals (high vs. low emotionally reactive selected Roman rats), and experimental manipulations (instrumental and consummatory reward devaluation and omission) (Donaire et al., 2018, 2022; Manzo et al., 2014). The consistently observed increase in alcohol intake after reward downshift seemed to result from its anxiolytic properties, as registered in the hole-board test for anxiety administered after the preference test (Donaire et al., 2020). Interestingly, the augmented alcohol intake induced by reward loss was reduced or absent in animals receiving partial reinforcement training before experiencing reward omission (Manzo et al., 2015b) or devaluation (Donaire et al., 2022), and in rats with concurrent access to a wheel for voluntary running (Castejón et al., under review). These results suggest that animals are able to detect emotional activation and display self-regulatory behaviors to reduce emotional distress (Torres & Papini, 2016).

Studies aimed at assessing the neurobiological basis of these pharmacological effects found that the SNC is attenuated by local infusion of the benzodiazepine diazepam into the amygdala (Liao & Chuang, 2003). Interestingly, no such effect was observed when diazepam was infused into the hippocampus, a region that may not be involved in cSNC (e.g., Flaherty et al., 1998).

Neurobiological evidence

Another key component in the emotional characterization of SNC is the neurobiology that governs the behavioral outcomes of unexpected reward downshifts through a neural circuit of connected brain areas. A theoretical model consisting of various brain regions argues for several integrated circuits underlying frustration in the cSNC situation (Ortega et al., 2017). These circuits are located in both the brainstem, which governs the taste-licking modal action pattern, and the diencephalon-telencephalon, which processes inputs and modulates outputs.

At the diencephalic-telencephalic level, a specific circuit has been theorized to control the negative emotional components of frustration, and includes areas such as the gustatory thalamus, insular cortex, basolateral amygdala, central amygdala, the anterior cingulate cortex, and possibly the lateral habenula. The brain areas involved in this negative-emotion circuit were determined largely by lesion studies investigating how individual brain areas regulate specific behaviors.

The central amygdala has often been associated with the acquisition and expression of conditioned fear as well as the development of behavioral responses to stressful stimuli (Davis & Whalen, 2001; Gilpin et al., 2015). In the context of SNC, electrolytic lesions of the centromedial amygdala have been shown to eliminate consummatory suppression after a 32%-to-4% sucrose downshift, while leaving unshifted controls unaffected (Becker et al., 1984). Transient inactivation of the centromedial amygdala using lidocaine immediately after downshift in a runway paradigm also reduced runway latencies in subsequent trials (Salinas et al., 1993). Additional experiments have suggested a distinct emotional mechanism. Transient inactivation of the centromedial amygdala with lidocaine microinfusions was shown to reduce consummatory suppression after sucrose downshift (Kawasaki et al., 2015). However, lidocaine infusion had no effect on anticipatory negative contrast (ANC). ANC involves a two-bottle paradigm in which access to 4% sucrose is followed by either 32% or 4% sucrose in every session (Flaherty & Checke, 1982). After training, licking for the first bottle of 4% sucrose is lower than licking for the following bottle when the second bottle contains 32% sucrose, than when the second bottle delivers 4% sucrose. In essence, rats suppress licking for the 4% sucrose in anticipation of the more valuable 32% sucrose, an effect that is resistant to anxiolytics, suggesting the lack of a negative emotional component (Flaherty, 1996). Thus, the elimination of consummatory suppression in cSNC while maintaining suppression in ANC after lidocaine infusions into the centromedial amygdala indicates an emotional role of this area in frustration and not just an effect on consummatory suppression. Kawasaki et al. (2015) also observed increased locomotor activity in an open field after centromedial amygdala inactivation suggesting a reduced fear of open spaces and further supporting the role of the central amygdala in the emotional coding of events. Subsequent studies involving chemogenetic inactivation confirmed that the central amygdala has control over the suppression of consummatory behavior in the cSNC task (Guarino et al., 2020b; Fig. 2).

The lateral habenula is another brain area of note given its specific function in reward processing. The lateral habenula plays a role in establishing negative values to rewards (Friedman et al., 2011; Proulx et al., 2014). Dysfunction of the lateral habenula has even been linked to mood disorders, such as major depression (Hu et al., 2020). Furthermore, studies investigating the role of the lateral habenula in reward loss have found that lesions with quinolinic acid delayed the onset of appetitive extinction in both instrumental and consummatory tasks (Donaire et al., 2019). These lesions could have interfered with the ability to detect reward omission and send the appropriate signals to other brain areas regulating the adjustment to the new reward value, such as the ventral tegmental area and the nucleus accumbens (Nair et al., 2013). Therefore, it can be hypothesized that the lateral habenula must contribute to the onset of frustrative nonreward after reward downshifts by assigning a negative emotional valence to the new reward value.

The negative emotional circuit within the larger neural circuitry that governs SNC provides a neurobiological framework to understand the behavioral outcomes shown after reward downshifts. Whereas lesion studies involving individual brain areas in different contexts of reward loss provide a foundation for understanding their role in these tasks, future studies will need to focus on specific pathways that connect different brain areas. This will provide a more wholistic framework to explain frustrative nonreward from a neurobiological perspective.

The SNC in mammals

Generality

Most research on SNC using reward downshifts comes from experiments conducted with rats, but other mammalian species also exhibit SNC effects. For example, Tinklepaugh (1928) trained rhesus monkeys (Macacca sp.) using bananas as the high-value reward and lettuce as the low-value reward. Although the banana was the preferred reward, monkeys would still eat the lettuce. However, monkeys experiencing banana-to-lettuce downshifts displayed searching behavior, rejected the lettuce, and reacted aggressively toward the experimenter. Using similar procedures involving sucrose downshifts, studies with human babies reported similar SNC effects. Newborns given alternated access to two 5-min periods of water and two 5-min periods of 15% sucrose showed a lower sucking rate during water periods than newborns given water for a 20-min session (Kobre & Lipsitt, 1972). SNC involving sucrose consumption was also found in mice (Mus musculus). Mice exposed to a 32%-to-4% sucrose downshift consume significantly less 4% sucrose solution during the postshift phase than mice always given access to the 4% sucrose solution. Moreover, treatment with the benzodiazepine anxiolytic diazepam eliminated this cSNC effect in mice (Mustaca et al., 2000a). Two marsupial species, the red opossum (Lutreolina crassicaudata) and the white-eared opossum (Didelphis albiventris), were also tested in a SNC situation using 32%-to-4% sucrose downshifts. In both species, downshifted animals displayed consummatory suppression to the 4% sucrose solutions relative to unshifted controls receiving the 4% solutions throughout the experimental sessions (Papini et al., 1988). Red opossums also showed an increase in running speed during appetitive extinction in the Y-maze task (Papini & Ramallo, 1990), a result similar to the frustration effect observed in rats under analogous conditions (Amsel & Roussel, 1952; Thomas & Papini, 2001). Finally, Bentosela et al. (2009; Dzik et al., 2019; Jakovcevic et al., 2013) reported that dogs (Canis familiaris) trained to receive dry beef liver (a higher-value reward) learned to maintain a longer gaze on the experimenter than dogs trained to receive dog pellets (a lower-value reward). Also, dogs downshifted from dry liver to dog pellets rejected food more often than unshifted dogs, decreased their gaze durations, and withdrew from the experimenter, a pattern of behavior consistent with SNC effects. Taken together, this evidence supports the view that reward loss can have emotional significance for mammals.

Behavioral displays and emotion

Changes in behavioral postures, facial and bodily expressions, and vocalization patterns have been identified as potential measures of emotion in a variety of mammals exposed to unexpected environmental changes. This information is useful for two reasons: first, because it may suggest SNC experiments with new species and therefore broaden the comparative database for mammals, and second, because species differences in behavior might highlight the adaptive value of frustrative nonreward.

For example, 2- to 8-month-old babies exhibited facial expressions associated with interest and joy when movement of their arms activated an audiovisual stimulus, but expressions related to frustration and anger when arm movements no longer activated the audiovisual stimulus (Lewis et al., 1990). Infants trained to kick a foot to move crib mobiles containing six or ten identical objects displayed increased kicking, decreased attention, and increased negative vocalizations when the number of objects was reduced to two (Mast et al., 1980).

Studies with terrestrial mammals suggest the presence of displays in response to stimuli that may have emotional significance. For example, dogs (Canis familiaris) display asymmetric tail wagging in the presence of different stimuli, such as right-side biased tail movements in the presence of their owner, but left-side biased tail movements in the presence of an unfamiliar conspecific or when tested alone (Quaranta et al., 2007). Moreover, dogs showed increased heart rate and anxious behavior when shown video images of conspecifics exhibiting left-side tail wagging relative to right-side tail wagging (Siniscalchi et al., 2013). In pigs, tail wagging is associated more often with rewarding situations than with aversive situations (Reimert et al., 2013, 2017). For example, pigs (Sus domesticus) displayed longer tail movement in the presence of enrichment devices than in their absence (Marcet-Rius et al., 2018). By contrast, changes in ear posture seem to be associated with increased negative emotions or decreased positive emotions in pigs (Reimert et al., 2013). The ear postures of sheep (Ovis arie) showed consistent changes in situations involving sudden changes, stimulus familiarity, and controllability (Boissy et al., 2011). Sheep maintained their ears in a horizontal posture in a neutral state, but they pointed their ears backward in unpleasant, uncontrollable, unfamiliar situations (fear-eliciting situations), upward in negative but controllable situations (anger-eliciting situations), and displayed asymmetric ear posture in situations involving unexpected, sudden changes (surprising situations).

Research with aquatic mammals also provides evidence of displays that can be associated with emotion. In dolphins, jaw clapping, pectoral flipper, tail, and head slapping, and S-shaped body postures have been suggested as indicators of frustration, anger, and aggression, whereas pectoral fin rubbing and slow-pace synchronized swimming have been observed in association with positive situations and identified as indicators for social bonding and affiliative interactions (see Delfour & Charles, 2021). However, observations of swimming patterns (i.e., circular, synchronous, contact, and social swimming) and speed, two behavioral patterns used to measure emotional states in dolphins, have produced ambiguous results. For example, swimming that is circular, fast, and social increases in positive social contexts and decreases in negative situations (Serres et al., 2020), although fast swimming has been observed both in stressful situations and in social play involving high energy behaviors (Delfour & Charles, 2021). Finally, bottlenose dolphins (Tursiops truncatus) remained farther away from negative-valence stimuli when they were presented in their left visual field, rather than in the right visual field (Delfour & Charles, 2021).

Observations of displays suggesting emotion in various mammals raises testable questions in the context of reward downshift research. Would dogs display tail-wagging asymmetries or sheep change ear postures when exposed to unexpected reward downshifts? Would pigs show increased ear movement and decreased tail movement? Would dolphins show swim patterns and behavioral displays associated with negative social contexts? Would dolphins and other mammals show visual lateralization in response to reward devaluation? Using these and similar behavioral indicators in the context of the SNC task or related behavioral paradigms involving unexpected reward downshifts will provide insights into the underlying mechanisms and brain processes. Such data will shed light on the variety of behavioral adjustments to reward loss in mammals and their possible adaptive significance.

Adaptive significance: The incentive disengagement hypothesis

Evolutionary background

The behavior systems framework assumes that the organization of behaviors connected to a specific function (e.g., feeding, sexual reinforcement, defensive behavior) originates in the adaptive benefits conferred to ancestors in a given lineage (Silva & Silva, 2022). Although reproductive success is difficult to estimate in learning experiments (see Gutiérrez & Domjan, 1996; Hollis et al., 1997), an educated guess about the evolutionary origin of a behavior system is still possible. As a starting point, consider the parallels between frustrative nonreward and fear learning. Fear is usually aligned to self-preservation in predatory encounters and intraspecific contests (Crump et al., 2020). The evolution of bodily structures associated with a dynamic lifestyle in early chordates (e.g., the tail and paired lateral muscles of Cambrian chordates, such as Pikaia and Yunnanozzon; McMenamin, 2019) and with a defensive function (e.g., the armors of Paleozoic agnathan and placoderm fish; Randle & Sansom, 2019) suggest that predatory pressures were intense. The neural mechanisms supporting the protective emotion of fear likely evolved as part of an antipredatory suite of characters.

The hypothesis that fear mechanisms evolve in early vertebrates as a protective emotion is consistent with the conservation of fear-related learning phenomena and brain mechanisms. For example, goldfish (Carassius auratus) and zebrafish (Danio rerio) rapidly acquire avoidance behavior and contextual fear conditioning, exhibit extinction when shocks are discontinued, and show modulation by factors that also influence mammalian learning under similar conditions (e.g., such as shock intensity and blockage of NMDA receptors); this learning depends on the dorso-medial telencephalon (Kenney et al., 2017; Lal et al., 2018; Overmier & Hollis, 1983; Overmier & Papini, 1986; Portavella et al., 2004). The dorso-medial telencephalon is considered homologous to parts of the mammalian amygdala (Northcutt, 2006), which is central to an understanding of fear and avoidance conditioning in mammals (Cain, 2019; Janak & Tye, 2015).

In goldfish, which exhibit rapid acquisition of active avoidance when training involves a single trial per day (Portavella et al., 2003), evidence of behavioral effects related to frustrative nonreward under analogous training conditions has been difficult to obtain. For example, reward downshift leads to a reversed SNC effect in goldfish (Couvillon & Bitterman, 1985; Lowes & Bitterman, 1967). A similar pattern of results involving evidence of avoidance learning and reversed SNC has been described in amphibians (Daneri et al., 2007; Muzio et al., 2011). This selective review is leaving many important points untouched (for more complete discussions, see Papini, 2003, 2014, 2021), but the conclusion still stands that unlike the case with fear, the selective pressures encouraging the evolution of the brain mechanisms underlying frustrative nonreward must have been weak in the Paleozoic environments of early vertebrates. Therefore, the frustration circuitry must have evolved after that of fear under more salient environmental pressures of Mesozoic vertebrates.

Incentive disengagement

High energy demands and activity levels in early mammals provided the pressure to evolve mechanisms for a rapid switch from previously successful appetitive behaviors to exploratory patterns increasing the chances of finding new sources of food. Stout et al. (2002, p. 255) suggested that the adaptive function of the brain mechanisms supporting frustration is to facilitate a switch “from previously successful responses that no longer work, to new responses that may bring the animal in contact with needed resources.” Switching away from previously successful responses requires a mechanism that would facilitate detachment, a notion similar to Klinger’s (1975) incentive disengagement mechanism.

Klinger (1975) described incentive disengagement as a cycle, which includes invigoration, aggression, depression, and recovery. Behavioral invigoration, especially relevant in the present context, occurs in several situations following surprising reward omissions, but especially when previously rewarded alternative behaviors are available. In the original demonstration, Amsel and Roussel (1952) reinforced rats to run in each of two runways arranged in tandem. Rats ran faster in the second runway following reward omission in the first runway. Variations of this effect with added controls demonstrate that response invigoration is in part due to the surprising omission of the reward (Dudley & Papini, 1995; Stout et al., 2003) and also that invigoration is eliminated by adrenalectomy (Thomas & Papini, 2001), a fact suggesting an aversive emotional state. Moreover, rats trained to press two levers paired with either a large or a small reward prefer the large-reward lever in a free-choice test. However, a reward downshift in the large-reward lever is accompanied by a switch in preference to the small-reward lever that remained unshifted throughout training – a phenomenon called Pavlovian SNC (pSNC; Conrad & Papini, 2018; but see Guarino et al., 2020a). A reward downshift in one lever either invigorates behavior toward the unshifted lever or revalues the alternative offered by the unshifted lever. In the absence of an alternative response, reward downshift leads to an increase in exploratory activity whether in the conditioning box, open field, or radial-arm maze (Flaherty, 1991; Flaherty et al., 1978; Pellegrini & Mustaca, 2000). Frustrative nonreward might also play a role in the behavioral response to situations involving relief after exposure to aversive reinforcers. For example, in a one-way avoidance task, a surprising reduction in safety time deteriorates performance (Cándido et al., 1992), and this effect is eliminated by treatment with benzodiazepine anxiolytics (Torres et al., 1994). Surprising nonreward conditions are also introduced when “test” trials are presented under nonreinforced conditions. For example, in the Morris water maze task, after learning to locate a hidden platform allowing a rat to emerge from the water, the platform is removed and the time spent in the same quadrant is recorded. Rats trained under continuous reinforcement (platform access on every trial) spent less time in the same quadrant than rats trained under partial reinforcement (Prados et al., 2008), a result consistent with frustrative nonreward.

The failure of behavioral invigoration to produce rewards sets the conditions for incentive disengagement, leading to other phases of the cycle. For example, under some conditions, unexpected reward downshift leads to an increase in aggressive behaviors (Gallup Jr., 1965) or a suppression of aggressive behavior that could reflect a depression-like state (Mustaca et al., 2000b). The cSNC task is specifically characterized as involving a transient emotional state of frustration with a distinct recovery profile with the downshifted reward (Flaherty, 1996). Overall, reward downshift leads to frustrative nonreward and a series of consequences that fit the adaptive scenario outlined previously, eventually leading to the activation of search behavior (Papini & Dudley, 1997). We suggest that frustration promotes incentive disengagement aimed at breaking an attachment to a site that used to provide rewards but has become devalued or empty, whereas the search component aims at increasing chances of finding alternative sources of reward (Papini, 2003).

Selective pressures, exaptation, and frustration

In evolutionary theory, articulating an adaptive explanation requires two components: pre-existing conditions that provide an initial bias for a trait to acquire a new function – referred to as exaptation (Gould & Vrba, 1982; traditionally called preadaptation) – and selective pressures that co-opt the trait and further adjust it to its new function. Consider first a possible exaptation. The large olfactory bulbs of Mesozoic mammals suggest they relied on olfaction for foraging, probably because they were mostly nocturnal (Jerison, 1973; Kielan-Jaworowska, 1986; Striedter, 2005). Whereas it is not possible to observe the internal brain circuitry in fossil specimens, basal living mammals exhibit direct connections between the olfactory system and limbic structures involved in incentive downshift, including the amygdala (e.g., Guarino et al., 2020b). For example, the size of the olfactory bulbs in living insectivores (generally considered basal placentals, although their taxonomy is complex; e.g., Madsen et al., 2001) correlates positively with the size of the centromedial amygdala, but not with nonlimbic structures, such as the vestibular and cochlear nuclei (Barton & Harvey, 2000). Olfactory inputs connect profusely with additional brain areas involved in emotion, including the hippocampus, orbitofrontal cortex, piriform cortex, and anterior cingulate cortex (e.g., Krusemark et al., 2013). Olfactory inputs into regions involved in emotion would thus provide the first indication that the expected rewards are either devalued or absent.

A potential selective pressure favoring incentive disengagement based on frustration in a foraging context relates to a number of novel characters that appeared in cynodonts, the Mesozoic precursors of mammals (Ruta et al., 2013). These characters seem to be linked to a stronger demand for energy intake. Cynodonts and early mammals exhibited differentiated teeth, suggesting a more elaborate processing of food (e.g., slicing, crushing, chewing); legs located underneath the trunk, rather than extending to the side as in most reptiles, suggesting higher levels of activity and enhanced oxygen inflow linked to higher metabolic levels; hair, suggesting a greater control over the internal temperature (e.g., endothermy); and larger relative brain size (Striedter, 2005). The energy demands of a high metabolic rate and large brain in Mesozoic mammals relative to their reptilian ancestors may have been in part responsible for the evolution of brain mechanisms supporting rapid detachment from devalued sources of food.

A dissociation of the mechanisms underlying fear and frustration in basal vertebrates, together with the well-known overlap in mechanisms underlying these two forms of emotional learning in mammals (Gray & McNaughton, 2000; Papini et al., 2015; Wagner, 1969), suggest an intriguing evolutionary hypothesis (Papini, 2003). The mechanisms underlying frustration could be seen as evolving from those underlying fear by gene co-option, so as to play a role in the context of foraging (see Fig. 3). Gene duplication and co-option have been postulated to explain such cases as the quadruplication of Hox genes in vertebrates, the evolution of feathers in birds, and the evolution of the enzymes involved in the development of the eye’s lenses (McLennan, 2008). Genes expressed during fear conditioning (e.g., Lori et al., 2019) and reward downshift tasks (e.g., Sabariego et al., 2013) are beginning to be identified. This evolutionary hypothesis suggests that there should be substantial overlap in gene expression during exposure to situations inducing fear and frustration. It remains to be determined whether such gene co-option leading to the evolution of incentive disengagement linked to frustration occurred only once in some mammalian ancestor. As discussed next, it could have occurred more than once given the pervasive nature of fear in vertebrates.

Fig. 3
figure 3

A hypothesis of the evolutionary connection between fear, frustration, and grief. Incentive disengagement based on frustration may have evolved in Mesozoic mammals or their direct ancestors in the context of foraging. These mammals, as well as living basal mammals (monotremes, marsupials, insectivores, etc.) tend to be solitary. No evidence of grieving for social losses has been described in these mammals. Highly social mammals (e.g., primates, cetaceans) display signs of emotional distress in cases of social separation or loss. Grieving mechanisms may have evolved independently from solitary ancestors. Convergent mechanisms for incentive disengagement may have also evolved in some groups of birds in relation to foraging and social loss, but strong evidence is still lacking (hence the question marks). Lineages leading to living reptiles diverged very early in tetrapod evolution and, therefore, they are distantly related to both living birds and living mammals (see Papini, 2021)

Full size image

Frustration and grief

There is little evidence suggesting that Mesozoic mammals were social species, in contrast to fossil evidence suggesting some degree of sociality in some dinosaurs of the same period (e.g., Cotton et al., 1998; Horner, 1992). A Cretaceous multituberculate mammal, Filikomys primaevus, apparently nested and burrowed in multigenerational colonies, providing evidence consistent with sociality (Weaver et al., 2021). More complex social behavior evolved with placental mammals, but the mother-infant bond based on milk and protection has probably been a constant feature in mammalian evolution. Such bonds offer a scenario besides that of foraging where reward loss can be experienced in terms of the separation or death of one of the pair members.

Many mammalian species have been described as developing an emotional state resembling human grief after the death of an infant, including elephants, macaques, gorillas, chimpanzees, orcas, and dolphins, among many others (e.g., Anderson, 2017; Reggente et al., 2016; Sharma et al., 2019). Rasmussen and Reite (1982) reported the case of an adult female macaque (Macaca sp.) who lost her first offspring at 5 months of age and subsequently had a failed pregnancy. After the miscarriage, the macaque showed depressive symptoms, including refusal to eat and social isolation. The monkey was treated with the antidepressant amitriptyline and within a week showed significant improvement in feeding and social behaviors. After full recovery, eating, resting, and locomotor behaviors were measured as a baseline record and then, to further explore her behavior after a different incident of social loss, her closest conspecific companion was removed for 2 weeks. During this separation period, the female macaque exhibited a similar reduction of feeding and activity, and an increase in resting behavior. These behaviors returned to baseline levels when she and her companion were reunited.

Extensive research on social isolation and mother-infant separation in primates also shows the impact these procedures have on behavior, physiology, and health (e.g., Zhang, 2017). These consequences occur in the context of social attachments involving individual recognition and separation anxiety. Species forming monogamous bonds are also subject to similar effects following separation. In prairie voles (Microtus ochrogaster), which form stable monogamous pairs, Sun et al. (2014) tested the effects of social loss by separating male and female voles for 4 weeks. They found evidence of anxiety induced by partner loss behaviors in the elevated plus maze and light-dark box, and increased depressive behaviors in the forced-swim test.

Observations in the natural environment are likely biased toward cases of strong attachments that persist days after the infant has died. An example that captured the attention of the general public was that of an orca (named “j35”) observed carrying a dead infant for up to 17 days and about 1,000 miles, off the coast of British Columbia, Canada, in July-August 2018 (https://www.whaleresearch.com/j35). Most people can empathize with the response of this mother to the loss of her infant. This behavior by the mother would seem to disprove the incentive disengagement hypothesis advanced here, which predicts a rapid readjustment of behavior that minimizes the disruption of vital functions (e.g., feeding, social interactions). Indeed, animal grieving behavior has been considered maladaptive from an evolutionary perspective (Archer, 2001). We argue here that these recorded instances of grief are likely isolated cases, rather than the species-typical response to a social loss. Interestingly, the website tracking this orca mother informs that she was observed “vigorously” chasing a school of salmon with other orcas after the dead body of the infant had been abandoned. But there surely are extensive individual differences in coping with a social loss of this magnitude. The same website informs that 75% of newborn orca calves have died in the last two decades, suggesting that loss events are more common that one would wish, given that this population of orcas has been classified as “endangered.” Measurable individual differences have been described in both fear extinction (Galatzer-Levy et al., 2013) and recovery from reward downshift (Papini et al., 2014). A relatively small percentage of animals expresses difficulty in either extinguishing freezing after fear conditioning or recovering normal levels of consummatory behavior after reward downshift, whereas the majority of animals show appropriate adjustments in behavior. Thus, it seems possible that the modal behavior during grieving is to break the attachment and switch to other behavioral functions. The mechanisms for incentive disengagement may have been co-opted from the original function in the context of foraging in relatively solitary mammals, to the realm of social loss in mammals that evolve more complex social behavior, such as primates and cetaceans (Papini, 2006). This is also represented in Fig. 3 as the evolution of a “novel” mechanism based on a pre-existing one – an exaptation.

A note on incentive disengagement in birds

Birds could offer a point of comparison to assess this adaptive hypothesis. Birds also evolved high activity levels, high metabolic rates, and relatively large brains (Brusatte et al., 2015). However, like most reptiles, their reliance on vision for foraging, courtship, and other functions may have biased them to evolve cognitive, rather than emotional, mechanisms for regulating the adjustment to situations involving frustrating nonreward (Stout et al., 2002). Still, from a behavioral perspective, birds pose a complex emotional puzzle. Papini et al. (2019) reviewed evidence of fear phenomena and mechanisms in birds showing similarities with mammalian fear. This review also cited evidence for behavioral phenomena potentially linked to frustration that have been reported in both birds and mammals, such as aggressive behavior in situations involving reward omissions and faster extinction after continuous, rather than partial reinforcement. But inconsistent results were also reported in terms of reward downshift. Whereas starlings (Sturnus vulgaris) provided evidence of cSNC after a downshift in reward quality (Freidín et al., 2009), pigeons (Columba livia) exposed to a downshift in reward magnitude yielded evidence of a reversed iSNC (Papini, 1997; see also Conrad et al., 2020; Pellegrini et al., 2008). These inconsistencies might reflect differences in terms of species (starlings vs. pigeons), reward parameters (quality vs. magnitude), or response parameters (consummatory vs. instrumental). Importantly, pigeons fail to exhibit response invigoration after surprising nonreward (Stout et al., 2002), a key component of the incentive disengagement hypothesis. Pigeons are known to increase responding in fixed-interval schedules in cycles that end in nonreinforcement, a phenomenon analogous to the frustration effect described previously (e.g., Amsel & Roussel, 1952). In a test of this hypothesis (Papini & Hollingsworth, 1998), pigeons did increase responding after nonreward when cycles were separated by short intervals, but the effect disappeared after longer intervals between cycles. Importantly, response invigoration was eliminated solely because key pecking increased after a rewarded cycle, a fact consistent with food-induced demotivation, rather than frustration-induced invigoration (see also Stout et al., 2002). In a second experiment using the peak procedure (Catania, 1970), peak performance was observed with a large number of nonreinforced trials, but not with occasional tests. When observed, peak performance was highest around the time scheduled for food delivered in training trials (i.e., timing), but no response invigoration was observed thereafter. Peak performance in pigeons was more consistent with timing and discrimination based on reduced generalization from training to test trials than with an emotional account based on frustrative nonreward (Papini & Hollingsworth, 1998). Clearly, more research is needed before the incentive disengagement hypothesis of frustration can be fully evaluated in birds; this possibility is acknowledged in Fig. 3 (see question marks).

Concluding comments

Substantial evidence suggests that mammalian behavior during episodes involving unexpected reward reductions or omissions is accompanied by an aversive emotional state traditionally referred to as frustrative nonreward (Amsel, 1992). Whereas knowledge of the mechanisms underlying frustrative nonreward is less detailed than that of other aversive emotions, such as fear, significant progress has been made due to the availability of experimental procedures that produce reliable behavioral effects. What is lacking is a similarly detailed vision of the potential adaptive significance of frustrative nonreward. In the spirit of Domjan’s (e.g., 1994) adaptive view of learning and behavior, we advance the hypothesis that frustration helps a mammal break an attachment to a site or stimulus that is no longer useful to procure resources such as food and fluid. Such detachment would encourage the search for alternative sources, thus minimizing the potential for starvation given the high energy demands of mammals. The explanation offered by the incentive disengagement hypothesis is consistent with several key aspects of mammalian behavior in reward downshift situations, but its application to other groups of vertebrates, especially to birds, has produced conflicting results. Future studies within an evolutionary-comparative framework, bringing new species and procedures into focus, will provide answers to the questions raised in this article.