Introduction

Attention deficit hyperactivity disorder (ADHD) is a common neurodevelopmental disorder characterized by elevated levels of inattention, overactivity and impulsivity that impair children’s daily functioning across settings (American Psychiatric Association 2013). The etiology of the disorder remains uncertain. However, altered sensitivity to reinforcement has been hypothesized to contribute to symptoms of ADHD (see Tripp and Wickens 2008; Luman et al. 2010 for reviews). Several studies have demonstrated a stronger preference for small immediate over larger delayed rewards in children with ADHD, compared to typically developing children (see Sonuga-Barke et al. 2008). Other studies have shown increased sensitivity to the performance enhancing effects of reward (Luman et al. 2008; Huang-Pollock et al. 2012; Dovis et al. 2012; Bubnik et al. 2015), reduced performance under partial reinforcement schedules (Douglas and Parry 1994; Barber et al. 1996; Aase and Sagvolden 2006; Frank et al. 2007) and increased associative learning when feedback is continuous (Luman et al. 2015) or paired with reward (Luman et al. 2009) in children with ADHD.

Response allocation studies indicate children with ADHD do not match their behavior to the available rewards as closely as typically developing children (Kollins et al. 1997a; Taylor et al. 2010). These studies evaluate the effect of reward delivery on subsequent behavior, with children expected to distribute their responses proportionally to the rates of reinforcement in operation during a task (Kollins et al. 1997b). Unlike other choice tasks, where children are asked to select between response alternatives of known value (e.g., delay discounting tasks), in response allocation studies children are not informed of the reinforcement contingencies operating. Response allocation studies also differ from instrumental-learning tasks in that reinforcement is not being explicitly used to shape behavior (Hauser et al. 2014; Chantiluke et al. 2015) or improve performance (Luman et al. 2009). In these studies, participants often remain uncertain of reward availability across response alternatives throughout the task (e.g., Alsop et al. 2016).

Two recent response allocation studies (Alsop et al. 2016; Furukawa et al. 2017) used signal-detection methodology to evaluate the sensitivity of children with ADHD to changes in reward availability. Children with ADHD are known to experience difficulty adjusting their behavior according to situational demands (Douglas and Peters 1979; Barkley 1997) and this is reflected in current and previous diagnostic criteria (American Psychiatric Association 2013). Both Alsop et al. (2016) and Furukawa et al. (2017) reported the behavior of children with ADHD did not track the changing reinforcement contingencies as closely as their typically developing peers, when rates of reinforcement were low and there was uncertainty over the association between an action and its outcome. In the Alsop et al. (2016) study, the ADHD group did alter their response allocation following the change in reinforcement contingencies, but to a smaller extent than was seen in the control group. It is unclear whether the children with ADHD were less sensitive or slower in adapting to the change in reinforcer availability. It is possible their behavior would have shown greater change with increased exposure to the reinforcement contingencies.

Here we evaluate how the behavior of Brazilian children with and without ADHD responds to unequal reward frequency, and how it adapts when reward availability changes. We employed a variant of the signal-detection task described by Alsop et al. (2016) and Furukawa et al. (2017), in which the children received greater exposure to the reinforcement contingencies before and after these changed. In the current task, children were asked to indicate which of two stimuli were presented on a computer screen by pressing the appropriate response button. The two types of correct discrimination were associated with different rates of positive reinforcement, with correct responses of one type rewarded four times as often as the other. Such unequal arrangement of reinforcement typically produces a systematic preference for the more frequently reinforced response alternative (McCarthy and Davison 1981). The response contingencies reversed after delivery of 30 rewards (Phase 1), the task continuing until a further 30 rewards were delivered (Phase 2). In the earlier studies (Alsop et al. 2016; Furukawa et al. 2017), the children were exposed to only 20 rewards before the response contingencies switched, and there were two contingency changes.

Based on earlier signal-detection studies (Tripp and Alsop 1999; Alsop et al. 2016), we expected both groups of children to show a preference (response bias) toward the initially more frequently reinforced alternative. The extent to which the children’s behavior allocation would track the change in reinforcer availability, following increased exposure, was less clear. If the behavior of Brazilian children with ADHD adapts less well to the change in reward availability than controls, we would expect a smaller shift in their response bias when the response contingencies change. If they are simply slower to respond to a change in the reinforcement schedule, then increased exposure to the contingencies might lead to a similar overall shift in response allocation for the two groups.

The cultural background of the current sample (Brazilian) should not influence the children’s behavioral sensitivity to reinforcement, assuming basic neurobiological processes underlie such sensitivity. The majority of studies comparing reinforcement sensitivity in children with and without ADHD has been undertaken in Western Europe and North America. The small number of studies with other cultural groups, one in South Africa (Aase et al. 2006) and three in Asia (Masunami et al. 2009; Yu et al. 2015; Furukawa et al. 2017), suggest altered sensitivity to reward is a common characteristic of ADHD. On the other hand, increased reward seeking, irrespective of associated risks, has been identified in Latin American countries with a relatively recent history of migration and associated variation in the dopamine D4 receptor allele frequencies (Chen et al. 1999). The findings from the current study will further contribute to our understanding of the cross-cultural continuity of altered reward processing in ADHD.

Method

Ethical approval for the current study was obtained from the Comitê de Ética em Pesquisa do Hospital Universitário Gaffrée e Guinle and the Ethics Committee for Research Projects Analysis of Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo in Brazil.

Participants

Data from 72 children, 32 meeting DSM-IV diagnostic criteria for ADHD (81.3% boys) and 40 typically developing children (57.5% boys), are included in the study (Table 1). Within the ADHD group, nine children were diagnosed with inattentive type, three with hyperactive/impulsive type and 20 with combined type ADHD; one child had comorbid oppositional defiant disorder and another child was diagnosed with dyslexia; 11 were prescribed stimulant medication which was discontinued for at least 24 h prior to study participation.

Table 1 Participant characteristics
Full size table

Inclusion criteria for the study were an estimated IQ of at least 70, normal or corrected vision, no past or current head injury, neurological disorder or psychosis. Children in the ADHD group were recruited through university neurology clinics, where they completed multi-method, multi-informant diagnostic assessments. Data from semi-structured diagnostic interviews [K-SADS-PL, Disruptive Behavior Disorder section (Kaufman et al. 1997; Brasil and Bordin 2010) in Rio de Janeiro; the clinic’s own interview procedure following the DSM-IV criteria in São Paulo], parent and teacher completed rating scales for ADHD symptoms [SNAP (Swanson 1995; Mattos et al. 2006)] and observations of the child’s behavior were used to make a clinical diagnosis of ADHD. Parent and teacher completed broadband rating scales [CBCL/TRF (Achenbach and Rescorla 2001; Bordin et al. 2013)] and background questionnaires screened for other behavioral and emotional problems, neurological and medical conditions. Cognitive functioning was assessed with the WISC-III (Wechsler 1991; do Nascimento and de Figueiredo 2002).

Assessments were carried out by a team including a licensed clinical psychologist, neurologists experienced in working with children with ADHD and supervised medical students at both sites. All assessments were overseen by EMC in Rio de Janeiro and EBC in São Paulo, EF consulted with both teams. Children were required to display six or more symptoms of inattention and/or hyperactivity/impulsivity in at least one setting (e.g., home, school) and functional impairment from symptoms, based on the ratings and semi-structured diagnostic interviews with a parent. Evidence of some symptoms in a second setting was checked during the parent interview, using teacher reports when available, and observations during the assessment. Symptoms were not summed across informants. The diagnosis of ADHD was applied conservatively as the two public university clinics served diverse populations, presenting with a variety of behavioral, emotional and cognitive difficulties. Whenever the comprehensive assessment for ADHD could not confirm an ADHD diagnosis, or exclude other explanations for symptoms, these cases were screened out. As a result, comorbidity rates are low.

Control group children were recruited through invitation letters sent home to parents through public schools, a public after-school program and by the students working at the data collection sites. These children completed an abbreviated IQ assessment (WISC-III Vocabulary/Block Design). Their parents and teachers completed the behavior rating scales, which were used to rule out the presence of ADHD or other significant behavioral or emotional disorders. Those demonstrating fewer than four symptoms of inattention or hyperactivity/impulsivity were included. The sample included two children, for whom the responses of a parent yielded a mild elevation on the CBCL Anxious-Depressed scale (T-score = 70 and 72),Footnote 1 no other indications to suggest psychiatric problems were reported for any other control subjects. Parent and teacher reports were reviewed for other inclusion criteria.

Independent t tests indicated the mean age was older (t = 4.45, p < .001) and estimated IQ was lower (t = − 3.05, p < .01) in the ADHD group, compared to the control group (Table 1). The percentage of males in the ADHD group was significantly higher than in the control group (χ2 = 4.61, p < .05). The majority of children in both groups were from lower to middle income families.

Experimental task

The children sat approximately 400 mm from a flat-screen monitor with a two-button response panel in front of them. Instructions and a short demonstration of the procedure were presented on the screen and read aloud by the experimenter (see Supplemental Method for the exact instructions). The children were told that a checkerboard of red and blue cartoon characters would appear, and that they should indicate whether there were “more red” or “more blue” characters by pushing the appropriate button. They were advised that correct responses onlysometimesearn rewards (tokens) that could be exchanged for a prize later. They were not informed of the reinforcement contingencies associated with the response alternatives or that the contingencies would change.

A multi-element reinforcement system was used to maximize reward effectiveness: the message “You won a token, Well done!” appeared on the screen (in Portuguese), together with a 3000 ms, randomly selected without replacement, animated cartoon; the experimenter gave verbal praise, e.g., “good” and placed a colored token in a clear plastic container next to the child. Following incorrect and non-rewarded correct responses, the screen was blank and the experimenter remained silent. All children received a prize at the end of the session irrespective of their performance on the task.

The experimental trials began following two practice trials. All trials began with a small animated cartoon character (juggler) in the center of the screen for 750 ms, which served as a fixation point and a warning that a stimulus was about to appear. The checkerboard of red and blue characters (10 × 10 arrays containing either more red or more blue faces in a ratio of 54:46) appeared 250 ms later and was presented for 2000 ms or until the child responded. If the child did not respond within 2000 ms, the screen went blank until a response was made. Correct responses scheduled for reinforcement received the consequences described followed by a 1000 ms inter-trial interval (ITI) before the next trial began. Correct responses not scheduled for reinforcement or incorrect responses were followed by the ITI. Successive blocks of eight trials contained an equal number of each array type, randomized within blocks.

The computer determined quasi-randomly which correct responses were reinforced. At the start of the session, for all participants, correct identifications of one stimulus (“more blue”) were rewarded four times more often than correct identifications of the other stimulus (“more red”). After the child had received 30 rewards with this distribution (Phase 1), the contingencies were reversed and correct identifications of “more red” stimuli were reinforced four times as often as correct identifications of the other stimulus until the child received another 30 rewards (Phase 2). Each successive block of 10 reinforcements contained 8 reinforcements for correct identification of one array type and 2 for correct identification of the other, randomized within each block. By reinforcing only some correct responses, the experimenters ensured each child obtained the same arranged distribution of reinforcers in each phase of the study (McCarthy and Davison 1984).

Data collection and analysis

On each trial, the computer recorded which stimulus was presented, which response was made, whether or not the response was reinforced and the child’s response time. Three measures of performance were calculated for each child: median response time, discriminability between stimuli (\( \log d \)), i.e., accuracy and response bias (\( \log b \)), i.e., the systematic preference for the more frequently reinforced alternative (McCarthy and Davison 1981).

Discriminability between the stimuli was calculated by the equation:

$$ \log d = \frac{1}{2}\log \left( {\frac{{{\text{Blue}}_{\text{Correct}} }}{{{\text{Blue}}_{\text{Incorrect}} }} \cdot \frac{{{\text{Red}}_{\text{Correct}} }}{{{\text{Red}}_{\text{Incorrect}} }}} \right) $$

and response bias by:

$$ \log b = \frac{1}{2}\log \left( {\frac{{{\text{Blue}}_{\text{Correct}} }}{{{\text{Blue}}_{\text{Incorrect}} }} \cdot \frac{{{\text{Red}}_{\text{Incorrect}} }}{{{\text{Red}}_{\text{Correct}} }}} \right) $$

where BlueCorrect denotes the number of correct responses following presentations of the “more blue” array, RedIncorrect denotes the number of incorrect responses following presentation of the “more red” array, and so forth. Response bias scores were calculated relative to the response alternative that was reinforced more frequently during the initial phase. Thus, we expected the response bias score to have a positive value during this first phase. During the second phase, after the contingencies were reversed, the response bias score was expected to decrease, possibly becoming negative. A bias score of 0 would indicate no systematic preference for one response alternative over the other.

Mean response discriminability and response bias scores and median response times were calculated for all trials completed to receive reinforcements 1–10, 11–20 and 21–30 (reward distribution 4:1) and reinforcements 31–40, 41–50 and 51–60 (reward distribution 1:4). The number of trials required to receive ten rewards was examined for each block.

Results

The signal-detection data were analyzed with mixed ANOVA using SPSS GLM for the three blocks of Phase 1, and the three blocks of Phase 2. To examine the total shift in response bias from the first to second phase, the bias scores for the last block of each phase were compared. As the mean age and estimated IQ scores for the groups were significantly different, correlations between age, IQ and performance were calculated separately for the ADHD and control groups. All correlations were nonsignificant, and results were unchanged when age and IQ were entered as covariates in the analyses; thus, the non-covaried results are presented.Footnote 2 The analyses were run with the entire ADHD sample and again including only those with combined type ADHD; the findings were unchanged and the analyses using all subjects are presented. Table 2 presents mean scores for discriminability, response bias, median response time and trial numbers for each block of 10 reinforcements. Table 3 presents the ANOVA results. The bias score data are also presented in Fig. 1.

Table 2 Means and standard errors for discriminability, response bias, median response time and number of trials for the ADHD and control groups during the first and second reinforcement phases
Full size table
Table 3 The results from mixed ANOVA for discriminability, response bias, median response time and number of trials for the three blocks of Phase 1, the three blocks of Phase 2, and examining the total shift in response bias from the first to second phase
Full size table
Fig. 1
figure 1

Mean response bias scores for the ADHD and control groups during the first and second reinforcement phases

Full size image

Phase 1

The ADHD and control groups demonstrated similar mean discriminability scores and trial numbers across the initial three blocks. The median response times for both groups declined with time on task [Block main effect: F(1.6, 111.1) = 11.69, p < .001, Greenhouse–Geisser correction]. The mean bias scores for both groups were significantly greater than zero during the first block [ADHD: t(31) = 3.01, p < .01; control: t(39) = 7.74, p < .001] and were stable across the initial phase. Mean bias scores were significantly smaller for the ADHD group compared to the control group [Group main effect: F(1, 70) = 4.62, p < .05].

Phase 2

Discriminability and median response time scores and trial numbers were similar for the two groups and remained stable over the three blocks of the second phase. The mean response bias for both groups approached zero during the first block following reversal of the reward distribution. Control group bias scores remained stable, i.e., close to zero, throughout this phase. The ADHD group developed a significant bias toward the now more frequently reinforced alternative by the third block of the second phase [t(31) = − 3.97, p < .01]. Mixed ANOVA comparing the bias scores of the two groups over the three blocks yielded a significant Group × Block interaction [F(1.8, 125.0) = 3.23, p < .05, Greenhouse–Geisser correction], reflecting the stability of the control group scores, while the ADHD group’s scores continued to decline, i.e., became negative, over time.

Shift in the response allocation from the end of Phase 1 to the end of Phase 2

Mixed ANOVA comparing the bias scores of the two groups at the end of the first and second phases yielded significant main effects for Phase [F(1, 70) = 45.71, p < .001] and Group [F(1, 70) = 8.04, p < .01]. Both the control and ADHD groups shifted their response allocation following the switch in the reinforcement contingencies, that is, their bias scores toward the initially more frequently reinforced response alternative declined. The extent of this shift was similar for the two groups (see Fig. 1).

Discussion

This study assessed the behavioral sensitivity of Brazilian children with and without ADHD to differences in the frequency of reward. It also evaluated the effect of a change in reward contingencies on the children’s response allocation. The behavior of both groups of children was sensitive to the asymmetric reward distribution. When the ratio of rewards for correct responses switched, subtle, but important group differences were observed.

Under the initial reinforcement conditions, the response bias scores of the children with ADHD were significantly smaller than those of the controls. This suggests the behavior of Brazilian children with ADHD may be less sensitive to unequal reward frequency than that of their typically developing peers. This finding is consistent with early suggestions that children with ADHD have a reduced sensitivity to reward (Wender 1971, 1972, 1974; Haenlein and Caul 1987; Barkley 1989). The behavior of Brazilian children with ADHD was not, however, insensitive to the unequal reward distribution, with their initial response bias scores similar to those reported by Alsop et al. (2016) for American and New Zealand children with ADHD.

Following reversal of the response contingencies, the bias scores of both groups dropped to near zero, demonstrating their behavioral sensitivity to the change in reward availability. Over the three blocks of trials, the bias scores of the children with ADHD continued to decline so that by the third block they showed a significant response bias toward the now more frequently reinforced response alternative. In contrast, the bias scores of the control group remained close to zero across all three blocks of this phase of the task, indicating similar behavioral allocation across the two response alternatives. The total change in response bias from the end of the first to the end of the second phase was not significantly different across groups.

The variation in the pattern of response bias for the two groups is not explained by differences in their ability to attend to, or complete, the signal-detection task. Discriminability scores for the two groups were stable across both phases of the task, and not significantly different from one another. The overall accuracy for both groups was between 74 and 78% across the task, indicating children were attending to, and able to discriminate between, the stimuli. Similarly, there were no group differences in response times or the number of trials completed. For both groups of children, longer task exposure led to faster response times during the initial phase of the task, with stable response times during the second phase.

The relative influence of recent rewards versus reward history on behavior might help to explain the observed differences in the groups’ response patterns. Tripp and Alsop (1999) showed children with ADHD were more sensitive to individual instances of reward than typically developing children, whose behavior was influenced more by their history of reward on a task. The response patterns of the control group children during the current study appear to reflect the integration of their experiences with reward throughout the task. Under the initial reinforcement contingencies, these children developed a stronger response bias based on the overall higher frequency of reward for one response alternative over the other. Occasional rewards on the less frequently reinforced alternative had little effect on their behavior. Subsequent exposure to the opposite reward distribution led to nearly equal behavioral allocation across the two response alternatives during the second phase. This could be interpreted as the amalgamation of their reward experiences across the entire task.

If children with ADHD are influenced more by individual instances of reward, than their reward history, then occasional rewards on the less frequently rewarded response alternative would act to reduce their bias toward the more frequently reinforced alternative during Phase 1. Following reversal of the contingences, responding to individual instances of reward would result in them allocating more of their responses to the alternative now providing more instances of reward. Reduced influence of overall reward history could lessen the effects of prior contingencies and help explain the gradual development of a significant response bias during Phase 2. The total number of trials did not allow reliable examination of bias separately for trials immediately following rewards on the more and less frequently rewarded alternatives to test such an hypothesis.Footnote 3 Although bias development during Phase 2 might be interpreted as evidence of increased sensitivity to the contingency change in the ADHD group, the absolute shift in bias from the first to the second phase was similar for both groups. The response pattern of the ADHD group in the current study does suggest children with ADHD are able to adapt their behavior to a change in reward availability given adequate exposure.

The current findings differ somewhat from those reported by Alsop et al. (2016). In their study, children with ADHD demonstrated similar sensitivity to the initial asymmetric reward distribution, but reduced sensitivity to changes in reward availability compared to their typically developing peers. These performance differences may arise from differences in the sample characteristics or the experimental paradigms used. The ADHD group in the currently study was older, and rates of comorbidity were lower than in Alsop et al. (2016). In the current study, children were exposed to each set of reinforcement contingencies for longer (more reinforcements) and these changed only once. Consistent with the findings from the Alsop et al. (2016)’s study, the ADHD group in the current study did not show a significant response bias after receiving 20 rewards following reversal of the contingencies. The development of a significant bias in the ADHD group, later in the second phase, suggests the behavior of these children continued to change with increased exposure to the new contingencies.

Conversely, the control group did not develop a significant bias after the reversal of contingencies in the current study. Their initial bias was higher than that reported for control group children by Alsop et al. (2016). This higher bias, together with increased exposure to the initial reward distribution, may have resulted in a stronger effect of the Phase 1 contingencies on their response allocation during Phase 2. The initial mean bias score of the typically developing children in the current Brazilian sample might suggest increased sensitivity to reward in this population (Chen et al. 1999). However, if this were the case, we would expect to a greater shift in their behavior after the contingencies changed.

In interpreting the current findings, it is important to consider the study’s strengths and limitations. Our sample sizes are moderate only. However, diagnostic criteria were applied conservatively, following best practice guidelines with oversight from senior clinicians. Behavioral and emotional difficulties were ruled out in the control group, allowing confidence in the membership of the two groups. Rates of comorbidity within the ADHD group were low, excluding other explanations for the observed differences in the way the two groups responded to manipulation of the reinforcement contingencies. We are thus confident that the observed differences reflect group membership based on ADHD symptomatology.

Importantly, the current findings highlight the subtlety and complexity of altered reinforcement sensitivity in ADHD. Children diagnosed with ADHD are not insensitive to the effects of reward; however, their response patterns do differ from those of typically developing children. The current results, together with findings from previous signal-detection studies (Tripp and Alsop 1999; Alsop et al. 2016; Furukawa et al. 2017), suggest the time course over which children with and without ADHD integrate or utilize reward information differs. Such an interpretation is consistent with earlier hypotheses emphasizing the importance of recent reward experiences on the subsequent behavior of children with ADHD (Barkley 1997; Tripp and Alsop 1999; Tripp and Wickens 2008). Sagvolden et al. (2005) suggested behavior sequences in closer proximity to a reward outcome are more strongly reinforced in children with ADHD. Working memory may also influence integration of reward information (Frank et al. 2007). The present findings also highlight the importance of reward exposure in understanding children’s sensitivity to changing reinforcement contingencies, especially when behavior-outcome associations are uncertain. Under lean, partial reinforcement schedules, prolonged exposure prior to a contingency change may increase the effects of reward history for typically developing children. For children with ADHD, increased exposure following a contingency change may help them adapt their behavior to the new contingencies. Researchers should be wary of assuming differences in reward sensitivity along a single dimension when studying motivational processes in the etiology and management of ADHD.

Clinically, the current findings are a timely reminder of the numerous and varied motivational influences on children’s behavior. These include rates of reinforcement, the presence of competing reinforcement schedules, prior experience with reward, and importantly how changes in expectations for behavior are signaled. Taking account of the current findings, we make the following recommendations: whenever possible give children with ADHD sufficient opportunity to transition when situational demands change; scaffold as necessary to help them identify changes in behavioral expectations/requirements; reinforce, i.e., praise, instances of desired behavior to increase and confirm the appropriateness of new behaviors. When rapid behavior change is expected, ensure children are explicitly informed of what is required and the associated consequences; ensure sufficient reinforcement is available to maintain behavior change. Caregivers should also be advised that children’s behavior is subject to concurrent reinforcement schedules, in addition to the ones they are implementing, and that these are likely to change across situations.