Task-specific prioritization of reward and effort information: Novel insights from behavior and computational modeling

Abstract

Efficient integration of environmental information is critical in goal-directed behavior. Motivational information regarding potential rewards and costs (such as required effort) affects performance and decisions whether to engage in a task. While it is generally acknowledged that costs and benefits are integrated to determine the level of effort to be exerted, how this integration occurs remains an open question. Computational models of high-level cognition postulate serial processing of task-relevant features and demonstrate that prioritizing the processing of one feature over the other can affect performance. We investigated the hypothesis that motivationally relevant task features also may be processed serially, that people may prioritize either benefit or cost information, and that artificially controlling prioritization may be beneficial for performance (by improving task-accuracy) and decision-making (by boosting the willingness to engage in effortful trials). We manipulated prioritization by altering order of presentation of effort and reward cues in two experiments involving preparation for effortful performance and effort-based decision-making. We simulated the tasks with a recent model of prefrontal cortex (Alexander & Brown in Neural Computation, 27(11), 2354–2410, 2015). Human behavior was in line with model predictions: prioritizing reward vs. effort differentially affected performance vs. decision. Prioritizing reward was beneficial for performance, showing striking increase in accuracy, especially when a large reward was offered for a difficult task. Counterintuitively (yet predicted by the model), prioritizing reward resulted in a blunted reward effect on decisions. Conversely, prioritizing effort increased reward impact on decision to engage. These results highlight the importance of controlling prioritization of motivational cues in neuroimaging studies.

Introduction

Adaptive behavior requires information in the environment to be processed promptly, evaluated, and integrated with one’s goals. The efficiency of such integration is critical for selecting appropriate behavioral responses, especially when achieving a goal requires exerting mental or physical effort. While difficult tasks are perceived as aversive (Kool, McGuire, Rosen, & Botvinick, 2010), humans are willing to engage in such tasks if a reward is expected upon completion (Shenhav, Botvinick, & Cohen, 2013; Westbrook & Braver, 2015). The factors that drive and support motivation while expecting or engaging in effortful behavior have received considerable attention in psychology and neuroscience research (Westbrook & Braver, 2015). Behavioral and neuroimaging findings show that the perceived value of rewards decreases as a function of the effort required to obtain it (Apps, Grima, Manohar, & Husain, 2015; Botvinick, Huffstetler, & McGuire, 2009; Croxson, Walton, O’Reilly, Behrens, & Rushworth, 2009) and that anticipating having to perform a more difficult task affects reward feedback evaluation (Gheza, De Raedt, Baeken, & Pourtois, 2018). While reward and effort are motivationally opposed, brain regions typically implicated in processing reward also are involved in registering task difficulty, suggesting a potential overlap between reward-related and effort-related motivational mechanisms (Vassena et al., 2014).

Generally, motivational information has differential effects on behavior depending on the valence of the information. Information regarding prospective rewards leads to faster reaction times and higher accuracy in tasks requiring cognitive control (Chiew & Braver, 2016; Janssens, De Loof, Pourtois, & Verguts, 2016; Krebs, Boehler, & Woldorff, 2010; Locke & Braver, 2008; van den Berg, Krebs, Lorist, & Woldorff, 2014), inhibition (Boehler, Hopf, Stoppel, & Krebs, 2012; Boehler, Schevernels, Hopf, Stoppel, & Krebs, 2014; Rosell-Negre et al., 2014), and perceptual discrimination (Schevernels, Krebs, Santens, Woldorff, & Boehler, 2014). Furthermore, reward information can affect high-level control strategies (Braem, 2017). Information regarding upcoming task-difficulty (and consequent required effort) influences task preparation (Krebs, Boehler, Roberts, Song, & Woldorff, 2012; Vassena et al., 2014; Vassena, Cobbaert, Andres, Fias, & Verguts, 2015; Vassena, Gerrits, Demanet, Verguts, & Siugzdaite, 2018), improving task performance when increased control is required (Aarts & Roelofs, 2010; Bugg & Smallwood, 2016). This process is likely mediated by increased attentional allocation (Maunsell & Treue, 2006), which enhances neural activity in task-related structures and results in performance improvements. This mechanism is known as proactive control (Braver, 2012) and has the function of adapting performance to internal goals. Information regarding effort also influences decision-making: typically when given a choice, people tend to avoid more demanding tasks (Kool et al., 2010) and prefer knowing in advance how much effort the task will require (Apps et al., 2015).

Although both reward and effort information is consistently observed to influence behavior and brain activity, it remains unclear how such information is integrated to guide behavior. Most studies rely on the assumption that information coming from multiple sources (i.e., benefit and cost, such as reward and task-difficulty) is processed simultaneously and integrated. Furthermore, relatively little attention has been given to the possibility that cost and benefit information may differentially influence task performance compared with decision-making. Comparisons between effects of cost and benefit information on decisions as opposed to task accuracy are scarce. We challenge the assumption of simultaneous processing and unbiased integration, by proposing that reward and cost information is processed in a serial fashion, and that depending on the task context, either reward or cost may be prioritized. One straightforward way of assessing information integration is manipulating the order in which such information is presented. Simultaneous presentation of multiple information sources leaves the subject free to process information in an unconstrained fashion. Presenting the same information serially enforces an order of processing, potentially reducing or increasing the impact of the piece of information coming first on task performance or decisions whether to engage in the task at all.

Evidence that order of processing affects behavior comes from judgment and decision-making literature: how information is presented can influence preferences, and therefore decisions. For example, framing a problem in terms of gains or losses modulates willingness of people to take risky decisions: a gain-frame makes people risk-averse, whereas a loss-frame encourages risk-seeking behavior (Gonzalez, Dana, Koshino, & Just, 2005; Kühberger, 1998; Kühberger, Schulte-Mecklenbeck, & Perner, 1999). Moreover, an anchor (i.e., numerical value presented before the task) typically influences subsequent numerical estimates of quantities or probabilities (Furnham & Boo, 2011; Jacowitz & Kahneman, 1995; Strack & Mussweiler, 1997). These effects are known as biases or heuristics, strategies deployed to process efficiently the information available in the environment when resources (such as time or concentration) are limited. Both framing and anchoring can be seen as contextual variables influencing processing of subsequent information (De Martino, Kumaran, Seymour, & Dolan, 2006; Kahneman & Tversky, 1979). Furthermore, pronounced order effects have been observed in delay-discounting tasks (Kirby & Maraković, 1996; Reynolds, 2006), where people make a series of decisions between immediate smaller rewards and delayed bigger rewards (Robles, Vargas, & Bejarano, 2009).

Several (neuro)computational models describing cognitive processes, such as working memory, prediction, and reasoning, also suggest that information processing and mapping may influence subsequent task-performance. In such models, different pieces of task-relevant information are processed in a serial fashion, and each feature is represented at one level of a neural, temporal, or representational hierarchy within prefrontal cortex (PFC) (Alexander, Vassena, Deraeve, & Langford, 2017; Badre, 2008; Badre & D’Esposito, 2007; Collins & Frank, 2012; Koechlin, Ody, & Kouneiher, 2003). The Hierarchical Error Representation model (HER, Alexander & Brown, 2015) suggests that different mappings of information are learned during task performance: information about context tends to be prioritized, i.e., processed first. Simulations of the HER model demonstrate that interfering with this learned prioritization can produce changes in behavior, including prolonged learning times and increased error rates (Alexander & Brown, 2015). Consequently, the HER model suggests that order effects may derive from differences in the mapping of information (as a function of which one information source is prioritized), with consequences on processing efficiency and task performance.

In summary, findings from judgment and decision-making psychology and computational modeling converge in suggesting that order of presentation may affect performance and decision. In the context of motivational information, these hypotheses have never been tested. Models, such as the HER, have been developed to account for the role of PFC in processing complex information, maintaining goals, and guiding decisions (Alexander & Brown, 2018; Alexander et al., 2017). Critically, neuroimaging studies investigating motivation have consistently implicated several subregions of PFC. Expecting to perform an effortful task is associated with increased activity in medial and lateral PFC, activity that partially overlaps with expectation of a larger reward (Vassena et al., 2014). Activity in lateral PFC also is consistently observed in studies manipulating effort and task load (Aarts & Roelofs, 2010; Locke & Braver, 2008; McGuire & Botvinick, 2010; Sohn, Albert, Jung, Carter, & Anderson, 2007; Vassena et al., 2018). Furthermore, activity in medial PFC scales with the subjective value of the expected reward discounted by the required effort (Apps & Ramnani, 2014; Chong et al., 2017). This functional overlap between motivational information coding and typical control and working-memory coding suggests an underlying shared computation across context. In other words, the same computational architecture that subserves task-relevant information in cognitive control also may explain the involvement of these regions in motivational information coding.

Generally, studies of value-based decision-making utilize simultaneous presentation of multiple information sources, assume that these sources are weighted equally and integrated, and do not address how this putative integration may occur. The goal of this study was twofold. First, we investigated the general question of whether manipulating the order of processing of motivational information could influence subsequent behavior in a manner consistent with serial information integration. Second, we tested the HER model as a candidate account of the mechanisms underlying information integration by simulating the model on Experiments 1 and 2 and compared model behavior to observed behavior. The HER model predicts that prioritizing one information source over the other (for example benefit over cost) may influence task performance and choice behavior, thus providing a novel hypothesis on how individuals process multiple sources of motivational information.

Existing accounts of value-based decision-making would favor the possibility that, when presented simultaneously, and with ample time in which to register information, task-difficulty and reward cues are processed to produce a net-value that governs specific behaviors, such as decisions to engage in a task or the level of effort to expend in task performance. One could learn, for example, the value of engaging in a task as the level of reward, which is then discounted by the required effort (as a function of task-difficulty), as suggested by neuroeconomic perspectives (Kivetz, 2003; Westbrook & Braver, 2015). Conversely, the net-value of engaging in a task could be conceived as the cost of performing the task (in terms of prospective effort expenditures), which is then offset by the prospective reward of successfully completing the task. While a rational model of value might suggest that these two disparate fashions of integrating reward and effort information might result in the same net-value, ample evidence, as described above, suggests that this is not necessarily the case. In line with the predictions of the HER model, such serial processing may reflect the prioritization of reward or cost information, with the variable processed first becoming a context for the second variable (e.g., effort may be processed first and influence subsequent processing of reward information), rather than simultaneous integration of equally weighted information sources (Vassena, Deraeve, & Alexander, 2017). Furthermore, it may be the case that disrupting prioritization strategies through manipulating order information may shift net-value estimates in a task-dependent manner. More specifically, enforcing an order consistent with a person's usual prioritization may have little effect on one task, while producing profound effects on other tasks.

To investigate the influence of prioritizing reward over cost information and vice-versa on behavior, we adapted a mental effort task previously used to investigate the neural correlates of reward and difficulty coding (Vassena et al., 2014, 2015). In Experiment 1, we investigated the effect of prioritizing reward over difficulty (cost) information on task performance. In Experiment 2, we tested the effect of prioritizing reward over difficulty information on the decision to engage or not in a proposed task. To investigate the hypothesis that motivational information may be processed serially, and prioritized depending on task context, we simulated each experiment using the HER model to derive qualitative predictions regarding behavioral performance, which we compared with actual behavioral patterns observed in our subjects.

Experiment 1

Materials and methods

Experiment 1. Participants

Forty-one healthy subjects participated in this experiment (mean age 19 ± 1 years; 7 males). Participants were recruited through the online platform Experimetrix among the students of the first-year bachelor of psychology. Participants received a credit for their participation, plus a monetary compensation depending on task-performance (up to 4 euros). This amount was calculated based on points accumulated at the end of the experiment. At the beginning of the session, each participant gave written, informed consent. Sample size was determined a priori based on previous studies investigating the effect of difficulty and reward on task-performance (Janssens et al., 2016; Krebs et al., 2010; Schevernels et al., 2014; Vassena et al., 2015) and on our unpublished data from a similar experiment investigating neural correlates of task-preparation and decision-making under different reward and difficulty conditions.

Experiment 1. Procedure

The goal of Experiment 1 was to investigate the influence of prioritization of difficulty and reward information on performance. A mental effort task was adapted from our previous work (Fig. 1c; Vassena et al. 2014, 2015). This paradigm was selected, because participants consistently report difficult trials to be more effortful compared with easy trials (both in pilot data and in our above-mentioned studies).

Fig. 1

Stimuli and trial-structure in Experiment 1. a Motivational cues used in both experiments. Vertical bars indicate the amount of potential reward (small, large). Horizontal bars indicate the difficulty level (easy or hard), and therefore required effort. b Example of cue presentation when difficulty information is prioritized (left panel) and when reward is information is prioritized (right panel). c Trial-structure in Experiment 1. Two cues are presented one after the other, indicating difficulty level and potential reward in the upcoming trial. Cues are followed by an arithmetic operation, with two possible results. Participants selected the correct response and received feedback

At the beginning of each trial, two motivational cues were presented serially. The motivational cues consisted of grey circles with vertical and horizontal black lines (Fig. 1a). The horizontal lines represented task difficulty on the current trial: a line in the lower part of the circle signaled an easy trial, and a line in the upper part of the circle signaled a difficult trial. The vertical lines represented potential reward in case of correct response: a line in the left part of the circle indicated no reward, and a line in the right part of the circle indicated high reward. This resulted in four possible combinations of task difficulty and potential reward. Furthermore, difficulty and reward information could be presented simultaneously (both first and second cue showing both lines) or sequentially (the first cue showing a horizontal line, the second a vertical line or vice-versa; Fig. 1b). The goal of this procedure was to compare simultaneous presentation of reward and difficulty information with serial presentation, keeping different orders of presentation into account.

The trial was structure as follows: At the beginning of every trial, a blank white screen was shown (500 ms), followed by the first cue presented centrally on the screen (2,000 ms). A short blank screen (50 ms) separated the first cue from the presentation of the second cue (500 ms), after which the task began. The task consisted of two arithmetic operations (2 additions or an addition and a subtraction). The operations remained on the screen for 1,500 ms, after which two possible results were displayed: one on the left and one on the right. Participants had to select the result that they thought was correct by pressing the left or the right response button depending on the position of the correct result. The response time limit was 1,000 ms. Feedback was then presented, showing the amount of obtained reward (points) in case of correct response (1,000 ms). In case of an incorrect response, the word “error” was shown on the screen. If no response was recorded within the time limit, the words “respond faster” were presented.

This setup resulted in a 3 x 2 x 2 design, with factors order (simultaneous, difficulty first, reward first), difficulty (easy, difficult), and reward (low, high). Thirty trials per conditions were presented to each participant in a randomized order (360 trials in total). Following a correct response, subjects received feedback indicating the number of points they earned (100 points for high-reward trials, 0 points for no-reward trials). The difficulty manipulation was implemented by exploiting the reliable effect of decade-crossing. In the difficult condition, both operations implied carrying or borrowing (e.g., 5 + 8 − 6). In the easy condition, none of the operations implied carrying or borrowing (e.g., 5 + 1 + 1). As shown in previous research, this procedure consistently shows difficulty effects (in either accuracy or reaction time data, and in ratings concerning perceived difficulty; Vassena et al., 2014, 2015). Before task execution, a short training phase was administered (12 trials) to acquaint participants with the cues, speed of presentation, and easy and hard trial types. Participants were instructed to respond as accurately and quickly as possible. During task execution, participants had nine breaks, during which they could rest briefly. The total duration of the task was approximately 35 minutes (not including breaks).

Experiment 1. Analysis

Data were analyzed with the generalized linear mixed models approach using the R package lme4 (Bates, Mächler, Bolker, & Walker, 2014). For binary outcome dependent variables (accuracy), we used generalized linear mixed models. For continuous dependent variables (choice RTs and performance RT), we used linear mixed models. RT data also were z-scored and log-transformed to prevent violations of the assumption of normality. Trials where participants made an error at the calculation task were excluded from the RT analysis. For each of the dependent variables of interest, we performed a model building procedure. First, we estimated a benchmark model, including a random intercept and a random slope for each subject. Subsequently, three models were estimated, each including the benchmark model plus one of the fixed effects for order, difficulty, and reward. Each of these three models was compared against the benchmark model following a likelihood ratio testing procedure. The goal of this procedure was to determine which factors in the model provided a significant contribution in explaining the variance in the dependent variable at hand. Subsequently, a new benchmark model was determined, including a random intercept and a random slope for subject and the fixed effects that proved significance in the previous testing. Each interaction was tested separately against this new benchmark model. Follow-up pairwise comparisons were used to assess significant interactions. To provide a numerical estimate of effect size, for each model we also calculated the Ω₀², an analogous of the R² for (generalized) linear mixed models, which compares the residual variance of the tested model against the residual variance of a (fixed) intercept-only null model (Xu, 2003). This calculation was performed on the model fits run on averages per participant per condition. It should be noted that the use of such measures in the linear mixed models framework is debated (Nakagawa & Schielzeth, 2013); however, it can provide an intuitive measure comparable to effect sizes computed in linear models. A further caveat is that given that the estimated model, including interactions, will always explain a larger portion of the variance compared with the simple model, including only the intercept. Therefore, this measure might not be optimal to judge effect size for interactions. Finally, 95% confidence intervals (CI) were computed with the Wald method implemented in the lme4 package. The hypotheses of the current study mainly focused on accuracy. RT results also are reported for completeness.

Experiment 1. Results

Statistical significance of the results was assessed by likelihood ratio testing. χ² and p values refer to comparisons between the benchmark model and the same model plus the fixed effect or interaction of interest (see Experiment 1, Methods, Analysis section). Likelihood ratio test results for accuracy are reported. We observed a significant fixed effect of difficulty and reward, with an additional nonsignificant trend for order (difficulty χ²₍₁₎ = 2108, p < 0.0001, Ω₀² = 0.39, reward χ²₍₁₎ = 36.36, p < 0.0001, Ω₀² = 0.44, order χ²₍₂₎ = 4.55, p = 0.10, Ω₀² = 0.34). Accuracy was higher for easier trials (M_easy = 0.97 ± 0.03, CI [ 2.54, 2.84 ], M_hard = 0.73 ± 0.11, CI [−2.84, −2.54]) and for high reward trials (M_{no reward} = 0.84 ± 0.08, confidence interval [CI] [−0.37, −0.19], M_{high reward} = 0.87 ± 0.06, CI [0.19, 0.37]). A significant difficulty X reward interaction was observed (χ²₍₁₎ = 11.402, p < 0.001, Ω₀² = 0.50). Follow-up comparisons revealed that the effect of high reward on accuracy was significant both in the easy (t₍₄₀₎ = 2.55, p ≤ 0.05), M_{high reward – no reward} = 0.01) and in the hard condition (t₍₄₀₎ = 5.88, p ≤ 0.0001, M_{high reward – no reward} = 0.05) but greater in the hard condition. Furthermore, a significant order X reward interaction was observed (χ²₍₄₎ = 19.39, p < 0.001, Ω₀² = 0.49). The effect of reward in improving accuracy was significant in the simultaneous (t₍₄₀₎ = 5.6, p < 0.0001, M_{high reward – no reward} = 0.05) and reward first condition (t₍₄₀₎ = 5.8, p < 0.0001, M_{high reward – no reward} = 0.12) but not in the difficulty first condition (t₍₄₀₎ = 0.54, p = 0.59, M_{high reward – no reward} = 0.01). Finally, a significant order X difficulty interaction was observed (χ²₍₄₎ = 12.76, p < 0.05, Ω₀² = 0.49). Accuracy was significantly reduced for hard trials in all conditions, showing the strongest impact when the difficulty cue was presented first (t₍₄₀₎ = −14.03, p < 0.0001, M_{hard – easy} = −0.28), average impact when presented simultaneously (t₍₄₀₎ = −12.99, p < 0.0001, M_{hard - easy} = −0.25), and the weakest impact when difficulty was presented last (t₍₄₀₎ = −11.9, p < 0.0001, M_{hard – easy} = −0.22) (Fig. 2).

Fig. 2

Experiment 1. a Average accuracies in each difficulty and reward condition, in the three possible orders (simultaneous, difficulty first, reward first). Errors bars show one standard error of the mean. The black circles show that when difficulty information is prioritized, reward effect is absent. b Upper panel: summary of confirmed model predictions. (see Computational model section below). Lower panel: average reward effect (differences in accuracy in the high reward vs. low reward condition) c. Average RTs at the calculation task in each difficulty and reward condition, in the three possible orders (simultaneous, difficulty first, reward first). Errors bars show one standard error of the mean

Analysis of RTs revealed a significant effect of difficulty (χ²₍₁₎ = 330.5, p < 0.0001, Ω₀² = 0.46), a trend for effect of reward (χ²₍₁₎ = 2.37, p = 0.12, Ω₀² = 0.13), and no effect of order (χ²₍₂₎ = 2.08, p = 0.35). Participants responded faster to easy trials (M = 434.37 ± 48.15, CI [−0.37, −0.30]) compared with hard trials (M = 543.14 ± 72.14, CI [0.30, 0.37]). Participants responded more slowly to high-reward trials (M = 491 ± 53 ms, CI [−0.06, 0.01]) compared with no-reward trials (M = 486 ±54 ms, CI [−0.01, 0.06]), perhaps due to more careful responding in light of the reward at stake (although this difference did not reach significance). A significant reward X difficulty interaction also was observed (χ²₍₂₎ = 6.8, p < 0.05, Ω₀² = 0.47) and a trend for the order X reward interaction (χ²₍₅₎ = 8.84, p = 0.12, Ω₀² = 0.47). To test whether these interactions were mainly driven by the marginally significant fixed effect of reward, we established a new benchmark model that included the trend-level fixed effect of reward. When tested against this benchmark, the reward X difficulty interaction added only a marginal contribution (χ²₍₁₎ = 3.6, p = 0.06), whereas the order X reward interaction was no longer significant (χ²₍₄₎ = 5.62 1, p = 0.23). For this reason, no follow-up comparisons were performed on these interactions.

Experiment 1. Interim discussion

Experiment 1 investigated effects of order of presentation of motivational information on task performance. We adapted a task previously used in neuroimaging research to investigate reward and effort anticipation during task-preparation (Vassena et al., 2014). Cues indicating reward amount and task difficulty were presented serially (in both possible orders), as well as simultaneously (as in the original fMRI experiment). As suggested by the HER model simulations, we predicted effects of prioritizing reward or difficulty information on task-related preparation and consequent influence on accuracy.

The results show a striking influence of prioritizing reward as opposed to difficulty information on task performance. Naturally, accuracy is higher in easy trials and in trials with high reward prospect. However, when reward cues are presented first, accuracy at the task is comparable to the simultaneous condition. Conversely, when effort is presented first, the otherwise present reward effect is abolished. The similarity between the Reward first and simultaneous confirms the model-based predictions.

Importantly, the second cue always lasted 500 ms. Previous studies have shown that reward information can be correctly processed when presented for as short as 200 ms before task onset (Janssens et al., 2016). Moreover, effect of reward on attention-related brain processes as measured through EEG can arise as early as 200 ms (P2) and 400 ms (P3, Schevernels et al., 2014). These time intervals are all shorter than the one used in the current experiment. This suggests that subjects in our experiment have sufficient time to process reward cues when presented second, and differences in behavior are due to the contextual influence of effort information on reward processing. It should be noted that the effect of difficulty we observe is partially driven by task difficulty itself. However, this alone could not explain the influence presentation order on performance.

Experiment 2

Materials and Methods

Experiment 2. Participants

Thirty-five healthy subjects participated in this experiment (mean age 24 ± 5 years; 8 males). Participants were recruited through the online platform Experimetrix and received a monetary compensation for participation. A portion of the amount was calculated based on points accumulated at the end of the experiment. The maximum possible remuneration was 13 euros. At the beginning of the session, each participant gave written, informed consent. Sample size was determined a priori, based on previous studies investigating the effect of difficulty and reward on task-performance (Janssens et al., 2016; Krebs et al., 2010; Schevernels et al., 2014; Vassena et al., 2015) and on our unpublished data from a similar experiment investigating neural correlates of task-preparation and decision-making under different reward and difficulty conditions.

Experiment 2. Procedure

The goal of Experiment 2 was to investigate the effect of prioritization on decision-making. The same paradigm from Experiment 1 was adapted to include a choice period where participants could decide to perform the proposed task or a default task with minimal effort and low reward (Fig. 3). As in Experiment 1, two cues were presented, indicating potential reward and task difficulty in three possible orders (simultaneous, difficulty first, reward first). In this experiment, however, the reward conditions were low reward (50 points) and high reward (100 points). The difficulty manipulation was equivalent to Experiment 1, and the same arithmetic operations were used. After the second cue, a screen was presented with the text “Accept?”. The response time limit was 1,000 ms. Half of the participants had to press the left button to accept (and the right to reject). This was counterbalanced in the other half of the participants, who had to press the right button to accept (and the left to reject). An “Accept” response meant deciding to engage in a trial with the potential reward and difficulty level indicated by the cues preceding the “Accept” screen. In case of “Reject” response, a standard trial type would occur instead, with a very easy task (easier than the easy level, i.e., 5 + 0 + 0), for a much smaller amount of points (5 points). The timing of the rest of the task was equivalent to Experiment 1.

Fig. 3.

Stimuli and trial-structure in Experiment 2. a Motivational cues used in both experiments. Vertical bars indicate the amount of potential reward (small, large). Horizontal bars indicate the difficulty level (easy or hard), and therefore required effort. b Example of cue presentation when difficulty information is prioritized (left panel), and when reward is information is prioritized (right panel). c Trial-structure in Experiment 2. Trial and time are the same as in Experiment 1, with one added display after the cues, asking participants if they are willing to “Accept,” namely engage in a trial with the proposed potential reward and difficulty level. In case of negative response, a very easy trial follows, for only 5 points reward.

Reward conditions and consequences of rejecting a trial were explained to the participants at the beginning of the experiment. Total number of trials, trials per condition, breaks, and total duration were the same as in Experiment 1. A training phase was included to acquaint participants with the cues, procedure, and difficulty levels. Participants were asked to consider carefully their choices and not always respond “Accept” or “Reject.” Moreover, they were instructed to be as fast as possible, because late responses would mean losing the chance to perform the trial (and earn points).

Experiment 2. Analysis

The same analysis procedure was used as in Experiment 1. Choice data (accept vs. reject) and choice RTs data were analyzed. Accuracy and RTs at the task also were analyzed. For binary variables (choice data and accuracy data), generalized mixed models were used. For RT data, linear mixed models were used. RT data were z-scored and log-transformed to prevent violations of the assumption of normality. Trials where participants rejected the task were excluded from RT analysis, as well as trials where participants made an error at the calculation task. As our hypothesis concerned decision-making, we mainly focus on the choice data.

Experiment 2. Results

Data from one participant were excluded, because this person always accepted the proposed task. Statistical significance was assessed by likelihood ratio testing. χ² and p values refer to comparisons between the benchmark model and the same model plus the fixed effect or interaction of interest (see Experiment 1, Methods, Analysis section). Likelihood ratio test results for choice data (accept vs. reject) are reported. The analysis revealed a significant fixed effect of difficulty and a fixed effect of reward, while order had no influence (difficulty χ²₍₁₎ = 212.83, p < 0.0001, Ω₀² = 0.39, reward χ²₍₁₎ = 448.13, p < 0.001, Ω₀² = 0.44, order χ²₍₂₎ = 1.07, p = 0.59). Participants were more likely to accept an easy option (M = 0.92 ± 0.13, CI [0.75, 0.99]) compared with a hard one (M = 0.84 ± 0.23, [−0.99, −0.75), and a high-reward option (M = 0.93 ± 0.10, CI [1.17, 1.43]) compared with a low-reward option (M = 0.82 ± 0.27, CI [−1.43, −1.17]). We additionally observed a significant difficulty X reward interaction (χ²₍₁₎ = 11.55, p < 0.0001, Ω₀² = 0.50). This interaction was driven by a larger effect of reward in the hard (M_{high reward - low reward} = 0.16) as opposed to easy condition (M_{high reward - low reward} = 0.06), although both differences were significant at follow-up t-tests (easy t = 2.26, p < 0.05, hard t = 4.38, p < 0.0001). Interestingly, the order X reward interaction also was significant (χ²₍₄₎ = 9.80, p < 0.05, Ω₀² = 0.48). This interaction was driven by a larger reward effect in the simultaneous condition (M_{high reward - low reward} = 0.13, t₍₃₃₎ = 4.63, p < 0.0001) compared with the difficulty first (M_{high reward - low reward} = 0.12, t₍₃₃₎ = 4.79, p < 0.0001) and especially reward first condition (M_{high reward - low reward} = 0.09, t₍₃₃₎ = 3.68, p < 0.001), although all three differences were statistically significant. The order X difficulty interaction did not show a significant effect (χ²₍₄₎ = 2.38, p = 0.67) (Fig. 4).

Fig. 4.

Experiment 2, decision-making. a Choice data (average probability of accepting the given level of difficulty and reward) per every condition. Error bars indicate one standard error of the mean. Black circles show the smaller reward effect when reward is prioritized, as compared to when difficulty is prioritized, and simultaneous presentation. b Upper panel: summary of confirmed model predictions (see Computational model section). Lower panel: average reward effect (differences in probability of accepting in the high reward vs. low reward condition) c. Average choice RTs per condition. Error bars indicate one standard error of the mean.

In our analyses for choice RTs, likelihood ratio test results yielded no significant fixed effects of order, difficulty, or reward (order χ²₍₂₎ = 2.62, p = 0.27, difficulty χ²₍₁₎ = 0.39, p = 0.53, reward χ²₍₁₎ = 0.10, p = 0.75). No significant interactions were observed. For completeness, the same analysis was rerun, including trials in which participants accepted the task at hand, but made an error at the calculation task. The only difference was a trend towards significant for the effect of order (χ²₍₂₎ = 5.21, p = 0.07).

Likelihood ratio tests for accuracy at the calculation task yielded a significant effect of difficulty (χ²₍₁₎ = 1819.3, p < 0.0001, Ω₀² = 0.77) and no effects of reward (χ²₍₂₎ = 0.09, p = 0.76) or order (χ²₍₂₎ = 0.24, p = 0.89). As a note, when rejected trials were included in the analysis (when participants received the very easy task for very low reward), the reward effect was significant (χ²₍₁₎ = 8.53, p < 0.01, higher accuracy also was higher in the high reward trials (M = 0.84 ± 0.06) compared with low-reward trials (M = 0.82 ± 0.07)). Participants were more accurate in the easy task (M = 0.97 ± 0.04, CI [2.83, 3.21]) compared with the hard task (M = 0.69 ± 10, CI [−3.21, −2.83]). The order X reward interaction was marginally significant (χ²₍₅₎ = 10.10, p = 0.07, Ω₀² = 0.77). As this replicated the results obtained in the accuracy data of Experiment 1, this effect was investigated further. High reward improved accuracy in the simultaneous (M_{high reward-low reward} = 0.02, t₍₃₃₎ = 2.37, p < 0.05) and reward first condition (M_{high reward-low reward} = 0.03, t₍₃₃₎ = 2.25, p < 0.05) but not in the difficulty first condition (M_{high reward-low reward} = −0.009, t₍₃₃₎ = −0.81, p = 0.43). This confirms our previous results showing that the reward effect was abolished in the difficulty first condition. Finally, the difficulty X reward interaction showed a trend towards significance (χ²₍₂₎ = 4.27, p = 0.12), whereas the order X difficulty interaction was not significant (χ²₍₄₎ = 5.09, p = 0.28) (Fig. 5).

Fig. 5.

Experiment 2, calculation task results. a Average accuracies at the calculation task in all conditions. The error bars show one standard error of the mean. Black circles show that reward effect is abolished when difficulty is prioritized. These results replicate experiment 1. b Average RTs at the calculation task in all conditions. The error bars show one standard error of the mean. c Average reward effect (differences in RT in the high reward vs. low reward condition)

Likelihood ratio test results for RTs at the calculation task produced a significant effect of difficulty and reward (difficulty χ²₍₁₎ = 85.5, p < 0.001, Ω₀² = 0.34, reward (χ²₍₁₎ = 7.79, p < 0.01, Ω₀² = 0.26) and no effect of order (χ²₍₂₎ = 0.17, p = 0.91). Participants responded faster in easy trials (M = 488 ± 78 ms, CI [−0.23, −0.15,]) compared with hard trials (M = 829 ± 130 ms, CI [0.15, 0.23]). Participants responded slightly slower in high-reward trials (M = 660 ± 94 ms, CI [0.02, 0.1]) compared with low-reward trials (M = 655 ± 92 ms, CI [−0.1, −0.02]), perhaps due to more careful selection of the response in light of the reward at stake. Furthermore, the difficulty X reward interaction was significant (χ²₍₁₎ = 8.94, p < 0.01, Ω₀² = 0.36), while the order X difficulty (χ²₍₄₎ = 0.87, 0.92) and order X reward (χ²₍₄₎ = 1.06, p = 0.9) interactions were not. High reward significant slowed down responses in the hard condition (M_{high reward-low reward} = 11 ms, t₍₃₁₎ = 2.24, p ≤ 0.05), while it had no effect in easy condition (M_{high reward-low reward} = 0.17 ms, t₍₃₁₎ = −0.04, p = 0.97).

Experiment 2. Interim discussion

Experiment 2 investigated the effect of order of motivational information on decision-making. We hypothesized that order might affect preference and consequent choice when participants could decide which trials to engage in. This would be in line with results from decision-making psychology, demonstrating the influence of order on choice behavior. Our results indeed showed that order affects decisions to engage in the task. Naturally, participants were more inclined to accept trials providing higher reward and requiring lower effort. Acceptance rates when difficulty was presented first were comparable to simultaneous presentation; however, when reward was presented first, the effect of reward on acceptance rates was reduced. This confirms that both first and second cues were adequately processed, as shown by the significant effect of difficulty and reward on decision both in the difficulty first and reward first conditions. Furthermore, this shows that prioritizing effort information makes decisions more dependent on the potential reward available. Crucially, and in contrast to our findings in experiment 1, this also suggests that during simultaneous presentation when one can choose to engage or not, task difficulty information is prioritized (whereas during performance, reward information appears to be prioritized). The similarity between the difficulty first and simultaneous condition confirm model-based predictions.

Importantly, accuracy during task performance results replicated our results from Experiment 1, even though the decision component was introduced, and the sample size was smaller. This confirms that order of presentation of motivational information affects task performance even with greater delay between cue presentation and task execution.

Computational model

Computation model. Methods

In line with the Hierarchical Error Representation (HER) model (Alexander & Brown, 2015) of dorsolateral and medial prefrontal cortex (PFC), our behavioral results indicated that manipulating prioritization of motivational information affected performance and decision-making. To test the validity of the HER model as a candidate underlying mechanism, we also simulated both tasks (code available upon request) to derive qualitative predictions regarding the possible influences of our order manipulation on accuracy during performance of the calculation task and decision-making.

The HER framework models how task-relevant information is processed in PFC by representing the likelihood of events (predictions), mismatches between predictions and actual events (prediction errors), the likelihood of contexts (higher-level links between stimuli and prediction error history attached to them), and mismatch between expected and actual context. The HER computational architecture suggests that PFC is organized in a hierarchical, layer-wise structure, with inferior layers representing concrete, response-related stimuli (putatively encoded by posterior-medial PFC), while superior layers represent context-related information (putatively encoded by anterior-lateral PFC). Previous simulations demonstrate differences in the model's behavioral performance during task conditions in which the model is forced to represent that task in an inefficient manner (Alexander & Brown, 2015). When two sources of information are required to determine a correct response, the model learns to map information that more directly identifies the correct response to lower hierarchical levels associated with concrete predictions. When this mapping is disrupted through, e.g., manipulations of the order of presentation of information sources, the model learns to perform the task less quickly and with more overall error.

To simulate the model on the effort and reward tasks described in this study, the following changes were made. First, to simulate performance on the calculation task (Experiment 1), one output was modeled, reflecting the level of effort exerted by the model to perform the calculation task. Model output was used as input to a sigmoid function, which represented the probability of the model performing the task correctly for a given level of effort:

$$ \mathrm{P}\left(\mathrm{correct}\right)=\frac{1}{1+{\mathit{\exp}}^{-\beta \left(E-D\right)}} $$

(1)

where E is the level of effort indicated by the model output, D is the difficulty of the task, β (value = 2) is a scale parameter. Large values of β indicate rapid increases in the probability of a correct response for modest increases in E. Feedback to the model was delivered as V = R-E, where R is the level of reward earned by the model on a specific trial. If the model delivered an incorrect response, R = 0. When the model correctly performed a trial, the value of R depended on the reward condition (Fig. 6b, x axis). When R was greater than E, the net-value V of the trial was positive, leading to an increased level of effort on the next trial in the same reward/effort condition. Conversely, when R was less than E, the net-value of the trial was negative, and less effort was expended on the next trial. The model thus adaptively modulated its behavior based on feedback over the course of the experiment.

Fig. 6.

The HER model. a Hierarchical mapping of information. When integrating two sources of information, the HER model learns to map each source to a distinct hierarchical level (Left and Middle frames). This mapping is determined automatically by the model. Sources of information that are prioritized (i.e., processed first) are mapped to the higher level. Sources of information that most reduce outcome uncertainty are mapped to the lower level. This information-to-level mapping may change depending on task demands. In the forced mapping “Reward 1^st” (left), the model represents reward at the higher level: this corresponds to prioritizing reward. In the forced mapping “Effort 1^st” (middle), the model represents effort at the higher level: this corresponds to prioritizing effort. In the flat version of the model (right), both sources of information are mapped to the same layer. b Depending on the task, the model learns different mappings of reward and effort information to hierarchical levels. In the decision task (upper frames), the model learns to map effort information to the higher level - the model's behavior is similar to prioritizing effort. Model’s behavior is different when reward information is force-mapped to the higher level (Fig. 6a, reward prioritized). This pattern is reversed for the performance task: the model learns to map reward information to the 2^nd level – the model’s behavior is similar to prioritizing reward. In the flat version of the model, behavior for all conditions is identical despite changes in salience of effort and reward information analogous to order manipulations. c. The model's behavior under free mapping conditions is more similar to the prioritizing effort condition in the decision task (left frame) relative to the prioritizing reward condition (computed as the sum of the squared difference between the model's choice/performance in the Reward 1^st/Effort 1^st conditions and the Free condition). The reverse is true for the performance task (right frame). Behavior is the same for all conditions using the flat model.

To simulate choice behavior (Experiment 2), four outputs were modeled. Two outputs reflected the model's prediction of the possible reward to be obtained from either engaging in a prospective task or accepting a lower, default reward, and two outputs reflected the model's prediction of the level of difficulty entailed in accepting the default and nondefault options. The net value of accepting the default or nondefault option was calculated as the difference between the predicted reward and predicted effort for each option. This net value was translated into choice behavior using a softmax function:

$$ P\left({O}_1\right)=\frac{{\mathit{\exp}}^{\beta {V}_1}}{\sum_{n=1}^2{\mathit{\exp}}^{\beta {V}_n}} $$

(2)

where V is the net predicted value (Reward minus Effort) of each option and β is a temperature parameter with that was set to 2.

For each task, the model was simulated in 3 different conditions (Fig. 6). In the first condition (Reward 1^st), the mapping of effort and reward information was manually specified such that reward information was represented in the higher layer, while effort was represented in the lower layer. This mapping is equivalent to forcing the model to prioritize reward information. In the HER model, this mapping forced the model to learn output predictions based on effort information, and these predictions were contextualized by reward-related information after the model had learned the overall average predictions based on the effort information alone. In the second condition (Effort 1^st), the forced mapping was reversed such that reward information was represented in the higher layer while effort was represented in the lower layer; in this case, effort information contextualized reward information. This mapping is equivalent to forcing the model to prioritize effort information. Finally, in the third condition (Simultaneous), the model was free to select the hierarchical level in which effort and reward information was mapped. This mapping is equivalent to the simultaneous presentation condition, i.e., the model is free to prioritize either effort or reward information.

Two factors influence how the HER model selects mappings of information to hierarchical levels (Alexander & Brown, 2015). The first factor concerns how effectively each source of information reduces uncertainty regarding the outcomes of behavior. When multiple sources of information are presented simultaneously, information that better reduces uncertainty regarding an outcome is preferentially represented at the lower hierarchical level, reflecting its utility in determining correct behavior. The second factor relates to the representation of information over time: stimuli presented at long delays before a response tend to be mapped to higher hierarchical levels, while temporally proximal stimuli that require an immediate response are mapped to lower hierarchical levels. Together, these factors suggest that contextual information, used to modulate processing of subsequent cues with immediate behavioral relevance, is prioritized, i.e., represented at superior hierarchical levels. The forced mappings in our simulations therefore correspond to experimental conditions in which reward information is prioritized (Reward 1^st) and effort information is prioritized (Effort 1^st). Our simultaneous mapping simulations correspond to the simultaneous condition in which subjects are free to view both sources of information ad libitum.

For both the performance and choice tasks, the model experienced two difficulty levels crossed with eight reward levels, allowing us to examine how the model's behavior changed with differences in reward and effort level. Eight reward levels were used (rather than the two used in the experiments) to ensure that model predictions were not due to a particular reward value. The model was simulated for 15,000 trials in each task and condition, for a total of 6 different simulation modes, and each mode was repeated 100 times.

To assess the importance of hierarchical organization on the behavior of the model, we simulated a flat version of the HER model with a single hierarchical layer. As with previous simulations (Alexander & Brown, 2015), the flat model was able to store multiple items in internal representation; thus, on each trial, the internal representations of the single hierarchical layer of the model included the current effort level and the current reward level. In order to simulate salience effects that may result from longer processing times for one stimulus component in our order manipulation, the internal representation in the model for the shorter-duration component was set to 0.5 (1.0 for the longer-duration component). In the simultaneous condition, both internal representations were set to 1.0.

Computational model. Simulations

The behavior of the model on both the choice and performance tasks differed between the forced mapping conditions “Reward 1st” and “Effort 1st” (Fig. 6b). It should be noted that, besides the manual mapping of reward and effort information to different hierarchical levels, all parameters and task contingencies were identical for these conditions; differences in behavior are therefore due entirely to the layer to which information was mapped. In the decision task, the model was generally more likely to accept effortful trials (i.e., select the nondefault option) in the “Reward 1^st” than in the “Effort 1^st” condition. Likewise, in the performance task, the model’s performance was generally higher in the “Reward 1^st” condition than the “Effort 1^st” condition. Generally, these findings suggest that prioritizing reward may attenuate the subsequently computed (prospective or experienced) effort.

Of additional interest is the model's behavior under the Free condition, in which effort and reward information were mapped automatically by the model to hierarchical levels. Here, we see that in the choice task, the model's behavior is more similar to the “Effort 1st” condition, while in the performance task, the model's behavior is more similar to the “Reward 1st” condition (Figs. 2c and 4c). Because the model was free to select the mapping of effort and reward information, the similarity in behavior implies that, for each task, the Free model selected mappings consistent with the prioritizing effort in the choice task, and prioritizing reward in the performance task. Analysis of the representations maintained at each level in the Free condition for each task confirmed this: the model is more likely to represent effort information in the higher level in the choice task and reward information in the performance task. Previous work (Alexander & Brown, 2015,2018) suggested that the mapping of information to hierarchical levels in the HER model is partly determined by how effectively each source of information reduces response or outcome uncertainty, with information more effective at reducing uncertainty mapped to lower hierarchical layers. In our Free simulations, reward information reduces uncertainty about choice more effectively than effort, while effort information more effectively reduces uncertainty about performance outcome.

In contrast to the above, simulations of the flat model showed no differences between simulation conditions in either the choice task, nor in the performance task. While observing no differences in behavior of the flat model does not necessarily confirm the importance of hierarchical structures in processing multidimensional stimuli, it does appear to rule out a simple salience account in explaining differences in behavior due to order effects. Hierarchical organization is one means by which order effects might be realized; alternately, order effects might also be due to lateral interactions between representational units (Botvinick, 2008).

Finally, our model simulations provide a theoretical framework for analyzing our empirical data. First, and most generally, our simulations suggest that the behavior of subjects under one of our order manipulations (Reward 1^st, Effort 1^st) may be more similar to behavior under the simultaneous condition (Free mapping for the model) than under the alternate order manipulation. In other words, this suggests that when reward and effort information are presented simultaneously, subjects may apply a preferred mapping of effort and reward information (equivalent to either prioritizing effort, or prioritizing reward).

Second, and more specifically, the model suggests the mapping of information to levels (Fig. 6a, “Reward 1^st” & “Effort 1^st”) may be task-dependent (Fig. 6c). Choice behavior in the Reward 1^st condition, for example, may be more similar to the simultaneous condition (reward is prioritized by the model), while performance behavior may be more similar for the Effort 1^st condition (effort is prioritized by the model). Finally, and most specifically, our simulations suggest how the specific tasks may be affected by the order manipulation: performance accuracy on the calculation task should be altered for the Effort 1^st condition, while choice behavior should be altered for the reward 1^st condition.

Computational model. Interim discussion

Results of model simulation of experiment 1 (performance task) were in line with subjects’ behavior: performance was altered primarily in the Effort 1^st condition (model: Fig. 6b, subjects: Fig. 2). Prioritizing reward had similar effects as simultaneous presentation (higher accuracy in the large reward condition), while prioritizing effort differed substantially, consistent with effects in the model derived from the forced-mapping of information to hierarchical layers (Fig. 6b-c). Additionally, the mapping learned by the model under the Free condition on the performance task corresponds to subject behavior in the simultaneous and Reward 1^st conditions, consistent with the hypothesis that task information is mapped to specific hierarchical layers in a task-dependent fashion (Fig. 6c).

Results of model simulation of experiment 2 were also in line with participants’ behavior. Choices were mainly altered in the Reward 1^st condition. Prioritizing effort led to similar choice behavior as in the simultaneous condition. Prioritizing reward resulted in increased effort acceptance.

Overall, the two qualitative predictions of the model, that prioritization may influence the hierarchical mapping of information and that prioritization of reward vs. difficulty may be task-dependent, are supported by our findings. Furthermore, the qualitative predictions describing the specific manner in which prioritization influences choice and performance also matched participants’ behavior.

General discussion

In this study, we investigated the hypothesis of a task-dependent hierarchical mapping of reward and difficulty information. We manipulated prioritization of cost and benefit information prior to task-performance and decision-making (either prioritizing reward or prioritizing difficulty information). We simulated the same tasks using the HER model, which postulates serial processing of information. Behavioral results confirmed the model predictions, showing task-dependent hierarchical mapping. In a performance task, reward information was prioritized, while in a decision-making task, difficulty information was prioritized. Furthermore, forced prioritization of costs reduced the effect or reward on performance accuracy, while forced prioritization of reward reduced the impact of rewards on choices. These results support both general and specific predictions of the HER model, suggesting that when multiple sources of information are simultaneously available, prioritizing one or the other source can dramatically affect behavior.

Previous evidence suggests that the order in which different information sources are presented affects subsequent behavior (Furnham & Boo, 2011; Kühberger, 1998), and the presence of order effects is well-known in the judgment and decision-making literature (Busemeyer, Wang, & Townsend, 2006; Furnham & Boo, 2011; Jacowitz & Kahneman, 1995; Kühberger et al., 1999). Despite the known influence of information order on subsequent behavior, there has been relatively little study of how order effects may influence motivationally salient information and if such effects would be consistent across different task-contexts, such as task-performance and choice. This is especially relevant to the field of cognitive neuroscience, in which extensive work has been conducted to identify regions of the brain that encode decision variables related to reward, cost, and value (Behrens, Woolrich, Walton, & Rushworth, 2007; Botvinick et al., 2009; Kolling, Behrens, Mars, & Rushworth, 2012; Kurniawan, Guitart-Masip, Dayan, & Dolan, 2013; O’Doherty, 2014; Rangel & Hare, 2010; Rushworth & Behrens, 2008; Rushworth, Kolling, Sallet, & Mars, 2012; Shenhav et al., 2013; Vassena et al., 2014; Verguts, Vassena, & Silvetti, 2015). Generally, studies of this kind provide information related to decision variables simultaneously and without controls on the subjective salience assigned to each information source, nor the behavioral task for which that information is to be used. One possibility raised by the present study is that associations of neural regions with a particular decision variable are confounded both by differences in the salience of various information sources, as well as the behavior that is to be informed by that information. A region identified as being related to reward representation, for example, may only represent reward due to salience-related processing differences, or due to the nature of the task (e.g., choice vs. performance).

Is there a reason to believe such confounds may exist? Generally, regions within PFC are identified as being critical to integrating information related to value-based decision-making, especially DLPFC and MPFC (Gläscher, Daw, Dayan, & O’Doherty, 2010; Lee, Shimojo, & O’Doherty, 2014; Rangel, Camerer, & Montague, 2008; Rushworth & Behrens, 2008; Vassena, Holroyd, & Alexander, 2017). Recent computational models of PFC postulate serial processing and hierarchical coding of such information (Badre, 2008; Badre & D’Esposito, 2007), where components of compound stimuli are coded at different levels of this neural hierarchy (Alexander & Brown, 2015, 2018; Collins & Frank, 2012; Collins & Frank, 2016; Koechlin, 2014). A consequent prediction of these models is that the different features composing complex stimuli are mapped on to different hierarchical levels, and the top-down integration of information along the hierarchy drives behavioral responses. However, it is frequently not specified how this mapping is realized, and it is generally declared by fiat that one dimension of a compound stimulus acts as a context variable modulating responses to the other dimension (Badre, Kayser, & D’Esposito, 2010). This assumption is clearly observed in the neuroeconomics literature in which the level of effort is frequently assumed to modulate subjective responses to reward (Botvinick et al., 2009; Botvinick & Rosen, 2009; Kivetz, 2003). Moreover, it may be that the optimal variable-to-level mapping varies depends on task requirements, and assumptions about the mapping that may be valid in one task context may be invalid in another, thus leading to inaccurate predictions. However, these assumptions had never been tested in the context of processing compound motivational information. Influential frameworks such as the Expected Value of Control theory (Shenhav et al., 2013) have proposed that cost and benefit may be integrated in a compound value signal (to determine the optimal level of control to be allocated) in so far as engaging in the effortful task leads to a reward. However, this account does not consider the possibility that prioritizing cost over benefit (over vice-versa) may alter behavior.

In experiment 1, our results demonstrate a clear improvement in accuracy when the high-reward cue is immediately available in the reward-first and simultaneous conditions, while presenting difficulty information first abolishes this reward effect. In experiment 2, prioritizing reward attenuates the difference in choosing to engage in hard vs easy trials. The comparison between the two experiments demonstrates that order of motivational information differentially affects different cognitive processes, namely task performance and decision-making. Separate instantiations of the HER model were used to model behavior for decision-making and performance, and it was observed that, for each task, the model learned a different mapping of effort and reward information to internal hierarchical levels. While we assumed that our order manipulations in experiment 1 and 2 promoted a forced mapping of effort and reward information (as in model simulation conditions Reward 1^st and Effort 1^st), our results remain ambiguous regarding how information is processed during the simultaneous condition. In the simultaneous condition, subjects were allowed an extended period of time (2,500 ms) to process both sources of information. This extended processing time might allow subjects to execute multiple processing strategies relevant to imminent tasks, i.e., subjects may be able to organize information into separate hierarchical structures related to choice behavior and task performance. While the HER model in its present form maintains hierarchically structured information about only a single task at any point, this does not rule out that multiple task-related hierarchical structures might coexist, either within the same region or in different hemispheres (Koechlin, Basso, Pietrini, Panzer, & Grafman, 1999), and that the mapping of information to structure is maintained independently for each active task hierarchy. In this case, one might expect that curtailing the time allowed for subjects to process information in the simultaneous condition could impose additional constraints on hierarchical mapping and potentially force subjects to prioritize processing strategies related to either choice or performance behavior. As a result, if subjects are only allowed time sufficient to execute one processing strategy during simultaneous presentation, the differences we observe in experiment 2 between the decision and performance phases may attenuate or disappear.

More generally, our results suggest that, during task performance, increased effort allocation may be promoted by presenting reward information first (i.e., participants performing better on high reward/hard trials). During decisions, however, optimal allocation seems to occur when effort information is presented first, as participants accept more often high reward/hard trials. This can be considered an adaptive behavior as task difficulty never exceeds participants’ capacities (i.e., the task is doable, with high overall accuracy) and such a choice grants a higher reward. In other words, presenting reward first seems beneficial for performance, while generally increasing acceptance during choice (irrespective of difficulty), which in some contexts may be considered an inefficient strategy. For example, in a limited resources context (i.e., patients with reduced attentional capacity or motivation), allocating effort only to circumstances granting more benefit might be especially relevant.

The future implications of these results are manifold. First, Experiment 1 suggests that changing the configuration in which information is presented—by prioritizing either benefit or cost information—can affect (and potentially improve) performance, and especially effort allocation. This might be particularly relevant for educational purposes in school settings. Moreover, this could be helpful to restructure task instructions for patients with disorders of motivation, such as depression. As a caveat, one should note that in the current experiments we only measured performance and choice, and we did not look at subjective effort investment, as typically evaluated through neurophysiological measures (Gendolla, 2012; Gendolla & Krüsken, 2001; Silvestrini & Gendolla, 2009). Future research could bring our results one step further, investigating for example effects of order of presentation on effort investment via measuring cardiovascular response. Moreover, using eye-tracking one could further specify strategic attentional allocation during cue presentation and task execution (both decision and performance), Second, we specifically designed and piloted this task to ensure high performance accuracy across easy and difficult conditions, under the assumption that participants know that they will successfully complete a trial if they decide to allocate effort and engage in it. However, it is possible that uncertainty about the outcome linked to a more difficult trial partially contributed to the measured effect, potentially leading to error avoidance effects. Error prediction has indeed been proposed as a mechanism affecting cognitive control exertion (Braver, 2012; Brown & Braver, 2007; Brown & Braver, 2005). To clarify this potentially contributing factor, future studies could exploit manipulations where accuracy is matched between easy and hard trials, specifically aiming at disentangling effects of prioritizing effort as compared to prioritizing outcome uncertainty on decisions. In any case, this factor could not account for the differences in choices as a function of prioritization.

Third, our results support the HER model as a plausible candidate mechanism underlying processing of task-relevant information across motivation and cognitive control contexts. Future work should directly test neural predictions of the HER model (for lateral and medial PFC activity), besides the behavioral predictions investigated here. Furthermore, neurophysiological data in animals suggests that dissociable subcortical circuits are implicated in cost-benefit decisions (the ventral tegmental area) and energization of effortful behavior (substantia nigra, Varazzani, San-Galli, Gilardeau, & Bouret, 2015). These task conditions are conceptually comparable with our choice and performance manipulation. Future studies should investigate in humans the possibility that differential effects of prioritization on performance versus choice may be mediated by different neural cortico-subcortical circuits, as recent computational work suggests (Silvetti, Vassena, Abrahamse, & Verguts, 2018).

Fourth, Experiment 2 indicates that prioritizing benefit over cost information influences preference and decision. Recent research showed altered effort-based decision-making in patients with depression (Silvia et al., 2016; Treadway, Bossaller, Shelton, & Zald, 2012; Yang et al., 2014), schizophrenia (Barch, Treadway, & Schoen, 2014; Culbreth, Westbrook, & Barch, 2016; McCarthy, Treadway, Bennett, & Blanchard, 2016), and bipolar disorder (Hershenberg et al., 2016). Generally, these populations tend either to avoid more effortful tasks or to show inefficient effort allocation. Training based on prioritization might reveal new strategies to improve efficiency in effort allocation.