Reward Responsiveness, Learning, and Valuation Implicated in Problematic Pornography Use — a Research Domain Criteria Perspective

Problematic pornography use (PPU) describes a pattern of behavior characterized by excessive time spent using or thinking about pornography and continued use despite negative consequences. To help advance the understanding of transdiagnostic underlying psychological and neurobiological mechanisms in PPU, we aim to review existing evidence on these mechanisms focusing on positive valence systems within the transdiagnostic Research Domain Criteria (RDoC) framework. Reward anticipation processes seem to be increased in individuals with PPU symptoms when they anticipate sexual stimuli compared with other rewards. Studies further suggest that the initial neural and attentional responses to sexual rewards compared with different control stimuli are also increased in individuals with PPU symptoms, as are conditioned responses in sexual reward learning paradigms. Sexual reward valuation studies point towards an increased neural value differentiation with increasing PPU symptoms. The current state of evidence indicates that positive valence systems are altered in persons with PPU. This framework of organizing evidence may aid in elucidating PPU development and maintenance as well as planning future studies.


Introduction
While most people experience the use of pornography as relaxing, fun, and sexually exciting [1,2], some individuals develop a problematic use pattern. Problematic pornography use (PPU) is mainly characterized by excessive time spent using or thinking about pornography, loss of interest in other activities, impaired self-control over the use, failed attempts at reducing the use, and continuation or escalation of use despite negative consequences resulting from the use [3,4]. It can be estimated that 0.8-8% of pornography users show symptoms of PPU [5]. PPU can be considered as a subtype of compulsive sexual behavior disorder (CSBD; [6]), and research has shown that the majority of patients seeking treatment for CSBD report PPU as their primary problematic sexual behavior [7]. Therefore, current opinions view PPU as a subtype of this construct [8,9] and also assume that many patients presenting with CSBD will show PPU as their primary problematic sexual behavior. Since the inclusion of CSBD in the International Classification of Disease (ICD-11; [10]) in 2019, CSBD Topical Collection on Sex Addiction. is now commonly used as an umbrella diagnosis for PPU symptom patterns.
Classifying PPU as CSBD may be useful for clinical settings, and the ICD-11 and DSM-5 provide the common ground necessary to facilitate communication among clinicians. However, the concept of disorders belonging to distinct categories (e.g., mood disorders, anxiety disorders, addiction) is not enough when the goal is to understand the underlying psychological and neurobiological mechanisms involved in the development of problematic behaviors such as PPU and to eventually develop effective treatments [11,12]. The Research Domain Criteria (RDoC; [13]) present an alternative framework that aims to describe disorders not by category but by underlying mechanisms at the genetic, molecular, physiological, and behavioral level. These mechanisms are represented in the domains (1) negative valence systems, (2) positive valence systems, (3) cognitive valence systems, (4) systems for social processes, (5) arousal/regulatory systems, and (6) sensorimotor systems (National Institute of Mental Health; [14]). In this review, we aim to summarize the current state of PPU literature from an RDoC perspective to benefit from its transdiagnostic dimensional approach. Specifically, we focus on the domain of positive valence systems and its subconstructs because most experiments published in connection with PPU/CSBD lie within this domain. Furthermore, a recent study that examined patients with CSBD and healthy control participants in behavioral tasks from the domains of negative valence, positive valence, cognitive systems, and sensorimotor systems found group differences in the positive valence domain only [15••]. For an overview of positive valence systems according to the RDoC, please see Table 1 (definitions adapted from the NIMH website; [14]).
In this review, we included all studies that employ reward in a controlled experimental setting and assess symptoms of PPU. We aimed to review clinical studies including patients with CSBD (assessed via clinical interview), as well as studies with general population samples, which conducted a dimensional PPU measure (for a review of measures see [16]). Correlations of dimensional PPU measures with reward processes in these samples may indicate mechanisms in early subclinical problematic behaviors [17].

Positive Valence Systems in PPU
In the following, we summarize and discuss PPU study results within the respective constructs, sorted based on the reported task design and analysis. For an overview of all included studies, as well as the dimensional indicators of PPU, please see Table 2.

Reward Responsiveness
Many studies have examined Reward Responsiveness in both clinical samples and in general population samples in connection with a dimensional PPU measure. We discuss studies concerning the subconstructs reward anticipation (sexual incentive delay (SID) task) and initial response to reward (functional magnetic resonance imaging (fMRI)) and behavioral passive viewing, approach-avoidance task, dot probe task, stroop task). We could not identify any studies examining reward satiation and PPU.

Reward Anticipation
The SID task is an adaption from a widely established paradigm to examine reward anticipation (monetary incentive delay task, [27,28]). The task generally contains simple visual cues (symbols, geometric shapes) that signal the possibility of "winning" a reward in the trial but only if the participant reacts quickly enough to a target that is presented after the cue. Different versions of the SID task have been used mostly in fMRI studies [29•, 30, 31] and one behavioral study [15••] to examine reward anticipation in connection with PPU symptoms. The informative SID variant [32] contains a sexual condition with a potential sexual picture reward, a money condition with a potential monetary reward, and a control condition with no potential reward. During target presentation, the participants press one of two buttons quickly, depending on one of two shapes that is presented as the target. Additionally, the sexual cue and monetary cue contain information concerning the magnitude (more or less explicit sexual picture, more or less money) and the winning probability (25%, 50%, or 75%) of the potential rewards. Due to this additional information in the cues, outcomes in this task likely reflect both reward anticipation and reward valuation processes. Two studies have used this informative SID task to examine anticipation and valuation processes in participants with PPU; here, we discuss the anticipation effects. An fMRI study [29•] found that participants with PPU and control participants differed in their striatal responses to sexual cues but not in their responses to sexual pictures themselves. Participants with PPU showed increased ventral striatal activity specifically for sexual cues but not for monetary cues compared to the control group. Across the whole sample, relative ventral striatal sensitivity to sexual cues versus monetary cues was significantly related to the increased behavioral motivation to view sexual images (reaction time difference between sexual and money trials) and several dimensional indicators of PPU. The shortest reaction times overall were found in participants with PPU in sexual trials. A behavioral study [15••] recently employed the informative SID to examine behavioral outcomes. They found interaction effects in reaction time and subjective ratings of the rewards: Reaction times of patients with CSBD were shorter in sexual trials, while those of control participants were shorter in monetary reward trials. Patients also rated all sexual rewards and all large rewards higher than the control group. A simpler version of the SID contains a sexual condition with a sexual video reward, a control condition with a control video reward (pleasant, non-sexual massage video), and another control condition with no potential reward [30]. This basic SID also uses only one target with a simple one-button reaction. In the basic SID, no relationship was found between the neural activity during sexual cues or sexual videos contrasted with the control condition with any dimensional indicators of PPU in a generally healthy sample [30]. Interestingly, the authors assessing the healthy sample under acute stress found that a dimensional PPU indicator moderated the effect of an acute cortisol stress response on neural activity related to the sexual cues and sexual videos [31]. Participants who reported more time spent on pornography use showed less neural reactivity to sexual cues and videos in the orbitofrontal cortex, the more stressed they were, measured by the individual cortisol response. Orbitofrontal cortex activity reflects the abstract value of rewarding stimuli [33,34], so to these participants, the anticipated and experienced stimuli may have been devalued in the stressful situation.
In summary, the results from the informative SID studies point to behavioral and neural reward anticipation processes that are sensitized towards sexual over monetary rewards in participants with PPU as the popular incentive sensitization theory of addiction proposes [35]. This theory postulates that repeated use of a substance sensitizes reward circuitry to cues associated with substance use, and attributes increased incentive effects to these cues. Transferred onto PPU, the reward circuitry would attribute increased incentive salience to cues that signal pornography use. In contrast, the basic SID did not show similar results with dimensional indicators of PPU -effects on reward anticipation only emerged when participants were under acute stress. One reason for these discrepancies between studies could be that incentive sensitization in PPU-related anticipatory processes are not yet visible in subclinical participants. Another likely reason are the differences between the two tasks that may have engaged slightly different underlying processes, such as the control condition (money reward versus massage video reward). The different results indicate that anticipation processes may be sensitized towards sexual compared to monetary reward but not compared to other immediate non-sexual rewards. A very interesting difference is the reward value information contained in the informative SID cues because effects of this information in connection with PPU were also found [29•]. Notably, effects of PPU on neural processing of subjective reward value in the subclinical sample were also found in the basic SID [36]. Since these results belong to the reward valuation construct, they are further discussed in "Reward Valuation." From the current state of evidence, motivational components of reward anticipation for sexual rewards seem to be increased in participants with PPU symptoms, visible in behavioral data and neural activity -but only under certain conditions. The results may indicate a potentially important mechanism involved in PPU, but further research is needed to disentangle how incentive sensitization may change in different situations and with increasing disorder severity and how it interacts with reward valuation.
Voon et al. [37•] found that patients with CSBD demonstrate greater activity in the ventral striatum, dorsal anterior cingulate cortex (dACC), and amygdala when viewing explicit sexual videos compared with videos of exciting sporting activities. Using pictures instead of videos, another study found increased activity in dACC, thalamus, dorsal striatum (caudate nucleus), and prefrontal cortex in patients in an explicit sexual-exciting sports contrast [38]. The authors further found the thalamic and prefrontal activity in this contrast to be positively related with dimensional indicators of PPU. Contrasting work found a negative correlation between time spent using pornography and dorsal striatal (putamen) reactivity to similar sexual pictures in a general population sample [39]. A series of behavioral studies examined the change in self-reported sexual arousal induced by viewing explicit sexual pictures in different samples. They found similarly high correlations between a dimensional indicator of PPU and task-related change in sexual arousal in heterosexual men [40], homosexual men [42], and heterosexual women [41].
The fMRI findings indicate that direct neural responses to sexual rewards are altered in patients with CSBD, likely reflecting increased motivational salience of sexual rewards [51][52][53][54][55][56]. The frequent use of pornography for instant sexual gratification, relaxation, and stress-relief can be seen as repeated learning sessions through which the association between visual sexual stimuli and a positive outcome is strengthened in patients with PPU. These increased neural responses may only be visible in clinical samples because for these individuals, we can assume a more excessive individual learning history, and thus increasingly firm associations between sexual stimuli and positive outcomes coming into the experiment. Furthermore, sexual videos rather than pictures are probably more similar to the stimuli that pornography users are familiar with [57] which may be why they seem to recruit more of the established conditioned reward regions [51]. The self-reported larger increase in sexual arousal in individuals with more PPU symptoms could reflect a similar mechanism. However, self-report as a measure of sexual arousal is probably influenced more strongly by cultural and societal norms regarding pornography use and individual attitudes towards sexuality [58][59][60] so it might be less reliable. Altogether, initial response to reward regarding direct physiological responses to sexual rewards is strongly involved in PPU. Most studies suggested increased responses to disorder-related rewards as opposed to only one study indicating blunted reward responses [39].
The other studies in this domain measure behavioral approach/avoidance biases (AAT; [61]) and attentional biases towards sexual rewards (dot probe, stroop; [62]). In this version of the AAT with sexual stimuli, participants are instructed to push stimuli away from or pull them towards themselves by means of a joystick attached to the computer [63]. Relative differences in times are interpreted as approach (i.e., shorter pull than push reaction times) or avoidance biases (i.e., shorter push than pull reaction times), respectively, for a given picture category. Sklenarik et al. [44] have reported an overall approach bias for sexual compared to neutral images and found a positive relationship between a dimensional indicator of PPU and the approach bias scores in a sample of heterosexual men from the general population. With the same task in a sample of heterosexual women from the general population, the authors reported a slightly smaller approach bias for sexual compared to neutral images [45], but also a positive relationship between the sexual approach bias and another indicator of PPU. Another study including men and women from the general population, using an AAT with negative and positive emotional pictures alongside sexual and neutral pictures, found no stronger overall approach bias for sexual pictures compared to the other pictures [46]. However, the approach bias for sexual pictures here was also positively correlated with an indicator of PPU. Another AAT study [43] did not find any linear relationship between indicators of PPU and approach or avoidance biases towards sexual compared with neutral stimuli but a u-shaped association with both approach and avoidance tendencies being associated with symptom severity. Some findings point to the fact that physiological conditions (i.e., alcohol intoxication) and/or psychological conditions (i.e., arousal) may foster biased responding towards sexual stimuli and that such biases are associated with sexual risk intentions [64]. Overall, results concerning the approach bias towards sexual stimuli as an indicator of compulsive sexual behavior are still mixed, and the potential mechanism is not completely understood. The dot probe task measures attentional bias by presenting two pictures, on either side of a screen; in this case, one sexual and one control picture. After the picture offset, a dot is presented in the location of one of the former pictures, and the participants are instructed to react quickly to this dot. Quicker reaction time to the dot when it occurs in the previous location of a sexual picture and slower reaction time when it occurs in the previous location of the control picture is interpreted as greater attentional bias towards sexual pictures. In such a paradigm (but with arrows instead of dots), Pekal et al. [48] found a dimensional indicator of PPU to be associated with the attentional bias towards sexual stimuli. A similar study [47] also found a greater attentional bias towards explicit sexual pictures in patients with CSBD compared to the control group. The attentional bias for sexual words measured via reaction time in a stroop task [49] was also found to be positively related to a dimensional indicator of PPU. Finally, in a more recent stroop task using sexual images with colored frames, Wang et al. found that a group with high scores on a PPU measure displayed a greater attentional bias towards sexual images via reaction time interference effects and event-related potentials compared to a group with low PPU scores [50].
The discussed evidence points towards increased approach and attentional biases for sexual stimuli associated with PPU symptoms. This would fit with the fMRI data, supporting the notion of increased sexual stimuli salience due to more prior learning experiences, but the results are difficult to integrate because of significant differences in designs. For example, AAT studies with the most straightforward results used pictures of neutral objects as the only control condition for sexual pictures [44,45]. The sexual approach bias from such designs could represent a genuine bias towards sexual content. Concurrently, it could also represent a mere bias towards pictures of humans. Hence, more rigorous study designs with different stimulus categories are essential to disentangle varying accounts. Furthermore, while the AAT and the dot probe task are useful proxies for attention and approach processes, it may be beneficial to use other, more direct measures to further assess behavioral initial response to reward in connection with PPU. One possibility would be to capture overt attention via eye tracking (e.g., dwell time, scan path) during free exploration of complex pictures with explicit sexual content.

Reward Valuation
Most studies that exist in this domain engage both reward responsiveness and reward valuation in the tasks they employ. Many classic reward valuation paradigms such as forced-choice preference, willingness-to-pay, risky decision-making, or probability discounting (for a review of valuation studies see [65]) have not yet been employed in connection with PPU. One delay discounting study [66] found stronger devaluation of monetary rewards due to delay in participants who spent more time on pornography use in a general population sample but did not include sexual rewards. No other studies concerned with delay or effort and PPU were identified. In the following, we discuss reward (ambiguity/risk) studies that used sexual rewards and examined reward valuation processes in task design or analysis [15••, 18, 29•, 36, 67].

Reward (Ambiguity/Risk)
The results of these studies point towards altered valuation of sexual rewards in participants experiencing PPU symptoms. As we already touched upon before, the cues in the informative SID [32] contain information on magnitude and probability of the following sexual and monetary rewards, i.e., reward value information. Gola et al. [29•] found that the generally stronger ventral striatal activity in response to sexual cues in patients with PPU was modulated by the magnitude presented in the cue. In the basic SID, the individual ratings of sexual videos and neural activity in ventral as well as dorsal (caudate nucleus) striatum were found to be correlated more strongly in participants who reported more PPU symptoms [36]. In another fMRI study, higher ventral striatal responses to preferred sexual pictures compared to nonpreferred sexual pictures were found in a general population sample [67]. This ventral striatal activity relative to preference was again correlated with self-reported PPU symptoms. Finally, behavioral studies found PPU symptoms to be correlated with sexual arousal ratings [18,68] and viewing times of preferred pictures [18] in general population samples. A self-selected PPU group also rated generally non-preferred sexual pictures (homosexual male/male pictures, heterosexual sample) higher than the control group [18].
The increased association between subjective value of sexual rewards and (ventral) striatal activity indicates that this region differentiates more strongly between differently preferred stimuli, the more PPU symptoms a participant experiences. The increased differentiation of incentive value signals in the striatum might reflect a heightened sensitivity to sexual reward value information, perhaps mediated by a greater need for seeking, identifying, and responding to preferred sexual stimuli during PPU development. A value differentiation effect like this was however not found in behavioral data [18]. The clinical (patients with CSBD) sample in the same study even seemed to show a generalization of preference across unexpected stimuli [18]. Altogether, reward valuation of sexual stimuli seems to be a crucial factor in PPU and should be researched further, with more studies directly targeting valuation processes (e.g., active choice paradigms; [65]). Value differentiation as well as value generalization effects should be given much attention in future studies, especially when looking at PPU development versus maintenance.

Reward Learning
The studies that examined reward learning as a factor in PPU have used classical appetitive conditioning paradigms [69•, 70, 71] and thus belong to the probabilistic and reinforcement learning subconstruct. Classical appetitive conditioning describes the process by which an initially neutral stimulus (conditioned stimulus; CS +) is repeatedly paired with a reward (unconditioned stimulus; UCS). In differential conditioning paradigms, another neutral stimulus (CS −) is never paired with the UCS. After few pairings, participants show increased responses to the CS + compared to the CS − , such as increased valence and arousal ratings, elevated skin conductance responses (SCRs), and activities of the brain reward circuitry [72][73][74][75]. Studies explicitly examining reward prediction error or habit in PPU could not be identified.

Probabilistic and Reinforcement Learning
Banca et al. [70] examined the reactions of patients with CSBD and control participants in conditioning paradigms inside the MRI scanner and outside. Outside of the scanner, participants took part in two differential behavioral conditioning tasks, one with a CS + money that announced an UCS money (£1) and one with a CS + sexual that announced an UCS sexual (photo of naked woman). Both tasks also contained a CS − that announced nothing/the absence of the UCS. Conditioned reactions to the CS + were probed afterwards in a forced-choice preference task. Patients with CSBD were more likely to select the CS + sexual versus the CS − compared to control participants. However, patients also showed greater preference towards the CS + money compared with CS − than the control group. Thus, patients showed greater conditioned preference overall, independent of UCS type. In the MRI, the conditioning task contained all three conditions (sexual, money, nothing). Two CS per condition (CS + sexual , CS + money , CS −) were paired with UCS sexual , UCS money , or nothing, similar to the behavioral task. There were no group differences in CS-related activity. However, activity associated with viewing the UCS sexual in the dACC fell steeper over the experiment than activity for UCS money in the CSBD group compared to control group. In the control group, there was no change in activity towards either UCS over the experiment. Another classical conditioning study, [69•] examined the neural activity during a task with no money condition and a more explicit UCS sexual , also in a clinical sample. Thus, the stimuli were CS + sexual , CS − , UCS sexual , and nothing/absence of the UCS. The authors found increased amygdala activity for the CS + sexual versus the CS − in the CSBD group compared to the control group. Additionally, they found decreased functional connectivity during the contrast CS + sexual > CS − between the ventral striatum and prefrontal cortex in the CSBD compared to control group. Finally, in a behavioral appetitive conditioning study [71], a CS + sexual was always paired with a UCS sexual and a CS − with a UCS neutral . The authors found that the more PPU symptoms a participant reported, the higher they rated the CS + sexual on sexual arousal after conditioning as compared to before.
The findings suggest enhanced conditioned responses in patients with CSBD, visible through behavioral preference, ratings, and amygdala activity. Furthermore, the decreased coupling between prefrontal cortex and ventral striatum indicates decreased inhibitory control, which may facilitate trigger-responses [76,77]. The UCS sexual -specific decrease in dACC activity in patients with CSBD [70] may be explained by a decrease in predicted action-outcome value over the experiment [55]. Therefore, from the studies we discussed here, appetitive conditioning seems to be an important factor in PPU. However, it is not yet clear if these effects are specific for learning situations that involve sexual UCS, due to patients' extensive experience with these stimuli before the experiment. It might also be the case that patients show a generally increased reward sensitivity as proposed by impulsivity theories of addiction [78][79][80] and supported by the behavioral data [70]. Another reward learning factor considered important in most other addictions is appetitive extinction [81,82]. Extinction describes the process whereby the association between CS + and UCS becomes weakened, and the individual learns that the CS + no longer signals the UCS. One important factor in PPU, which some authors conceptualize as a disorder due to addictive behaviors and not as subtype of CSBD, is that patients keep using pornography, even when the once positive outcomes can no longer be expected; hence, extinction deficits can be assumed to play an important role in explaining the maintenance of this behavior [3,83]. Banca et al. [70] included an extinction task after conditioning where all CS were shown again without the UCS. However, no group effects were found during extinction, so the role of this process in PPU remains inconclusive at this point. Altogether, reward learning, classical conditioning in particular seems to be an important mechanism in PPU, but there is still much left to shed light on such as the roles of temporal difference learning, instrumental conditioning, extinction, and generalization.

Limitations
Some factors that limit the conclusions we can draw regarding positive valence systems for PPU are sample characteristics, the diversity of sexual and control stimuli, and the diversity of PPU measures. There is a focus on heterosexual men in the studies we identified. Only in the subconstruct initial response to reward, we found studies also including heterosexual women [41,45,46,48,49] and one study including homosexual men [42]. Therefore, findings related to reward anticipation, reward valuation, and reward learning while promising are for now still limited to one demographic subgroup. Notably, sexual orientation is assessed dimensionally (via the Kinsey scale; [84]) and reported transparently in most studies, which is not the case for gender identity. The current lack of measurement and reporting of sample gender information [85] makes it difficult to know for example whether trans and/or nonbinary individuals were included and whether the authors assessed sex assigned at birth (suggested by the use of "male"/ "female" terminology) or gender identity ("men"/ "women" terminology). More transparent and rigorous reporting of gender and inclusion of more than one gender and sexual orientation group in studies on PPU is especially important when we consider that these person characteristics have been shown to interact with other behavioral addictions as well as substance abuse [86][87][88]. An issue complicating the integration of findings across experiments is the diversity of stimuli. Experimental control stimuli ranged from generally rewarding pleasant or arousing stimuli over neutral person images to objects such as furniture or household items. Stimuli in the sexual condition ranged from pictures of dressed women over images of naked people to explicit images or videos of sexual intercourse. Comparisons between these varying types of sexual and control stimuli may reflect a variance of underlying mechanisms. A similar point can be made for the range of different dimensional indicators of PPU that were used. Many different PPU questionnaires are in use currently [16] but especially the assessment of only time spent using pornography as a proxy for problematic use patterns is debated [26].

Conclusion
The current state of literature indicates that the RDoCpositive valence systems are important factors in PPU. For reward anticipation, the evidence indicates incentive sensitization towards stimuli announcing sexual rewards in patients with PPU, but it is still unclear which situational factors contribute to this process. Similarly, most studies found increased initial response to reward, as shown in neural responses and attentional/approach biases to sexual rewards associated with PPU symptoms. This could be due to patients' prior learning history with these rewards. Reward valuation studies reported an increased differentiation of sexual reward values associated with PPU. This might reflect heightened sensitivity to sexual reward value information which could be a precursor or consequence of PPU. Finally, reward learning studies have shown increased conditioned responses to stimuli associated with sexual reward in classical conditioning paradigms. It is still unclear whether this is specific for sexual rewards or if patients with PPU are characterized by a general disposition to reward learning due to overall increased reward sensitivity.
Funding Open Access funding enabled and organized by Projekt DEAL. The work on this article was carried out in the context of the Research Unit ACSID, FOR2974, funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) -411232260.

Conflict of Interest
The authors declare no competing interests.
Ethics Approval This article does not contain any studies with human or animal subjects performed by any of the authors.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.