In this Registered Report, we assessed the utility of the affective priming paradigm (APP) as an indirect measure of food attitudes and related choice behaviour in two separate cohorts. Participants undertook a speeded evaluative categorization task in which target words were preceded by food primes that differed in terms of affective congruence with the target, explicit liking (most liked or least liked), and healthiness (healthy or unhealthy). Non-food priming effects were tested as a manipulation check, and the relationship between food priming effects and impulsive choice behaviour was also investigated using a binary food choice task. As predicted, priming effects were observed for both healthy and unhealthy foods, but there was no difference in the magnitude of these effects. This may suggest that the paradigm is most sensitive to affective, but not cognitive, components of attitudes (i.e., healthiness), but alternative theoretical explanations and implications of this finding are discussed. Food and non-food priming effects were observed in both reaction time (RT) and error rate (ER) data, but contrary to expectations, we found no association between food RT priming effects and choice behaviour. All findings from confirmatory analyses regarding RT and ER priming effects, and the absence of the expected correlations between priming effects and impulsive food choices, were successfully replicated in the online cohort of participants. Overall, this study confirms the robustness of the APP as an indirect measure of food liking and raises questions about its applied value for research of eating behaviours.
There is an emerging need for a greater understanding of attitudes towards foods that may drive unhealthy eating behaviours, such as overeating. Attitudes reflect “object-evaluation associations” that can be retrieved from memory and influence behaviour towards the attitude object (Klauer & Musch, 2003). For example, individuals may respond positively to a food that contains intrinsically rewarding ingredients (e.g., sugar, fat), with the positive evaluation automatically activated by the learned association between reward and consumption. Evaluations of foods arise from both affective and cognitive components of attitudes (Marty et al., 2017). The affective component reflects an individual’s hedonic reaction to the sensory properties of foods, commonly referred to as food liking, which is a central determinant of dietary choice (Eertmans, Baeyens, & Van den Bergh, 2001). The cognitive component may involve thoughts about the nutritional value of a food item and potential health consequences (Trendel & Werle, 2015). This study examined the methodological validity of an indirect measure of attitudes—the affective priming paradigm (APP; Fazio & Olson, 2003; Fazio, Sanbonmatsu, Powell, & Kardes, 1986; Hermans, De Houwer, & Eelen, 2001; Klauer & Musch, 2003)—and the extent to which priming measures were sensitive to affective (i.e., liking) and cognitive (i.e., healthiness) components of food attitudes. The association between priming measures and food-choice behaviour was also investigated.
The interplay between affective and cognitive components of attitudes may be paramount to the understanding of eating behaviours, including food choices. Appetitive foods and their cues, such as sight or smell, can induce positive affective reactions (Blechert, Meule, Busch, & Ohla, 2014) and activate the brain’s reward circuits associated with “wanting” and “liking” (Berridge, Ho, Richard, & DiFeliceantonio, 2010). In food-rich societies, where high-calorie foods are heavily promoted, such cue-evoked positive reactions are frequent and can drive impulsive food choices (Zoltak, Veling, Chen, & Holland, 2018) that likely contribute to overeating and other unhealthy eating behaviours (Berridge et al., 2010; Lawrence, Hinton, Parkinson, & Lawrence, 2012; Sato, Sawada, Kubota, Toichi, & Fushiki, 2016). These impulsive food choices are not guided by deliberate processes, such as the consideration of consequences (Veling et al., 2017). Cognitive components of attitudes include social norms and individual beliefs about the attitude object (i.e., foods), such as nutrition and health consequences, and should be considered as determinants of eating behaviours (Eertmans et al., 2001). Interestingly, cognitive and affective components of attitudes can interact, as implicit measures can be influenced by various sources of valence, such as caloric content, economic cost, and effects on one’s health (Verhulst, Hermans, Baeyens, Spruyt, & Eelen, 2006). For example, unhealthy foods can be perceived to be tastier than healthy foods and chosen for consumption more frequently, even if individuals are not consciously aware of the association between healthiness and tastiness (Ackermann & Palmer, 2014).
The APP has been previously applied to the food domain as an implicit, or indirect, measure of attitudes (e.g., Lamote, Hermans, Baeyens, & Eelen, 2004; Roefs, Herman, MacLeod, Smulders, & Jansen, 2005a). The current study employed a variant of the APP where attitude objects are presented as primes and are unrelated to the primary task of identifying the evaluative connotation of target words presented after the primes (Fazio & Olson, 2003). Participants were asked to perform an evaluative categorization task, identifying target words as either positive or negative when preceded by either most liked (i.e., positive) or least liked (i.e., negative) food primes (see Figs. 1 and 2). Here, the main outcome of interest is the affective priming effect, which manifests as faster responses (and/or lower error rates) on affectively congruent (i.e., most liked food-positive target or least liked food-negative target) than incongruent trials (i.e., most liked food-negative target or least liked food-positive target). In contrast to other indirect measures of (food) attitudes, such the implicit association test (Greenwald, McGhee, & Schwartz, 1998), this APP task variant does not require an evaluative response towards the prime, and participants are explicitly instructed to not pay attention to the primes (pictures or words). Affective priming effects can be explained by response competition/facilitation processes (Fazio & Olson, 2003; Wentura & Degner, 2010; but see Discussion section), and in the food domain they are often utilized as a measure of liking or preferences. We posit that such priming measures may be influenced by both affective and cognitive components of attitudes, and their association with food-choice behaviour should be examined further.
The APP has been shown to capture the evaluation of foods (i.e., liking) through observed priming effects for both reaction times and error rates (Lamote et al., 2004), even when attitudes were only recently acquired in laboratory settings (Verhulst et al., 2006). Although the affective component of food attitudes has been successfully investigated using the APP, previous studies have yielded mixed evidence for its utility in identifying the influence of cognitive components, such as health-related values, on implicitly measured food attitudes. While some studies have found that healthiness or fat content may have no influence on the affective priming effect (Becker, Jostmann, Wiers, & Holland, 2015; Roefs, Herman, et al., 2005a), other evidence suggests that priming can reflect preference for low-fat over high-fat palatable foods, potentially attributed to health concerns (Roefs, Stapert, et al., 2005b).
Overall, there has been moderate evidence to suggest that the APP can tap into the affective components of foods. This study aimed to address three questions that are central to establishing the methodological utility of the APP in eating behaviour. First, can priming effects be obtained for most liked and least liked foods, as expected by previous findings? Second, is this paradigm sensitive to cognitive components of attitudes, such as the healthiness of the foods? Finally, are priming effects for foods that vary in liking and healthiness associated with impulsive choices to consume these foods?
The study tested several confirmatory hypotheses regarding the utility of the APP as an indirect measure of food attitudes. Priming effects were examined using both median reaction times for correct responses (RTs) and error rates (ERs). The relationship between priming measures and impulsive food choices were also investigated. A schematic diagram of the APP contrasts and selected hypotheses is shown in Fig. 1. In the Preregistered Analyses section, statistical tests for four categories of predictions (H1–H4) are outlined. These were the exclusive set of a priori hypotheses. For confirmatory analyses, all hypotheses were tested and reported with no changes to the specified independent variables, dependent variables, or any other variables, variable derivations, stated statistical transformations, or data exclusions within each test. The hypotheses, analyses, manipulated, and nonmanipulated variables, and measurements in the Methods and Analyses sections were therefore complete, necessary, immutable, and exclusive for all preregistered confirmatory outcomes.
H1. Positive priming effect for non-food primes as a manipulation check for the APPFootnote 1
H1a. RTs would be, on average, faster in congruent than in incongruent non-food prime trials.
H1b. ERs would be, on average, lower in congruent than in incongruent non-food prime trials.
H2. Priming effects (RTs) for healthy and unhealthy foods
H2a. RTs would be, on average, faster in congruent than in incongruent food prime trials.
H2b. RTs would be, on average, faster in congruent than in incongruent healthy food prime trials, specifically.
H2c. RTs would be, on average, faster in congruent than in incongruent unhealthy food prime trials, specifically.
H2d. The priming effect (RT difference scores) would be, on average, greater for unhealthy than for healthy most liked food prime trials (see Measures and Indices section for priming effect calculation).
H3. Priming effects (ERs) for healthy and unhealthy foods
H3a. ERs would be, on average, lower in congruent than in incongruent food prime trials.
H3b. ERs would be, on average, lower in congruent than in incongruent healthy food prime trials, specifically.
H3c. ERs would be, on average, lower in congruent than in incongruent unhealthy food prime trials, specifically.
H3d. The priming effect (ER difference scores) would, be on average, greater for unhealthy than for healthy most liked food prime trials.
H4. Relationship between food choices and observed priming effects (RTs)
H4a. The probability of choosing a most liked over a least liked food from within a pair of healthy food stimuli would positively correlate with the priming effect (RTs) in healthy food prime trials.
H4b. The probability of choosing a most liked over a least liked food from within a pair of unhealthy food stimuli would positively correlate with the priming effect (RTs) in unhealthy food prime trials.
H4c. The probability of choosing an unhealthy over a healthy most liked food would positively correlate with the difference in priming effects (RTs) between unhealthy and healthy most liked food prime trials.
Preregistered hypotheses for priming effects were proposed for both speed (RT) and accuracy (ER) measures. In response priming procedures without strict time windows (e.g., 300–450 ms) priming effects are most commonly observed in RTs (Wentura & Degner, 2010), but we assume that such effects may be observed in either speed and/or accuracy performance (RTcon < RTinc and/or ERcon < ERinc). In addition, accuracy data should be inspected for potential speed–accuracy trade-offs. For example, participants could purposefully delay their responses on incongruent trials to improve accuracy, producing a priming effect for RTs, but a reverse effect for error rates (i.e., ERcon > ERinc). Therefore, support for observed priming effects would be dependent on both speed and accuracy hypotheses, as shown in the expressionFootnote 2 below, where there should be no effects in the opposite direction (RTcon > RTinc or ERcon > ERinc), and there should be evidence for either RT or ER priming effects (RTcon < RTinc or ERcon < ERinc). A contingent analysis plan for testing these hypotheses (i.e., follow-up tests) when the effects were not in the expected direction was preregistered (see Preregistered Analyses).
Data collection protocol
The study was undertaken in both laboratory (group testing) and online settings using Inquisit 5 (http://www.millisecond.com). The study protocol was matched for the two collected data sets, which were analyzed and reported separately. The primary data set stemmed from the laboratory setting, as this would allow us to examine consequential food choices (see Food-Choice Task section). The online data set would directly replicate any findings on the APP as an indirect measure of food attitudes (H1–H3) and examine whether priming measures were associated with nonconsequential food choices (i.e., choices are not motivated by the offer of real food at the end of the experiment). This data collection protocol would also provide insights into data quality and potential differences in the utility of the APP between laboratory and online settings (see Data Quality Checks in Supplementary Material).
Sampling plan and participants
A sequential Bayes factor (SBF) design (Schönbrodt, Wagenmakers, Zehetleitner, & Perugini, 2017) was employed with a predefined minimum sample size (nmin = 40) and a maximum number of participants (nmax = 200) for each study setting (laboratory and online). A threshold of BF10 ≥ 10 would indicate strong evidence for the alternative hypothesis (H1) compared with the null (H0), whereas a threshold of BF01 ≥ 10 would correspond to strong evidence for H0 relative to H1 (see Lee & Wagenmakers, 2013). For every 10 participants collected, data were inspected for potential exclusions (see Data Exclusions section), and interim analyses were conducted to check whether these evidential thresholds were met for all confirmatory hypotheses. If not, data from another 10 participants were collected, and this process was repeated until either the desired level of evidence for all confirmatory hypotheses was obtained, or nmax had been reached.
Although frequentist power analysis was not appropriate for an SBF design, a Bayes factor design analysis (BFDA; see Fig. S1 in the Supplementary Material) was conducted to assess the probability of the proposed design generating misleading evidence (Schönbrodt & Wagenmakers, 2018). Analyses were performed for all preregistered hypotheses, as in directional t tests for priming-related hypotheses (H1–H3) and directional correlations for food-choice task predictions (H4). The design priors were consistent with the analysis priors that would be employed for Bayesian t tests and correlations (see Preregistered Analyses section). Only the BFDA results were considered for the design of the study, and no other power analyses were performed.
Recruitment was conducted via advertisements at Cardiff University and ProlificFootnote 3 (https://www.prolific.ac/), and data were collected in both laboratory and online settings (see Study Setting section). We recruited 254 individuals via Prolific, and excluding 18 recorded drop-outs, 30 recruited individuals were not eligible to participate and quit the study (see Fig. 4). In laboratory settings, a total of 205 participants were recruited. When the maximum number of participants was reached for APP analyses (H1–H3), we had to recruit additional participants to also reach nmax for H4 because of different data exclusion criteria (see Sample Characteristics section; Fig. 4). A total of 134 participants recruited via the Experimental Management System (EMS) received course credits when eligible (e.g., undergraduate students), and 71 participants not eligible for course credits received monetary reimbursement (£6). Participants performing the study via Prolific were rewarded £4.50 upon completion.Footnote 4
The complete and exhaustive set of inclusion and exclusion criteria for participation in the study were as follows. Eligible participants were at least 18 years of age, had normal or corrected-to-normal vision, including normal colour vision, and spoke English as their first or second language. Exclusion criteria included being on a diet and/or have recently been taking diet pills, a past and/or current history of eating disorders and food allergies and/or intolerances. Screening survey questions can be found at https://osf.io/n36cg, and all criteria were based on self-report. Further post hoc exclusions of participants from preregistered analyses are presented in the Data Exclusions section.
The study was approved by the local Research Ethics Committee at the School of Psychology, Cardiff University. All eligible participants provided informed consent and were debriefed. The study employed a within-subjects design, and blinding of participants and/or experimenters was not applicable. However, participants were not made aware of the study aims before completion. Also, the contact between the experimenter and participants was minimized as data was collected online and in group laboratory settings.
Recruited participants confirmed their eligibility and proceeded to provide their consent and choose their study setting (laboratory or online). Participants also indicated their dominant, or preferred, hand for performing the study tasks. A schematic of the study procedure is shown in Fig. 2. The prime selection process required participants to complete a rating task where they rated how much they like food and non-food stimuli (see Prime Selection section). Participants completed a short APP practice block (16 trials), where they received feedback on both the speed and accuracy of their responses. Participants completed eight blocks of the task in total, with short breaks in between and instruction reminders.
After the APP, participants performed a food-choice task (FCT; see Food-Choice Task section), consisting of two blocks in total. In laboratory settings, participants received a food item chosen during the task at the end of the study. In online settings, food choice was not consequential in terms of real consumption. Ratings for all primes and targets (see Prime and Target Ratings section) were provided after the FCT for exploratory analyses. Participants were presented with three short questionnairesFootnote 5 (see Questionnaires section). The total duration of the study per participant was 40–50 minutes, after which participants were debriefed.
Affective priming paradigm
The food primes were selected from 25 healthy and 25 unhealthy foods that were rated on liking, as measured using a visual analogue scale ranging from −100 (strongly dislike) to 100 (strongly like). Four unhealthy and four healthy foods that had the maximum rating were selected as “most liked” primes, and four unhealthy and healthy foods that had the minimum rating were chosen as “least liked” primes. For each selected food category (e.g., apples for healthy most liked), there were two exemplars in the APP. Instructions highlighted that “the rating task includes foods that could be either liked or disliked” to minimize the potential of social desirability bias whereby participants consistently rate foods on the positive end of the scale. Non-food primes were selected from 25 positive images from various categories, such as animals, that comprised several items (e.g., kitten, puppy, panda). The food ratings were always presented first, and the order of healthy and unhealthy food rating blocks was randomized across participants. Foods in each block were presented in a random order. More details about the food and non-food stimuli can be found in the Supplementary Material.
The APP involved an evaluative categorization task (see Fig. 2) in which participants categorized target words as either positive or negative. The targets were preceded by either “positive” or “negative” food primes, as well as positive non-food primes (manipulation check). The food prime trials involved a 2 × 2 × 2 design, with the manipulated variables of healthiness (healthy vs. unhealthy), affective congruence (congruent vs. incongruent), and liking (most liked vs. least liked). Non-food prime trials differed only in affective congruence. Each block of 40 trials consisted of 16 healthy and 16 unhealthy food prime trials as well as 8 non-food prime trials. Congruent and incongruent prime–target pairs appeared with equal probability for all trials. There were 32 positive and 32 negative targets in total (see Supplementary Material), which appeared randomly with equal probability across two consecutive blocks for food prime trials. Targets for non-food prime trials were presented randomly across eight blocks.Footnote 6
Participants were instructed to categorize the words as quickly and as accurately as possible. Participants responded using the “G” and “H” keys, as explained in Fig. 2. Each trial commenced with a central fixation cross followed after 1,000 ms by the prime. Following a 233-ms interstimulus interval (ISI), the prime was succeeded by a backward mask (17 ms) to limit subjective awareness of the primes, constructed from a mosaic of various food stimuli with different colour compositions (Wentura & Degner, 2010). The stimulus-onset asynchrony (SOA) between prime and target was 250 ms. The response window begun on target onset (i.e., 1,250 ms), and participants had a maximum reaction time of 1,500 ms. Each trial ended either when a response was registered or when the maximum total trial duration was reached (2,750 ms). A trial was considered incorrect if the target categorization was wrong or participants did not respond within 1,500 ms. All stimuli were presented centrally, and pictures had their relative dimensions set to 40% of the vertical and horizontal width of the presentation window. The targets and fixation cross (+) were presented in black, bold Arial fonts. Words were presented in uppercase letters against a uniform grey background.
The FCT involved binary food choices, adapted from previous literature (Veling et al., 2017; Zoltak et al., 2018). Participants were instructed to choose the foods that they would prefer to eat at the end of the experiment. To measure consequential food choices, in laboratory settings participants were instructed that one of their choices would be selected by the researcher(s) and they would be given the food item they had chosen on that occasion. The selection of the food was not random due to the unsuitability of certain foods for laboratory storage (e.g., fast decay of fruits). The proposed selection process was in line with instructions used in previous literature (Veling et al., 2017). The researcher(s) selected an item from the list of suitable foods and restricted selection to foods rated as “most liked” by the participants (see Supplementary Material). In online settings, participants did not receive a food item at the end of the study, and thus choices were not consequential. In the laboratory, we also provided participants with bottled water after screening to minimize the potential impact of thirst levels on food choices.
Each trial in the FCT (see Fig. 3a) involved the simultaneous presentation of two food items on the left and right of a central fixation cross, which participants would choose between using the “C” and “M” keys.Footnote 7 A response had to be registered within a maximum of 1,500 ms, and participants would then be presented with response feedback (500 ms) where their confirmed choice would be highlighted (i.e., a yellow frame around the selected food). A central fixation cross was presented during the intertrial interval (1,000–2,000 msFootnote 8).
Participants were instructed to make their choices quickly, and time pressure would help ensure that food choices were not deliberate, reducing the probability of demand characteristics (Veling et al., 2017). Feedback was presented if participants did not respond within 1,500 ms, instructing them to choose faster (“Please try to choose faster”—1,000 ms). To avoid loss of data, missed trials were repeated, and only one repetition per trial was allowed. For each design cell of the APP (healthiness × liking) there were four food categories included in the FCT. All food prime categories were included in the FCT and represented by the primary exemplars (i.e., stimuli used in prime selection). Two main types of trials were presented, and each type had two categories (see Fig. 3b). The FCT comprised 128 binary choices in total and was split into two blocks of 64 trials with a short intervening break.
Prime and target ratings
Participants explicitly evaluated all prime categories and targets for exploratory analyses. Food primes were evaluated for valence, arousal, perceived healthiness, and frequency of cravings. Non-food primes were also evaluated for valence and arousal. Ratings were only obtained for the primary exemplars. All targets were evaluated for valence and arousal (see https://osf.io/n36cg).
Trait and state variables
An initial questionnaire recorded several trait and state variables that could be associated with eating behaviours and related information (available at https://osf.io/n36cg). These variables included how long ago participants had their last meal, whether they followed a specific diet, and hunger levels. Self-reported height and weight was recorded to calculate the participants’ body mass index (BMI: kg/m2). Participants also indicated their gender and ethnicity (optional). Participants then proceeded to complete the short version of the Food Cravings Questionnaire—Trait-reduced (FCQ-T-r; Meule, Hermann, & Kübler, 2014). FCQ-T-r consists of 15 items scored on a 5-point scale (strongly disagree to strongly agree).
Follow-up study questionnaire
At the end of the study, participants completed a follow-up study questionnaire (see Supplementary Material), where they were asked to answer questions about their performance in the APP (e.g., response strategies). Participants also indicated the number of occasions they were interrupted during the word task (see Waters & Li, 2008). The survey included an instructional manipulation check to examine whether participants were paying attention to the instructions as well as a questionnaire attention check measure (Kees, Berry, Burton, & Sheehan, 2017). Participants’ performance on the data quality assurance measures would later be compared for online and laboratory settings in exploratory analyses (see Supplementary Material).
Measures and indices
All planned comparisons are outlined in the section below, where RTcon and RTinc denote the sample means of individual median correct RTs in congruent and incongruent trials, and ERcon and ERinc refer to the mean error rates in congruent and incongruent trials, respectively. At the level of participants, median RTs were used, as they are less sensitive to outliers and may provide a more accurate measure of central tendency in positively skewed distributions.Footnote 9 The median RTs were computed for each participant, and then a Bayesian paired-samples t test was conducted for the alternative hypothesis that the population mean of the difference in median RTs is smaller than zero (or greater than zero for H2d and H3d). The difference in median RTs for each participant between congruent and incongruent trials (medianRTinc − medianRTcon) was then calculated for further testing of RT priming effects. The sample means of these difference scores were then compared across conditions (e.g., ∆RTunhealthy > ∆RThealthy in H2d) and are referred to as ∆RT. Similarly, ∆ER was defined as the priming effect for error rates, where ∆ER = ERinc − ERcon. For the calculation of error rates at the participant level, accuracy is recoded as 1 = incorrect and 0 = correct.
With regard to FCT analyses, p(unhealthy|most liked) refers to the conditional probability of choosing an unhealthy food in the unhealthy versus healthy food-choice trials when most liked food pairs were presented (see Fig. 3 for trial types). Accordingly, p(most liked|healthy) denotes the conditional probability of choosing a most liked food in the most liked versus least liked trials where healthy food pairs are presented, and p(most liked|unhealthy) indicates the conditional probability of choosing a most liked food on trials where the unhealthy food pairs were presented. Choices were recoded according to trial types to compute these probabilities. For example, in trials where participants chose between most liked and least liked foods, and the foods presented were healthy, choices were coded as 1 = most liked and 0 = least liked. Then, the mean was calculated and denoted the probability that participants chose a most liked food in these most liked versus least liked (healthy) choice trials, that is p(most liked|healthy). Probability values were calculated from the number of completed trials. The difference in priming effects (RTs only) between unhealthy and healthy most liked food prime trials is represented by ∆RTunhealthy − ∆RThealthy.
Bayesian paired-samples t tests (Rouder, Speckman, Sun, Morey, & Iverson, 2009) employed a prior with the √2/2 scale parameter for the half-Cauchy distribution. Bayesian correlation pairs had a stretched beta prior with width γ = 1, which corresponds to a uniform prior (Wagenmakers, Verhagen, & Ly, 2016). Analyses were conducted separately for the online and laboratory data sets, and results were reported independently (see Study Setting section). The evidential value and hence interpretation of the results was exclusively based on Bayes factors, but frequentist statistics have also been reported (a = 0.05). H1, H2 and H3 were exclusively tested using directional Bayesian paired-samples t tests, as outlined below.
H1a. RTcon < RTinc for non-food prime trials
H1b. ERcon < ERinc for non-food prime trials
H2a. RTcon < RTinc for food prime trials
H2b. RTcon < RTinc for healthy food prime trials
H2c. RTcon < RTinc for unhealthy food prime trials
H2d. ∆RTunhealthy > ∆RThealthy for most liked food primes
H3a. ERcon < ERinc for food prime trials
H3b. ERcon < ERinc for healthy food prime trials
H3c. ERcon < ERinc for unhealthy food prime trials
H3d. ∆ERunhealthy > ∆ERhealthy for most liked food primes
H4 was only examined via directional Bayesian correlation pairs, as shown below. The reported correlation coefficient was Pearson’s rho. Definitions of probabilities have been described in detail above (see Measures and Indices section).
H4a. ∆RThealthy for most liked primes positively correlates with p(most liked|healthy)
H4b. ∆RTunhealthy for most liked primes positively correlates with p(most liked|unhealthy)
H4c. ∆RTunhealthy − ∆RThealthy (for most liked primes) positively correlates with p(unhealthy|most liked)
As a contingent analysis plan, Bayes factors for H1, H2 and H3 in the opposite direction would also be reported if differences between means were descriptively in the unexpected direction, such as RTcon > RTinc for food prime trials. The decision to report the positive one-sided tests would be based on descriptive values and not on Bayes factors, as support for the null in a directional Bayesian t test does not exclude the possibility that there is greater evidence for an effect in the opposite direction. For example, even if there is adequate evidence for H0 and the null hypothesis is preferred to the negative hypothesis (RTcon < RTinc), the positive hypothesis (RTcon > RTinc) may still be favoured over the null (Morey, 2014). More details about the preregistered analysis plan (e.g., software, data transformations, reported statistics, effect size calculation) can be found in the Supplementary Material.
Error rates in the APP were inspected for food and non-food prime trials separately, and participants with ERs greater or equal to 0.4 from within either set of trials were excluded from all respective analyses. This obviated the need for further inspection of the distribution of missed or inaccurate responses across conditions. The FCT data were inspected for missed responses, where participants did not respond within 1,500 ms. Analyses conducted for H4a, H4b, and H4c would not include participants who had more than 50% of missed trials across the two blocks in any trial type examined under H4 (i.e., <16 out of 32 trials).
Data were also inspected for timing delays in trial events in the APP due to the possible occurrences of technical issues during online testing (e.g., slow broadband). Timing delays were defined as trial events that last two or more screen refreshes than originally programmed. The trial events that were inspected were the presentation of the prime (233 ms) and mask (17 ms), and trials with timing delays would be removed from analyses. If a participant had more than 25% of trials removed, they would then be excluded from all analyses.
The final sample size for APP analyses (H1–H3) was 202 for both the laboratory and online cohorts (see Fig. 4). Due to nmax not being reached for FCT analyses (H4) after data exclusions, additional participants were recruited, resulting in different sample sizes for APP and FCT analyses. The final sample size for FCT analyses (H4) was 200 for both cohorts. Using preregistered data quality assurance measures (see Follow-Up Study Questionnaire section), we found that, descriptively, the online study setting overall matched the controlled laboratory environment in terms of data quality (see Data Quality Checks in the Supplementary Material).
Descriptive statistics of demographics and other sample characteristics are shown in Table 1. Overall, the laboratory and online cohorts were approximately matched, but online testing generated a more diverse sample in terms of gender and age. Participants in both cohorts were generally not very hungry at the time of the study. In the laboratory sample, 58% of participants self-reported eating 1–3 hours before the study, and 21% of participants had a meal just before the study (“Less than 1 hour ago”). In the online sample, 53.5% of participants self-reported eating 1–3 hours before the study, and 21.2% of participants had a meal less than 1 hour before the study. A total of 161 participants from the laboratory cohort (80.5%) reported that they were not following any specific diet, while only 20 (10%) were vegetarian and 12 pescatarian (6%). Similarly, the online cohort consisted of 177 participants who did not follow a specific diet (89.4%), and only 14 participants reported a vegetarian diet (7.1%). A noteworthy difference between the two cohorts was participants’ BMI, as calculated by self-reported height and weight. The mean BMI in the online cohort trends towards the overweight category (≥25 kg/m2). The average FCQ-T-r total scores indicate that neither the laboratory, or online cohorts had “clinically relevant” levels of trait food cravings (Meule, 2018).
Findings from preregistered analyses
Interpretation of outcomes
The relative evidence for the confirmatory hypotheses was interpreted based on Bayes factors. BF10 denotes evidence for the alternative hypothesis (H1) compared with the null (H0), and BF01 reflects the relative evidence for the null. We have adopted the guidelines reported in Lee and Wagenmakers (2013) to describe the strength of relative evidence for each hypothesis. Regarding calculated effect sizes (Cohen’s dav), we acknowledge that commonly employed benchmarks are often arbitrary and require careful consideration of the specific research context and relevant literature (Lakens, 2013). Here we follow previous guidelines on Cohen’s d (Cohen, 1988) for ease of interpretation.
We first report the results of the manipulation check for the APP as stated in H1 (i.e., non-food priming effects). Extreme evidence was obtained for the expected RT priming effects on non-food prime trials (H1a), as presented in Tables 2 and 3. In the laboratory cohort, participants were faster to respond in congruent (M = 549.5ms, SD = 64.8 ms) compared with incongruent non-food trials (M = 568.5 ms, SD = 63.2 ms) [dav = −0.30, 95% CI for dav = −0.39, −0.21]. This relatively small effect was replicated in the online cohort, with participants having, on average, lower median RTs on congruent (M = 563.6 ms, SD = 71.4 ms) rather than incongruent non-food trials (M = 585.9 ms, SD = 73.3 ms) [dav = −0.31, 95% CI for dav = −0.39, −0.23]. RT priming effects from non-food prime trials are shown in Fig. 5.
In support of H1b, a small priming effect for error rates in the expected direction was observed in both samples. In the laboratory cohort, there was extreme evidence for lower error rates (proportion of errors) on congruent (M = 0.06, SD = 0.07) compared with incongruent non-food trials (M = 0.09, SD = 0.08) [dav = −0.35, 95% CI for dav = −0.51, −0.20; W = 3,477.50, pW < .001].Footnote 10 Similarly, in the online cohort there was very strong evidence for error rates being reduced from congruent (M = 0.04, SD = 0.05) to incongruent non-food trials (M = 0.06, SD = 0.09) [dav = −0.32, 95% CI for dav = −0.50, −0.14; W = 3,063.00, pW < .001]. These findings confirm the success of the manipulation check.
Food priming effects
Findings from primary laboratory cohort
The results of all statistical tests for preregistered hypotheses H2 and H3 from the laboratory cohort (N = 202) are presented in Table 2. As preregistered, RTs (and RT difference scores) for all comparisons under H2 were log-transformed due to the violation of the normality assumption for H2a, H2c, and H2d. Bayesian and frequentist t tests were therefore conducted using log-transformed data. We report the nontransformed sample means of median RTs here for a more convenient interpretation of mean differences. We obtained extreme evidence for a small RT priming effect across food prime trials (H2a), as RTs were on average faster on congruent (M = 549.1 ms, SD = 58.5 ms) compared with incongruent trials (M = 565.4 ms, SD = 60.2 ms) [dav = −0.27, 95% CI for dav = −0.33, −0.21]. A small effect was observed in healthy food prime trials (H2b), whereby RTs were faster on congruent (M = 549.2 ms, SD = 59.6 ms) compared with incongruent trials (M = 565.6 ms, SD = 61.5 ms) [dav = −0.27, 95% CI for dav = −0.34, −0.19]. There was extreme evidence for a small RT priming effect across unhealthy food prime trials (H2c), as shown by RTs in congruent (M = 549.3 ms, SD = 59.3 ms) and incongruent trials (M = 566.3 ms, SD = 61.1 ms) [dav = −0.28, 95% CI for dav = −0.35, −0.21]. Observed priming effects for healthy and unhealthy food prime trials have been visualized using raincloud plots (Allen, Poggiali, Whitaker, Marshall, & Kievit, 2018, 2019; see Fig. 6). In contrast to our prediction that the RT priming effect would be greater for unhealthy compared with healthy most liked food prime trials (H2d), there was moderate evidence for the null hypothesis compared with the alternative. The RT priming effect for unhealthy most liked food primes (∆RTunhealthy; M = 17.8 ms, SD = 43.8 ms) was not greater than the RT priming effect for healthy most liked food primes (∆RThealthy; M = 13.6 ms, SD = 48.1 ms) [dav = 0.08, 95% CI for dav = −0.07, 0.22].
As explained in the Hypotheses section, we expected support for any observed priming effects to be evident in both speed-related and accuracy-related hypotheses. There was extreme evidence for a medium ER priming effect across food prime trials (H3a). Participants made fewer errors on congruent (M = 0.06, SD = 0.04) compared with incongruent food prime trials (M = 0.09, SD = 0.07) [dav = −0.54, 95% CI for dav = −0.68, −0.41; W = 2,581.50, pW < .001]. A medium effect was also observed in healthy food prime trials (H3b), as error rates were lower on congruent (M = 0.06, SD = 0.05) relative to incongruent trials (M = 0.09, SD = 0.07) [dav = −0.49, 95% CI for dav = −0.63, −0.34; W = 3,960.00, pW < .001]. There was also extreme evidence for a medium ER priming effect in unhealthy food prime trials (H3c). We found the expected differences in error rates between congruent (M = 0.06, SD = 0.05) and incongruent unhealthy food prime trials (M = 0.09, SD = 0.07) [dav = −0.51, 95% CI for dav = −0.66, −0.37; W = 2,849.00, pW < .001]. Priming effects for error rates were in the expected direction across food prime trials, and therefore we can conclude that any RT effects were not observed due to strategic responding or speed–accuracy trade-offs. Contrary to predictions about differences in ER priming effects between healthy and unhealthy most liked food primes (H3d), there was moderate evidence for the null compared with the alternative hypothesis. The ER priming effect was not, on average, greater for unhealthy (M = 0.02, SD = 0.07) compared with healthy most liked food primes (M = 0.01, SD = 0.08) [dav = 0.10, 95% CI for dav = −0.07, 0.27; W = 8822.00, pW = .105].
Direct replication: Findings from online cohort
The results of all statistical tests for preregistered hypotheses H2 and H3 from the online cohort (N = 202) are presented in Table 3. RTs for all comparisons under H2 were log-transformed (logRTs) due to the violation of the normality assumption for H2b, in line with the preregistered analysis plan, and nontransformed sample means are reported here for convenience. Reaction time and error rate priming effects were replicated in the online cohort. First, there was extreme evidence for a small RT priming effect across food prime trials (H2a), as on average RTs on congruent trials (M = 568.6 ms, SD = 71.3 ms) were faster compared with RTs on incongruent food prime trials (M = 580.5 ms, SD = 70.7 ms) [dav = −0.18, 95% CI for dav = −0.22, −0.13]. A small RT priming effect was also observed in healthy food prime trials (H2b). RTs were faster on congruent (M = 568.4 ms, SD = 71.4 ms) compared with incongruent healthy food prime trials (M = 580.1 ms, SD = 74.0 ms) [dav = −0.17, 95% CI for dav = −0.22, −0.11]. Extreme evidence was obtained for a small RT priming effect in the expected direction for RTs on congruent (M = 568.7 ms, SD = 73.6 ms) and incongruent (M = 581.0 ms, SD = 71.0 ms) unhealthy food prime trials [H2c; dav = −0.18, 95% CI for dav = −0.24, −0.13]. The results from laboratory and online cohorts converge for H2d as well, as there was moderate evidence that the RT priming effect for most liked unhealthy foods (∆RTunhealthy; M = 14.5 ms, SD = 44.7 ms) was not greater than the RT priming effect for most liked unhealthy foods (∆RThealthy; M = 12.8 ms, SD = 43.0 ms) [dav = 0.05, 95% CI for dav = −0.10, 0.19].
In line with the findings from the laboratory cohort, priming effects were observed in terms of error rates ruling out the possibility of strategic performance trade-offs. However, in the online cohort, we found small, and not medium, ER food priming effects. There was extreme evidence for a small ER priming effect across food prime trials (H3a). Error rates on congruent trials (M = 0.05, SD = 0.05) were, on average, lower compared with error rates on incongruent food prime trials (M = 0.06, SD = 0.07) [dav = −0.28, 95% CI for dav = −0.40, −0.15; W = 4,238.00, pW < .001]. In healthy food prime trials (H3b), results for error rates on congruent (M = 0.05, SD = 0.05) and incongruent trials (M = 0.06, SD = 0.07) were in the same direction [dav = -0.30, 95% CI for dav = −0.43, −0.17; W = 3,110.00, pW < .001]. Strong evidence was also obtained for a small ER priming effect in unhealthy food prime trials (H3c), as error rates were on average lower on congruent (M = 0.05, SD = 0.05) compared with incongruent trials (M = 0.06, SD = 0.08) [dav = −0.23, 95% CI for dav = −0.37, −0.09; W = 4,942.50, pW < .001]. Consistent with the results for H3d in the laboratory cohort, there was strong evidence that the ER priming effect was not greater for unhealthy (M = 0.01, SD = 0.08) compared with healthy most liked food primes (M = 0.01, SD = 0.07) [dav = −0.01, 95% CI for dav = −0.15, 0.13; W = 6,302.50, pW = .779].
Bayesian correlation pairs for hypotheses H4a and H4b have yielded conclusive evidence regarding the absence or presence of the expected linear positive correlations. There was strong evidence that the probability of choosing a most liked food over a least like food from within a pair of healthy food stimuli (M = 0.97, SD = 0.06) did not positively correlate with the RT priming effect in healthy food prime trials [H4a; BF01 = 21.89; r = −.073, p = .849, 95% CI = −0.210, 0.066]. Similarly, there was strong evidence that the probability of choosing a most liked over a least liked food from within a pair of unhealthy food stimuli (M = 0.96, SD = 0.05) did not positively correlate with the RT priming effect in unhealthy food prime trials [H4b; BF01 = 15.98; r = −.035, p = .686, 95% CI = −0.172, 0.105]. Moderate evidence was obtained for the null compared with the alternative hypothesis for H4c. The probability of choosing an unhealthy over a healthy most liked food (M = 0.60, SD = 0.28) did not positively correlate with the difference in RT priming effects between unhealthy and healthy most liked food prime trials [H4c; BF01 = 6.70; r = .041, p = .283, 95% CI = −0.098, 0.179].
In the online cohort, there was moderate evidence for the lack of a positive correlation between the probability of choosing a most liked food over a least like food from within a pair of healthy food stimuli, or p(most liked|healthy) (M = 0.96, SD = 0.08), and the RT priming effect in healthy most liked food prime trials [H4a; BF01 = 4.88; r = .061, p = .195, 95% CI = −0.078, 0.198]. There was also moderate evidence for the absence of a positive linear correlation between the probability of choosing a most liked over a least liked food from within a pair of unhealthy food stimuli, or p(most liked|unhealthy) (M = 0.94, SD = 0.10), and the RT priming effect in unhealthy food prime trials [H4b; BF01 = 7.62; r = .032, p = .328, 95% CI = −0.108, 0.170]. As a further validation of the FCT, both probabilities of choosing a most liked food item in the most liked vs least liked trials were very high and above 0.5. Consistent with the laboratory cohort, there was moderate evidence for the lack of a positive linear correlation between the probability of choosing an unhealthy over a healthy most liked food, or p(unhealthy|most liked) (M = 0.55, SD = 0.33), and the difference in RT priming effects between unhealthy and healthy most liked food prime trials [H4c; BF01 = 6.43; r = .044, p = .270, 95% CI = −0.096, 0.181].
The primary aim of this study was to assess the utility of the affective priming paradigm (APP) as an indirect measure of food liking and related choice behaviour. Using a variant of the APP that requires evaluative categorization that is semantically unrelated to the content of the primes, participants responded as quickly and as accurately as possible to the valence of word targets. Affective congruence was manipulated so that both healthy and unhealthy foods that had been selected as most liked or least liked via an initial rating task were paired with both positive and negative targets. After the APP, participants completed a binary food-choice task (FCT), and impulsive food-choice probabilities for different food pairs were measured. The three main research questions of the study were tested via preregistered confirmatory hypotheses in laboratory settings and replicated in a second cohort of participants who completed the experiment online. The findings for each of these research questions and their implications are discussed at length below, together with directions for future research.
Can priming effects for foods be obtained with the APP?
Yes. In line with previous findings (e.g., Lamote et al., 2004), robust priming effects were observed across food prime trials (most liked and least liked foods) for both speed (RTs) and accuracy (ERs). Effects were also shown to be robust for both healthy and unhealthy foods (see Fig. 6), providing conclusive evidence that the APP can be used as an indirect measure of food liking. The reaction time (RT) priming effects were relatively small, and although effect sizes for direct comparisons are not commonly reported in the literature, mean differences between RTs on congruent and incongruent trials seem to be consistent with previous studies that employed similar paradigms (e.g., Lamote et al., 2004; Roefs, Herman, et al., 2005a; Verhulst et al., 2006). The interpretation of the findings was strengthened by the success of the manipulation check for the APP, which assessed priming effects for most liked non-food stimuli. Importantly, all results from the laboratory cohort (N = 202) were directly replicated in the online cohort (N = 202). Statistical tests were repeated under different data aggregation/reduction criteria adopted from previous literature (Lamote et al., 2004; Verhulst et al., 2006) in order to establish the robustness of the observed priming effects. There were no discrepancies between the results based on the preregistered analysis plan and the alternative analyses (see Table S4 in the Supplementary Material), which suggests that our findings were not influenced by the aggregation and outlier removal criteria employed in this study.
Is the APP sensitive to cognitive components of food attitudes?
Possibly not, but if cognitive components have an effect on task performance, this is likely to be small. The sensitivity of the APP in capturing both affective and cognitive components of attitudes was investigated by comparing the RT and ER priming effects for healthy and unhealthy most liked foods. Previous literature employing the APP indicates that a greater priming effect for healthy (or low-fat) compared with unhealthy (or high-fat) foods may indicate that the APP taps into cognitive components of attitudes (i.e., food healthiness). For example, Roefs, Stapert, et al. (2005b) suggested that health concerns may have determined observed priming effects in two groups of participants who differed in terms of BMI and dietary restraint (Experiment 2). Specifically, the authors report that participants in both groups showed a “preference” for low-fat over high-fat foods.Footnote 11 Previous research also suggests that priming effects are sensitive to changes in context/environment and attentional focus, such as participants performing the study in a local hospital instead of a laboratory and experimentally manipulating the focus of participants’ attention on either the palatability or healthiness of food stimuli before the task (Roefs et al., 2006; Roefs, Stapert, et al., 2005b).
Although we obtained conclusive evidence that healthiness did not influence the magnitude of the RT and ER priming effects for most liked foods, we could not conclude that this is because the APP is only sensitive to affective components of attitudes. Future research could assess how healthiness affects the relative strength of observed priming effects when individuals self-report a preference of healthy over unhealthy foods, which could be attributed to social desirability or health concerns. In both study cohorts there were no descriptive differences in explicit liking between healthy and unhealthy most liked foods that could transfer to the APP. Although healthiness may not affect priming effects in an explicit manner that would undermine its validity as an indirect measure of liking—as, for example, when these reflect concerns related to health, weight-related goals or social norms (Czyzewska & Graham, 2008)—healthiness could influence task outcomes through implicit healthiness attributes that are automatically retrieved from memory (e.g., see Rangel, 2013; Trendel & Werle, 2015). We specifically assumed that if individuals had greater automatic affective reactions towards unhealthy foods (e.g., “unhealthy = tasty” intuition; Raghunathan, Naylor, & Hoyer, 2006; also see Werle, Trendel, & Ardito, 2013), this would manifest as a positive difference in the magnitude of RT/ER priming effects, even when food primes were matched on explicit liking.Footnote 12 However, even if unhealthy food primes induce stronger affective reactions compared with healthy food primes, there is not enough evidence to suggest that the APP would capture such subtle differences. The seminal study by Lamote et al. (2004) actually indicated that prime extremity (e.g., moderate vs. strong liking) does not influence observed priming effects, which could mean that only the overall valence of the primes (liked/positive, disliked/negative) determines task outcomes.
Overall, our findings are consistent with the study by Becker et al. (2015), which did not report any differences for the food prime contrasts (healthy, unhealthy, control) in their affective priming paradigm (Study 2). Similarly, Roefs, Herman, et al. (2005a) provided evidence for a priming effect for palatable (most liked) and unpalatable (least liked) foods, but found that fat content did not influence the results. Our results suggest that affective priming effects for most liked food primes in this study were not influenced by healthiness in any observable manner (i.e., differences between ΔRTs or ΔERs in H2d and H3d). Nevertheless, we recommend that the sensitivity of the APP to affective and cognitive components of attitudes is explored further in target populations (e.g., restrained eaters and individuals with a BMI in the overweight and/or obese category; Cserjesi, De Vos, & Deroost, 2016; Papies, Stroebe, & Aarts, 2009; Roefs, Herman, et al., 2005a; Roefs, Stapert, et al., 2005b).
Are priming effects associated with impulsive food-choice behaviour?
No, at least in this design. A novel contribution of this study was the investigation of impulsive food choices using a binary reaction time task, adapted from previous literature (Veling et al., 2017; Zoltak et al., 2018; but also see Verhulst et al., 2006). Food liking has a paramount role in dietary choices, which are often impulsive and not guided by deliberate thoughts (Eertmans et al., 2001; Veling et al., 2017). If positive affective reactions towards foods can influence impulsive food choices (Zoltak et al., 2018), priming effects obtained via the APP could in theory be associated with the probability of choosing appetitive foods under different conditions, such as choosing a most liked unhealthy food when most liked and least liked foods are presented. Alternatively, if we assume that impulsive choices are driven by “wanting” and not “liking” for food cues in the environment (Berridge et al., 2010), performance in the APP would not be predictive of choice behaviour in the laboratory.
Confirmatory analyses provided conclusive evidence for the absence of positive linear correlations between RT priming effects for most liked foods and the probability of choosing a most liked food from both healthy and unhealthy items. These null findings were replicated in the online cohort. A potential limitation of the FCT design for most liked versus least liked food pairs was that food-choice probabilities were very high and there was not enough variability in participants’ responses, and that could have influenced the tested correlations. An even more meaningful question was whether the difference between RT priming effects for most liked healthy and unhealthy foods was associated with the probability of choosing an unhealthy food in trials where most liked healthy and unhealthy foods were presented. Again, however, there was strong support for the null hypothesis compared with the alternative in both laboratory and online cohorts. One notable methodological difference between the two cohorts was that in online testing settings the FCT did not involve offering participants food items for consumption, which meant that impulsive food choices were not consequential. However, as mentioned above, results converged fully between the two cohorts and although their choices were not consequential, participants in the online cohort had very high probabilities of selecting most liked foods on FCT trials where healthy and unhealthy food pairs were presented.
One issue that remains unclear is whether the strict time limit for these food choices between two most liked food items caused a choice uncertainty that could not be easily resolved, leading some participants to respond randomly or arbitrarily in these FCT trials. Follow-up experiments could measure both impulsive and deliberate food choices or use alternative time windows (e.g., short vs. long) and self-report questionnaires to discard alternative explanations for the absence of a positive correlation between RT priming effect difference scores and food-choice probabilities. We also recommend that future studies employ informative analysis priors for Bayesian correlations, as we believe that if a relatively weak relationship exists between priming effects and food-choice behaviour, the number of observations required to capture this would be very large with the current choice of prior distribution (e.g., see Fig. S1b in the Supplementary Material).
Considerations for future research
Most participants in both laboratory and online cohorts were healthy-weight individuals with self-reported frequency and intensity of food craving experiences that did not indicate unhealthy eating behaviours, such as binge eating (Meule, 2018). It is possible that the absence of differences between RT priming effects for most liked healthy and unhealthy foods was because participants, on average, did not have stronger affective, or hedonic, reactions towards unhealthy foods. This research question could be addressed in a sample of individuals that are overweight and/or obese or show eating disorder symptomatology. In such cases, the distinction between cognitive and affective components of food attitudes may be more informative due to increased approach and/or attentional bias towards appetitive cues and the conflict of this bias with health-related goals, such as losing weight (Kakoschke et al., 2015). Accordingly, impulsive food choices that are driven by strong affective reactions towards unhealthy foods should be examined further in a representative sample of individuals that exhibit unhealthy eating behaviours, such as overeating.
Another next step in this line of research could be to employ different variants of the APP to disentangle theoretical explanations of priming effects and attempt to replicate and extend the presented findings. Affective priming effects can be explained by response competition/facilitation processes, as the primes can be defined as being congruent or incongruent to the required response to the target (Fazio & Olson, 2003; Wentura & Degner, 2010). Theoretically, however, it is also possible that the perception of the prime activates the “object-evaluation association” from memory, increasing the accessibility of valence for the targets when these are congruent with the prime compared with incongruent (see Fazio, 2001; Herring et al., 2013). In the evaluative categorization task, this distinction between the encoding and response perspectives cannot be inferred from observed priming effects. The pronunciation or naming task variant of the APP (see Herring et al., 2013, for discussion) involves responding to targets irrespective of their valence and can therefore exclude the response perspective from the explanation of any obtained priming effects.
The compatibility between the prime and the required response to the target may further reduce the “implicitness” of the measure if participants are aware of its effects. The standardized follow-up study questionnaire results (see Supplementary Material) showed that many participants were aware of the effects of affective congruence on their performance. For example, they believed that the content of the picture influenced their performance when the word they had to categorize was negative and the preceding picture depicted a food they liked the most. In addition to APP performance, potential confounds for primes and targets should be considered. Even in cases where robust priming effects fail to be obtained, participants’ individual explicit ratings of target valence and prime characteristics can have substantial diagnostic value and provide the basis for the investigation of individual differences. To bridge the gap between explicit and implicit measures of food attitudes, it may be worth pursuing the use of semistructured questionnaires that measure participants’ subjective awareness of performance differences in critical trials (e.g., congruent vs incongruent in the APP) as well as tailored stimulus selection (e.g., selecting primes via an initial rating task).
Exploratory analyses (see Data Quality Checks in Supplementary Material) also indicated that data from laboratory and online settings did not differ in terms of quality and precision. On the contrary, it is possible that participants in laboratory studies are more aware of experimental procedures, which could lead to increased bias in responses (e.g., demand characteristics; Podsakoff, MacKenzie, Lee, & Podsakoff, 2003). Where data quality assurance measures are in place (e.g., attention checks), online testing provides a fruitful avenue for studies requiring larger and more diverse samples and direct replications.
Open practices statement
All raw and processed data are publicly available at https://osf.io/73xfr. Study materials can be found at https://osf.io/sjcx7. All codes necessary for the reproduction of confirmatory analyses is publicly available at https://osf.io/73xfr. The Stage 1 protocol received in-principle acceptance on 20/02/2019 and was registered on the Open Science Framework at https://osf.io/y2tus.
The robustness of the manipulation check has been demonstrated in a series of unregistered pilot experiments (see Paradigm Development at https://osf.io/z6nmx/).
Logical operators: ¬ = “not”; ⋁ = “or”; ⋀ = “and”
Prolific requires prescreening of participants, and the current country of residence was set to UK for two reasons: (1) consistency of subject pools between laboratory and online settings and (2) food brands included in the behavioural tasks might not have been popular outside the UK.
A £6.00/hr rate was used for both Prolific and EMS participants. On Prolific, the estimated time of completion was 45 minutes, and, in the laboratory, the study was expected to last 60 minutes due to the coordination of group testing settings and the time it would take to provide all participants with the chosen food items (see Procedure).
Questionnaire items may prime participants to pay attention to health-related or weight-related information, and therefore were presented after the behavioural tasks had been completed.
Because of the separate randomization of targets for food and non-food prime trials, certain targets may have appeared twice in a block.
We purposefully deviated from the previous literature on single-hand responses because of the possibility of learned associations (e.g., between the index/middle finger commands for “positive” and “negative” in the APP and food choices). Here, participants responded using both hands, by placing their index fingers on the “C” and “M” keys.
Random selection in steps of 100 ms.
Robustness checks were conducted to ensure that findings from confirmatory analyses were not sensitive to the choice of data exclusion and aggregation criteria (e.g., mean instead of median RTs). The preregistered robustness checks and the outcomes of all analyses can be found in the Supplementary Material.
In this section, pW denotes the p value from the Wilcoxon signed-rank tests for clarity. In line with the preregistered analysis plan, these were conducted for tests under H3 if normality was violated (see Supplementary Material).
Note that the interaction between prime fat content and target affect (positive vs. negative) for palatable foods was not strong enough to contradict the present findings (η2 = 0.05, p = .09).
We could also presume that both healthiness and tastiness attributes form a learned evaluation that is automatically retrieved from memory. In this case, attitudes would be either positive or negative (healthiness and tastiness covary), without separate and accessible components.
Ackermann, C.-L., & Palmer, A. (2014). The contribution of implicit cognition to the theory of reasoned action model: A study of food preferences. Journal of Marketing Management, 30(5–6), 529–550. doi:https://doi.org/10.1080/0267257X.2013.877956
Allen, M., Poggiali, D., Whitaker, K., Marshall, T., & Kievit, R. (2018). RainCloudPlots tutorials and codebase (Version v1.0) [Computer software]. doi:https://doi.org/10.5281/zenodo.1402959
Allen, M., Poggiali, D., Whitaker, K., Marshall, T., & Kievit, R. (2019). Raincloud plots: A multi-platform tool for robust data visualization [Version 1; peer review: 2 approved]. Wellcome Open Research, 4(63). doi:https://doi.org/10.12688/wellcomeopenres.15191.1
Becker, D., Jostmann, N. B., Wiers, R. W., & Holland, R. W. (2015). Approach avoidance training in the eating domain: Testing the effectiveness across three single session studies. Appetite, 85, 58–65. doi:https://doi.org/10.1016/j.appet.2014.11.017
Berridge, K. C., Ho, C.-Y., Richard, J. M., & DiFeliceantonio, A. G. (2010). The tempted brain eats: Pleasure and desire circuits in obesity and eating disorders. Brain Research, 1350, 43–64. doi:https://doi.org/10.1016/j.brainres.2010.04.003
Blechert, J., Meule, A., Busch, N. A., & Ohla, K. (2014). Food-pics: An image database for experimental research on eating and appetite. Frontiers in Psychology, 5, 617. doi:https://doi.org/10.3389/fpsyg.2014.00617
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, N.J: L. Erlbaum Associates.
Cserjesi, R., De Vos, I., & Deroost, N. (2016). Discrepancy between implicit and explicit preferences for food portions in obesity. International Journal of Obesity, 40(9), 1464–1467. doi:https://doi.org/10.1038/ijo.2016.91
Czyzewska, M., & Graham, R. (2008). Implicit and explicit attitudes to high- and low-calorie food in females with different BMI status. Eating Behaviors, 9(3), 303–312. doi:https://doi.org/10.1016/j.eatbeh.2007.10.008
Eertmans, A., Baeyens, F., & Van den Bergh, O. (2001). Food likes and their relative importance in human eating behavior: Review and preliminary suggestions for health promotion. Health Education Research, 16(4), 443–456.
Fazio, R. H. (2001). On the automatic activation of associated evaluations: An overview. Cognition and Emotion, 15(2), 115–141. doi:https://doi.org/10.1080/02699930125908
Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition research: Their meaning and use. Annual Review of Psychology, 54(1), 297–327. doi:https://doi.org/10.1146/annurev.psych.54.101601.145225
Fazio, R. H., Sanbonmatsu, D. M., Powell, M. C., & Kardes, F. R. (1986). On the automatic activation of attitudes. Journal of Personality and Social Psychology, 50(2), 229–238.
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. (1998). Measuring individual differences in implicit cognition: The implicit association test. Journal of Personality and Social Psychology, 74(6), 1464–1480.
Hermans, D., De Houwer, J., & Eelen, P. (2001). A time course analysis of the affective priming effect. Cognition and Emotion, 15(2), 143–165. doi:https://doi.org/10.1080/0269993004200033
Herring, D. R., White, K. R., Jabeen, L. N., Hinojos, M., Terrazas, G., Reyes, S. M., … Crites, S. L. (2013). On the automatic activation of attitudes: A quarter century of evaluative priming research. Psychological Bulletin, 139(5), 1062–1089. doi:https://doi.org/10.1037/a0031309
Kakoschke, N., Kemps, E., Tiggemann, M., Kakoschke, N., Kemps, E., & Tiggemann, M. (2015). Combined effects of cognitive bias for food cues and poor inhibitory control on unhealthy food intake. Appetite, 87, 358–364. doi:https://doi.org/10.1016/j.appet.2015.01.004
Kees, J., Berry, C., Burton, S., & Sheehan, K. (2017). An analysis of data quality: Professional panels, student subject pools, and Amazon’s Mechanical Turk. Journal of Advertising, 46(1), 141–155. doi:https://doi.org/10.1080/00913367.2016.1269304
Klauer, K. C., & Musch, J. (2003). Affective priming: Findings and theories. In J. Musch & K. C. Klauer (Eds.), The psychology of evaluation: Affective processes in cognition and emotion (pp. 9–50). Mahwah, NJ: Erlbaum.
Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4. doi:https://doi.org/10.3389/fpsyg.2013.00863
Lamote, S., Hermans, D., Baeyens, F., & Eelen, P. (2004). An exploration of affective priming as an indirect measure of food attitudes. Appetite, 42(3), 279–286. doi:https://doi.org/10.1016/j.appet.2003.11.009
Lawrence, N. S., Hinton, E. C., Parkinson, J. A., & Lawrence, A. D. (2012). Nucleus accumbens response to food cues predicts subsequent snack consumption in women and increased body mass index in those with reduced self-control. NeuroImage, 63(1), 415–422. doi:https://doi.org/10.1016/j.neuroimage.2012.06.070
Lee, M. D., & Wagenmakers, E.-J. (2013). Bayesian cognitive modeling: A practical course. Cambridge, England: Cambridge University Press. doi:https://doi.org/10.1017/CBO9781139087759
Marty, L., Miguet, M., Bournez, M., Nicklaus, S., Chambaron, S., & Monnery-Patris, S. (2017). Do hedonic- versus nutrition-based attitudes toward food predict food choices? A cross-sectional study of 6- to 11-year-olds. The International Journal of Behavioral Nutrition and Physical Activity, 14(162), 1–10. doi:https://doi.org/10.1186/s12966-017-0618-4
Meule, A. (2018). Food cravings in food addiction: Exploring a potential cut-off value of the Food Cravings Questionnaire-Trait-Reduced. Eating and Weight Disorders—Studies on Anorexia, Bulimia and Obesity, 23(1), 39–43. doi:https://doi.org/10.1007/s40519-017-0452-3
Meule, A., Hermann, T., & Kübler, A. (2014). A short version of the food cravings questionnaire-trait: The FCQ-T-Reduced. Frontiers in Psychology, 5(190), 1–10. doi:https://doi.org/10.3389/fpsyg.2014.00190
Morey, R. D. (2014). BayesFactor: Software for Bayesian inference: Bayes factor t tests, part 1 [Blogpost]. Retrieved from http://bayesfactor.blogspot.com/2014/02/bayes-factor-t-tests-part-1.html
Papies, E. K., Stroebe, W., & Aarts, H. (2009). Who likes it more? Restrained eaters’ implicit attitudes towards food. Appetite, 53(3), 279–287. doi:https://doi.org/10.1016/j.appet.2009.07.001
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method biases in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88(5), 879–903. doi:https://doi.org/10.1037/0021-9010.88.5.879
Raghunathan, R., Naylor, R. W., & Hoyer, W. D. (2006). The unhealthy = tasty intuition and its effects on taste inferences, enjoyment, and choice of food products. Journal of Marketing, 70(4), 170–184. doi:https://doi.org/10.1509/jmkg.70.4.170
Rangel, A. (2013). Regulation of dietary choice by the decision-making circuitry. Nature Neuroscience, 16(12), 1717–1724. doi:https://doi.org/10.1038/nn.3561
Roefs, A., Herman, C. P., MacLeod, C. M., Smulders, F. T. Y., & Jansen, A. (2005a). At first sight: How do restrained eaters evaluate high-fat palatable foods? Appetite, 44(1), 103–114. doi:https://doi.org/10.1016/j.appet.2004.08.001
Roefs, A., Stapert, D., Isabella, L. A. S., Wolters, G., Wojciechowski, F., & Jansen, A. (2005b). Early associations with food in anorexia nervosa patients and obese people assessed in the affective priming paradigm. Eating Behaviors, 6(2), 151–163. doi:https://doi.org/10.1016/j.eatbeh.2004.10.001
Roefs, A., Quaedackers, L., Werrij, M. Q., Wolters, G., Havermans, R., Nederkoorn, C., … Jansen, A. (2006). The environment influences whether high-fat foods are associated with palatable or with unhealthy. Behaviour Research and Therapy, 44(5), 715–736. doi:https://doi.org/10.1016/j.brat.2005.05.007
Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Psychonomic Bulletin & Review, 16(2), 225–237. doi:https://doi.org/10.3758/PBR.16.2.225
Sato, W., Sawada, R., Kubota, Y., Toichi, M., & Fushiki, T. (2016). Unconscious affective responses to food. PLOS ONE, 11(8). doi:https://doi.org/10.1371/journal.pone.0160956
Schönbrodt, F. D., & Wagenmakers, E.-J. (2018). Bayes factor design analysis: Planning for compelling evidence. Psychonomic Bulletin & Review, 25, 128–142. doi:https://doi.org/10.3758/s13423-017-1230-y
Schönbrodt, F. D., Wagenmakers, E.-J., Zehetleitner, M., & Perugini, M. (2017). Sequential hypothesis testing with Bayes factors: Efficiently testing mean differences. Psychological Methods, 22(2), 322–339. doi:https://doi.org/10.1037/met0000061
Trendel, O., & Werle, C. O. C. (2015). Distinguishing the affective and cognitive bases of implicit attitudes to improve prediction of food choices. Appetite, 104, 33–43. doi:https://doi.org/10.1016/j.appet.2015.10.005
Veling, H., Chen, Z., Tombrock, M. C., Verpaalen, I. A. M., Schmitz, L. I., Dijksterhuis, A., & Holland, R. W. (2017). Training impulsive choices for healthy and sustainable food. Journal of Experimental Psychology: Applied, 23(1), 1–14. doi:https://doi.org/10.1037/xap0000112
Verhulst, F., Hermans, D., Baeyens, F., Spruyt, A., & Eelen, P. (2006). Determinants and predictive validity of direct and indirect measures of recently acquired food attitudes. Appetite, 46(2), 137–143. doi:https://doi.org/10.1016/j.appet.2005.11.004
Wagenmakers, E.-J., Verhagen, J., & Ly, A. (2016). How to quantify the evidence for the absence of a correlation. Behavior Research Methods, 48(2), 413–426. doi:https://doi.org/10.3758/s13428-015-0593-0
Waters, A. J., & Li, Y. (2008). Evaluating the utility of administering a reaction time task in an ecological momentary assessment study. Psychopharmacology, 197(1), 25–35. doi:https://doi.org/10.1007/s00213-007-1006-6
Wentura, D., & Degner, J. (2010). A practical guide to sequential priming and related tasks. In B. Gawronski & B. K. Payne (Eds.), Handbook of implicit social cognition: Measurement theory, and applications (pp. 95–116). New York, NY: Guilford Press.
Werle, C. O. C., Trendel, O., & Ardito, G. (2013). Unhealthy food is not tastier for everybody: The “healthy = tasty” French intuition. Food Quality and Preference, 28(1), 116–121. doi:https://doi.org/10.1016/j.foodqual.2012.07.007
Zoltak, M. J., Veling, H., Chen, Z., & Holland, R. W. (2018). Attention! Can choices for low value food over high value food be trained? Appetite, 124, 124–132. doi:https://doi.org/10.1016/j.appet.2017.06.010
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work was supported by the H2020 European Research Council Consolidator grant 647893-CCT, held by C.D.C. The authors declare no conflicts of interest. Link to accepted Stage 1 protocol: https://osf.io/y2tus
Electronic Supplementary Material
About this article
Cite this article
Tzavella, L., Maizey, L., Lawrence, A.D. et al. The affective priming paradigm as an indirect measure of food attitudes and related choice behaviour. Psychon Bull Rev 27, 1397–1415 (2020). https://doi.org/10.3758/s13423-020-01764-1