The Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998) has attracted an enormous amount of research interest in recent years. It was designed to assess automatic implicit associations between two target concepts and an attribute dimension by using participants’ reaction times. On the IAT, participants have to sort stimuli that appear consecutively in the middle of the computer screen into four different categories: (a) two contrasted target concept categories that form the target dimension and (b) two contrasted attribute categories that form the attribute dimension.

The IAT procedure using the extraversion IAT as an example

On an extraversion IAT (Back, Schmukle, & Egloff, 2009), as used in the present reanalysis, the target dimension includes self-relevant versus non-self-relevant words (e.g., me vs. others), and the attribute dimension includes extraversion-related versus introversion-related words (e.g., talkative vs. shy). The IAT consists of seven blocks altogether, of which Blocks 1, 2, and 5 are the so-called single or practice blocks, which introduce the target or attribute discrimination. In these blocks, the categories of either the target concepts or the attribute concepts are presented in the upper corners of each side (i.e., left and right) of the display screen. Participants are instructed to respond to exemplars of each category by pressing a key on the same side as the label. Blocks 3 and 4 as well as 6 and 7 are the so-called combined blocks, in which the attribute discrimination is paired with the target discrimination (i.e., participants must assign words from all four categories in these blocks). Thus, on the extraversion IAT, in Blocks 3 and 4 (the compatible phase), participants must respond to self-relevant and extraversion-related words with one key and to non-self-relevant and introversion-related words with the other key. In Blocks 6 and 7 (the incompatible phase), participants must respond to introversion-related and self-relevant words with one key and to the extraversion-related and non-self-relevant words with the other key.Footnote 1

The traditional IAT effect

The rationale behind the IAT is that the sorting task should be easier and thus completed more quickly when the two concepts that share one response key are strongly associated. If two concepts are only weakly associated, sorting them into one category should be more difficult and should therefore be conducted more slowly. The traditional IAT effect (i.e., the so-called D measure; see Greenwald, Nosek, & Banaji, 2003a, 2003b) is computed as the difference in reaction times between the incompatible phase and the compatible phase divided by their overall standard deviation. It is used as an indicator of the strength of the association between the concepts (e.g., self and extraversion for the extraversion IAT).

The problem associated with the traditional IAT effect

Although the validity of the IAT for measuring automatic associations has been documented in a number of studies (e.g., Banse, Seise, & Zerbes, 2001; Bar-Anan & Nosek, 2014; Gawronski, 2002; Greenwald et al., 1998; Hofmann, Gawronski, Gschwendner, Le, & Schmitt, 2005), the traditional IAT effect is associated with a relevant problem. It has been criticized for containing not only variance related to the construct but also method-specific variance (e.g., Back, Schmukle, & Egloff, 2005; McFarland & Crouch, 2002; Mierke & Klauer, 2003) and faking-related variance (e.g., De Houwer, Beckers, & Moors, 2007; Fiedler & Bluemke, 2005; McDaniel, Beier, Perkins, Goggin, & Frankel, 2009; Röhner, Schröder-Abé, & Schütz, 2011, 2013; Steffens, 2004). Unfortunately, using the traditional IAT effect (i.e., the D measure; see Greenwald et al., 2003a, b) does not allow the researcher to separate these sources of variance from each other. In other words, all variance will usually be treated as construct-related variance.

However, recent research by Klauer, Voss, Schmitz, and Teige-Mocigemba (2007) has indicated that so-called diffusion model analyses allow the user to decompose the traditional IAT effect (i.e., the D measure; Greenwald et al., 2003a, b), and thus may be useful for separating these different sources of variance from each other.

The diffusion model

The diffusion model (Ratcliff, 1978, 2014; Ratcliff, Gomez, & McKoon, 2004; Ratcliff & Rouder, 1998, 2000) represents a stochastic model for binary decision tasks. It enables researchers to estimate several parameters associated with people’s decision processes. Accordingly, it has been successfully applied to data from a variety of decision tasks such as recognition memory (e.g., Spaniol, Madden, & Voss, 2006), lexical decisions (Ratcliff, Gomez, et al., 2004; Ratcliff, Thapar, Gomez, & McKoon, 2004), perceptual discrimination (Voss, Rothermund, & Brandstädter, 2008; Voss, Rothermund, & Voss, 2004), priming effects (Voss, Rothermund, Gast, & Wentura, 2013), the IAT (Klauer et al., 2007; Schmitz & Voss, 2012; van Ravenzwaaij, van der Maas, & Wagenmakers, 2011) and others (for a review, see Wagenmakers, 2009).

The diffusion model is built on the assumption that people continuously accumulate information (i.e., response evidence) from the stimuli presented in a task in order to make their decisions. After collecting sufficient information, they make their decision.

Here, we will explain the process underlying the diffusion model by visually representing a sample decision path for one person on one trial of a decision task (see Fig. 1).Footnote 2 The x-axis represents time. As can be seen in Fig. 1, the diffusion model distinguishes between the actual decision process (represented by parameters v and a) and a nondecision period (represented by parameter t0). During the actual decision process, the participant makes his or her decision on the basis of the information presented by the stimulus. The nondecision period includes processes that are applied before and after the actual decision (e.g., the perceptual encoding of the stimulus and the motor execution of a key press to indicate the decision).

Fig. 1
figure 1

Diffusion process underlying the diffusion model (cf. Schmitz & Voss, 2012). The diffusion model distinguishes between the nondecision parameter (parameter t0) and parameters from the actual decision process (parameters v and a). The x-axis represents time and is read from left to right. The y-axis represents the response-related decision with two response criteria, which are placed at 0 (for incorrect responses) and a (for correct responses). The counter begins fluctuating as a function of the information that accumulates with time at the postulated point z. The accumulation of information includes systematic as well as random influences. As soon as one of the two response criteria is crossed, the decision process is terminated, and the respective response is initiated. In the sample path for one trial, the participant accumulates enough information to provide the correct response. Parameter v is the mean amount of accumulated information for one participant across a certain number of trials (e.g., one phase in an IAT). The distributions of correct and incorrect responses are displayed outside their respective response criteria

Full size image

The person’s decision process (see the sample decision path in Fig. 1) begins at a point z (i.e., beginning at point z, the person accumulates systematic and random information over time). Accordingly, the sample decision path runs like an internal counter that changes over time. The counter thereby runs in a corridor between two response criteria, which are located on the response-related decision axis (i.e., the y-axis). The two response criteria are placed at 0 and a. Each criterion represents one of the two response alternatives. In our example, the upper criterion (i.e., a) corresponds to the correct response (i.e., Response C), whereas the lower criterion corresponds to the incorrect response (i.e., Response I). The counter is driven in opposite directions by information supporting the two different decisional outcomes (i.e., the response alternatives). As soon as one of the two response criteria (i.e., the upper criterion a or the lower criterion 0) is crossed, the decision process is terminated, and the response linked to the respective response criterion is initiated. In our sample, the upper criterion is crossed and thus, the correct response is initiated (see Klauer et al., 2007; Ratcliff, Gomez, et al., 2004; Ratcliff, Thapar, et al., 2004; Schmitz & Voss, 2012). Decision tasks typically include more than one trial. Hence, after the first decision has been made, a second and even some additional decisions will usually follow. The person will thus provide the response related to the first decision and will then begin again by encoding the next stimulus by accumulating information until one of the criteria is satisfied. The person will then provide the next response, and so on. Parameter v represents the mean amount of information accumulated by one participant across a certain number of decision processes (e.g., several trials on an IAT). The distributions of correct and incorrect responses on several hypothetical trials are displayed outside their respective response criteria in Fig. 1.

A valuable quality of the diffusion model is that it provides detailed information about the cognitive processes underlying performance on decision tasks as used in the IAT (see Voss, Nagler, & Lerche, 2013). Through the exhaustive use of available performance data (i.e., latencies of correct and incorrect responses and the accuracy of responding; see Schmitz & Voss, 2012), the diffusion model decomposes the cognitive process involved in completing the IAT into a number of meaningful components. These components are represented by several model parameters.

The parameters of the diffusion model

As can be seen in Fig. 1, the diffusion model distinguishes between several parameters. Consistent with previous research on diffusion models (Klauer et al., 2007; Ratcliff & Rouder, 1998; Schmitz & Voss, 2012; Voss et al., 2004), three parameters can be considered to be most informative: namely, two parameters from the actual decision process (parameters v and a), and one parameter from the nondecision period (parameter t0). The validity of these parameters has already been successfully demonstrated in various experiments (e.g., Ratcliff, Thapar, et al., 2004; Ratcliff, Thapar, & McKoon, 2001; Thapar, Ratcliff, & McKoon, 2003; Voss et al., 2004). To make the meaning and function of these three parameters more accessible to the reader, we will exemplarily describe the decision-making process on the compatible and incompatible phases of the extraversion IAT for a fictitious participant named Peter in the following paragraphs.

Peter’s task is to press the left or the right key in order to assign stimuli (e.g., the words “talkative”, “me”, “shy”, “others”) to one of four categories (extraversion-related vs. introversion-related words and self-relevant vs. non-self-relevant words). The categories are presented in either the upper left or upper right corner of the computer screen. To fulfill the task, he has to make a decision about whether the consecutively presented stimuli belong to one of the two left or two right categories. To make this decision, he samples information. Once he has sampled enough information, his decision process is terminated, and he will initiate the pressing of the respective response key. By using Peter’s reaction times and errors, it is possible to estimate the following parameters.

The construct parameter v

Parameter v refers to Peter’s performance in the actual decision process. It quantifies the efficacy with which Peter accumulates response-related information. The higher Peter’s value on parameter v, the faster Peter has reacted while simultaneously committing only a few errors—in other words, the easier it was for Peter to make his decision.

The response caution parameter a

Parameter a refers to the amount of evidence Peter accumulates before he makes a decision in the actual decision process. It quantifies Peter’s cautiousness when making his response, called response caution. Response caution is a non-ability-related personality characteristic that contributes to speed–accuracy settings (Schmitz & Voss, 2012). People differ in whether they prefer a conservative response mode (i.e., respond more slowly but with high accuracy) or a more liberal response mode (i.e., respond more quickly by accepting the risk of increased errors). The higher Peter’s value on parameter a, the more information he samples before he makes his decision—in other words, the more conservative his response mode is.

The nondecision parameter t0

In contrast to the above-mentioned parameters a and v, parameter t0 refers not to the actual decision process, but to the nondecision components in Peter’s reaction time (cf. Klauer et al., 2007; Schmitz & Voss, 2012). This includes, for example, the perceptual encoding of the stimuli, task preparation, task switching, and the execution of motor responses (e.g., Klauer et al., 2007; Schmitz & Voss, 2012). The last item, in particular, has been given a great deal of empirical support (e.g., Ratcliff, Thapar, & McKoon, 2006; Voss et al., 2004). Thus, the higher Peter’s value on parameter t0, the more time he took to press a response key (e.g., due to his ability to execute this motor response).

The new IAT effects decomposed by diffusion model analyses

The parameters (i.e., v, a, and t0) that result from diffusion model analyses can be computed for the compatible phase as well as for the incompatible phase of the IAT. They can be used to decompose the traditional IAT effect into the following three dissociable IAT effects: IAT v , IAT a , and IAT t0 (Klauer et al., 2007). These dissociable IAT effects can be computed as the compatibility effects for the parameters v, a, and t0 (Klauer et al., 2007) by subtracting the estimated parameters from the compatible phase from the estimated parameters from the incompatible phase in each case (i.e., IAT v = parameter v in incompatible phase – parameter v in compatible phase, IAT a = parameter a in incompatible phase – parameter a in compatible phase, and IAT t0 = parameter t0 in incompatible phase – parameter t0 in compatible phase). In contrast to the traditional IAT effect, these IAT effects newly developed by Klauer et al. (2007) seem to be able to differentiate between different sources of variance in the IAT results, as we explain in the following paragraphs.

The construct-related IAT effect (IATv)

In two studies, Klauer et al. (2007) demonstrated that IAT v from a political-attitude IAT was significantly related to attitude ratings of political standpoints and, thus, was significantly associated with the construct-related variance of the IAT. In addition, construct-related variance was not mapped onto the two other new IAT effects: IAT a and IAT t0 (Klauer et al., 2007), indicating that both contain some variance from the IAT effect other than the construct-related variance.

The response-caution-related IAT effect (IATa)

As was shown by Klauer et al. (2007), IAT a is significantly related to so-called control IATs (i.e., IATs developed to contain method-specific variance only) and, hence, is significantly associated with the method-specific variance of the IAT effect. IAT v and IAT t0 were found to play only small roles in accounting for method-related variance (Klauer et al., 2007), supporting the idea that both are due to variance other than method-specific variance.

The non-decision-related IAT effect (IATt0)

Last but not least, Klauer et al. (2007) demonstrated that IAT t0 is not significantly associated with either construct-related or method-related variance in the IAT. Thus, it appears to capture an additional, third source of variance in the IAT.

In sum, Klauer et al. (2007) successfully showed that it is possible to use diffusion model analyses to separate construct-related variance from method-specific variance. However, until now, no empirical research has investigated whether it may also be possible to use diffusion model analyses to separate construct-related variance from faking-related variance. Before we explain how faking-related variance might influence the new IAT effects, it is important for the reader to understand how people fake the IAT.

Faking strategies on the IAT

Recent research has indicated that faking on the IAT is driven by different faking strategies (Röhner et al., 2013). To investigate faking strategies on the IAT, Röhner et al. (2013) instructed participants either to fake or not to fake their IAT scores. The faking instructions differed according to the requested faking direction (i.e., faking low scores vs. faking high scores) and according to whether the participants had or had not been previously informed about possible faking strategies (i.e., naïve vs. informed faking). Instructing participants to fake represents the most common methodology for investigating faking behavior (Smith & McDaniel, 2012), since this methodology provides valuable insight into the extent to which people can fake and into the strategies that people apply when asked to fake (see Smith & Ellingson, 2002; Smith & McDaniel, 2012).Footnote 3

The results of Röhner et al.’s (2013) study accordingly provided important insights into the variability of IAT faking strategies. Röhner et al., (2013) found that the choice of faking strategy depended on whether participants were requested to fake low scores or high scores (i.e., the faking direction) and on participants’ preexisting knowledge of faking strategies (i.e., whether participants had or had not been informed about possible faking strategies). Successful faking is driven by strategically slowing down or accelerating on either the compatible or the incompatible phase (Röhner et al., 2013; see also Fiedler & Bluemke, 2005). Naïve fakers (i.e., participants asked to fake without being informed about possible faking strategies) successfully fake low scores on the IAT by slowing down on the compatible phase and fake high scores by accelerating on the compatible phase (Röhner et al., 2013). Informed fakers (i.e., participants asked to fake after being informed about possible faking strategies) slow down on the compatible phase to fake low scores and slow down on the incompatible phase to fake high scores (Röhner et al., 2013).

The influence of faking on the new IAT effects

How might faking-related variance affect the newly developed IAT effects? This has not been investigated until now. Nevertheless, some ideas can be drawn from the literature.

Faking should not affect IATv

Since the IAT v represents the IAT’s construct-related variance, it should not be affected at all by faking-related variance. In line with Klauer et al.’s (2007) suggestions, faking might have an impact on parameter a and its compatibility effect IAT a (i.e., participants’ response caution), as well as on parameter t0 and its compatibility effect IAT t0 (i.e., the time needed to provide a motor response outside of the decision process). By contrast, faking should not affect IAT v , because both of the parameters that are likely related to faking variance (i.e., parameters a and t0) are automatically partialed out of IAT v (see also Klauer et al., 2007). In this vein, Wagenmakers (2009) also supposed that when diffusion model analyses are used, unwanted strategic variance (e.g., faking-related variance) is filtered out of IAT v .

Faking might affect IATa

Faking strategies (i.e., the acceleration or slowing down of reaction times in certain IAT phases; see Röhner et al., 2013) might be caused by the adaptation of speed–accuracy settings (see Fiedler & Bluemke, 2005). To be more specific, fakers might slow down their reaction times in an IAT phase by responding especially cautiously on it, and might accelerate their reaction times in an IAT phase by applying a more liberal response caution to it. Naïve and informed fakers of low scores tend to use the strategy of slowing down on the compatible phase (Röhner et al., 2013). They might slow down on the compatible phase by applying a higher response caution on that phase than on the incompatible one (see Fiedler & Bluemke, 2005; Klauer et al., 2007). This, in turn, should result in a higher parameter a in the compatible phase and a lower parameter a in the incompatible phase. As a result, the compatibility effect IAT a should decrease for participants who are asked to fake low scores. Naïve fakers of high scores tend to use the strategy of accelerating on the compatible phase, whereas informed fakers of high scores tend to use the strategy of slowing down on the incompatible phase (Röhner et al., 2013). Again, participants might do this due to variability in response caution in those phases (see Fiedler & Bluemke, 2005; Klauer et al., 2007). Both strategies (i.e., accelerating in the compatible phase and slowing down in the incompatible one) should result in a higher parameter a in the incompatible phase than in the compatible phase. Thus, faking high scores on the IAT should lead to an increase in the value of IAT a , because the faking strategies are mirror inversions of those that result from faking low (see Röhner et al., 2013). As a consequence, faking might affect the compatibility effect IAT a , because it should decrease for participants asked to fake low scores and should increase for those asked to fake high scores.

Faking might affect IATt0

Faking strategies might also be caused by motor-response adaptations that occur outside the actual decision process (see Klauer et al., 2007). To be more specific, fakers might slow down their reaction times in an IAT phase by delaying their key-pressing motor response, and might accelerate their reaction times in an IAT phase by pressing the key as quickly as possible. Naïve and informed fakers of low scores might implement the strategy of slowing down in the compatible phase by delaying their response execution in that phase more than in the incongruent one. This should cause higher t0 parameters in the compatible phase and lower t0 parameters in the incompatible phase. As a result, the IAT t0 compatibility effect should decrease for participants who are asked to fake low scores. By contrast, naïve and informed faking of high scores on the IAT should lead to increases in the value of IAT t0, because the faking strategies are mirror-inverted versions of those for faking low (see Röhner et al., 2013). Thus, faking might affect the IAT t0 compatibility effect, because it should decrease for participants asked to fake low scores and should increase for those asked to fake high scores.

The IAT represents a very popular measure (Bosson, Swann, & Pennebaker, 2000; Rudolph, Schröder-Abé, Schütz, Gregg, & Sedikides, 2008), and its fakeability has already been well-documented (e.g., Fiedler & Bluemke, 2005; Röhner et al., 2011, 2013; Steffens, 2004). Thus, it would be useful to be able to separate construct-related variance from faking-related variance on the IAT. However, no previous research has tested this possibility by investigating the impacts of faking on the three dissociable IAT effects. In our study, we thus tried to use diffusion model analyses to assess whether it would be possible to separate construct-related variance from variance caused by faking. Our study’s hypotheses are summarized in the following points.

  1. 1.

    Since IAT v is supposed to be influenced exclusively by construct-specific variance (see Klauer et al., 2007; Wagenmakers, 2009), we expected no impact of faking instructions on IAT v (i.e., on the construct-specific variance of the IAT).

  2. 2.

    By contrast, since participants typically adapt their reaction times in order to fake the IAT (Röhner et al., 2013), and this manipulation of reaction times might be due to an adaptation of their response caution (i.e., parameter a), we expected the faking instructions to impact the compatibility effect IAT a . IAT a was expected to decrease for participants asked to fake low scores, due to higher response caution in the compatible than in the incompatible phase. By contrast, IAT a was expected to increase for participants asked to fake high scores on the IAT, due to higher response caution in the incompatible than in the compatible phase.

  3. 3.

    Since reaction time adaptations can also be caused by slower or faster motor responses that hail from outside the decision process (i.e., parameter t0), we furthermore expected the faking instructions also to impact IAT t0. IAT t0 was expected to decrease for participants asked to fake low scores, because they were expected to delay their motor responses more in the compatible phase than in the incompatible phase. By contrast, it was expected to increase for participants asked to fake high scores on the IAT, because they were expected to delay their motor responses more in the incompatible phase than in the compatible phase.

Method

Data for reanalysis

To examine the process components in the IAT under faking, we reanalyzed a published data set that was collected to investigate the behavior of fakers on the IAT (Röhner et al., 2013). We decided to reanalyze this data set for several reasons: First, this previous study had investigated the faking of high and low scores on the IAT. Since we were interested in the impact of faking low and high scores on the dissociable IAT effects, it was necessary that both possible faking directions were contained in the primary data set. Second, naïve faking and faking after being informed about faking strategies were included in the data set. Since we wanted to investigate the impact of knowledge about faking strategies on the dissociable IAT effects, we needed a primary data set in which naïve and informed faking could be compared with each other. Third, the IAT used in the previous study (i.e., the extraversion IAT) is a frequently used and very popular IAT (e.g., Grumm & von Collani, 2007; Schmukle, Back, & Egloff, 2008; Steffens & Schulze-König, 2006). Last but not least, as 84 participants were included in this study an a priori power analysis using G*Power 3.1.7 (Faul, Erdfelder, Lang, & Buchner, 2007) revealed a power of nearly 100% (.998) for ANOVAs with repeated measures to detect a moderate effect size. The minimum acceptable sample size was determined to be N = 51 related to a power level of .950.

As is detailed in Röhner et al. (2013), a total of 84 volunteers (64 female, 20 male; 74 students) from Chemnitz University of Technology participated in the study in exchange for personal feedback and partial course credit. Their mean age was 22.37 years (SD = 4.45). Participants were randomly assigned to one of three conditions: (a) a control group, (b) a faking condition LH (faking low scores first, and then faking high scores), or (c) a faking condition HL (faking high scores first, and then faking low scores). All participants completed an extraversion IAT (Back et al., 2009) a total of three times. After all participants completed the IAT once without faking instructions (i.e., baseline assessment), participants in the control group completed the IATs two more times without further instructions. Participants in the faking conditions were asked to fake the IAT first with no information about the IAT’s rationale or faking strategies (naïve faking). Afterwards, they were told how to fake the IAT and were asked to fake the IAT again (informed faking). Specifically, participants in the LH faking condition faked low scores under the naïve faking condition and high scores under the informed faking condition. Participants in the HL faking condition faked high scores under the naïve faking condition and low scores under the informed faking condition.

Analytical approach

We decided to use the command-line program fast-dm (Voss & Voss, 2007, 2008), which can be downloaded from the website www.psychologie.uni-heidelberg.de/ae/meth/fast-dm/. We did not use the program called DMAT (Vandekerckhove & Tuerlinckx, 2007, 2008) because DMAT, in contrast to fast-dm, requires a sufficiently high number of correct and incorrect trials (see Voss, Nagler, et al., 2013), whereas its efficiency is comparably low. The program EZ (Wagenmakers, van der Maas, Dolan, & Grasman, 2008; Wagenmakers, van der Maas, & Grasman, 2007), in contrast to fast-dm, does not estimate the relevant parameters but rather estimates only the scores that correspond to them. Thus, we used fast-dm because it met our needs best.

Pretreatment of the data set

In fast-dm, the degree of correspondence between the observed cumulative distribution and the predicted cumulative distribution is quantified by the KS statistic (Kolmogorov, 1941), which is known to provide robust estimates even in the presence of minor outliers (Voss, Nagler, et al., 2013; Voss & Voss, 2008). Thus, we followed the recommendation (see Voss, Nagler, et al., 2013; Voss & Voss, 2008) to remove outliers from the individual reaction time distribution only if participants had reaction times below 200 or above 5,000 ms. This led to the exclusion of 156 trials (i.e., 0.3% of the trials).

Parameter estimation

We used fast-dm to estimate the independent diffusion models for each participant (N = 84) and each combined phase type (i.e., compatible vs. incompatible phase) within every measurement occasion (i.e., baseline, retest/naïve faking, and retest/informed faking). Altogether, we thus computed 504 diffusion model analyses (i.e., 84 participants × 2 combined phase types × 3 measurement occasions). Each diffusion model analysis was based on about 96 trials (i.e., exactly 96 trials in the absence of outlier trials and an average of 92 trials when outliers were excluded).

On the basis of the respective trials, we used fast-dm to estimate the values that best fit the parameters a, v, and t0 to explain the observed reaction-time distributions for correct and incorrect responses. Voss et al. (2004) provide the mathematical details of parameter estimation in fast-dm in their study’s Appendix.

We followed advice from existing studies on diffusion models to ensure the reliability of our model parameters. Thus, to reduce uncertainty in the parameters, only parameters v, a, and t0 were allowed to vary freely, whereas the starting point z was fixed to a/2 (Schmitz & Voss, 2012) and the difference in response-execution speed (i.e., parameter d) was fixed to zero (Voss, Voss, & Klauer, 2010). Given that the intertrial-variability parameters of the extended diffusion model (i.e., parameters szr, sv, and st0) are usually not very reliable (Schmitz & Voss, 2012), they were excluded as we needed only the basic diffusion model parameters for our investigation. To exclude those parameters, we set them to zero before running fast-dm.

Checking the model fit

To verify whether the estimated parameters could explain the empirical data, a Kolmogorov backward equation was used to test the plausibility of the data. Here, small values of p indicate that the diffusion model cannot account for the data (Voss, Nagler, et al., 2013; for further details, see Voss & Voss, 2007, 2008). On the basis of Voss, Nagler, et al. (2013), we discarded a participant’s data before running further analyses if they showed a poor fit to the model (i.e., p values < .05). Altogether, three participants were excluded because their data showed a poor fit. This led to a remaining sample of N = 81 participants (i.e., n = 28 in the control group, n = 25 in the LH faking condition, and n = 28 in the HL faking condition). We used fast-dm to multiply all incorrect responses by –1 and to summarize the empirical and theoretical reaction time distributions of correct and incorrect responses in one graph. Graphical displays of the model fits are presented in Fig. 2a and b.

Fig. 2
figure 2figure 2

a Overlaid predicted (parameter-based) and observed (empirical)cumulative distribution functions (cdfs) for all experimental groups and measurement occasions in the compatible IAT phase. Continuous lines represent the predicted cdfs and dashed lines represent the observed cdfs. The plotted functions are joint distributions of correct and incorrect responses. Negative values on the x-axis are the latencies of error responses (multiplied by –1) and are plotted on the left side. Positive values on the x-axis are the latencies of correct responses and are plotted on the right side. b Overlaid predicted (parameter-based) and observed (empirical) cumulative distribution functions (cdfs) for all experimental groups and measurement occasions in the incompatible IAT phase. Continuous lines represent the predicted cdfs, and dashed lines represent the observed cdfs. The plotted functions are joint distributions of correct and incorrect responses. Negative values on the x-axis are the latencies of error responses (multiplied by –1) and are plotted on the left side. Positive values on the x-axis are the latencies of correct responses and are plotted on the right side

Full size image

Computation of the compatibility effects

On the basis of the parameters estimated in the diffusion model analyses, we computed the compatibility effects IAT v , IAT a , and IAT t0 by subtracting the estimated parameters for the compatible phase from those for the incompatible phase (see Klauer et al., 2007). Since we computed these compatibility effects for each participant at every measurement occasion, a total of 243 compatibility effects resulted from these computations.

Analyzing the compatibility effect

After fitting the data to the diffusion model, estimating the parameters, and computing the compatibility effects, we used analyses of variance (ANOVAs) with repeated measures to test our hypotheses. For all ANOVAs, we set α to .05. First, we computed 3 (measurement occasion) × 3 (experimental group) ANOVAs with repeated measures on the respective compatibility effects (i.e., IAT v , IAT a , and IAT t0) to investigate whether faking would have an impact on those IAT effects. Second, to take a closer look at the results, we additionally computed 3 (measurement occasion) × 3 (experimental group) ANOVAs with repeated measures on each of the parameters (i.e., parameters v, a, and t0), separately for the compatible and incompatible phases in each case.

Results

Effects of faking on the construct-related IAT effect (IAT v )

To determine whether faking would affect IAT v , we computed a 3 (measurement occasion) × 3 (experimental group) ANOVA with repeated measures on IAT v . As expected, the main effect of group, F(2, 78) = 1.11, p = .335, ω2 = .00, was nonsignificant. Interestingly, the main effects of measurement occasion, F(1.89, 147.67) = 3.67, p = .030, ω2 = .06, and the interaction effect, F(3.79, 147.67) = 16.91, p < .001, ω2 = .43, were medium to large in size and significant.

Between- and within-group comparisons can be found in Table 1. An examination of the between- and within-group comparisons revealed that naïve faking of low and high scores as well as informed faking of high scores had influences on IAT v . In those cases, IAT v increased or decreased according to the requested faking direction (i.e., low or high scores, respectively).Footnote 4 Interestingly, one condition showed no (significant) influence of faking on IAT v (i.e., informed faking of low scores). Naïve faking of low scores led to significantly higher values of IAT v , whereas naïve faking of high scores led to significantly lower values of IAT v . Whereas informed faking of high scores led to significantly lower values of IAT v , informed faking of low scores did not lead to (significantly) higher values of IAT v . Thus, the impact of faking on IAT v depended on the specific faking condition (i.e., naïve vs. informed faking and faking low vs. high scores).

Table 1 Post hoc comparisons of the construct-related IAT effect (IAT v ) and the construct parameter v from both IAT phases in the diffusion model of the extraversion IAT
Full size table

To take a closer look at the results, we additionally computed 3 (measurement occasion) × 3 (experimental group) ANOVAs with repeated measures on parameter v for the compatible and incompatible phases. For parameter v in the compatible phase, the main effects of measurement occasion, F(1.87, 145.57) = 13.11, p < .001, ω2 = .22, and group, F(2, 78) = 4.90, p = .010, ω2 = .09, and the interaction effect, F(3.73, 145.57) = 25.44, p < .001, ω2 = .53, were medium to large in size and significant. For parameter v in the incompatible phase, the main effect of measurement occasion, F(1.92, 149.34) = 4.68, p = .012, ω2 = .08, and the interaction effect, F(3.83, 149.34) = 2.67, p = .036, ω2 = .07, were medium in size and significant, too. The main effect of group, F(2, 78) = 2.25, p = .112, ω2 = .03, remained nonsignificant. These results highlight the idea that faking affects parameter v in the compatible as well as the incompatible phase. However, the effect of faking on parameter v is more pronounced in the compatible phase.

Examining the between- and within-group comparisons for parameter v provided additional information about the influence of faking on parameter v (see Table 1). In the compatible phase, faking low led to a significantly lower parameter v under naïve faking, but not under informed faking. Faking high led to a significantly higher parameter v under both naïve and informed faking (see Table 1). In the incompatible phase, faking low led to a significant decrease in parameter v under naïve but not under informed faking. Both naïve and informed faking of high scores led to a nonsignificant decrease in parameter v.

Thus, IAT v could not be considered to be an exclusive indicator of construct-related variance, because faking intentions for the most part had an impact on parameter v, and therefore also on IAT v .

Effects of faking on the response-caution-related IAT effect (IAT a )

To determine whether faking would affect IAT a , we computed a 3 (measurement occasion) × 3 (experimental group) ANOVA with repeated measures on IAT a . As expected, the main effect of measurement occasion, F(1.67, 129.96) = 4.82, p = .014, ω2 = .07, the main effect of group, F(2, 78) = 7.84, p = .001, ω2 = .14, and the interaction, F(3.33, 129.96) = 4.50, p = .004, ω2 = .13, were medium to large in size and significant.

The between- and within-group comparisons (see Table 2) showed that, surprisingly, IAT a was affected by only one faking condition. As expected, IAT a significantly decreased when informed participants were asked to fake low scores. Surprisingly, the informed faking of high scores, as well as the naïve faking of low and high scores, did not lead to significant differences in IAT a .

Table 2 Post hoc comparisons of the response-caution-related IAT effect (IAT a ) and the response-caution parameter a from both IAT phases in the diffusion model of the extraversion IAT
Full size table

To take a closer look at the results, we additionally computed 3 (measurement occasion) × 3 (experimental group) ANOVAs with repeated measures on parameter a for the compatible and incompatible phases. For parameter a in the compatible phase, the main effect of measurement occasion, F(1.48, 115.13) = 1.30, p = .271, ω2 = .01, and the main effect of group, F(2, 78) = 2.94, p = .059, ω2 = .04, remained nonsignificant. However, the interaction effect, F(2.95, 115.13) = 4.05, p = .009, ω2 = .10, was medium in size and significant. For parameter a in the incompatible phase, the main effect of measurement occasion, F(1.89, 147.50) = 4.90, p = .010, ω2 = .08, and the main effect of group, F(2, 78) = 4.73, p = .012, ω2 = .08, were medium in size and significant. The interaction effect, F(3.78, 147.50) = 2.05, p = .094, ω2 = .05, remained nonsignificant.

The results of the between- and within-group comparisons for parameter a document the influence of the informed faking of low scores on IAT a . As expected, the informed faking of low scores led to a significant increase in parameter a in the compatible phase and to a decrease in the incompatible phase (see Table 2). Surprisingly, the informed faking of high scores as well as the naïve faking of low and high scores did not affect parameter a in either IAT phase.

Thus, IAT a appeared to capture some faking-related variance. Surprisingly, it did so only when participants had been informed about faking strategies on the IAT and were asked to fake low scores. However, the informed faking of high scores as well as the naïve faking of low or high scores did not affect IAT a .

Effects of faking on the non-decision-related IAT effect (IAT t0)

To determine whether faking would affect IAT t0, we computed a 3 (measurement occasion) × 3 (experimental group) ANOVA with repeated measures on IAT t0. The main effect of measurement occasion, F(1.85, 143.98) = 4.10, p = .021, ω2 = .07, was medium in size and significant. Interestingly, the main effect of group, F(2, 78) = 1.41, p = .251, ω2 = .01, and the interaction, F(3.69, 143.98) = 2.06, p = .095, ω2 = .04, remained nonsignificant.

The results for the between- and within-group comparisons (see Table 3) showed that, unexpectedly, there were no significant differences between and within groups on IAT t0 in any of the faking conditions (i.e., naïve vs. informed faking and faking high vs. low scores).

Table 3 Post hoc comparisons of the non-decision-related IAT effect (IAT t0) and the nondecision parameter t0 from both IAT phases in the diffusion model of the extraversion IAT
Full size table

To take a closer look at the results, we additionally computed 3 (measurement occasion) × 3 (experimental group) ANOVAs with repeated measures on parameter t0 for the compatible and incompatible phases. For parameter t0 in the compatible phase, the main effects of measurement occasion, F(1.75, 136.25) = 2.70, p = .078, ω2 = .04, and group, F(2, 78) = 0.51, p = .602, ω2 = .00, and their interaction, F(3.49, 136.25) = 0.83, p = .493, ω2 = .00, remained nonsignificant. For parameter t0 in the incompatible phase, the main effect of measurement occasion, F(1.85, 143.93) = 1.49, p = .230, ω2 = .01, the main effect of group, F(2, 78) = 0.27, p = .761, ω2 = .00, and the interaction, F(3.69, 143.93) = 0.84, p = .497, ω2 = .01, also remained nonsignificant.

The between- and within-group comparisons for parameter t0 illustrated that there was no influence of faking on IAT t0. In other words, parameter t0 was not influenced by naïve faking, informed faking, the faking of high scores, or the faking of low scores in the compatible or the incompatible phase (see Table 3). Thus, interestingly, IAT t0 did not appear to capture any faking-related variance at all.Footnote 5

Discussion

In the present study, we investigated whether diffusion model analyses could be used to separate construct-related variance from faking-related variance on the IAT. The results showed the advantage of using diffusion model analyses to compute the dissociable IAT effects IAT v , IAT a , and IAT t0, since they allowed important insights into the different sources of variance that are related to the IAT effect.

Does faking have an impact on the construct-related IAT effect (IAT v )?

Interestingly, our results showed that faking instructions had an impact on IAT v . However, faking did not affect IAT v in the same way in every faking condition. Faking had an impact on IAT v when participants were asked to fake low scores or high scores without being informed about possible faking strategies, or when they had been informed about possible faking strategies and had to fake high scores. In those cases, faking low led to an increase and faking high to a decrease in IAT v . Only one condition showed no (significant) influence of faking on IAT v (i.e., informed faking of low scores).

Parameter v in the compatible and incompatible phases provided additional information about the results. First, the impact of faking was more pronounced for parameter v in the compatible than in the incompatible phase. This finding is in line with recent research that has shown that in most cases, people who are able to fake successfully use strategies to manipulate their reaction times in the compatible phase (see Röhner et al., 2013). Second, the naïve faking of low scores was associated with decreases in parameter v in the compatible as well as in the incompatible IAT phase. The latter might sound counterintuitive, since it does not mirror a successful faking strategy in this condition (Röhner et al., 2013). However, it might be explained by Röhner et al.’s (2013) results, which showed that naïve fakers of low scores also (unsuccessfully) tried to fake low scores by committing errors in an unsystematic manner (i.e., on both IAT phases). Taken together, these results indicate that parameter v and IAT v seem to capture some of the variance caused by faking strategies.

These results on the impact of faking on IAT v might at first glance appear somewhat disappointing, since the construct-related IAT effect was shown to be affected by faking. However, they provide important insights into the faking process itself and into the faking strategies that are used on the IAT. The results show that, in addition to the preexisting implicit associations that are supposed to be measured with the IAT (see Greenwald et al., 1998), faking intentions also influence the ease of decision-making on the IAT (i.e., captured by IAT v ). How might this influence of faking on IAT v be explained? Faking might to some extent result from temporary changes in people’s accessible mental associations. For example, people could try to take on the role of an extraverted person by telling themselves, “For the next few minutes, I will be an extraverted person”, or the role of an introverted person by telling themselves, “For the next few minutes, I will be an introverted person”. It has already been shown that it is possible to create temporary mental associations (De Houwer et al., 2007). Such temporary mental associations may be helpful for faking, especially under difficult faking conditions (e.g., faking without preexisting knowledge about faking strategies). A temporary change in mental associations as a result of faking might in turn influence the ease of decision-making on the IAT, and may consequently also influence IAT v . Thus, under faking, IAT v not only contains construct-related variance, but also faking-related variance, since the way that people’s faking intentions contribute to the ease of decision-making may be similar to the way that preexisting implicit associations do.

However, it is interesting that IAT v was affected by faking in only three out of the four different faking conditions. The informed faking of low scores had no (significant) impact on IAT v . How might this result be explained? Several studies have revealed that faking with a recommended strategy is easier than faking without a recommended strategy (e.g., Fiedler & Bluemke, 2005; Kim, 2003; Röhner et al., 2011) and that faking low scores is easier than faking high scores (Röhner et al., 2011). Thus, faking low scores with a recommended strategy for how to do so represents the easiest faking condition. Most likely, participants in this (easy) faking condition did not need to establish a temporary mental association to fake successfully, and hence, faking had no (significant) impact on IAT v in this faking condition.

Does faking have an impact on the response-caution-related IAT effect (IAT a )?

The impact of faking on IAT a was a mirror inversion of the impact of faking on IAT v . IAT a was not affected by faking when participants faked low scores or high scores without being informed about possible faking strategies, or when they were informed and were asked to fake high scores. In other words, IAT a was affected only when participants faked low scores after being informed about faking strategies (i.e., the easiest faking condition). In this condition, faking low scores was reflected by the expected decrease in the value of IAT a . As a consequence, IAT a may indeed capture some of the faking-related variance, but only for informed fakers of low scores (i.e., not for informed fakers of high scores or naïve fakers of low and high scores).

The results for parameter a in the compatible and incompatible phases might provide additional information about this result. IAT a was affected by the informed faking of low scores (i.e., the easiest faking condition) because of a more conservative speed–accuracy setting in the compatible phase and a more liberal one in the incompatible phase. By contrast, parameter a in the compatible and incompatible phases was not significantly influenced by faking in the more difficult faking conditions (i.e., faking without being informed about possible faking strategies or faking high scores).

How can this be explained? Again, this result sheds more light on the faking process itself. The finding reveals that when participants were confronted with a comparably easy faking condition (i.e., the informed faking of low scores), they did not have to temporarily change their mental associations to fake successfully (as in the more difficult faking conditions, such as naïve faking or faking high scores). Instead, they were able to simply manipulate their response caution in order to fake successfully (i.e., to apply higher response caution in the compatible than in the incompatible phase).

Does faking have an impact on the non-decision-related IAT effect (IAT t0)?

Surprisingly, IAT t0 was not affected at all by faking in any faking condition (i.e., under naïve or informed faking of low or high scores). The results for parameter t0 in the compatible and incompatible phases demonstrated that IAT t0 values (i.e., the response time outside of the actual decision process) were nearly equal across all experimental conditions. Thus, IAT t0 does not capture any faking-related variance at all.

Remembering that IAT t0 is assumed to reflect components that lie outside of the actual decision process, this result again provides important insights into the faking process. The result indicates that fakers do not manipulate their reaction times outside of the decision process, but instead manipulate the decision process itself by temporarily changing their accessible mental associations (thus affecting IAT v ) or, under comparably easy faking conditions, by manipulating their response caution (thus affecting IAT a ). This finding in turn might to some extent explain why it is so difficult to detect fakers (see Fiedler & Bluemke, 2005; Röhner et al., 2013). On the one hand, we already know that faking indices have a high risk of misclassification, since they are based solely on detecting some kind of deceleration in a person’s response rate (i.e., a strategy not applied by all fakers; see Röhner et al., 2013). On the other hand, as was shown in this study, fakers do not manipulate their reaction times outside of the actual decision process (e.g., by delaying their motor execution of the key pressing), but instead manipulate the decision process itself (e.g., by manipulating the ease of decision-making by establishing helpful mental associations). Thus, the risk of misclassification associated with existing faking indices (see Röhner et al., 2013) might additionally be explained by the possibility that faking behavior and nonfaking behavior may simply not be distinctive enough to be clearly separated from each other by faking indices.

The assets and drawbacks of using diffusion model analyses

From an applied point of view, researchers might be somewhat deterred from using diffusion model analyses to compute these newly developed IAT effects. In fact, as compared with the computation of the traditional IAT effect, this new procedure is more complex and time-consuming, which might be seen as two relevant drawbacks. In addition, our results indicate that a supposed relevant advantage of this new method does not hold true, because faking surprisingly affected the construct-related IAT effect IAT v . Thus, as is true for the traditional IAT effect, IAT v also has to be interpreted with caution, since it can be confounded with faking-related variance.

Still, diffusion model analyses are also associated with at least two relevant assets. One first, great benefit of using diffusion model analyses to compute model parameters and compatibility effects is that they can be applied in order to screen data to identify some first indications of faking. This is especially important because a multitude of research has shown that the IAT can be faked (e.g., De Houwer et al., 2007; Fiedler & Bluemke, 2005; McDaniel et al., 2009; Röhner et al., 2011; Steffens, 2004). Recent research has additionally indicated that identifying IAT fakers is much more difficult than had previously been thought (Fiedler & Bluemke, 2005; Röhner et al., 2013). The naïve faking of low scores might stand out because of decreases in the p values associated with parameter estimation. The informed faking of low scores might be indicated by effects on IAT a , but the naïve and informed faking of high scores might not be indicated, because faking high scores only affects the construct-related IAT effect (i.e., IAT v ). From a practical standpoint, faking low is indeed the more likely faking condition, because low scores often represent the associations that are not socially stigmatized in particularly sensitive areas, such as pedophilia, racism, stereotypes, or sexism (e.g., Agerström & Rooth, 2011; Banse, Schmidt, & Clarbour, 2010; Banse et al., 2001; Carlsson & Björklund, 2010; Gray, Brown, MacCulloch, Smith, & Snowden, 2005; Greenwald & Banaji, 1995; Greenwald et al., 1998; Latu et al., 2011).

A second important asset of diffusion model analyses, as revealed by our results, is that they provide detailed information about the cognitive processes underlying the faking process on the IAT. Although some research has tried to uncover what people actually do to fake the IAT (e.g., Agosta, Ghirardi, Zogmaister, Castiello, & Sartori, 2011; Cvencek, Greenwald, Brown, Gray, & Snowden, 2010; Röhner et al., 2013), the processes behind faking on the IAT still represent a sort of black box. The diffusion model has allowed us to peer into this black box to reveal at least three important insights into the faking process. First, people are able to change their mental associations in order to fake successfully, at least to some extent. They do so especially in difficult faking conditions (i.e., faking high scores or naïve faking conditions). Thus, in difficult faking conditions, establishing appropriate mental associations may help people to fake successfully. Second, in comparably easy faking conditions (i.e., faking low scores), fakers do not need to establish such helpful mental associations, but instead they may simply respond more cautiously or liberally in order to fake successfully. Third, faking represents a process that takes place within the decision process and not outside of it—an important finding that in turn helps to explain why detecting fakers is so difficult (see Fiedler & Bluemke, 2005; Röhner et al., 2013).

Limitations

Of course, this study is limited by the fact that we used only an extraversion IAT. The results should be replicated using IATs that measure other constructs. In addition, the participants in this reanalyzed data set were students with an average age of 22.37 years. However, age is related to reaction times and errors (e.g., Endrass, Schreiber, & Kathmann, 2012). Since both are used for parameter estimation in diffusion model analyses, future research should replicate and extend the findings of our study by using a sample that includes older participants. Last but not least, we used only diffusion model analyses (Klauer et al., 2007). Of course, other models, such as the quad model (Conrey, Sherman, Gawronski, Hugenberg, & Groom, 2005) or the discrimination–association model (Stefanutti, Robusto, Vianello, & Anselmi, 2013), have been proposed too for decomposing the processes underlying the IAT effect. However, only within the diffusion model analyses (Klauer et al., 2007) the impact of faking on the model parameters was considered.

Summary and conclusion

At present, it is not possible to clearly separate construct- and faking-related variance on the IAT by using diffusion model analyses. As is also true for the traditional IAT effect, the construct-related IAT effect IAT v is influenced by faking. Nevertheless, diffusion model analyses can help us understand and interpret IAT scores. First, the use of diffusion models enabled us to determine why it is not yet possible to separate construct-related variance from faking-related variance on the IAT. We showed that the ease of decision making (as captured by the construct-related IAT effect IAT v ) was systematically manipulated by fakers. To prevent faking from having an impact on the construct-related IAT effect, future research should investigate how IAT v can be assessed more purely. Second, by using diffusion model analyses it might be possible to screen data to obtain some indication of faked IAT scores. Whereas fakers of high scores did not stand out, poorly fitting models might indicate that participants have tried to fake low scores in a naïve manner, and fakers of low scores who have knowledge about possible faking strategies might stand out because of their values on IAT a . Last but not least, our results clearly showed that the diffusion model represents a useful tool for investigating the faking process and for expanding our understanding of how people fake the IAT.