Two of the major questions in our quest to understand how humans interact with their visual environments are: (1) When are visual features combined and represented in memory in order to alter our behavior to later instances of them (“feature integration effects”; Hommel, Proctor, & Vu, 2004)?, and (2) When is attention biased against or in favor of specific features/regions in the visual field (“orienting effects”; Hilchey, Pratt, & Christie, 2018)? These questions are not necessarily mutually exclusive, and on occasion the paradigms addressing them intersect (cf. Gabay, Chica, Charras, Funes, & Henik, 2012; Hilchey, Rajsic, Huffman, & Pratt, 2017a, 2017b; Hommel, 1998; Hommel & Colzato, 2004; Hommel et al., 2004; Lupiáñez, Milán, Tornay, Madrid, & Tudela, 1997; Mordkoff, 2012; Notebaert & Soetens, 2003; Notebaert, Soetens, & Melis, 2001; Pratt & Abrams, 1999; Rajsic, Bi, & Wilson, 2014; Stürmer, Leuthold, Soetens, Schröter, & Sommer, 2002; Tanaka & Shimojo, 1996; Taylor & Donnelly, 2002; Terry, Valdes, & Neill, 1994). Yet, interpretations of the data from such paradigms vary considerably, depending on whether the data are seen through the lens of frameworks that place greater emphases on feature integration (e.g., Hommel, 2004; Lupiáñez, 2010, for reviews) or attentional processes (e.g., Klein, 2004; Lupiáñez, Klein, & Bartolomeo, 2006, for reviews). The success of both frameworks under certain circumstances (e.g., Hilchey, Rajsic, Huffman, Klein, & Pratt, 2018) invites us to consider, broadly, the limit(s) and transition point(s) between feature integration and attentional orienting effects.

In this article, we first highlight the phenomena historically ascribed to feature integration effects and those ascribed to attentional orienting effects, while also outlining popular approaches for studying them. We then review a subset of the attentional orienting literature that is particularly relevant to feature integration. We then describe an experiment in line with the bedrock attention literature that was capable of providing clear evidence for or against feature integration effects. Our goal in doing so is straightforward. On the one hand, if the data from the bedrock attention literature reveal integration effects, then there is the distinct possibility that various “attentional effects” may be subsumed by the principles of feature integration frameworks. On the other hand, if feature integration effects are absent across these data, the bedrock attention research will remain intact, while revealing a limit on feature integration frameworks.

Feature integration effects

The basic paradigms for studying feature integration effects are often conveniently simple. One particularly well-known example is the partial-repetition-costs paradigm (Hommel, 1998; Hommel & Colzato, 2004; Hommel, Memelink, Zmigrod, & Colzato, 2014). Usually, in this paradigm, a left or right arrow appears at fixation, indicating that a left- or right-handed response should be prepared. Next, a stimulus (S1) appears randomly somewhere in parafoveal or peripheral vision, to which the prepared response is made (R1). This is followed by another stimulus that randomly matches the previous stimulus’s location, color, shape, and so forth (S2). The participants then discriminate the color or form of S2 with an arbitrary keypress response (R2). The paradigm allows for the evaluation of how performance varies across all orthogonal combinations of stimulus, location, and response repetition (i.e., “repetition effects”).

In the partial-repetition-costs paradigm, participants respond fastest and most accurately when S2 and R2 match S1 and R1 in all respects. When S2 and R2 match some, but not all, of the aspects of S1 and R1, response times and error rates increase. Finally, when S2 and R2 match no features of S1 and R1, response times and error rates are typically similar to when S2 and R2 fully repeat S1 and R1. The increase in response times and error rates when S2 and R2 repeat only some of the features from S1 and R1 is known as a partial-repetition cost. Such findings are not confined to the partial-repetition-costs paradigms. Rather, feature integration effects seem to occur in many tasks that require stimulus discrimination on the basis of color or form (Hilchey, Pratt, & Christie, 2018; Hilchey et al., 2017a, 2017b; Hommel, 1998; Hommel & Colzato, 2004; Notebaert & Soetens, 2003; Rajsic et al., 2014; Taylor & Donnelly, 2002) and have become foundational evidence supporting feature integration across perception and action.

In explaining such integration effects, the theory of event coding (TEC; Hommel et al., 2001) argues that information (color, shape, response, or location) is linked together to form an implicit memory representation, metaphorically referred to as an “event file” (i.e., an object file that also includes response features; see Schumacher & Hazeltine, 2016, for a further extension that includes task features). If S2 and/or R2 repeat(s) some part of S1 and/or R1, the existing event file is retrieved. When S2 and R2 fully repeat S1 and R1, the retrieved event file matches the current stimulus–response combination, allowing participants to respond quickly and accurately. When, however, the second stimulus–response combination only partially repeats the first, the retrieved event file conflicts with the current stimulus processing and response requirements. This conflict leads to relatively slow responses. With color or form discrimination tasks—even when simplified further, such that there are only two stimuli in each trial, each requiring either shape or color processing prior to the keypress response (Hilchey et al., 2017a, 2017b)—feature integration frameworks account for the keypress effects very well (but see Hilchey, Rajsic, et al., 2018, for eye movement data). There is often little need to appeal to attentional frameworks or constructs (Hilchey et al., 2017a; Hommel, 2005).

Attentional orienting effects

When the aforementioned paradigms are modified slightly, such that serially presented stimuli can be detected or localized, irrespective of their color or form, attentional frameworks are historically invoked to account for the findings (e.g., Maylor, 1985; Maylor & Hockey, 1985; Posner & Cohen, 1984; Vaughan, 1984). With the basic approach, a simple keypress or left/right localization response is made to the abrupt appearance of any stimulus, regardless of its identity. When the interval is greater than a fraction of a second but less than several seconds, responding is usually slower whenever the target location repeats instead of switches. This effect is often attributed to inhibition of return (Posner, Rafal, Choate, & Vaughan, 1985).

Generally speaking, the concept behind inhibition of return is that a constellation of mechanisms bias attentional orienting against previously attended locations or objects, (Klein, 2000; Klein & Redden, in press). These mechanisms can bias attentional orienting by reducing the salience or priority of previously attended objects or inputs, or by making people more reluctant to respond to previously attended locations, depending on the involvement of the oculomotor system (Grison, Kessler, Paul, Jordan, & Tipper, 2004; Hilchey, Klein, & Satel, 2014; Hilchey, Pratt, & Christie, 2018; Satel, Hilchey, Wang, Story, & Klein, 2013; Taylor & Klein, 2000). Regardless of whether the slower responses for target location repeats relative to switches are due to degradation in the quality of the input signal (e.g., Smith, Ball, & Ellison, 2012) or to a reluctance to respond in the direction of the prior target location (e.g., Hilchey, Dohmen, Crowder, & Klein, 2016), the repetition of the target’s form and/or color should not modify the effect.

Although we prefer the aforementioned account, certain complex alternatives stress the potential for contributions from joint feature integration and stimulus detection processes. Sometimes, inhibition of return is alternatively described as a “detection cost” that is presumed to occur in precisely the kinds of detection or localization tasks that we have just highlighted. According to the detection cost theory of inhibition of return (e.g., Lupiáñez, 2010; Lupiáñez, Martín-Arévalo, & Chica, 2013), repeating a stimulus location leads to effects that are independent of orienting. At its core, the detection cost theory appears to recognize a distinction between “attentional orienting”—which, in classic spotlight theories of attention, refers to the alignment of the senses, either covertly or overtly, to a location in order to improve the extraction of input from it—and “attentional selection”—which refers to the ease with which said input can be extracted or selected from said space, once it is oriented to (Posner & Petersen, 1990; see also, e.g., Hilchey, Rajsic, et al., 2018). The detection cost theory envisions that at least some of the “inhibition of return” effect is more closely related to selection than orienting.

Behind the detection cost is the idea that the visual system is hardwired for detecting new events or objects. Toward this end, the visual system encodes, maintains, and has access to the features (e.g., location, color, shape, etc.) of recently encountered stimuli. This account asserts that the closer the resemblance between a recent and a current stimulus, the more difficult it becomes to detect the “new” stimulus, an effect that expresses itself in the form of slower responses. The circumstances in which the detection cost can be seen in behavior are somewhat unclear, since there are apparent contradictions between studies (Lupiáñez, 2010; cf. Lupiáñez et al., 2013). According to Lupiáñez et al. (2013, p. 245), the detection cost is in effect negligible in target–target tasks, largely because the aforementioned TEC principles are presumed to dominate. If this is the case, we need not worry too much about detection costs and can instead remain focused on the ability of the TEC to account for the findings. However, according to Lupiáñez (Lupiáñez, 2010, p. 27), the detection cost is related to integration processes, such that the likelihood of integrating the second target into the representation of the first increases, the greater the perceived resemblance between them. It is assumed that integrating the second target into the representation of the first is a more time-consuming operation than simply forming a new representation. To the extent that a detection cost is in effect, the clearest prediction from the detection cost theory is that repeating the target form/color should increase the magnitude of “inhibition of return.”

To recap quickly: On the one hand, according to inhibited-reorienting accounts, target location repetition costs should be unaffected by whether nonspatial features repeat or switch across stimuli. On the other hand, variations of the detection cost theory and the TEC suggest that target location repetition effects will be critically dependent on the repetition of form and/or color. Therefore, if location repetition effects are critically modified by nonspatial feature repetition, the bedrock attentional orienting literature could be threatened or subsumed by feature integration frameworks. If these “inhibition of return” effects are not, however, critically modified by feature repetition, then there is a clear limit on feature integration frameworks. To find this out, we now review the relevant basic attentional orienting literature.

Attentional orienting literature review

Although attentional orienting has been studied using many different variations on the same basic procedure (Posner & Cohen, 1984), we are concerned with a narrow subset of those studies. To cleanly examine feature integration when nonspatial stimulus properties are unambiguously irrelevant, we looked at experiments that used target–target designs—that is, designs in which simple detection or localization responses were made to each stimulus. Furthermore, within those target–target studies, we examined those that manipulated the targets’ locations and nonspatial properties. We excluded any localization or detection experiment in which nonspatial feature discrimination was necessary before making the response (e.g., Morgan & Tipper, 2007). We excluded the intertrial-position-priming literature on visual search with localization and detection responses (e.g., Campana & Casco, 2009) and the spatial negative-priming literature, in which a single distractor is usually pitted against a target that needs to be localized (e.g., Christie & Klein, 2001). Finally, we excluded studies implementing cue–target designs (studies that present an irrelevant stimulus, the cue, that the participants are asked to ignore, followed by a second stimulus that the participants must respond to) that manipulated whether the nonspatial properties of the cue repeated (e.g., Hu, Fan, Samuel, & He, 2013; Pratt, Hillis, & Gold, 2001). In each of these types of studies, discrimination of stimulus form or color either was or might be necessary to differentiate targets from distractors or cues.Footnote 1

These restrictions limited our search to a surprisingly small number of studies: six, consisting of 14 experiments in total (Fox & de Fockert, 2001; Kwak & Egeth, 1992; Pratt & Castel, 2001; Tanaka & Shimojo, 1996, 2000; Taylor & Donnelly, 2002). See Table 1 for an overview of these studies’ general methods. Within each experiment, we looked at the location repetition effect and examined whether that effect varied on the basis of whether the nonspatial feature repeated or switched. The TEC and attentional orienting accounts both make specific, and differing, predictions for these experiments. Considering only the relationship between target location and response repetition, the TEC predicts a null result when stimulus location repeats instead of switches for localization responses. For detection responses, the prediction is a bit ambiguous, but either way it is contrary to inhibited-reorienting accounts. If the location repeats in the detection task, so does the response, leading to a full repeat. If the location switches, the response still repeats, perhaps leading to a partial mismatch between the past and present location and response codes. The TEC either predicts a null result or faster responding when the target location repeats, depending on whether the response location is considered relevant (Hommel, 2007). Inhibited-reorienting accounts predict slower responses when the target location repeats (Posner et al., 1985). Second, considering only the relationship between target location and nonspatial feature repetition, the TEC expects that any effect of the stimulus location repeating from one moment to the next will be modified by whether the nonspatial features also repeat. For instance, in the event of binding between a nonspatial property and space, stimulus location repetition costs might be expected if the stimulus shape changes; stimulus location repetition advantages might be expected if the stimulus shape also stays the same (e.g., Hommel, 2005). Classic inhibited-reorienting accounts predict that the repetition of a nonspatial feature will be inconsequential for inhibited spatial reorienting.

Table 1 An overview of the studies analyzed in the present review
Full size table

Our findings are summarized in Fig. 1.Footnote 2 The data paint a clear picture. Repeating stimulus locations (and responses) versus switching stimulus locations significantly slowed responding in all analyzed studies (M ± SE = – 27 ± 4 ms), with each experiment finding a significant main effect of stimulus location repetition. We next looked at whether this stimulus location repetition effect changed depending on whether the nonspatial properties also repeated, by comparing the location repetition effects in the full-repetition conditions with those in the full-switch conditions. This revealed that the location repetition effects were no different in the full-repetition (– 26 ± 6 ms) than in the full-switch (– 23 ± 5 ms) conditions. There was one anomalous study. Pratt and Castel (2001) found that in the full-repeat and full-switch conditions there were no location repetition effects, but in the partial-repetition condition they found a significant cost for repeating locations. Note that although visually it appears that Tanaka and Shimojo (2000) may have found a similar interaction, it was not significant, and its direction does not fit the predictions of feature integration approaches or the general pattern in this literature. Finally, the data from Tanaka and Shimojo (1996) could not be considered, because none of the relevant means were reported or visualized, but statistically they reported slower responding when the target location repeated than when it switched, regardless of whether the nonspatial properties of the stimulus repeated.

Fig. 1
figure 1

Location repetition costs as a function of nonspatial feature repetition type. Note that only Pratt and Castel (2001) and Tanaka and Shimojo (2000) included a partial-repetition condition and reported the relevant data for including that condition in our analysis

Full size image

To summarize this literature, location repetitions led to slower response times in almost every case, an effect that was not modified by repeating or switching target features from one moment to the next (e.g., color). As we noted, there was one exception: Pratt and Castel (2001; n = 8) demonstrated that same-location costs were eliminated whenever the nonspatial features fully repeated or fully switched, and were restored when a single nonspatial feature repeated, a pattern consistent with feature integration if fully unified stimulus representations are assumed (and inconsistent with inhibited reorienting). This exception is all the more striking because Pratt and Castel used the same design as Tanaka and Shimojo (1996; n = 6). Both studies tested four response–stimulus intervals (100, 300, 500, and 1,200 ms) and rectangles that differed in either color (red or green) or orientation (horizontal or vertical). Both studies manipulated task, and in separate sessions the targets had to be either detected with single keypresses, localized with a left or right keypress, discriminated on the basis of orientation, or discriminated on the basis of color with a left or right keypress. Evidently, none of these factors were responsible for the discrepancy.

We decided to reevaluate Pratt and Castel (2001) by conducting a quick conceptual replication and quadrupling the sample size (n = 32). We used the layout of a standard spatial-orienting paradigm (Chica, Martín-Arévalo, Botta, & Lupiánez, 2014). As in Pratt and Castel (2001) and Tanaka and Shimojo (1996), the targets were red and green rectangles (1.1 × 0.7 dva), matched on luminance (17 cd/m2) and oriented either vertically or horizontally. These targets appeared randomly at one of two marked locations (2 × 2 dva white outline boxes), centered 6 deg to the left or right of a fixation cross (0.15 × 0.15). We tested at 900- and 1,200-ms response–stimulus intervals, and each participant completed 384 trials in the localization and color discrimination tasks, the order of which was counterbalanced. In our analysis of the localization task, trials containing errors (2%), and response times less than 100 ms or greater than 1 s (2.8%) were excluded. Only location repetition mattered, F(1, 31) = 22.85, MSE = 928, p < .001, ηp2 = .424 (Fig. 2),Footnote 3 such that there was a target location repetition cost, regardless of whether anything else repeated, consistent with the pattern common in the literature (see Fig. 1). All other ps > .22.Footnote 4

Fig. 2
figure 2

Response times in location switch and location repeat conditions as a function of nonspatial feature repetition condition. Error bars represent the within-subjects SEs (Cousineau, 2005)

Full size image

Summary

In the present study we investigated whether the feature integration effects expected by the TEC are found in the detection and localization tasks that form the backbone of the attentional orienting literature. In terms of stimulus location repetition effects, the TEC in its most general form predicts a relative benefit or a null result for location and response repetitions, yet target–target detection and localization tasks have consistently revealed similar costs for such full repetitions. Second, with the exception of Pratt and Castel (2001), none of the experiments showed partial-repetition costs; the location repetition effects were unaffected by the status of the nonspatial features. Furthermore, our attempt to replicate Pratt and Castel led to data consistent with the other studies in our sample. Taken together, we have found no compelling evidence for feature integration effects in detection and localization tasks, at least when there are virtually no nonspatial selection demands. This finding has implications for both the feature integration and attentional orienting literatures.

As a feature integration framework, one of the most impressive aspects of the TEC (Hommel et al., 2001) is its proposed generalizability. Whereas many theoretical frameworks are developed to account for relatively narrow sets of findings, the TEC was proposed as a general theory of perception and action interactions, cutting across specific paradigms. Here, however, we have resurrected data from a paradigm closely related to the partial-repetition-cost paradigm that limit this generalizability. In these most basic tasks with minimal nonspatial processing demands, there is no evidence of feature integration effects. These data contrast with the predictions of the TEC, thus revealing a clear limit on it.

Of course, none of the tasks we have discussed may be said to be completely free of nonspatial processing. Indeed, one might suggest that simple detection and localization tasks require stimulus evaluation that, according to our hypothesis, should lead to feature integration effects, yet there are none (Tanaka & Shimojo, 1996, 2000; Taylor & Donnelly, 2002). It is true that in localization tasks, individuals need to evaluate whether a stimulus is to the left or the right of fixation, with the key point being that they can determine where the stimulus is or how to act on it, without having to process what it is, which may rely on different neural pathways. Our hypothesis pertains specifically to participants making evaluations concerning the stimulus itself, not merely where it is in space (see also Hilchey, Rajsic, et al., 2018).

Our study also speaks to the role of spatial attention in feature integration. According to the canonical feature integration theory (Treisman, 1986; Treisman & Gelade, 1980), spatial attention is needed to combine features in order to form object representations. This belief carried forward into the object preview paradigm (a precursor to the partial-repetition paradigm; Kahneman, Treisman, & Gibbs, 1992). In contrast, Hommel (2005) rejected attention as being necessary for feature integration. In the present study, we examine scenarios in which it is highly likely that each stimulus was attended and found little evidence of integration. This leaves open a couple of possibilities regarding spatial attention’s role in feature integration. One possibility is that spatial attention is insufficient for feature integration. Instead, paying attention to forms and colors on some level may be necessary simply because the task demands it. A second possibility is that attending a stimulus’s location always leads to the creation of an event file, but that this event file is only selectively retrieved, depending on what kind of nonspatial processing is necessary to forming a response (e.g., Hommel et al., 2014). We are partial to the first possibility, mainly out of concern that prior tasks have, at least at some point during their sequences of event, required shape or color processing.

To this point, we have been relatively clear in our belief that location repetition costs reflect an attentional orienting bias (inhibition of return). However, we acknowledge that this effect can become linked to landmarks in the environment and that inhibited reorienting, as a construct, may be unnecessary in such circumstances. We believe this partly because object-based inhibition-of-return effects tend to be more robust in covert than in overt attention tasks (e.g., Hilchey, Pratt, & Christie, 2018), making it difficult to determine directly whether orienting is inhibited. At least part of the reason for slower responses with target location repetitions than with switches may be due to effects on target selection, independent of orienting, as suggested by the detection cost theory of IOR (Lupiáñez, 2010; Lupiáñez et al., 2013). According to this view, a concession could be made to allow the location repetition cost to involve a feature integration process that relates more closely to target selection or extraction than to orienting. For example, perhaps an event file is created of the first target that includes only some of its associated features. Accordingly, one could presume ad hoc that color and shape are not included in the representation. Then, following Lupiáñez’s (2010) integration process, if the second target appears at the same location as the first, the visual system would treat the second target as an updated version of the first, leading to an updating cost and thereby slowing down responses. Granted, we must be careful to ensure that the theories do not become metaphysical to the point of becoming untestable. In any event, in the present context, without considering eye movements, we remain agnostic about whether the observed same-location costs are related to inhibition of return (orienting) or to some other mechanism related to target detection costs (selection). However, when the oculomotor system responsible for rapid eye movements is involved (Klein & Hilchey, 2011), we maintain that there are clear orienting biases against previously attended locations (e.g., Bichot & Schall, 2002; Hilchey et al., 2016; Hilchey et al., 2014; Hilchey, Pratt, & Christie, 2018; Hilchey, Rajsic, et al., 2018; Posner et al., 1985; Pratt & Abrams, 1999; Taylor & Klein, 2000).

In the present review, we have focused on the target–target paradigm and found no effect of nonspatial feature repetition on target location repetition effects. That said, another common paradigm used by attentional orienting researchers is the cue–target paradigm, in which only the second stimulus in the sequence is detected or localized. Interestingly, in that paradigm there is some evidence that repeating a stimulus form or color can interact with the target location repetition effects, such that the location repetition cost is greater when the nonspatial feature repeats (Hu, Samuel, & Chan, 2011; Klein, Wang, Dukewich, He, & Hu, 2015). Why nonspatial feature repetitions interact with location repetition effects in the cue–target, but not the target–target, paradigm remains an open question. One possibility is that, because cues do not require a response and are visually similar to the targets (and, in some cases, to other distractors; Hu et al., 2011; Klein et al., 2015), some level of feature discrimination is necessary to distinguish the cue from the target. This may make stimulus identity processing more relevant, perhaps leading to those interesting interactions between stimulus location and identity repetition.

The findings from our review support the arguments from previous researchers that task type and/or the participants’ intentions should alter feature integration effects (Memelink & Hommel, 2013; Schumacher & Hazeltine, 2016). Indeed, in contrast to previous examples of such modulations (Hazeltine, Lightman, Schwarb, & Schumacher, 2011; Hommel et al., 2014), in which feature integration effects were reduced, we found a particularly striking example in which feature integration effects were altogether absent (see also Moeller & Frings, 2017). Not only does this support the notion that task type or participants’ intentions can modulate integration effects, but it also highlights a potential tool for future researchers to use in investigating how task-based changes affect feature integration effects. Given that feature integration effects were absent in the examined detection and localization tasks, they may serve as a suitable baseline for future research looking at how task-based changes can lead to feature integration effects; rather than looking for task-based changes to modulate an interaction, one can look for the presence as compared to the absence of an interaction. For example, if only stimuli of a certain color must be detected or localized, nonspatial feature integration effects may once again play a more prominent role (Wilson, Castel, & Pratt, 2006). Finally, we note that because feature integration effects are absent from target–target paradigms in which decisions about the stimulus’s nonspatial features are unnecessary, these tasks are particularly useful for studying location repetition effect(s), since they are largely unaltered by nonspatial feature repetition. This is so regardless of whether the effect is on selection or orienting, and it stands in contrast to paradigms in which decisions must be made about nonspatial features, in which case orienting biases can be obscured entirely (Hilchey, Rajsic, et al., 2018).

Author note

This project was supposed by the Natural Sciences and Engineering Research Council of Canada, through a discovery grant (480593) awarded to J.P. and a postdoctoral scholarship awarded to M.D.H.