The present study provides the first empirical evidence for an interaction between physical and numerical size during visual search; that is, targets with congruent physical and numerical size were detected faster compared to targets with an incongruent configuration. Perceptual differences of the stimuli that might account for the observed differences in search performance (cf. Wong & Szücs, 2013) were controlled by a condition in which the target item was defined by the colour and not by the size, showing no difference between the numerical stimuli (Experiments 2), and by demonstrating a modulation of semantic distance on the size congruity effect, using different search sets with LCD-like stimuli that were perceptually matched by mirroring and rotation (Experiment 3).
The observation of a size congruity effect during visual search provides a substantial advancement over previous number processing research by demonstrating that an interaction between numerical and physical size can also occur outside the experimental specifics of classical size congruity paradigms. To be more precise, classical size congruity paradigms are centred around an explicit comparison task; that is, participants have a binary choice of which of two presented digits is numerically larger (e.g. Besner & Coltheart, 1979; Henik & Tzelgov, 1982; Pansky & Algom, 1999) or whether a single presented digit is numerically larger than a pre-defined standard (e.g. Schwarz & Heinze, 1998; Schwarz & Ischebeck, 2003). Participants then indicate their decision by pressing one of two response buttons, each representing one of the two choice alternatives. Several authors have argued that the specifics of this experimental set-up might explain the observed apparent interaction between numerical and physical size (e.g. Risko, Maloney, & Fugelsang, 2013; Santens & Verguts, 2011). For instance, Risko et al. (2013) argued that in an explicit binary comparison task reaction times are subject to attentional capture effects which lead to a temporal congruity effect (Schwarz & Stein, 1998), rather than to a size congruity effect. Furthermore, Santens and Verguts (2011) pointed out that in a typical size congruity paradigm, congruity can be defined only relative to the comparison task at hand: if the right of two presented stimuli is physically larger and also numerically larger, both the task-relevant physical and the task-irrelevant numerical magnitude will activate a ‘right larger’ code, leading to a faster response than in a situation where only one magnitude is activating a ‘right larger’ code, while the other is activating a ‘left larger’ code. Given these task-specific explanations of the nature of the classical size congruity effect, a demonstration of a similar interaction between numerical and physical size in an experimental paradigm that does not entail an explicit comparison between two stimuli would indicate that size congruity is not limited to occur within the experimental constraints of a binary comparison task. The current study is an instance of such a demonstration, since the response latencies revealed a size congruity effect in a visual search task in which a target item had to be found in a display with many stimuli. Since the congruity effect in the current study was measured with the same responses—the release of the start button once the target had been detected—for both large and small targets, any explanation relying on response competition (cf. Santens & Verguts, 2011) cannot account for the present congruity effect.
Moreover, with the current visual search paradigm we exclude the presence of a temporal congruity effect (Schwarz & Stein, 1998) resulting from attentional capture (cf. Risko et al., 2013), since temporal congruity applies merely to situations in which two stimuli (one numerically small and one numerically large digit) are processed sequentially. Even if one supposes that items were processed strictly sequentially and that the target digit in the present paradigm was always processed first, the account of temporal congruity requires the additional assumption that participants stopped their visual search after they have processed the first distractor. First, this assumption of an early termination is in conflict with the set size effects found in all experiments. Second, even if the processing would sometimes be restricted to the target and one distractor, this distractor was in 43 to 47 % of the cases of the same numerical size as the target and could not induce any cognitive interference. We therefore consider this explanation for the observed size congruity effect as very unlikely.
The present findings are, however, in line with different recent general proposals which assume that attention and the coding of magnitude information are two mutually dependent processes (e.g. Risko et al., 2013; Fischer et al., 2003). While Fisher et al. (2003) argued that numerical size has an influence on spatial attention, Risko et al. (2013) further discuss a possible influence of attention on magnitude judgements. More specifically, they speculated that if different types of magnitude share a common code (Walsh, 2003), then ‘bias[ing] attention to one type of magnitude […] could produce a bias to attend to a similar dimension of other types of magnitude’ and ‘looking for larger objects might bias one to attend to large numbers’ (Risko et al., 2013, p. 1146). The present study now provides the first direct evidence for exactly this notion: directing attention to the physically larger item during a visual search seems to produce an unintentional bias to attend to the numerically large items as well. Importantly, this was not the case if the visual search was guided by stimulus features that are not size-related (e.g. colour), emphasising that the current finding represents an interaction between two sources of magnitude information. Moreover, magnitude interaction was enlarged when targets and distractors were numerically more distant, even if perceptual features were kept constant. Based on earlier findings which demonstrated that semantic distance between numbers affects a numerical size comparison between them (Moyer & Landauer, 1967; Dehaene et al., 1990), an enlarged congruity effect between numerical and physical size for the numerically more distant target and distractor items in the search array indicates that numerical size is being processed and affects the visual search. Together, these findings are reassuring us that the search time differences observed in the two experiments are not the consequence of an advantage of numerically larger target digits in visual search, but are indeed a result of an interaction between number meaning and physical size, that is, a size congruity effect in visual search. The current findings are therefore in line with the notion that numerical information is processed by a generalised magnitude system (Walsh, 2003) which originally emerged to serve perception and action.
While the inclusion of different search set sizes was initially motivated by gaining more insights over participants engagement in the experimental task (since larger search sets should lead to longer search timesFootnote 1; Sagi & Julesz, 1987), the observed interaction of the size congruity effect with the search set size in Experiment 2 is not in conflict with the notion of a generalised magnitude system. Since larger set sizes result in longer search times, the modulation of the size congruity effect is most parsimoniously explained by differences in processing times: longer processing times should lead to a deeper processing of the task-irrelevant numerical information and should therefore cause a stronger size congruity effect (see Schwarz & Ischebeck, 2003). Alternatively, one could argue that this finding is in line with studies showing that the amount of items in a display automatically activates numerical representations (Naparstek & Henik, 2010). Search set size could therefore be conceived as a third source of magnitude information in the visual search task, and it could be speculated that this third magnitude affects the processing of the two other magnitudes (physical size and numerical size). Any interaction between set size and numerical and physical magnitude information might hence be interpreted as an additional instance of within magnitude interference (cf. Walsh, 2003). However, the impact of search set size on the congruity effect as well as on physical size and numerical size did not manifest itself as a consistent pattern across the three experiments and we belief that future experimentation will be needed in order to better understand the underlying mechanisms.
The finding of a size congruity effect during visual search shows an impact of numerical size congruity on early visual attentional processes. As known from several studies on visual perception, top-down guidance of attention towards a certain stimulus feature (e.g. physical size) enhances the visual saliency of objects containing this feature (e.g. Wolfe, 1994; Proulx & Egeth, 2008; Kiss & Eimer, 2011). Recent evidence for the notion that numerical information guides visual search comes from Schwarz and Eiselt (2012), who demonstrated that the performance to find a target number among distractor digits is systematically influenced by the numerical distance between the target and the distractors. The current data extend this finding by showing that a visual search for a target that is solely defined by its physical size is also affected by task-irrelevant information about numerical size; that is, when attending to one particular type of physical target size (i.e. a physically large or a small target), the numerical size of the same target seems to guide attention as well, with faster search times if the numerical size matches the current target’s physical size. The role of non-visual stimulus features like semantic information in guiding spatial attention is still controversially discussed (Wolfe & Horowitz, 2004; Moores, Laiti, & Chelazzi, 2003; Belke, Humphreys, Watson, Meyer, & Telling, 2008). The present findings of size congruity in visual search might therefore additionally contribute to this ongoing debate in visual attention research by showing that semantic knowledge about number symbols under some conditions affects the performance in a visual feature search.
Importantly, in contrast to the vast majority of research on attentional effects of number processing (e.g. Fischer et al., 2003; Ranzini et al., 2009; Bonato et al., 2009), the present study examined the modulations of visual attention caused by non-spatial number features. It therefore provides empirical evidence for a number–perception interaction driven by a congruity between physical and numerical size of an item during visual search, irrespective of its spatial location. The additional analyses of spatial effects revealed that search performance in the present paradigm was not affected by spatial–numerical associations (see Hubbard et al., 2005 for a review), that is the congruency between the horizontal position of a target item and its numerical value. While earlier research showed that numerical size can bias attention to spatial positions (Fisher et al., 2003), the current finding suggests that participants do not use these spatial–numerical associations (e.g. Dehaene, 1992) to guide a visual search and that digits of different physical and numerical size are found equally well at all spatial positions.
To our surprise, a congruity effect between the physical and numerical size of the target was only observed when searching for a physically large target. An explanation for the lack of a size congruity effect in the small target condition is speculative at this point. One might assume that this finding reflects an impaired automatic processing of the target’s semantic meaning if the target digit is displayed in a small physical size. This weaker semantic activation of the number meaning could be due to the higher perceptual demands to process the detailed visual pattern of small symbols compared to large symbols. Alternatively, the impaired semantic processing of physically small targets might be driven by the fact that the feature search was performed significantly faster in this condition, compared to the colour and large target condition. Searching for the small target was thus perceptually easier and possibly more bottom-up driven by global visual stimulus features (e.g. total covered area or changes in luminance). In both cases, an impaired semantic processing would result in a delayed interaction between physical and numerical size. This notion receives empirical support by our data, since a size congruity effect could be found when looking only at the trials with the longest search times in Experiment 2. The replication of the same pattern of results with the doubled stimulus size (Experiment 2) further suggests that the impaired semantic processing of smaller stimuli is not the result of a too small absolute physical size. Instead, it seems that it was the relative size difference to the surrounding distractors which resulted in an impaired semantic processing of the relatively smaller target, possibly due to stimulus-driven attention to the many larger stimuli in the set (Proulx, 2010). This more general phenomenon might explain that the semantic information of targets was not processed to an extent that affected behaviour, if all distractor items were of larger size. Furthermore, in the design of the current experiments, the distance between the centre of each target and the centre of the distractors was fixed, leading to smaller distractor–target distances when the target was physically large, and larger distractor–target distances when the target was physically small. The resulting disproportionately in crowding could have also persistently undermined the congruity effect with the small targets (cf. Whitney & Levi, 2011). Taken together, it seems plausible to assume that semantic effects of number meaning predominately emerge in the present paradigm if the target is a larger symbol among smaller distractors. Future research with a specific focus on the effects of relative size differences between target and distractors on semantic processing in visual search is needed to better understand the details of the underlying cognitive mechanisms.
Eventually, the presented experiments give some new insights about the origin of behaviourally observed interactions between numerical and physical size (see, e.g. Schwarz & Heinze, 1998; Cohen Kadosh & Walsh, 2009; Santens & Verguts, 2011 for this debate). In general, two opposing accounts have been formulated: The first account holds that numerical and physical size interact at an early processing stage at which both stimulus features are coded into a common analogue magnitude representation (e.g. Schwarz & Heinze, 1998). Reaction time differences between congruent and incongruent configurations are thought to reflect a difference in the cognitive demand to create a common representation in the case that numerical and physical size are of different magnitude, compared to when they convey the same relative size (i.e. both small or both large). Empirical evidence for an interaction at an early representational stage comes from electrophysiological data, suggesting that the facilitations and interference in a size congruity task arise quickly with onsets well before 300 ms (Schwarz & Heinze, 1998; Szücs & Soltész, 2008).
The second account of the size congruity effect holds, however, that interference effects do not emerge at the level of magnitude representation. For instance, Cohen Kadosh and Walsh (2009) argue for a dual-code model of magnitude representation in which first, fast automatic representations are thought to be non-abstract and dependent on the notation or modality, while only later, slower intentional abstract representations might follow, depending on task and context. This notion of dual magnitude codes would not predict an interaction between physical and numerical size in early perception, because the automatic and unintentional processing of physical and numerical size is assumed to be initially based on independent representations. However, in contradiction to this prediction, the current study suggests the presence of an early interference effect as the presence of task-irrelevant numerical size automatically and unintentionally affected the detection of a target defined by its physical size. Furthermore, Santens and Verguts (2011) assume that, similar to Cohen Kadosh and Walsh (2009), different sources of magnitude information from different domains are represented entirely separately and interact only at later response-related stages of processing. The authors pointed out that the classical size congruity paradigm relies on a one-to-one mapping between the two choice alternatives (i.e. ‘left larger’, ‘right larger’) and the two motor responses (‘left’, ‘right’) and proposed a dual-route model, assuming a parallel processing of task-relevant and task-irrelevant stimulus dimensions of the digits that after a certain time results in a co-activation of both visual and numerical size information. In congruent trials, both size-related stimulus features activate the same response code, while in incongruent trials, the two stimulus dimensions map onto different response codes, resulting in a conflict at the level of response selection. This conflict is accompanied by longer response times. Recent evidence for such an explanation of the size congruity effect, which rejects the assumption of shared representations of numerical and physical size, comes from ERP and fMRI studies that suggest the presence of interference during response selection (e.g. Cohen Kadosh, Cohen Kadosh, Henik, & Linden, 2008; Szűcs & Soltész 2007).
In contrast to a classical size congruity paradigm, the visual search task used in the present study comprised several simultaneously presented digits and a single motor response (releasing the start button) to mark detection. While the finding of a size congruity effect in the longest search times only when searching for a physically small target (see ‘Experiment 2’) could in principle be interpreted as an indication for a relatively late effect, in our view, it is unlikely that the numerical and physical dimensions of each of the up to 18 digits in our experiments pre-activated up to 18 different motor responses, and hence more plausible that the observed cognitive conflict is not originating from response-related stages of processing. Notably, all experiments included an additional pointing response to the (masked) target position, and it can be argued that participants prepared the pointing response not as a second step, but as part of the initial response of releasing the start button. An interpretation of the results in terms of a conflict of pre-activated responses, however, assumes the rather unlikely parallel pre-activation of 8 (small search set) or even 18 (large search set) different responses. Furthermore, opposed to a classical size congruity paradigm, the task-irrelevant information of numerical size would not pre-activate one single response of these 8 or 18, but multiple ones (since half of the digits in the search display were of large numerical size and the other half was numerically small). The task-relevant information of physical size, on the other hand, would pre-activate exactly one response (since there was only one physically larger digit). This would lead to contradicting pre-activations in both congruent and incongruent situations. Given these substantial differences between the current visual search task and the classical size congruity paradigm, it is difficult to assume that the classical size congruity paradigm constitutes a visual search with only two items (a target and a distractor). Nevertheless, even if one interprets a classical size congruity paradigm this way, the current demonstration of a size congruity effect in a visual search with 8 and 18 items is a substantial extension of former findings, as it has been suggested that the processes underlying detection performance in small display sizes (e.g. 2 items close to each other as in the classical size congruity paradigm) differ from those underlying detection performance in larger displays (e.g. 18 items in a larger circular array as in the current study; see Meinecke & Donk, 2002).
Taken all together, the results of the present study seem to rather support an interaction between numerical and physical size on an early than on a late level. Eventually, more research will be needed to answer this question with sufficient certainty. For instance, an important open question the current study cannot answer is whether the interaction between numerical and physical size affects the initial allocation of attention or the stage of accepting or rejecting an item as the target in a serial process (see also Moores et al., 2003; Belke et al., 2008). The new visual search paradigm presented here, combined with eye-tracking techniques should stimulate future research in this direction.