Abstract
Affordances represent features of an object that trigger specific actions. Here we tested whether the presence and orientation of a handle on a cup could bias grasping movements towards it in conditions where subjects were explicitly told to ignore the handle. We quantified the grip aperture profile of twelve healthy participants instructed to grasp a cup from its body while it either had no handle, a handle pointing towards, or away from the grasping hand (3 ‘move’ conditions, with large grip aperture). To ensure the smaller grip aperture afforded by the handle was implicitly processed, we interspersed trials in which participants had to grasp the cup from its handle or a handle not attached to a cup with a small grip aperture. We found that grip aperture was smaller in the presence of a handle in the ‘move’ conditions, independently of its orientation. Our finding, of an effect of the handle during the execution of a grasp action, extends previous evidence of such an influence measured during motor preparation using simple reaction times. It suggests that the specific action elicited by an object’s attribute can affect movement performance in a sustained manner throughout movement execution.
Similar content being viewed by others
Introduction
The concept of ‘affordance’ was introduced by Gibson1 to describe how features of objects (such as their physical properties) may be linked to a functional goal and the physical properties of an actor. Several behavioural studies have demonstrated that affordances modulate human movement performance, independent of the intention to act2,3,4,5,6,7,8,9,10. For example, Tucker and Ellis2 instructed participants to make finger presses with their right or left hand in response to the orientation (up or down) of a handled object on a screen. They observed faster response times if the orientation of the handle on an object was compatible with the responding hand, reflecting the ‘automatic’ activation of an action to grasp the handle. Further studies have demonstrated that object features may ‘automatically’ trigger components of specific actions, such as reaching and grasping3,4,5.
Nevertheless, the exact nature of affordance ‘effects’ remains elusive. Some authors have argued they represent spatial compatibility effects reported in other contexts6,7,8,9.
A study by Tipper et al.10, described attentional effects using distractor-target combinations which demonstrated spatial- and arm-specific biases during reaching. Later studies investigating distractor-target perturbations extended this work by providing evidence of similar processes in prehension11,12,13,14. It is noteworthy that even though the behavioural consequences of affordances may be related to attentional mechanisms, these effects are behaviourally separable to visuospatial attention. Indeed, they are present even when the object is irrelevant to the task2 and when visuospatial attention is directed away from the object15,16,17.
Further work on the nature of affordance effects suggests that affordances can trigger motor representations18,19. Studies in non-human primates have identified areas in the dorsal stream, which are responsible for selecting the appropriate motor programs for object-directed actions20,21,22. A ‘dorso-medial’ network, comprising areas in the posteromedial portion of the intraparietal sulcus and dorsal premotor cortex, implicated in on-line reaching and action selection, interacts with a ‘dorso-lateral’ network comprising area F5 and anterior intraparietal regions (AIP), implicated in grasp selection21,22,23. Recent studies suggest that computations for reach and grasp actions might be integrated24. The degree of interaction between them depends on task demands, the degree of online control and visual feedback required by an action25,26. Human homologues of these areas have been demonstrated in functional neuroimaging studies27,28,29,30. Additionally, motor representations triggered by affordances have been elicited with techniques such as TMS31,32 and EEG33,34,35. A study by Buccino et al.31 demonstrated greater motor excitability in the affording effector specifically when participants were presented with images of handled objects in which the handle was intact (and could therefore be grasped by the tested hand) and not in images of these objects in which the handle was ‘broken’.
The studies mentioned above have described ‘affordance’ effects mostly during action planning. However, the use of behavioural measures, such as reaction times, to measure these effects, has been criticised for failing to represent the full nature of the action triggered by objects6. One way to remedy this would be to study affordance effects during action execution.
This was done in an original study derived from the neuropsychology literature36. A patient with alien-limb syndrome due to cortico-basal degeneration (CBD) was asked to pick a cup on the right or left side of the table using the hand that was on the same side of the table as the object, irrespective of the orientation of its handle. When the cup was on the left, and its handle oriented to the right, the patient tended not to grasp the object with her left hand (the required response) but rather grasped it with her right hand, i.e. the grasp action being cued by the orientation of the object in relation to the patient’s preferred hand. This grasp action preference was felt to demonstrate an affordance of the handle on the cup, and was specific to this action as it was not elicited when the patient was asked to point to the cup36,37.
Based on these early findings we sought to investigate whether affordances, whose effects have mainly been described on movement planning, could elicit movement representations during the execution phase of a reach-to-grasp action to an object, in healthy human volunteers. We used quantifiable kinematic measures of movement, namely the grip aperture (i.e. the distance between thumb and index finger) to investigate the effect of the presence and orientation of a handle on a cup, whilst participants were asked to grasp the object from its rim in order to ‘move’ it to another location.
We hypothesised that grip apertures would be smaller in the presence of a handle on the cup, particularly if this was oriented in the direction of the grasping hand, as in previous studies investigating the role of affordances on actions2,3,4,5,6. However, there was a concern that due to the need for many repetitions of these trials, any affordance effect of the handle on the cup would not be detected if participants were not engaged in grasps targeted to the handle of the cup as well. To this end, trials with a ‘move’ instruction were interspersed with ones where participants had to grasp the handle of the cup, in order to ‘drink’ from it, to ensure the small precision grip afforded by the handle was implicitly processed. An additional ‘drink’ condition was included, which involved grasping a handle on its own whose purpose was to investigate affordance effects of the body of the cup on the handle in the drink conditions17 (Fig. 1b, see also videos in supplementary material). This generated a total of five conditions: two for the ‘drink’ actions, and three for the ‘move’ actions: one with the handle directed towards the grasping hand, one with the handle away, and one with no handle.
Grasping a cup to ‘move’ or to ‘drink’ from it. (a) Shows a picture of the set-up: 1) PLATO glasses (closed in this case), 2) starting position of the hand (with thumb and index finger pinched and position sensors in place), 3) TrakSTAR transmitter device, 4) platform for placing cup in the ‘move’ conditions, 5) cup with the handle oriented towards the grasping hand. (b) Demonstrates task conditions: 12 Participants were presented with one of three objects: a cup with a handle (that was either pointing ‘towards’ or ‘away’ from the grasping hand), a cup with no handle and a ‘handle only’. They were either instructed to grasp a cup to ‘move’ it; or to pincer grip the handle to ‘drink’ from it. The former required them to grasp the cup from its top end and move it from a central position on a table to a box that was adjacent and to the back of the object. The latter instruction was only provided for the cup with the handle oriented toward the grasping hand (‘handle on cup’) or with the ‘handle only’ that had the same dimension as the handle of the cup. Instructions were provided verbally at the same time as the object was presented by opening of PLATO-occlusion glasses. Each of the 5 conditions was presented randomly and repeated over 20 trials.
Participants were instructed to hold their index and thumb fingers together (in a pincer grip) at the start of every movement before making the grasp action required for the task: namely a small precision grip to grasp the handle when instructed to ‘drink’, or a larger precision grip to grasp the cup from its rim when instructed to ‘move’ it. Each movement was self-paced and the start of the movement was indicated by the opening of PLATO-occlusion spectacles (see Methods).
In addition to investigating affordance effects at the time of maximum grip aperture prior to grasping the object, we examined the time course of these effects throughout the movement execution phase by quantifying the grip aperture profile in 10% steps from movement onset until maximum grip aperture38.
Results
Kinematic data from a total of 12 participants were analysed for grip apertures relating to the five conditions of our task, leading to a large precision grip (in the three ‘move’ conditions) and a small precision grip (in the two ‘drink’ conditions). The results of these analyses are outlined below and in Figs 2 and 3.
Grip Apertures during transport to grasp the object. (a) Shows the grip apertures averaged over participants for each of the 5 conditions from start until just after MGA (100%, line) is reached. Participants started with their hand pinched. Their grip apertures were wider in the ‘move’ compared to the ‘drink’ conditions. In the former conditions, we observed significant modulation in grip aperture relating to the presence of a handle, even though the dimensions of the cup that was moved were identical. The MGA was largest in the move conditions when the cup had no handle. It was significantly reduced when the cup had a handle oriented away and when this was oriented toward the grasping hand. The grip apertures for the two ‘drink’ conditions, namely with a ‘handle on cup’ or a ‘handle only’ showed no significant modulation. (b) Represents the grip aperture histograms for each of the 3 move conditions. The data has been grouped in 10% of MGA time points, demonstrating significant differences in grip aperture between conditions over time (handle toward vs no handle occurring at 60% until 100% of MGA, handle away vs no handle occurring at MGA, see results section). *p = < 0.05, **p < 0.001.
Maximum grip aperture differences between subjects across the 3 ‘move’ conditions. This figure shows the modulation in maximum grip aperture (MGA) in the three ‘move’ conditions, according to presence and orientation of a handle on the cup. The three bars represent the MGA in the condition with a handle pointing towards the grasping hand, away from it and no handle, respectively. Each line represents a single subject and bars represent averages across subjects with error bars indicating standard errors, demonstrating consistent effects across participants. *p = < 0.05, **p < 0.001.
Errors
The error rates were analysed using a one-way repeated-measures analysis of variance (ANOVA) with five conditions (three ‘move’ and two ‘drink’ conditions) entered for each participant to identify whether error rates were different between conditions. There were 2.9% of errors across all trials, which were excluded from the overall analysis. These included errors caused by the experimenter (e.g. positioning the wrong cup, 0.33%), or errors made by the participant. In the latter category, there were errors in failing to initiate the movement with a pincer grip – leaving an open gap between the thumb and the index finger (0.25%) – these trials were excluded due to difficulty estimating the maximum grip aperture. The remaining errors (1.81%) related to errors made by participants in the different trial types (e.g. performing a small precision grip instead of a large one when asked to ‘move’ the cup or vice versa when asked to ‘drink’). The ANOVA revealed no significant main effect [F(1,4) = 1.14; MSE = 0.39; η 2 = 0.35; p = 0.09], suggesting these errors were distributed in approximately equal numbers across all conditions.
Grip apertures
The maximum grip apertures (MGA) for each of the five conditions were entered into another one-way repeated-measures ANOVA with the factor ‘HANDLE’ (‘towards’, ‘away’, ‘no handle’ in the three ‘move’ conditions, ‘handle only’ and ‘handle on cup’ in the two ‘drink’ conditions). This ANOVA, with a Greenhouse-Geisser correction, showed significant differences in MGA between all the handle conditions [F(1.29, 14.15) = 379.13; MSE = 64.16; η 2 = 0.97; p < 0.0001], as illustrated in Figs 2 and 3.
We describe changes in grip aperture in the ‘move’ and ‘drink’ conditions separately, in the paragraphs below. In the post-hoc t-tests described below, we applied a Bonferroni correction for multiple comparisons across all 5 conditions (see Methods).
Grip apertures in the ‘move’ conditions
The MGAs for ‘move’ conditions were all significantly larger than those for the ‘drink’ conditions, and were also significantly different from each other (Figs 2 and 3).
The smallest MGAs in the ‘move’ group were the ones with a handle. Specifically, the ‘move’ condition with the handle pointing ‘towards’ the grasping hand (Mean = 100.04 mm, SEM = 2.01 mm), led to similar grip apertures to the one with the handle pointing ‘away’ (Mean = 101.18 mm, SEM = 2.06 mm; t(11) = −2.9, p = 0.14). Both grip apertures were significantly smaller compared to the grip aperture when there was ‘no handle’ on the cup (Mean = 102.77 mm, SEM = 2.25 mm; towards versus no handle: t(11) = −6.2, p = 0.00069; away versus no handle: t(11) = −4.4, p = 0.0099). These results were consistent across participants and are outlined in Fig. 3.
Effects at earlier time points
Figure 2a shows the grip aperture profile averaged across participants in each task condition from the start of movement to the time just after participants reached their MGA. Traces were normalised relative to the time subjects reached MGA (100%).
In order to identify whether the MGA modulation induced by the handle in the ‘move’ conditions was also present earlier during movement execution, we analysed the grip aperture profile at several time points leading to the MGA time. Intervals of 10% were taken from 10-100% of time to MGA and analysed in a 10 (%MGA step) × 3 (HANDLE) repeated measures ANOVA. The Greenhouse-Geisser corrected main effects of ‘%MGA step’ was significant [F(1.83,20.1) = 388.43; MSE = 319.9; η2 = 0.97; p < 0.0001]; as was the main effect of ‘HANDLE’ [F(2,22) = 10.50; MSE = 31.89; η2 = 0.49; p = 0.001], and the interaction between the two [F(3.04,33.5) = 3.56; MSE = 11.42; η2 = 0.24; p = 0.024]. The differences in MGAs between the three ‘move’ conditions were present at early stages. Post-hoc comparisons with Bonferroni correction revealed the first difference in grip aperture to emerge between the ‘handle toward’ and ‘no handle’ conditions when participants had reached 60% of their MGA before grasping the object. This effect persisted until participants reached MGA. The difference in grip aperture between the ‘handle away’ and the ‘no handle’ conditions appeared at a later stage when participants reached MGA. The details of these differences (including statistical significance) throughout the reaching phase are highlighted in Table 1 and in Fig. 2b.
Grip apertures in the ‘drink’ conditions
As mentioned in the introduction, ‘drink’ trials requiring a small precision grip were interspersed with the ‘move’ trials, requiring a small precision grip. The purposes of these trials were 1) to ensure the implicit manipulation afforded by the handle in the ‘move’ trials, as in previous studies investigating affordances2,3,4,5,6 and 2) to investigate whether the large precision grip prompted by the cup led to larger grip apertures when preparing a small precision grip towards the handle. Two conditions, involving a small precision grip directed to the object, were generated to enable this: one involving grasping the handle of the cup, which in this case was always oriented in the same direction as the grasping hand (‘handle on cup’), and one involving grasping a handle on its own, with no cup (‘handle only’). Both conditions were preceded by the instruction to ‘drink’, indicating that a small precision grip was to be used.
A paired samples t-test comparing the two ‘drink’ conditions revealed no significant differences in MGA between the ‘handle only’ (Mean = 54.48 mm, SEM = 2.17 mm) and ‘handle on cup’ conditions (Mean = 55.03 mm, SEM = 1.99 mm; t(11) = −0.47, p > 0.1). For this reason, we did not analyse changes in grip aperture at earlier time points.
Influence of the previous trial on grip aperture
We investigated whether there was an influence of the grip aperture performed in the previous trial on the affordance effects induced by the handle, described above. To this end, we performed a further two-way repeated-measures ANOVA with factors: ‘grip type in the previous trial’ (large (in the ‘move’ conditions) or small (in the ‘drink’ conditions) precision grip) and ‘HANDLE’ (5 conditions, as above).
This ANOVA again revealed a main effect of ‘HANDLE’ [F(1.27,13.9) = 373.32; MSE = 133.28; η2 = 0.97; p < 0.0001], which was mostly driven by differences between large and smaller grip apertures for drink and move conditions (see supplemental material 2). The main effect of ‘grip type in the previous trial’ [F(1,11) = 0.06; MSE = 5.83; η2 = 0.005; p = 0.81], and the interaction between the two [F(4,44) = 2.06; MSE = 2.68; η2 = 0.16; p = 0.15] were not significant. This suggests that the effects we observed were not related to a carry-over effect or priming influence of the previous trial on the next one, but were genuinely relating to the influence of the presence and orientation of the handle on the cup.
Other performance measures influenced by affordances
We investigated the influence of the handle and previous grip on the speed of movement preparation and movement execution, using reaction and movement times, respectively (shown in Table 2).
Reaction times (RTs), defined as the time between the opening of the glasses and movement onset, were entered in a further one-way ANOVA with five conditions. This revealed a significant main effect of ‘HANDLE’ on RTs [F(4,44) = 8.86; MSE = 0.001; η2 = 0.45; p = 0.001].
Post-hoc t-tests revealed this was driven by a difference in the presence of a handle. RTs in the move condition with ‘no handle’ were the shortest (see Table 1) compared to all the other conditions. Significant differences in RTs were identified between the conditions ‘handle on cup’ and ‘handle only’ (both ‘drink’ conditions) versus the ‘no handle’ condition (t(11) = 5.5, p = 0.002 and t(11) = −7.4, p = 0.00013, respectively).
Taken together, these results showed that RTs for large targets, in the absence of a handle, were shorter, whereas RTs in the drink conditions were longer. This reflects speed-accuracy trade-offs previously described when grasping an object with smaller width39.
A further ANOVA was performed for the movement times, defined as the time between movement onset and movement offset (MTs, see Methods section), which were entered in a similar one-way ANOVA with 5 conditions for ‘HANDLE’. This revealed no significant main effect (F(1.45,15.9) = 2.4; MSE = 0.004; η2 = 0.18; p = 0.13), suggesting no difference in MTs between conditions.
Discussion
In this study, the kinematic measures of grasp (namely grip aperture) were modulated by the presence of a handle on a cup. The maximal grip aperture (MGA) was smaller when grasping a cup with a handle, compared to grasping a cup with no handle, despite the fact both objects were of the same size at their grasping points (specifically, they had the same height, weight and diameter, see Methods). This effect was observed in the ‘handle toward’, ‘move’ condition, where it was present early on and prior to reaching maximum grip aperture (from the time the grip aperture was at 60% of MGA), when reaching to grasp the cup. To our surprise, there was also an effect of the handle in the ‘handle away’ compared to the ‘no handle’, ‘move’ conditions, even though the handle in this condition was oriented away from the grasping hand. This effect was observed at the time of maximal grip aperture.
Participants had been instructed that in the ‘move’ trials they had to grasp the cup from its rim, between their thumb and index finger, to move it to a location behind. There was no requirement to grasp the cup from its handle on those conditions. There was no significant difference in errors according to trial type, suggesting that participants understood and followed this instruction accurately.
The presence of a handle in the ‘drink’ conditions compared to ‘move’ condition with ‘no handle’ revealed longer reaction times. Previous studies, have reported longer reaction times when grasping an object with smaller width39. The effects identified in MGA and RTs suggest that the handle on the cup afforded the action to grasp the cup from the handle, in addition to the action instructed by the task, which required a larger precision grip.
Further analyses investigating the effects of the previous trial on the grip aperture in the next one demonstrated that the affordance effects we identified were not a consequence of carry-over effects.
Taken together, these findings support our hypothesis that execution of the grasp in the ‘move’ conditions likely incorporated the representation of a smaller grip targeted to its handle, when this was present on the cup. Interestingly, by using kinematic measures (namely grip apertures) we were able to obtain an insight into how these processes evolve dynamically over time, in a way that has not been previously described in the literature. We discuss our findings in light of previous literature on the effects of affordances on action, with a particular focus to paradigms eliciting affordances implicitly, comparing these to stimulus-distractor paradigms in which they may be elicited more explicitly in order to make a choice between two actions. We end our discussion with some literature on the cognitive control of affordances, which may modulate their effects.
Implicit representations of grasp actions triggered by affordances
There are several accounts in the literature on the influence of affordances on action representations (see Introduction)1,2,3,4,5,6,12,13,17. Studies have argued affordances influence movement preparation in reaching tasks, possibly by eliciting motor representations of target locations10,11,40. Our results extend these findings and suggest that presence of salient, graspable features on a real everyday object also influence grasping, and that they can be identified during movement execution – quantified here by the grasp aperture profile taken at MGA and earlier time points.
They concur both with recent results demonstrating affordance effects in reaching tasks40,41, and with the original study by Riddoch et al., which had demonstrated an effect of object handle on the choice of grasp in a patient36. Similar results have also been obtained with studies investigating kinematic changes in grip aperture in the context of distractor-target paradigms11,12,13,14,42. The main difference between our study and these latter paradigms, was that our affordance effects are unlikely to be attributable to an explicit process of competition between actions19. As in original studies of affordance2,3,4,5,6 the effects of the handle of the cup on grasp representations was observed in the ‘move’ conditions, despite the fact participants were not explicitly required to grasp the handle itself. Even though our task did not explicitly invoke a choice between actions, both small and large precision grips were randomly elicited in response to the instruction and object to use, in the same way as both left and right button presses were required for performance of Tucker and Ellis’ original task demonstrating effects of affordances2. Hence our findings suggest that movement representations to grip the handle might have been elicited implicitly and in the absence of a requirement for their use2,6,17. Conversely, stimulus-distractor studies such as the ones described by Castiello et al.11,12,13,14 demonstrated interference of grip apertures to target objects when distractor objects of similar size versus different sizes where positioned in near, as opposed to distant locations, suggesting these representations occurred in the context of choice or (‘forced choice’11) action selection.
These effects might represent a mechanism by which participants could reprogram their grasp actions in response to changes in the environment. Such processes have recently been shown to also occur implicitly43. This mechanism would allow them to adjust their grasp in response to contextual factors and to adapt their choice of grasp to objects of different sizes, shape and orientation.
Some unexpected findings
It is noteworthy that we observed an affordance effect even when the handle was oriented in the direction opposite to that of the grasping hand. This supports our conjecture that the handle may be triggering a motor representation implicitly to form a small precision grip in this condition, even if this would have led to an uncomfortable grasp posture. Affordance effects have recently been shown to interact with factors affecting biomechanical constraints during grasp actions, measured, for example, with the end state comfort effect44,45. These studies suggest that although both affordances and end comfort aim to reduce the manifold of possible actions that can be elicited by providing perceptual or biomechanical constraints, respectively, they are indeed separable and interact with one another such that affordances may, in some instances, override biomechanical constraints to facilitate movement selection under some task conditions44.
In our study, we interspersed trials in which participants performed a larger precision grip to the rim of the cup in order to ‘move’ it with trials in which participants performed a precision grip either targeted to the handle of the cup or to a handle on its own. However, unlike the modulation in grip aperture we observed in response to the presence of a handle on the cup in the ‘move’ conditions, there was no modulation of the cup on the handle in the ‘drink’ condition.
One possibility for this finding may be that the grip apertures for the precision grips to the handle in the ‘drink’ conditions were too small to identify subtle differences between conditions using the kinematic measures we opted for, despite the fact it might have been present. Another possibility is that the precision grip performed on the handle in order to ‘drink’ from the cup may be more salient than the action to move it using a larger precision grip to its rim. This would entail that in addition to the affordance effect induced by object features, there might have been a semantic effect provided by the knowledge of the object and its function that reinforces its affordance42,46,47,48,49,50,51. If this is the case, our findings might suggest that participants would consider the affordance of grasping the cup from its body to a lesser extent than grasping it by its handle. Further investigations, using these paradigms in a functional neuroimaging environment, might help identify the underlying mechanisms for these.
The cognitive control of actions elicited by affordances
Several accounts have, as in our results, identified motoric representations driven by affordances, which were independent of task instruction2,3,4,5,6,7,8,9,10. Although most of them argue for their automatic elicitation, there is evidence to suggest that affordance effects can also be contextually driven6,17,20,52.
Original studies suggesting that actions could be triggered ‘automatically’ derive from the neuropsychology literature, where in some conditions affecting inhibitory control, perception may, indeed, automatically influence action performance. An example of this is in patients with frontal lobe lesions, who display ‘utilization behaviour’53. Similarly, patients with ‘Alien-limb syndrome’ demonstrate an inability to suppress actions elicited by objects or the environment36,54,55,56. However, more recent accounts of these behaviours have challenged this assumption in patient populations, suggesting that the perceptual effects observed on these patients’ actions may depend on the observer’s intentional set and task requirements6,17,54. For example, in the study by Riddoch et al.36, the finding of object-directed actions in a CBD patient was specific to actions that involved grasping of the cup, as opposed to actions involving pointing to it. In addition, they were attenuated when the cup was presented upside down. These results agree with more recent studies in which TMS was used to elicit motoric representations triggered by affordance under conditions of high versus low working memory load. These studies have demonstrated that affordance effects are likely under a form of cognitive control617,57, possibly involving dopaminergic systems in the brain58.
In conclusion, this study is the first to suggest an influence of ‘affordance’ during grasp actions, using kinematic measures. The affordance effects observed here were sustained throughout the reaching phase when grasping an object, and despite there not being a requirement from the task to use an alternative grasp. This extends findings from previous studies, which demonstrated affordance effects using reaction time differences during movement planning2,3,4,5,6,7,8,9,10, by providing evidence that this effect is also present during movement execution. Further studies will be required to investigate the nature of these effects and whether different responses to affordances in grasp may be sub-served by different brain networks involving dorsal and ventral stream pathways21,22,23,24,25,26,42,48,49,50,51.
Methods
Participants
Twelve right-handed participants aged 23-35 years old (mean age 27.5; 6 Males, 6 Females) with normal or corrected-to-normal vision participated in this experiment after providing informed consent, based on previous studies which used a similar number of participants19,26. Eight participants were tested at KU Leuven and 4 additional participants underwent the same procedure at the University of Oxford. The experimental procedures were approved by the ethics committee at KU Leuven and the University of Oxford (Central University Research Ethics Committee), respectively, and were carried out in accordance with the approved guidelines.
Stimuli
The experimental set-up is outlined in Fig. 1. Participants sat in front of a table on which an object was positioned at a distance of 40 cm in front of them (distance from the edge of the table to object in the midline). The object was placed in a position located centrally, over an area measuring 16 cm2 on the table in front of the participant. Behind this was a rectangular cardboard box measuring 10 cm × 15 cm and 3 cm in height, upon which the participants had to place the object when they heard the instruction to ‘move’ it.
One of three objects were presented to participants (see Fig. 1b): a cup with no handle measuring 6.5 cm in height, 4.5 cm in width at the opening and 80 g in weight; a cup with handle of identical dimensions (the handle diameter was 2.5 cm), height of 6.5 cm and width of 4.5 cm at the opening, also weighing 80 g; and a handle made up of a stick and tape measuring 6.5 cm in height and 2.5 cm in diameter (i.e., identical dimensions to the handle of the cup) and weighing 10 g. The cup with a handle was either presented with the handle pointing to the right or the left side. The former (right side) was in the same direction as the grasping hand.
One of two verbal instructions was provided (Fig. 1b and video – supplementary material): either to ‘move’ the object or to ‘drink’ from the object. In the ‘move’ condition, participants had to perform a large precision grip to pick the cup from its rim, between their index finger and their thumb, in order to position it on the box located immediately behind the object. In the ‘drink’ condition they had to perform a small precision grip to pick the handle (only) or the handle of the cup and bring it to their mouth, as if to drink from the object.
Apparatus
Participants wore liquid-crystal PLATO-occlusion glasses (Milgram, 1987) that allowed the timely start and suppression of vision during the experiment. An electromagnetic tracking system was used to obtain kinematic measures of grasp actions in this task (trakSTAR [NDI Europe, GMbH], sampling rate 240 Hz). Two sensors were used to measure grasp aperture and hand direction. These were positioned 1) on the thumbnail, and 2) the nail of the index finger (Fig. 1a). Informed consent has been obtained from the healthy participant who took part in this experiment, and is depicted in Fig. 1a, to be photographed and filmed during performance of the task and for this to be made available in an online open-access publication.
Procedure
Participants rested their right hand on the side (against a foam located 20 cm from the midline) with their fingers positioned in a pincer grip (closed). A trial started with the instruction provided by the experimenter. This occurred at the same time as the experimenter pressing a computer key, which triggered the opening of the liquid crystal occlusion glasses, allowing visualisation of the object.
They reached to grasp one of three objects: a cup with a handle, a cup with no handle or a handle only. There were two possible actions they were instructed to make: either to ‘move’ or to ‘drink’. The conditions to ‘move’ were only presented with the cup with a handle pointing to the right (handle towards condition) or the left (handle away) or with the cup with no handle (no handle). In these conditions, participants had to move the cup from a central position to a box located behind, by grasping the object between their thumb and index finger from the top. In the ‘drink’ conditions, which were only presented for the cup with a handle pointing to the right (i.e. the same direction and the grasping hand; handle on cup condition) or for the handle on its own (handle only condition), participants had to grasp the cup from the handle (or the handle by itself) with a pincer grip, in order to bring it to their mouth.
The glasses remained open for 5 seconds. Participants were allowed to perform the instructed action at their own pace. The glasses closed allowing the next trial instruction to be given by the experimenter. Kinematic measures of their grasp were obtained from the start of the movement to the time they grasped the object.
The objects and instructions were presented in random order; the equipment was controlled by a computer program using code written in MATLAB (MathWorks). A total of 100 trials (20 repetitions of each condition) were obtained for each participant.
Analysis
Kinematics were analysed off-line using custom-written scripts in MATLAB (MathWorks). Maximum grip aperture (MGA) was measured as this occurs before any interaction with the object takes place and is therefore independent of the exact object dimensions, thus providing a good indication of the influence of perception on action17. The MGA was calculated as the maximum distance between the thumb and index fingers prior to the object being grasped.
Movement onset was defined as the point one of the fingers reached a velocity 120 mm/s that continued to increase to at least 300 mm/s to account for initial noise from small movements of the hand before initiating the grasp. Movement offset was the point after the reach, where both fingers’ velocities dropped below 120 mm/s again, which occurred at the time the object was being grasped. Before calculating the velocity, the position data was filtered with a bidirectional low-pass Butterworth filter (second-order, 15 Hz cut-off frequency).
For comparison of the reach traces across trials, movements were aligned in time at the start of the movement and the time point when participants’ MGA was reached. The traces were resampled in 10% steps that reflected the percentage of time to MGA (where 0 was the movement onset and 100% was time of MGA – Fig. 2a and b).
For data analysis, repeated measures Analysis of Variance (ANOVAs) were used to analyse the following dependent variables: MGA, Reaction Times (RTs), Movement Times (MTs), and error rates. All analyses were initially performed using a one-way ANOVA with five levels (HANDLE, which included the 3 ‘move’ conditions and the 2 ‘drink’ conditions). The data were normally distributed, however in some of these ANOVAs (namely MGA), the assumption of sphericity (Mauchly’s test) was violated, so a Greenhouse-Geisser correction was applied to test for statistical significance. All ANOVAs presented here were performed using IBM SPSS Statistic 22 for Windows software (SPSS Inc., Chicago, IL). We used a significance level of 𝛼 = 0.05 both for the kinematic, reaction and movement time and error rate analyses. An additional 2 (previous grasp, large or small) × 5 (HANDLE) ANOVA was performed for MGA to investigate the possibility of carry-over effects from the previous trial.
Moreover, in the presence of a statistically significant effect, post-hoc, paired samples t-tests were performed comparing differences in mean MGA between respective conditions, with a Bonferroni correction.
To further investigate effects of interest, namely differences in MGA over time, we also carried out a comparison on the ‘move’ conditions at different times relative to MGA in a 10 (% MGA step) × 3 (HANDLE) repeated measures ANOVA. The % MGA steps were here the grip apertures at 10–100% of MGA time (Fig. 2b).
References
Gibson J. J. The ecological approach to visual perception (Boston ed., Houghton-Miffin (1979).
Tucker, M. & Ellis, R. On the relations between seen objects and components of potential actions. Journal of Experimental Psychology: Human Perception and Performance 24(3), 830–846 (1998).
Ellis, R. & Tucker, M. Micro-affordance: the potentiation of components of action by seen objects. British Journal of Psychology 91, 451–471 (2000).
Craighero, L., Fadiga, L., Rizzolatti, G. & Umilta, C. Action for perception: a motor-visual attentional effect. JEP Human Perception and Performance 25(6), 1673–92 (1999).
Symes, E. T. M., Ellis, R., Vainio, L. & Ottoboni, G. Grasp preparation improves change detection for congruent objects. JEP Human Perception and Performance 34(4), 854–871 (2008).
Bub, D. N. & Masson, M. E. J. Grasping beer mugs: On the dynamics of alignment effects induced by handled objects. JEP: Human Perception and Performance 36, 341–358 (2010).
Phillips, J. C. & Ward, R. S-R correspondence effects of irrelevant visual affordance: time course and specificity of response activation. Visual Cognition 9, 540–558 (2002).
Kornblum, S. & Lee, J.-W. Simulus-response compatibiliy with relevant and irrelevant stimulus dimensions that do and do not overlap with response. JEP: Human Perception and Performance 21, 855–875 (1995).
Riggio, L. et al. The role of attention in the occurrence of the affordance effect. Acta Psychol (Amst) 127, 449–458 (2008).
Tipper, S. P., Lortie, C. & Bayliss, G. C. Selective reaching: evidence for action-centered attention. JEP: Human Perception and Performance 18(4), 891–905 (1992).
Castiello, U., Bennett, K. M. & Stelmach, G. E. Reach to grasp: the natural response to perturbation of object size. Experimental Brain Research 94(1), 163–178 (1993).
Castiello, U. Grasping a fruit: selection for action. JEP: Human Perception and Performance 22(3), 582–603 (1996).
Castiello, U. Attentional coding for three-dimensional objects and two-dimensional shapes. Differential interference effects. Experimental Brain Research 123(3), 289–97 (1998).
Castiello, U., Bennett, K. & Chambers, H. Reach to grasp: the response to a simultaneous perturbation of object position and size. Experimental Brain Research 120, 31–40 (1998).
Corbetta, M. & Shulman, G. L. Control of goal-directed and stimulus-driven attention in the brain. Nature Reviews Neuroscience 3, 201–215 (2002).
Riggio, L. et al. The role of attention in the occurrence of the affordance effect. Acta Psychol (Amst) 127, 449–458 (2008).
Masson, M. E. J., Bub, D. N. & Breuer, A. T. Priming of reach and grasp actions by handled objects. Journal of Experimental Psychology: Human Perception and Performance 37(5), 1470–84 (2011).
Cisek, P. & Kalaska, J. F. Neural correlates of reaching decisions in dorsal premotor cortex: specifications of multiple direction choices and final selection of action. Neuron 45(5), 801–814 (2005).
Cisek, P. Cortical mechanisms for action selection: the affordance competition hypothesis. Phil Trans Royal Soc London B Biol Sci 362(1485), 1585–1589 (2007).
Young, G. Are different affordances subserved by different neural pathways? Brain Cogn 62(2), 134–42 (2006).
Fagg, A. H. & Arbib, M. A. Modeling parietal-premtor interactions in primate control of grasping. Neural Networks 11, 1277–1303 (1998).
Jeannerod, M., Arbib, M. A., Rizzolatti, G. & Sakata, H. Grasping objects: the cortical mechanisms of visuomotor transformation. Trends Neurosci 18, 314–320 (1995).
Rizzolatti, G. & Matelli, M. Two different streams form the dorsal visual system: anatomy and functions. Experimental Brain Research 153(2), 146–157 (2003).
Fattori, P. et al. The dorsomedial pathway is not just for reaching: grasping neurons in the medial parieto-occipital cortex of the macaque monkey. Journal of Neuroscience 30(1), 342–349 (2010).
Grol, M. J. et al. Parieto-frontal connectivity during visually guided grasping. Journal of Neuroscience. 27(44), 11877–87 (2007).
Verhagen, L., Dijkerman, H. C., Grol, M. J. & Toni, I. Perceptuo-motor interactions during prehension movements. Journal of Neuroscience 28(18), 4726–35 (2008).
Grafton, S. T., Fadiga, L., Arbib, M. A. & Rizzolatti, G. Premotor cortex activation during observation and naming of familiar tools. Neuroimage 6, 231–236 (1997).
Chao, L. L. & Martin, A. Representation of manipulable man-made objects in the dorsal stream. Neuroimage 12, 478–484 (2000).
Grezes, J., Tucker, M., Armony, J., Ellis, R. & Passingham, R. E. Objects automatically potentiate action: an fMRI study of implicit processing. Eur J Neurosci 17(12), 2735–40 (2003).
Binkofski, F. et al. A fronto-parietal circuit for object manipulation in man: evidence from an fMRI study. European Journal of Neuroscience 11(9), 3276–86 (1999).
Buccino, G., Sato, M., Cattaneo, L., Roda, F. & Riggio, L. Broken affordances, broken objects: a TMS study. Neuropsychologia 47(14), 3074–8 (2009).
Franca, M. et al. Corticospinal facilitation during observation of graspable objects: a transcranial magnetic stimulation study. PLoS One 7, 1–9 (2012).
Muthuhkumaraswamy, S. D., Johnson, B. W. & McNair, N. A. Mu rhythm modulation during observation of an object-directed grasp. Brain Res Cogn Brain Res 19, 195–201 (2004).
Proverbio, A. M., Adorni, R. & D’Aniello, G. E. 250ms to code for action affordance during observation of manipulable objects. Neuropsychologia 49, 2711–2717 (2011).
Proverbio, A. M. Tool perception suppresses 10–12 Hz mu rhythm of EEG over somatosensory area. Biol Psych 91, 1–7 (2012).
Riddoch, J. M., Edwards, M. G., Humphreys, G. W., West, R. & Heafield, T. Visual affordances direct action: neuropsychological evidence from manual interference. Cognitive Neuropsychology 15(6–8), 645–683 (1998).
Humphreys, G. W. & Riddoch, J. M. How to define an object: evidence from the effects of action on perception and attention. Mind and Language 22(5), 534–547 (2007).
Chinellato, E., Castiello, U. & Sartori, L. Motor interference in interactive contexts. Frontiers in Psychology 6, 791 (2015).
Bootsma, R. J., Marteniuk, R. G., MacKenzie, C. L. & Zaal, F. T. J. M. The speed-accuracy trade-off in manual prehension: effects of movement amplitude, object size and object width on kinematic characteristics. Experimental Brain Research 98, 535–541 (1994).
Stewart, B. M., Gallivan, J. P., Baugh, L. A. & Flanagan, J. R. Motor, not visual, encoding of potential reach targets. Current Biology 24, 19 (2014).
Gallivan J. P., Barton K. S., Chapman C. S., Wolpert, D. M. & Flanagan, J. R. Action plan co-optimization reveals the parallel encoding of competing reach movements. Nature communications, doi:10.1038/ncomms8428 (2015).
Gentilucci, M. & Gangitano, M. Influence of automatic word reading on motor control. Eur J Neurosci 10, 752–756 (1998).
Ocampo, B., Al-Janabi, S. & Finkbeiner, M. Direct evidence of cognitive control without perceptual awareness. Psychon Bull Rev 22, 1083–88 (2015).
Herbort, O. & Butz, M. V. Habitual and goal-directed factors in (everyday) object handling. Experimental Brain Research 213(4), 371–82 (2011).
Rounis, E., Zhang, Z., Pizzamiglio, G., Duta, M. & Humphreys, G. Factors influencing planning of a familiar grasp to an object: what it is to pick a cup. Experimental Brain Research 235(4), 1281–1296 (2017).
Chainay, H. & Humphreys, G. W. Neuropsychological evidence for a convergent route model for action. Cognitive Neuropsychology 19(1), 67–93 (2002).
Creem, S. H. & Proffitt, D. R. Grasping objects by their handles: a necessary interaction between cognition and action. Journal of Experimental Psychology: Human Perception and Performance 25, 1673–1692 (2001).
Goodale, M. A. & Milner, A. D. Separate visual pathways for perception and action. Trends in Neuroscience 15(no.1), 20–25 (2000).
Daprati, E. & Sirigu, A. How we interact with objects: learning from brain lesions. Trends Cogn Sci 10(6), 265–70 (2006).
van Polanen, V. & Davare, M. Interactions between dorsal and ventral streams for controlling skilled grasp. Neuropsychologia 79(PtB), 186–191 (2015).
Sakreida, K. et al. Affordance processing in segregated parieto-frontal dorsal stream sub-pathways. Neuroscience & Biobehavioral Reviews 69, 89–112.
Tipper, S. P., Paul, M. A. & Hayes, A. E. Vision-for-action: the effects of object property discrimination and action state on affordance compatibility effects. Psychonomic Bulletin & Review 13, 493–498 (2006).
Lhermitte, F. ‘Utilization behaviour’ and its relation to lesions of the frontal lobes. Brain 106, 237–55 (1983).
Shallice, T., Burgess, P. W., Schon, F. & Baxter, D. W. The origins of utilization behaviour. Brain 112, 1587–98 (1989).
McBride, J., Sumner, P., Jackson, S. R., Bajaj, N. & Husain, M. Exaggerated object affordance and absent automatic inhibition in alien limb syndrome. Cortex 49, 2040–2054 (2013).
Fisher, C. M. Alien hand phenomena: a review with the addition of six personal cases. The Canadian Journal of Neurological Sciences 27(3), 192–203 (2000).
Freeman, S. M., Itthipuripat, S. & Aron, A. High working memory load increases intracortical inhibition in primary motor cortex and diminishes the motor affordance effect. The Journal of Neuroscience 36(20), 5544–55 (2016).
Friston, K. J. et al. Dopamine, affordance and active inference. PLOS Computational Biology 8, 1–24 (2012).
Acknowledgements
E.R. was supported by grants from the British Medical Association (Helen Lawson) and the Academy of Medical Sciences for the equipment costs used to test participants, V.P. and M.D. are supported by grants from a BBSRC (BB/J014184/1) David Phillips fellowship (UK) and a Fonds Wetenschappelijk Onderzoek grant (FWO Odysseus, Belgium, G/0C51/13N).
Author information
Authors and Affiliations
Contributions
E.R., V.P. and M.D. designed the study, V.P. wrote the data collection and analysis scripts, E.R. and V.P. collected and analysed the data, E.R. wrote the main manuscript text. All authors reviewed and edited the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare that they have no competing interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Rounis, E., van Polanen, V. & Davare, M. A direct effect of perception on action when grasping a cup. Sci Rep 8, 171 (2018). https://doi.org/10.1038/s41598-017-18591-5
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-017-18591-5
- Springer Nature Limited