Recent research shows that our perceptions of space and time are closely linked (Cai & Connell, 2012; Casasanto & Boroditsky, 2008). Many studies further suggest that time is perceived as flowing in a certain direction. For instance, Miles, Betka, Pendry, and Macrae (2010) asked participants to classify times (e.g., 11 a.m.) as past or future by clicking in a top corner of the screen and found that the mouse movements of English speakers deviated to the left when the time was in the past and to the right when it was in the future. English speakers were also faster responding with their left hand to names of celebrities famous prior to their birth than to names of celebrities famous after their birth, and the reverse with the right hand (Weger & Pratt, 2008; see also Santiago, Lupiáñez, Pérez, & Funes, 2007; Ulrich & Maienborn, 2010).

However, it remains a question whether temporal directionality is imposed by the use of written or spoken linguistic materials in the task. In languages such as English, the act of reading or writing associates the past with the left (what has just been read/written) and the future with the right (what is yet to be read/written), and reading during a task may evoke this directional spatial frame, which is then borrowed for temporal representation. Support for this idea comes from the finding that temporarily exposing Dutch speakers to a right-to-left reading direction (by training them to read mirror-reversed phrases) reversed their mental timeline (Casasanto & Bottini, 2010). Some studies have tried to remove the requirement for participants to read by presenting spoken words referring to the past or future (Ouellet, Santiago, Israeli, & Gabay, 2010) or by asking participants to arrange pictures chronologically (e.g., from earliest to latest; Fuhrman & Boroditsky, 2010). However, since literate adults activate orthographies (i.e., the letters and spelling of words) even during spoken language processing (Chéreau, Gaskell, & Dumay, 2007; Perre & Ziegler, 2008), it means that spoken category labels (e.g., past, yesterday, earlier) are likely to evoke the same spatial frame as written words. Temporal tasks need to avoid both written (including onscreen feedback; e.g., Vallesi, Binns, & Shallice, 2008) and spoken linguistic materials in order to avoid evoking the spatial frame of reading/writing direction.

In addition to the directional representation of temporal references (e.g., past, future, earlier, later), another aspect of the mental timeline stipulates that temporal duration accumulates with increasing magnitude from left to right such that shorter durations are placed toward the left and longer durations toward the right. For example, Fabbri, Cancellieri, and Natale (2012) found that, when asked to determine whether a target duration was shorter or longer than a reference duration, Italian speakers responded more quickly with a left-hand key to short target durations but with a right-hand key to long target durations. Similarly, Frassinetti, Magnani, and Oliveri (2009; see also Vicario, Caltagirone, & Oliveri, 2007) showed that people underestimated duration when their attention was directed to leftward space but overestimated it when their attention was directed to rightward space. These findings therefore support a left-to-right directional mental timeline. However, these studies are still subject to the linguistic influences discussed above and, hence, to the possibility that the directional effects are due to evoking the spatial frame of reading/writing direction. For instance, Fabbri et al. presented letters or numbers onscreen during each trial, Vicario et al. explicitly instructed participants to decide whether a duration should be categorized as either short or long, and Frassinetti et al.'s use of color swatches may have automatically and obligatorily primed color labels (MacLeod, 1991).

Finally, direction aside, there is some suggestion that movement itself may not be necessary for the spatial representation of temporal information. Boroditsky and Ramscar (2002; see also Miles et al., 2010; Núñez, Motz, & Teuscher, 2006) argued that spatial movement is a necessary part of our representation of time; we perceive either ourselves as moving toward time or time as moving toward us. However, in the transverse axis, movement does not appear to be necessary. Cai and Connell (2012) used physical sticks to manipulate visual spatial distance and found that time estimates increased with stick length, while Casasanto and Boroditsky (2008) found similar effects with static lines. Thus, regardless of directionality, it remains unclear to what extent the representation of temporal information in space depends on movement.

The present study reports three experiments using a fully nonlinguistic paradigm that allows us to test the effect of both left-to-right and right-to-left movement on English speakers' temporal judgments. Each trial required participants to watch a video where an actor sang a musical note while making a long- or short-distance gesture, from left to right (as viewed onscreen, Experiment 1), making one from right to left (Experiment 2), or making no movement (Experiment 3). Since studies of gesture production show that speakers frequently use both transverse moving and static gestures to depict temporal information (Cooperrider & Núñez, 2009), gestural stimuli allowed us to embed spatial information in a naturalistic context to which people are sensitive, but in a less obtrusive manner than pairing temporal duration with (for example) lines or other geometric shapes (see also Connell, Cai, & Holler, 2012). Following Cai and Connell (2012), we expected long-distance gestures to lengthen subjective durations, relative to short-distance gestures. Critically, if time is directionally represented in space and English speakers have a preferential left-to-right mental timeline, this effect of spatial distance on temporal duration should be greater for left-to-right than for right-to-left moving gestures. On the other hand, if the spatial representation of time is nondirectional in the absence of linguistic materials, we would expect both directions of gesture to affect temporal duration to the same extent. Finally, by comparing moving and static gestures, we can infer whether movement causes or enhances spatial effects on time perception.

Experiment 1

Our aim in the first experiment was to replicate previous findings (Cai & Connell, 2012; Casasanto & Boroditsky, 2008) as a proof-of-concept demonstration of our gesture paradigm by showing that a long-distance gesture lengthens subjective duration, relative to a short-distance one. In this and the following experiments, we also administered a mental rotation test so that we could take into account participants’ spatial capacity in between-experiment comparisons.

Method

Participants

Sixteen native speakers of English from the University of Manchester community were paid £4 to take part. They were all right-handed, had no hearing impairment, and were not musically trained. Three were replaced for having unusually low mental rotation scores ( < 2).

Materials

The materials consisted of videos where a male or female professional singer sang a note of a particular duration (1.0, 1.2, 1.4, 1.6, 1.8, and 2.0 s) at the pitch of 133 Hz (C3, for the male) or 376 Hz (F#4, for the female), while making a gesture with the right (dominant) hand from left to right. The gesture covered either a short distance in the singer’s gesture space (i.e., moving the hand within the width of the body) or a long distance (i.e., moving the hand from beyond the left side of the body to beyond the right side). Both long- and short-distance gestures lasted for the same set of durations as the vocal notes. Because our right-handed singers naturally gestured from their left to their right, we flipped the videos horizontally so that the gestures appeared onscreen (i.e., from our participants' perspective) as left-to-right movement (see Fig. 1, left). To ensure auditory consistency between conditions, we overdubbed the relevant vocal note for each duration on both the long- and short-distance gestures of that duration, resulting in 24 items (six durations in each distance for each singer). We put all 24 items into a block and presented this block 3 times, resulting in 72 items in total. Items within each block were randomly presented. Participants were instructed not to attempt to count out the duration and to, instead, use their subjective estimate of time.

Fig. 1
figure 1

Examples of gestures in Experiments 1–3, where image A in each case shows the short-distance gesture and image B the long-distance gesture. For moving gestures (Experiments 1 and 2), the video stills capture the endpoint of the gestures, and the red arrow represents the direction of movement and distance covered

Full size image

Procedure

Participants were tested individually in a laboratory cubicle, sitting in front of a laptop and wearing headphones. They performed four practice trials before the main experiment. On each trial, participants first saw a central fixation cross for 500 ms, and then the singer's video appeared in an area of 14.3 × 10.8 cm onscreen. After the video, a screen appeared with the prompt “+,” and participants held down the space bar for the duration of the singing they had just heard. When participants pressed the space bar, the prompt “+” changed to “+++” and remained onscreen until participants released the space bar. There was a blank of 1 s between trials. After the experiment, participants were given Philips and Rawles’s (1979) mental rotation test. The whole experiment lasted about 15 min.

Results and discussion

The dependent variable was the reproduced duration in milliseconds. We first trimmed the data by replacing data points beyond 3 SDs from the mean per duration condition with the relevant cutoff RT (less than 1 % of data affected) and then conducted an ANOVA with note duration and gesture distance as within-subjects factors (Fig. 2). As predicted, gesture distance had a main effect on the temporal task, F(1, 15) = 35.07, p < .001, η 2 = .70, where reproduced durations were longer following long-distance than following short-distance gestures (1,523 vs. 1,355 ms). There was also a main effect of note duration, F(2.55, 38.23) = 118.79, p < .001, η 2 = .89 (throughout the paper, Greenhouse–Geisser correction was used when sphericity was violated), as longer note durations resulted in longer reproduced durations. There was no interaction between note duration and gesture distance, F < 1, η 2 = .06. In short, results validate our paradigm by replicating previous findings: Participants were sensitive to the spatial information in gesture when reproducing temporal duration, such that a note had a longer reproduced duration if it was accompanied by a long-distance gesture than when accompanied by a short-distance one.

Fig. 2
figure 2

Reproduced durations as a function of note duration and gesture distance in Experiment 1. The regression lines were fitted through the average reproduced durations for long- and short-distance gestures, respectively. Error bars show SEs

Full size image

Experiment 2

Experiment 1 established that left-to-right moving gestures affected perceived temporal durations. In Experiment 2, we tested whether and to what extent right-to-left moving gestures do the same thing. To our knowledge, only two published studies have previously examined the effect of right-to-left movement on time perception, but their effects are equivocal. In Casasanto and Boroditsky (2003, Experiment 4), leftward-growing lines led to a numerically smaller effect on reproduced time than did rightward-growing lines in earlier experiments. However, there was no reported comparison of effect sizes, nor were the spatial abilities of participants controlled from one experimental group to another, meaning that any apparent divergence may have emerged from individual differences in spatial processing. Vicario et al.'s (2007) optokinetic stimulation paradigm found that habituating participants to a leftward-scrolling visual display led them to underestimate durations in a comparison task to the same extent as a no-stimulation baseline, while rightward-scrolling led to overestimates. However, as was discussed earlier, asking people to categorize durations as shorter or longer might mean that Vicario et al.'s effects were dependent on linguistic labels evoking the left-to-right spatial frame of reading/writing direction. The present experiment thus represents the first attempt to test systematically, in the absence of linguistic label or spatial ability confounds, whether direction of spatial movement differentially affects time perception.

If the left-to-right transverse mental timeline in English speakers reflects a fundamental directionality of temporal representation (regardless of stimuli or task), English speakers should conceive of time as flowing from left to right in nonlinguistic paradigms, as well as in linguistic ones. As such, because right-to-left gestures move in an incongruent direction to the “default” mental timeline, they would have a reduced or negligible effect on time perception, as compared with the left-to-right gestures in Experiment 1. If, on the other hand, the left-to-right timeline is simply an artifact of temporal processing borrowing a spatial frame already activated by written or spoken linguistic materials, English speakers in our nonlinguistic paradigm should be able to represent time as flowing from right to left just as easily as flowing from left to right. We would therefore expect gesture movement to affect time representation, regardless of direction, such that the right-to-left moving gestures in this experiment affect time perception just as strongly as the left-to-right moving gestures in Experiment 1.

Method

The methodology was identical to that in Experiment 1, except that the videos were mirrored so that gestures moved from right to left onscreenFootnote 1 (see Fig. 1, middle). Sixteen new participants from the same pool were paid £4 for participation.

Results and discussion

The same trimming procedure as that in Experiment 1 was adopted and affected less than 1 % of the data. A main effect of gesture distance again emerged, F(1, 15) = 32.02, p < .001, η 2 = .68, with long-distance gestures inducing longer reproduced durations than did short-distance gestures (1,567 vs. 1,417 ms; see Fig. 3). As in Experiment 1, there was a main effect of note duration, F(5, 75) = 150.21, p < .001, η 2 = .91, but no interaction between note duration and gesture, F < 1, η 2 = .05. Even though English speakers read from left to right, right-to-left movement still influenced time perception so that longer spatial distance led to longer perceived temporal duration.

Fig. 3
figure 3

Reproduced durations as a function of note duration and gesture distance in Experiment 2. The regression lines were fitted through the average reproduced durations for long- and short-distance gesture, respectively. Error bars show SEs

Full size image

In order to test whether direction of movement induced different effects, we compared Experiments 1 and 2. Importantly, participants did not differ across experiments in their spatial abilities, with equivalent mental rotation scores (10.3 vs. 11.2), t(30) = 0.66, p = .51. We therefore conducted an ANOVA using note duration and gesture distance as within-subjects factors and experiment as a between-subjects factor. Overall, long-distance gestures led to longer temporal reproduction (1,545 ms) than did short-distance gestures (1,386 ms), F(1, 30) = 67.07, p < .001, η 2 = .69. Importantly, the effect of gesture distance was not mediated by experiment, F < 1, η 2 = .008, which suggests that Experiment 1 (with left-to-right moving gestures) and Experiment 2 (with right-to-left moving gestures) had similar distance effects (i.e., the RT difference between long- and short-distance gestures was 168 and 149 ms, with effect sizes of η 2 = .70 and η 2 = .68, in Experiments 1 and 2, respectively). There was also a main effect of note duration, F(3.34, 100.15) = 265.76, p < .001, η 2 = .90, which did not differ across the two experiments, as indicated by the nonsignificant interaction between note duration and experiment, F < 1, η 2 = .005. There was no main effect of the experiment factor, F < 1, η 2 = .01, no two-way interaction between note duration and gesture distance, F(5, 150) = 1.31, p = .26, η 2 = .04, and no three-way interaction, F < 1, η 2 = .02. The results are therefore consistent with the idea that people's transverse mental timeline is nondirectional: Time can just as easily flow from left to right (Experiment 1) or from right to left (Experiment 2) when neither the task nor the stimuli involve presentation of written or spoken linguistic materials. We will return to this issue in the General Discussion section.

Experiment 3

The aim of Experiment 3 was to test whether movement itself is necessary for spatial information to influence time perception by using static gestures where the palms face each other a certain width apart. Static gestures can thus convey the same long and short distances as the sweeping gestures of our earlier experiments, but without the need for any movement. We know from previous studies that static spatial distance can affect time estimation (Cai & Connell, 2012; Casasanto & Boroditsky, 2008, Experiment 6), but it remains unclear whether it affects time as strongly as spatial movement. Casasanto and Boroditsky (2008) found that static lines led to a numerically smaller effect on reproduced time than did moving stimuli, but their analysis of effect sizes showed no reliable difference. However, since their analysis (across six experiments, five of which used moving stimuli and one static lines) may have lacked the power to determine an underlying moving versus static difference, and since there was no control of spatial abilities between experimental groups, it therefore remains an open question whether movement contributes to the influence of space on time. If movement is completely irrelevant to the spatial representation of time, static and moving gestures would be equally effective in biasing subjective durations. Alternatively, it is also possible that movement is not necessary to the spatial representation of time but may, instead, contribute to it by drawing spatial attention to the demarcated distance; if so, static gestures should influence temporal perception, but to a lesser degree than moving gestures.

Method

The methodology was identical to that in Experiment 1, except that the videos showed static rather than moving gestures. The singers held out their hands with palms facing each other, demarcating either a short or a long distance (i.e., the palms were held within or beyond the width of their body) (Fig. 1, right). The short and long static distances were equivalent to the moving distances in Experiments 1 and 2. Another 16 participants were paid £4 to take part.

Results and discussion

The same trimming procedure as in the previous experiments affected less than 1 % of the data. As before, there was a main effect of gesture distance, F(1, 15) = 16.14, p = .001, η 2 = .52, with long-distance gestures inducing longer reproduced durations than did short-distance gestures (1,560 vs. 1,490 ms; see Fig. 4). There was a main effect of note duration, F(2.07, 31.03) = 175.72, p < .001, η 2 = .92, but no interaction between the two factors, F(2.90, 43.49) = 1.024, p = .39, η 2 = .06. These results thus suggest that static gestures biased participants’ perception of temporal durations: Even in the absence of movement, long-distance gestures lengthened subjective duration, relative to short-distance ones.

Fig. 4
figure 4

Reproduced durations as a function of note duration and gesture distance in Experiment 3. The regression lines were fitted through the average reproduced durations for long- and short-distance gestures, respectively. Error bars show SEs

Full size image

But do static gestures influence time perception to the same extent as moving gestures? To answer this question, we compared this experiment's static gestures in turn with left-to-right moving gestures from Experiment 1 and right-to-left moving gestures from Experiment 2. Importantly, analysis of the mental rotation scores showed that participants in Experiment 3 (M = 11.0) had equivalent spatial abilities to those in both Experiment 1 (M = 10.3), t(30) = 0.55, p = .59, and Experiment 2 (M = 11.2), t(30) = 0.12, p = .91. For comparison with Experiment 1, we analyzed gesture distance and note duration as within-subjects factors and experiment as a between-subjects factor. Gesture distance produced a significant effect, F(1, 30) = 51.08, p < .001, η 2 = .63, which, importantly, was mediated by experiment, F(1, 30) = 8.80, p = .006, η 2 = .23, meaning that the gesture distance effect (i.e., longer reproduced durations following long- than following short-distance gestures) was greater with left-to-right moving gestures in Experiment 1 (a difference of 168 ms, η 2 = .70) than with static gestures in Experiment 3 (a difference of 70 ms, η 2 = .52). The experiment factor itself was nonsignificant, F(1, 30) = 1.26, p = .27, η 2 = .04. Note duration had a main effect, F(2.53, 75.54) = 290.47, p < .001, η 2 = .91, but did not interact with experiment, F(2.53, 75.54) = 2.25, p = .099, η 2 = .07, or with gesture distance, F < 1, η 2 = .03, nor did a three-way interaction emerge, F(5, 150) = 1.18, p = .319, η 2 = .04. A similar pattern was obtained in comparing the present experiment with Experiment 2. Gesture distance produced a significant effect, F(1, 30) = 48.09, p < .001, η 2 = .62, and critically interacted with experiment, F(1, 30) = 6.36, p = .017, η 2 = .18, meaning that the gesture distance effect was greater with right-to-left moving gestures in Experiment 2 (a difference of 149 ms, η 2 = .68) than with static gestures in Experiment 3 (a difference of 70 ms, η 2 = .52). The experiment factor had no effect, F < 1, η 2 = .005. Note duration produced a significant effect, F(2.73, 81.89) = 325.69, p < .001, η 2 = .92, that did not interact with experiment, F(2.73, 81.89) = 1.95, p = .134, η 2 = .06, or note duration, F < 1, η 2 = .03, nor was there a three-way interaction, Fs < 1, η 2s = .03. Altogether, the findings in Experiment 3 and between-experiment comparisons indicate that, although spatial movement is not necessary to the spatial representation of temporal information, it enhances the effect of space on time perception.

General discussion

Results from three experiments show that an auditory note was perceived as longer in duration if it was accompanied by a long-distance concurrent gesture than by a short-distance one. Moreover, left-to-right and right-to-left moving gestures elicited similar distance effects on time perception in English-speaking participants, and moving gestures elicited a greater distance effect than did static gestures. These results suggest that while time is spatially represented (i.e., the effect of gesture distance), the spatial representation of time is nondirectional in the transverse axis (i.e., comparable effects of left-to-right and right-to-left gestures). By asking participants to view a video of singing/gesture and to hold down a key to reproduce the singing duration, the present study involved no linguistic stimuli, labels, or feedback that might evoke the reading/writing direction. Thus, it suggests that temporal magnitudes are not by default represented directionally (e.g., left to right for English speakers) along a mental timeline and further implies that directional effects in the mental timeline (i.e., for both temporal magnitude and reference paradigms) are restricted to experimental contexts that evoke reading/writing direction through the use of spoken or written linguistic materials in the task.

Even when language use activates a directional mental timeline, this representation is not fixed. Although participants in Casasanto and Bottini (2010) were able to reverse their timeline to a right-to-left direction after exposure to mirror-reversed Dutch orthography, such a reversal appeared only in the second half of mirror-writing trials and was weaker than the original left-to-right timeline evoked by normal writing, suggesting that once a spatial frame has been evoked by reading/writing in a particular direction, it appears to take some effort to undo it. If people's mental timelines were directional by default (i.e., the spatial frame of reading/writing direction extends to nonlinguistic tasks), we would have seen weaker or even nonexistent effects of spatial distance on time in Experiment 2, where the right-to-left gestural movement conflicted with the left-to-right reading/writing direction of English speakers. This did not occur. Rather, we found equally strong effects of spatial distance on time in Experiments 1 and 2, which indicates that, in our nonlinguistic paradigm, participants had no “default” direction of mental timeline to undo.

A nondirectional representation of time is also consistent with the proposition that time, together with other quantifiable dimensions such as space and number, is represented as magnitude, which itself is nondirectional (ATOM; Bueti & Walsh, 2009; Walsh, 2003). Some researchers have extended ATOM into a directional transverse timeline that places small numbers, magnitudes, and durations on the left and large numbers, magnitudes, and durations on the right, in order to explain findings that directing attention to leftward space led to underestimates of duration, while directing attention to rightward space led to overestimates (Frassinetti et al., 2009; Vicario et al., 2007). However, our data do not fit this directional extension of ATOM. If it were the case, right-to-left movement in Experiment 2, which would act to draw attention to leftward space, would have led participants to underestimate duration to a greater extent than in Experiment 1, where left-to-right movement drew attention to rightward space. However, this did not occur, indicating that any magnitude representations involved in representing time were inherently nondirectional.

Time can also be spatial without movement. In our final experiment, we found that moving stimuli (sweeping gestures) led to longer perceived durations than did stationary stimuli (static gestures). One possible explanation for our pattern of results is one of attention, whereby movement enhanced the visuospatial acuity of the distance demarcated between the start and end points of a sweeping gesture, relative to the perception of constant distance between the palms in a static gesture (see Tse, Intriligator, Rivest, & Cavanagh, 2004, for findings that attention to a concurrent stimulus lengthens subjective time). The spatial difference between long- and short-distance moving gestures was therefore subjectively larger in Experiments 1 and 2 and had a correspondingly larger effect on time representation than did the difference between static gestures over the same distances in Experiment 3. We have shown elsewhere that the ability of distance to affect estimated duration depends on the spatial acuity of the perceived distance (i.e., high spatial acuity in visual perception allows space to affect time, whereas low spatial acuity in haptic perception reverses the effect; Cai & Connell, 2012), and the present findings are consistent with this idea. Alternatively, it could be argued that the spatial effects from our moving gestures reflected a speed effect; that is, faster gestural movements resulted in longer time estimations than did slower gestural movements. However, this possibility is not compatible with the findings of Experiment 3, where the distance between the hands in static gestures also produced a spatial effect on time. Hence, we conclude that the spatial effects we observed with moving gestures are due to distance rather than speed.

In sum, the present study demonstrates that the spatial representation of time is inherently nondirectional in the transverse axis, and not even dependent on movement, when examined in a paradigm that avoids evoking the reading/writing direction of participants. Future research should use duration and other psychophysical paradigms in order to examine the fundamental representation of time because, while studies of temporal reference can tell us how language affects the time/space relationship, they cannot tell us how time itself is represented without introducing linguistic confounds.