Judgment of the location of a previously viewed moving or stationary target is often displaced from the actual (final) location of that target, and one component of displacement that is frequently observed is a shift in the direction of implied gravitational attraction. In other words, judgments of target locations are often biased toward the direction of implied gravitational attraction. This has been referred to as representational gravity (Hubbard, 1995b, 1997). The effects of physical gravity are ubiquitous in the experience of organisms that live on Earth, and the existence of representational gravity is consistent with speculation that neural systems might have evolved to exploit such invariant physical properties. Such exploitation might involve creation of internal models that mimic the effects of physical variables or properties on physical objects in order to provide estimates or predictions regarding the future behavior of those objects (cf. Angelaki, Shaikh, Green, & Dickman, 2004; Grush, 2005; Tin & Poon, 2005); indeed, there has been considerable speculation regarding the existence of an internal model of gravity (e.g., Lacquaniti et al., 2015; McIntyre, Zago, Berthoz, & Lacquaniti, 2001; Zago & Lacquaniti, 2005). Although an internal model of gravity might be related to perceptual and motor biases other than representational gravity (e.g., see Jörges & López-Moliner, 2017; Zago, 2018), representational gravity would nonetheless be a key element or component of any internal model of gravity or internal model of motion or localization more generally.

The focus here will be on empirical findings and theoretical suggestions relevant to representational gravity and on how representational gravity is related to other perceptual and cognitive variables, processes, and tasks. Part I examines variables that could potentially influence the presence or magnitude of representational gravity, and these include the size/mass of a target, velocity of a moving target, distance traveled by a moving target, orientation of a target, modality of a target, retention interval, response measure, height of a target in the picture plane, nontarget stimuli or cross-modal information, oculomotor behavior, body orientation of the observer, and psychopathology. Findings from studies that explicitly identify representational gravity, as well as from studies that do not explicitly identify representational gravity, but that provide evidence relevant to representational gravity, are considered. Additionally, suggestions for variables not yet examined, but that could be predicted to influence representational gravity, are provided. Part II considers relationships and contributions of representational gravity to physical gravity, perception of subjective visual vertical, perception of size/mass and weight, other spatial biases (e.g., representational momentum, landmark attraction), catching and intercepting moving targets, an internal model of gravity, naïve physics, a gravity heuristic, art and aesthetics, and ecological and representational approaches to perception. Part III provides a summary and conclusions; it is suggested that effects of gravity have been incorporated into perceptual and cognitive representation, and some potential properties of that incorporation are proposed.

Part I: Variables that influence representational gravity

There are several variables that influence representational gravity, and empirical findings regarding the effects of these variables are considered (see Table 1). These variables are classified as involving characteristics of the target, display, context, or observer. Effects of variables that have been studied within each of these groups are reviewed, and other variables within each group whose potential influence on representational gravity has not been examined, but that could be hypothesized to influence representational gravity, are identified.

Table 1 Findings and claims regarding to representational gravity
Full size table

Characteristics of the target

There are many potential characteristics of the target that might be predicted to influence representational gravity. Characteristics that have been investigated include the size/mass of a target, velocity of a moving target, distance traveled by a moving target, orientation of a target, and modality of a target. Other potential characteristics that have not yet been investigated include the shape of a target and the identity of (and other semantic information related to) a target.

Size/mass

Physical gravity influences moving objects and stationary objects, and so effects of representational gravity should be observed on localization of moving targets and of stationary targets. The majority of studies of representational gravity for moving targets presented two-dimensional pictorial displays (in which a computer-generated target would not have possessed physical mass per se), and so differences in mass are usually implied or suggested by differences in size.Footnote 1 Hubbard (1997, 1998) presented square targets of different sizes that moved horizontally or vertically, and displacement was measured along the axis of target motion and the axis orthogonal to target motion. Target size influenced displacement only along the axis aligned with implied gravitational attraction (O displacement for horizontally moving targets and M displacement for vertically moving targets; see below) regardless of the direction of target motion; larger vertically moving targets exhibited larger forward displacement when descending and smaller forward displacement when ascending, and larger horizontally moving targets exhibited larger downward displacement. Additionally, larger targets were rated as more massive, weighing more, and requiring more effort to move than were smaller targets. De sá Teixeira and Oliveira (2014) found that larger horizontally moving targets exhibited larger downward displacement, and for vertical motion, they also found that effects of target size on displacement were influenced by whether participants could visually track the target. De sá Teixeira and Oliveira suggested that effects of target size on representational gravity were related to foveal bias rather than to an analogue of weight or heaviness.

Hubbard (1997) found that target size did not influence downward displacement of stationary square targets presented in isolation. Hubbard and Ruppel (2000) presented a stationary square near a larger stationary landmark, and displacement toward the landmark was increased or decreased for larger targets above or below, respectively, the landmark; this pattern is consistent with a combination of representational gravity and a landmark attraction effect (i.e., when the target was above the landmark, representational gravity and landmark attraction operated in the same direction, and so they summed and displacement toward the target was relatively larger; when the target was below the landmark, representational gravity and landmark attraction operated in opposite directions, and so they partially canceled and displacement toward the landmark was relatively smaller). Additionally, larger targets to the left or right of the landmark exhibited larger representational gravity. The effects of target size for targets to the left or right of the landmark are consistent with an effect of implied weight; however, it is not clear how foveal bias could account for such an effect, as those targets were vertically aligned with the center of the landmark (and the fovea was presumably aligned with the center of the landmark). Also, studies of effects of target size/mass on representational gravity have only presented simple geometric stimuli, and so it could be useful to consider whether semantic knowledge regarding target identity might influence effects of target size/mass on representational gravity (cf. effects of target identity on representational momentum; Hubbard, 2005b).

Velocity

Velocity does not usually influence downward displacement of horizontally moving targets (de sá Teixeira, Hecht, & Oliveira, 2013; Hubbard, 1990; Hubbard & Bharucha, 1988), although de sá Teixeira, Hecht, and Oliveira (2013) noted a slight tendency for faster horizontally moving targets to exhibit smaller downward (more upward) displacement. However, velocity does influence forward displacement for vertically moving targets (de sá Teixeira & Hecht, 2014a; Hubbard, 1990; Hubbard & Bharucha, 1988). The effect of velocity on representational gravity of moving targets appears to be limited to when the direction of target motion is aligned with the axis of gravitational attraction; in such a case, though, it is not clear whether the differences in forward displacement reflect an effect of velocity on representational momentum or on representational gravity. However, the former is perhaps more likely, as increases in velocity result in increases in representational momentum (for review, see Hubbard, 2005b, 2014, 2018b), and there is no a priori reason to predict an effect of velocity on representational gravity. A lack of an effect of velocity on representational gravity would be consistent with physical laws (see Part II), as objects of different masses all accelerate due to gravity (i.e., fall) at the same rate (ignoring differences in air resistance, etc.; but see Oberle et al., 2005). Even so, findings of representational gravity for stationary objects (i.e., objects with a zero velocity) demonstrate that motion of the target (i.e., a nonzero velocity) is not necessary in order for representational gravity to be exhibited.

Distance traveled

An increase in the distance traveled by a horizontally moving target does not influence forward displacement of that target (de sá Teixeira, 2016a; Hubbard & Bharucha, 1988), but does increase downward displacement of that target (de sá Teixeira, Hecht, & Oliveira, 2013). Forward displacement for an ascending target or a descending target decreases or increases, respectively, with increases in vertical distance traveled by the target (Hubbard, 2001), but as discussed below, studies of the effect of distance traveled by vertically moving targets often involved a confound of distance traveled and height in the picture plane. Given that the gravitational attraction between two objects is in part a function of the distance of those two objects from each other (see Part II), horizontal motion of an object would not change the distance between an object and the (center of the) Earth, and so an effect on representational gravity of distance traveled in the horizontal direction is not completely consistent with physical gravity. However, an increase in downward displacement with increases in horizontal distance traveled is consistent with the observation that an unpowered horizontally moving object would fall along a parabola with increasing horizontal distance traveled. Vertical motion of an object would change the distance between that object and the (center of the) Earth, and so representational gravity for vertical motion is consistent with physical gravity. Also, most ascending objects decelerate as they rise (unless orbital or escape velocity is attained) and accelerate as they fall, and increases in forward displacement with decreases in height in the picture plane for ascending motion and descending motion are consistent with this.

Target orientation

There have been several studies of the effects of direction of gravitational attraction on perception of object orientation (e.g., Dyde, Jenkin, Jenkin, Zacher, & Harris, 2009) and stability (e.g., Lopez, Bachofner, Mercier, & Blanke, 2009), but few studies of the effects of target orientation on representational gravity. The majority of studies on representational gravity presented symmetrical geometric targets (e.g., circles, spheres), but Vinson et al. (2014) presented experimental participants with a target based on a silhouette of a human body. Their stimuli involved a frozen-action depiction of a person near a cliff; the description suggested the person either fell from or jumped off the cliff, and the person’s body was oriented to face either upward or downward. Memory for the location of the target was displaced consistent with representational momentum and representational gravity, but no effects of description or target orientation were observed in the vertical component of displacement. One possible account for the lack of an effect of target orientation on downward displacement in Vinson et al. is that the orientation of the long axis of the target relative to the suggested direction of motion was approximately the same in upward and downward orientations, and so there might not have been a difference in perceived resistance to motion as a function of target orientation (cf. Hubbard’s, 2005a, finding that forward displacement of a rectangular target was larger when direction of motion was parallel to the long axis of the target than to the short axis, which was suggested to reflect differences in perceived resistance to motion).

Target modality

Almost all of the studies examining representational gravity presented visual stimuli. Although there are many visual cues regarding the effects of physical gravity on physical objects in the environment, physical gravity per se is not a visual phenomenon nor directly visually perceivable. Indeed, if perception of gravity were linked to a specific sense, then vestibular or perhaps proprioceptive or kinesthetic senses might be the most appropriate; although the vestibular system has been linked to representational gravity (e.g., de sá Teixeira & Hecht, 2014b; de sá Teixeira et al., 2017), no studies involving effects of representational gravity on proprioceptive or kinesthetic targets have yet been reported. Consistent with the possibility of proprioceptive or kinesthetic involvement, one recent model of phenomenal causality suggests that visual perception of force is based on a linking of haptic and visual information (White, 2012; see also Hubbard, 2012, 2013b). To the extent that gravitational attraction is perceived as a force, such an account is suggestive. Only a single study of representational gravity with nonvisual stimuli has been reported, and that study involved auditory stimuli; when participants were presented with descending or ascending series of auditory pitches, memory for the initial pitch exhibited a larger forward displacement (in auditory pitch space) for descending pitch sequences than for ascending pitch sequences (Hubbard & Ruppel, 2013). Given the possibility of representational gravity with visual, nonvisual, or cross-modal (see below) stimuli, it could be hypothesized that effects of gravity are represented amodally or by a more abstract higher-order process or mechanism.

Characteristics of the display

There are many potential characteristics of the display that might be predicted to influence representational gravity. Characteristics that have been investigated include the retention interval between when a target vanishes and observers indicate the judged (final) location of that target, the response measure, and the height of a target in the picture plane. Other potential characteristics that have not yet been investigated include the surface form of a target (i.e., stimulus format, e.g., a smoothly moving target or a single static image), eccentricity of a target, and whether the participant controls (i.e., has action plans regarding) target behavior.

Retention interval

The magnitude of forward displacement for horizontally moving targets increases with increases in retention interval from zero to 1,200 milliseconds (de sá Teixeira, 2016a; de sá Teixeira, Hecht, & Oliveira, 2013). Interestingly, the forward and downward components of displacement for horizontally moving targets (i.e., representational momentum and representational gravity, respectively) appear to follow different time courses. During the first 150–300 milliseconds after the target has vanished, forward displacement increases and downward displacement is relatively small, and then forward displacement stabilizes and downward displacement increases. The magnitude of forward displacement for descending targets increases with increases in retention interval up to 600 milliseconds, whereas the magnitude of backward displacement of ascending targets increases slightly with increases in retention interval (de sá Teixeira & Hecht, 2014a). Consistent with this, representational gravity was observed in other studies that presented stationary targets and used a retention interval of 1,000 milliseconds (Hubbard & Ruppel, 2000), but not 250 milliseconds (Hubbard, 1997). Downward displacement of horizontally moving targets increases throughout the first 1,000 milliseconds if measured by probe judgment, but downward displacement does not begin to increase until after 600 milliseconds if measured by cursor positioning (de sá Teixeira et al., 2019). The time course but not the presence of representational gravity is influenced by vestibular stimulation during target presentation (de sá Teixeira et al., 2017), suggesting that vestibular input contributes to representational gravity (cf. Indovina et al., 2005).

Response measure

The most common response measure used to study representational gravity is cursor positioning (more recently referred to as mouse pointing by da sá Teixeira, Kerzel, and colleagues). After the target has vanished, participants use a computer mouse, trackpad, or trackball to position a cursor at the display coordinates of the remembered final location of the target, and by clicking on or near the mouse, trackpad, or trackball, the judged display coordinates are recorded. These judged coordinates are then compared with the actual final coordinates of the target. For moving targets, differences between the judged and actual coordinates are typically calculated along the axis of motion, referred to as M displacement, and along the axis orthogonal to motion, referred to as O displacement (see Fig. 1). For stationary targets, displacements are usually measured along the x-axis and y-axis, although these might have different study-specific names as a function of experimental conditions. Another common response method involves probe judgment. After the target has vanished, a stationary probe the same size and shape as the target is presented, and participants judge whether the location of the probe is the same as the remembered final location of the target. Responses involving cursor positioning (and relatedly, reaching toward and touching the stimulus display) are considered motor responses, and responses involving probe judgment are considered perceptual responses. In other studies that do not explicitly address representational gravity, but which often provide data relevant to representational gravity, participants intercepted (caught or hit) a rolling or falling target, and the timing and accuracy of interceptive behavior was measured.

Fig. 1
figure 1

Examples of M displacement and O displacement for rightward, leftward, descending, and ascending targets. M displacement is along the axis of motion, and M displacement is positive for locations in front of the actual target final location and negative for locations behind the actual target final location. O displacement is along the axis orthogonal to target motion; O displacement for rightward and leftward motion is positive for locations above the axis of motion and negative for locations below the axis of motion, and O displacement for descending and ascending targets is positive for locations to the right of the axis of motion and negative for locations to the left of the axis of motion.

Full size image

Although cursor positioning and probe judgment response measures both provide evidence of representational gravity, the two methods suggest slightly different time courses. As noted earlier, de sá Teixeira et al. (2019) reported representational gravity increased during the first 1,000 milliseconds with probe judgment, but was not present until after 600 milliseconds with cursor positioning. They interpreted this as reflecting differences between perceptual (probe judgment) and motor (cursor positioning) systems, and they suggested that representational gravity was stronger with perceptual responses than with motor responses. Such a difference suggests that representational gravity might result from multiple distributed mechanisms rather than from a single mechanism; indeed, the distinction between perceptual and motor responses is reminiscent of differences between dorsal and ventral visual streams. Curiously, de sá Teixeira et al. reported the opposite pattern for representational momentum (see also Kerzel, 2003), that is, representational momentum was stronger with motor responses than with perceptual responses. Additionally, de sá Teixeira (2016a) reported that higher and lower gains in mouse sensitivity in cursor positioning increased response times, and they speculated that increased response times for higher gains and for lower gains reflected an increased need for hand precision and arm effort, respectively. Although not explicitly discussing representational gravity, much of the research on interception of moving targets (see below) is relevant to representational gravity and can be considered to involve the motor system, as these studies involve catching or striking the moving target.

Height in the picture plane

Forward displacement is typically smaller for vertically moving targets higher in the picture plane than for vertically moving targets lower in the picture plane, and this is consistent with an influence of representational gravity; descending targets would presumably accelerate as they fell, and ascending targets would presumably decelerate as they rose (Hubbard, 2001). Even though a descending target low in the picture plane would be expected to stop descending (e.g., by hitting the ground or another barrier), it could continue to accelerate rather than decelerate until the moment it stopped, and so forward displacement would be increased (cf. Finke, Freyd, & Shyi, 1986). However, and as noted earlier, height in the picture plane is often confounded with distance traveled. Also, differences in the strength of gravitational attraction (between Earth and an object on or near the surface of the Earth) at different heights in the picture plane that would be encountered in everyday life are negligible, and so the effect of height in the picture plane on forward displacement of vertically moving targets is more likely due to the effects of gravitational attraction on velocity and representational momentum than to the absolute levels of gravitational attractional per se. Consistent with this, displacement of stationary targets in the upper half of the picture plane decreases with increases in retention interval (up to 600 milliseconds), whereas downward displacement stops after approximately 300 milliseconds for targets in the lower half of the picture plane (de sá Teixeira & Hecht, 2014a). The latter result might reflect a floor effect, as targets in the lower half of the picture plane might be expected or perceived to have less vertical distance through which to continue descending.

Characteristics of the context

There are many potential characteristics of the context that might be predicted to influence representational gravity. Characteristics that have been investigated include the presence of nontarget intramodal stimuli and cross-modal components of a single stimulus. Other potential characteristics that have not yet been investigated include attributions regarding why a target is stationary or in motion and expectations regarding changes in target behavior.

Nontarget intramodal stimuli

The presence of nontarget intramodal stimuli can influence representational gravity for a target, and this occurs regardless of whether a nontarget stimulus appears to contact the target. Freyd et al. (1988) presented a picture of an object (e.g., flowerpot) that was supported by another object (e.g., sitting on a table or hanging from a hook). If the supporting object was removed, the judged location of the object was displaced downward consistent with effects of gravity. Interestingly, downward displacement was decreased when the object was initially shown in isolation (e.g., the flowerpot was presented on an otherwise blank field; cf. stationary targets in Hubbard, 1997). In other words, representational gravity was larger when participants perceived removal of a nontarget object that had apparently counteracted the effects of implied gravity on the target object. Bertamini’s (1993) finding that the remembered location of an object on an inclined plane was mislocalized down the inclined plane, and that the magnitude of mislocalization was larger with steeper inclined planes, is also consistent with an effect of context on representational gravity for the target. Masuda et al. (2011) reported that illusory kappa motion was influenced by representational gravity when the context suggested kappa motion was up or down a slope, but not influenced by representational gravity when the context suggested kappa motion was horizontal. Representational gravity might have been more salient in the former case because physical gravity would be more salient in moving up or down a slope than in moving horizontally.

Hubbard (1995a) presented horizontally moving targets that moved above or across the surface of a larger stationary object or between the upper and lower surfaces of larger stationary objects below and above the target, respectively. Relative to a control target of the same size, velocity, and direction of motion, but presented on a blank background, representational gravity was increased when a target moved along the surface of a larger stationary object below the target (forward displacement of a target that moved along a surface was also decreased, and this was referred to as representational friction; see Hubbard, 1995a, 1995b, 1998). When a horizontally moving target moved along the upper and lower surfaces of larger stationary objects below and above the target, respectively, downward displacement was decreased (and representational momentum was decreased below that observed when the target moved along a single surface). One possibility is that a landmark attraction effect from a single surface below the target combined with representational gravity to increase downward displacement, whereas landmark attraction effects from surfaces above and below the target partially canceled out. Analogous effects of contact with surfaces to the left and/or right of a vertically moving target have also been found (Hubbard, 1998). Relatedly, when a target moves parallel to but does not contact a surface, the target is displaced along its orthogonal axis toward that surface (Hubbard & Ruppel, 1999), and so downward displacement of the target when nontarget intramodal stimuli are present appears more likely to also reflect landmark attraction or memory averaging with the nontarget stimuli and not just representational gravity for the target.

Cross-modal components

As noted earlier, the majority of studies examining representational gravity presented visual stimuli, although one study reported representational gravity for the initial location of auditory targets moving in auditory pitch space (Hubbard & Ruppel, 2013). Representational gravity is found cross-modally with visual and auditory stimuli; when a horizontally moving visual target was accompanied by ascending or descending motion in auditory pitch space, representational gravity for the visual target was larger when descending auditory pitch motion occurred than when ascending auditory pitch motion occurred (Hubbard & Courtney, 2010). Cross-modal stimuli involving visual target motion and auditory pitch motion, as well as effects of representational gravity on the representation of auditory pitch motion, suggest an extension of representational gravity or of an internal model of gravity to include a more abstract space in which physical gravity would not be expected to have an effect (e.g., auditory pitch space).Footnote 2 Visual information can be used and interact with vestibular information in determination of the perceived direction of gravitational attraction (e.g., Lacquaniti et al., 2013; Lacquaniti et al., 2015; Pfeiffer, Grivaz, Herbelin, Serino, & Blanke, 2016; Zago, La Scaleia, Miller, & Lacquaniti, 2011) and presumably the direction of representational gravity. Harris, Jenkin, Jenkin, Zacher, and Dyde (2017) reported that astronauts on the International Space Station exhibited a decreased reliance on visual cues in judgments of the subjective visual vertical and the perceptual upright, but given that visual information could be misleading in a weightless environment, such a decrease would be adaptive.

Bodily mechanisms such as the vestibular system (Angelaki, Wei, & Merfeld, 2001) and the tactile system (Trousselard, Barraud, Nougier, Raphel, & Cian, 2004) are involved in judgments of the direction of gravitational attraction. Relatedly, given the close associations of proprioceptive and kinesthetic information with physical gravity (as body movements are usually in a direction in opposition to or consistent with physical gravity), it could be predicted that proprioceptive or kinesthetic information presented concurrently with a visual or auditory target should influence representational gravity for that target. As noted earlier, such a pairing of visual information and gravitational force is similar to the development of a more general visual perception of force suggested by White (2012). Similarly, Tajadura-Jiménez et al. (2018) reported that manipulation of auditory and tactile cues related to a falling object could influence representation of the body. Participants are generally able to intercept a target even if a portion of that target’s visual trajectory is occluded (e.g., La Scaleia et al., 2015), and this is consistent with the notion that observers might use some type of internal model to predict the motion of the target and that such an internal model includes the effects of gravity. As perception of physical gravity involves inputs from multiple modalities, it is thus not surprising that representational gravity can be influenced by information from multiple modalities and that information in one modality can be influenced by information regarding the direction of gravitational attraction in another modality.

Characteristics of the observer

There are many potential characteristics of the observer that might be predicted to influence representational gravity. Characteristics that have been investigated include oculomotor behavior, body orientation relative to the environmental axis of gravity, and the presence of psychopathology (in the form of schizophrenia). Other potential characteristics that have not yet been investigated include the age of the observer, allocation of attention, effect of error feedback or explicit knowledge of representational gravity, and effects of psychopathology other than schizophrenia.

Oculomotor behavior

Whether experimental participants visually track a moving target or fixate a separate stationary point some distance away from a moving target does not influence representational gravity for that target, and the presence or absence of pursuit eye movements does not influence representational gravity across retention intervals ranging from zero to 1,200 milliseconds (de sá Teixeira, Hecht, & Oliveira, 2013). De sá Teixeira and Hecht (2014a) concluded that oculomotor behavior does not play a causal role in representational gravity, but was merely an epiphenomenon of an internal model of gravity. Also, representational gravity occurs for stationary targets (e.g., de sá Teixeira & Hecht, 2014a; Freyd et al., 1988), which further demonstrates that representational gravity does not depend upon pursuit eye movements. De sá Teixeira (2016b) conducted a Fourier decomposition of displacement data and reported that constraining pursuit eye movements had no effect on the presence or time course of representational gravity,Footnote 3 and given these findings, it was suggested that representational gravity might reflect an internal model of gravity. Also, de sá Teixeira and Oliveira (2014) reported that foveal bias can influence the effect of target size on representational gravity for moving targets, with larger targets exhibiting a larger displacement toward a fixated location than did smaller targets (see discussion below). As noted above, the presence of nontarget stimuli can influence displacement of a target, but research on potential effects of oculomotor behavior on displacement usually does not consider the possibility that a fixation point can function as a landmark and influence spatial localization of the target even if that landmark is not fixated.

Body orientation

Nagai et al. (2002) had participants view displays in which targets moved upward or downward or appeared to approach or recede, and their participants sat upright or lay prone (face downward); in the former case, the body axis and environmental gravity axis were aligned, and in the latter case, the body axis and environmental gravity axis were not aligned. Effects consistent with representational gravity occurred only when target motion was aligned with the environmental gravity axis regardless of the orientation of the body axis. However, a role of orientation of the body axis was found by de sá Teixeira and Hecht (2014b), who had participants view displays in which targets moved horizontally or vertically, and their participants sat upright or lay on their left sides. Previous findings regarding representational gravity were replicated with upright posture, but not with side posture, and de sá Teixeira and Hecht concluded that dynamic representation of gravity was suspended when the body axis was not aligned with the environmental gravity axis. There are several methodological differences that might account for the differences in findings (e.g., Nagai et al., 2002, presented implied motion, used probe judgments, and had participants lay face-down [i.e., the environmental gravity axis was aligned with the line-of-sight], whereas de sá Teixeira & Hecht, 2014b, presented smooth motion, used cursor positioning, and had participants lay on their left sides [i.e., the environmental gravity axis was not aligned with the line of sight]), and it is not clear yet which of these methodological differences (if any) might account for the differences in findings.

De sá Teixeira (2014) varied the degree of body tilt of experimental participants relative to the environmental direction of gravitational attraction. Increases in misalignment between the body axis and environmental gravity axis disrupted the time course of downward displacement; representational gravity was exhibited along the body’s main axis, and it was suggested this reflected an increased uncertainty regarding the actual direction of gravitational attraction. Further uncertainty regarding the direction of gravitational attraction was introduced by de sá Teixeira et al. (2017), who had participants judge the remembered final location of a target moving in one of several different possible directions while those participants were being spun in a centrifuge. Effects of representational gravity on displacement appeared to occur along the axis aligned with the main axis of the body. Of course, in everyday life any uncertainty regarding the direction of gravitational attraction that resulted from a lack of alignment between the body axis and the environmental gravity axis can usually be at least partially compensated for by visual and other cues (e.g., Lacquaniti et al., 2013). A final type of variable that might contribute to whether the body axis or the environmental gravity axis is more critical for representational gravity involves individual differences (e.g., field dependence/independence), but individual differences have not received much consideration in displacement literature. Also, the importance of the orientation of the body axis for representational gravity is consistent with the importance of an embodiment of cognitive processes and with the possibility of action-specific perception.

Psychopathology

There has been only one study of representational gravity in a population diagnosed with psychopathology. In that study, no differences in M displacement for vertically moving targets or in O displacement for horizontally moving targets were found between patients diagnosed with schizophrenia and control participants (de sá Teixeira, Pimenta, & Raposo, 2013). The presence of representational gravity in patients diagnosed with schizophrenia is consistent with the hypothesis that representational gravity does not depend upon oculomotor behavior, as patients diagnosed with schizophrenia often exhibit abnormalities in eye movements (e.g., Levy, Sereno, Gooding, & O’Driscoll, 2010; Sweeney et al., 1994). Potential effects of a psychopathology other than schizophrenia on representational gravity have not yet been reported. Based on studies of the related literature on representational momentum (for review, see Hubbard, 2014, 2018b), it could be predicted that patients diagnosed with intellectual disabilities might exhibit reduced representational gravity and that patients diagnosed with neglect might exhibit differences in representational gravity as a function of the visual field of target presentation. Also, it could be especially useful to consider representational gravity in patients diagnosed with vestibular disorders (e.g., vertigo) or who exhibit damage to vestibular cortex, but such studies have not yet been reported. Similarly, given the potential relationship of representational gravity to proprioceptive and kinesthetic information, it could be useful to consider representational gravity in patients diagnosed with different types of motor disorders.

Part II: Contributions and relationships of representational gravity

Representational gravity might contribute or be related to several different processes, tasks, or theories. Contributions and relationships considered here include those involving physical gravity, linear acceleration, the subjective visual vertical, size/mass and weight, other biases in spatial localization, catching or intercepting a moving target, an internal model of gravity, naïve physics, a gravity heuristic, art and aesthetics, and ecological or representational theories of perception and cognition.

Physical gravity

In order to understand representational gravity, it is useful to consider physical gravity. Newton’s equation for universal gravitation is

$$ F=\left({Gm}_1{m}_2\right)/{r}^2 $$
(1)

in which F is the force of gravity between two objects, G is the gravitational constant, m1 and m2 are the masses of the two objects, and r is the distance between the (centers of the) objects. For organisms living on Earth, the Earth would be one of the two objects, and the other object could be any other physical object (including the organism) in the environment. Accordingly, acceleration due to gravity on the surface of the Earth on some object is

$$ g= GM/{r}^2 $$
(2)

in which g is the gravitational force on an object on the surface of the Earth, G is the gravitational constant, M is the mass of the Earth, and r is the radius of the Earth (or more precisely, the distance between the center of the Earth and the center of the object). Representational gravity has been suggested to reflect weight, and the acceleration due to gravity from Eq. 2 can be used to determine the weight of an object,

$$ w= mg $$
(3)

in which w is the weight of the object, m is the mass of the object, and g reflects the acceleration due to gravity. With acceleration due to gravity being constant for organisms that live on Earth, weight is a constant proportion of mass, and there is no opportunity for such organisms to experience a dissociation of weight and mass.

An important element in each of Eqs. 1, 2, and 3 is mass. As noted earlier, the influence of target mass on displacement occurs only along the axis perceived to be aligned with gravitational attraction, and this is consistent with the notion that representational gravity might reflect the experience of weight rather than an effect of mass; even so, effects of mass (weight) along the axis of gravitational attraction are not completely consistent with physical gravity (e.g., larger representational gravity for larger targets is not consistent with the principle that objects of different masses fall at the same velocity). Effects of other variables that influence representational gravity are more consistent with physical gravity. For example, an increase in representational gravity with increases in retention interval is consistent with continued falling of a physical object, and an increased representational gravity after 300 milliseconds (de sá Teixeira, Hecht, & Oliveira, 2013) is consistent with acceleration during continued falling. Also, the effect of height in the picture plane on representational gravity for vertically moving targets is consistent with physical gravity, although differences in physical gravity between different heights in the picture plane typically encountered in everyday life would be far too small to be discriminable. However, effects of other variables on representational gravity are less consistent with physical gravity (e.g., effects of body orientation). Thus, representational gravity is not completely consistent with Newton’s laws. Such a conclusion is consistent with conclusions of Lacquaniti, McIntyre, Zago, and others that an internal model of gravity could be used to predict the most likely path of an object but would not necessarily incorporate Newton’s laws.

Linear acceleration

According to Einstein’s principle of equivalence, the effect of accelerating in a specific direction and the effect of gravitational attraction would be perceptually equivalent (Haugen & Lämmerzahl, 2001). Humans perceive their direction of motion and the direction of gravitational attraction with their vestibular systems, and a nonlinear combination of information from both otoliths and semicircular canals is often necessary for discriminating between acceleration and gravitational attraction (e.g., Angelaki, McHenry, Dickman, Newlands, & Hess, 1999; Angelaki et al., 2001). Additionally, humans can use a variety of visual cues in determining the direction of gravitational attraction (e.g., orientations of other humans, plants, architecture; Lacquaniti et al., 2013). Even so, an equivalence of linear acceleration and gravitational attraction, and the consequences of this equivalence for representational gravity, can be observed in experiments in which participants view displays in which targets are moving in various directions while those participants are being spun in a centrifuge. De sá Teixeira et al. (2017) found that placing participants in a spinning centrifuge disrupted the time course of representational gravity; they rejected the possibility that oculomotor effects of centrifugation contributed to the disruption, and they suggested that if there is a conflict between perceptual reference frames (e.g., acceleration in one direction and gravitational attraction in a different direction), then the internal model of gravity produces displacements “downward” along the idiotropic vector (i.e., the direction of gravity is perceived along the body axis with which it is aligned in everyday life).

Subjective visual vertical

The direction of gravitational attraction (on the surface of Earth) is usually aligned with the vertical, and so it is useful to consider how representational gravity is related to perception of the vertical. The most common assessment of the perceived vertical is the subjective visual vertical (SVV) test, in which an individual in a darkened environment adjusts a luminous rod to be aligned with the perceived vertical (for review, see Zago, 2018). An important component of the SVV is head and body orientation (Mittelstaedt, 1983). One study in which participants were seated upright or laying face-down suggested that representational gravity was aligned with the environmental gravity axis rather than the body axis (Nagai et al., 2002); however, other studies suggested body tilt relative to the external environment increases uncertainty regarding the direction of “down,” and in such cases, “down” was interpreted to be along the body axis or to reflect a combination of the body axis and environmental gravity axis (de sá Teixeira, 2014; de sá Teixeira & Hecht, 2014b; de sá Teixeira et al., 2017). When the body axis is tilted relative to the environmental gravity axis, judgments of SVV are biased away from the environmental gravity axis and toward the body axis (the Aubert effect; Trousselard et al., 2004). Given the reasonable suppositions that SVV should be aligned with the direction of gravitational attraction, and that representational gravity should occur in the direction of gravitational attraction, it could be predicted that tilted observers should exhibit representational gravity that is aligned with SVV rather than aligned with the environmental gravity axis.

If the body axis (and head) is not aligned with the environmental gravity axis, the normal time course of representational gravity is disrupted, and representational gravity is reduced to a constant displacement along the body’s main axis (de sá Teixeira et al., 2017). This pattern has been suggested to highlight the importance of vestibular input in the production of representational gravity. Indeed, the vestibular cortex is activated by visual motion coherent with natural gravity, but is not activated by other types of visual motion (Indovina et al., 2005). It might be that representational gravity is exhibited in the direction perceived to align with gravitational attraction (i.e., the perceived vertical), and that the perceived direction is not necessarily the actual direction (i.e., not the actual vertical, e.g., possible effects of field dependence/independence on the direction of representational gravity). Of course, in everyday life, the perceived and actual directions of gravitational attraction are generally aligned. It might be that SVV and representational gravity are both influenced by an internal model of gravity and that such a model is influenced by contextual variables in addition to the actual direction of gravitational attraction. This is consistent with the effects of context noted in Part I and effects of context on other displacements attributed to environmentally invariant principles (e.g., forward displacement of a rotating rectangle is increased or decreased if the orientation of a surrounding square frame is rotated slightly forward or slightly backward, respectively, from the final orientation of the target; Hubbard, 1993).

Size/mass and weight

As noted earlier, Hubbard (1997) found that target size influenced displacement only along the axis aligned with implied gravitational attraction and that larger targets were rated as more massive, weighing more, and requiring more effort to move than were smaller targets. Coupled with the notion that weight is experienced (and defined; see Eq. 3) as occurring only in the direction of gravitational attraction, it was suggested that representational gravity reflected effects of perceived weight. De sá Teixeira and Oliveira (2014) replicated the finding that larger horizontally moving targets exhibited larger representational gravity. Additionally, they reported that effects of target size on O displacement for vertically moving targets were not influenced by whether participants fixated a point to the left of the target trajectory, that the time course of downward displacement for horizontally moving targets was not influenced by target size, and that the rate of downward drift in displacement was not influenced by target size. Based on these latter findings, de sá Teixeira and Oliveira (2014) suggested that effects of target size on representational gravity were related to foveal bias rather than to an analogue of weight or heaviness. However, it is not clear how foveal bias might account for effects of target size on representational gravity (or for downward displacement more generally) of stationary targets to the left or right of a landmark and aligned with the vertical center of that landmark (e.g., see Hubbard & Ruppel, 2000). Moreover, given that oculomotor behavior does not influence representational gravity, but that foveal bias would presumably only be exhibited when a participant fixates away from the target, it is not clear how foveal bias could influence representational gravity when participants visually track (and hence foveate) the target.

Other spatial biases

Representational gravity is one of several spatial biases, and it has been suggested to combine with other spatial biases in determining the observed overall displacement of a target. Hubbard (1990, 1995b, 1997) suggested that for horizontally moving targets, representational momentum (M displacement) and representational gravity (O displacement) operate along different axes, but for vertically moving targets, representational momentum and representational gravity operate along the same axis (M displacement). Of particular relevance, the observed displacement of a given target reflects a combination of these two influences (e.g., when representational momentum and representational gravity operate in the same direction [descending target motion], they sum and forward displacement of the target is larger; see Fig. 2). However, even though representational gravity can combine with representational momentum in determining the observed overall displacement of a target, representational gravity is a separate and distinct form of spatial bias. Motes et al. (2008) conducted a principle components analysis of displacement data for horizontally moving targets that revealed M displacement and O displacement loaded on different factors and were statistically independent. De sá Teixeira (2014, 2016b) applied a Fourier decomposition to displacement data and found differences in effects of oculomotor behavior on representational momentum and representational gravity, and de sá Teixeira et al. (2019) found differences in the relative strength of perceptual responses and motor responses in measurement of representational momentum and representational gravity; these findings were interpreted as suggesting representational momentum and representational gravity arose from different mechanisms.

Fig. 2
figure 2

Examples of how representational gravity and representational momentum combine to produce the observed displacement of a moving target. When representational momentum and representational gravity operate in orthogonal directions (rightward motion, leftward motion), the resultant displacement reflects shifts in each direction. When representational momentum and representational gravity operate in the same direction (descending motion), they sum and the resultant forward displacement is relatively larger, and when representational momentum and representational gravity operate in opposite directions (ascending motion), they partially cancel and the resultant forward displacement is relatively smaller.

Full size image

Representational gravity can combine in analogous ways with other forms of spatial bias, including representational friction (Hubbard, 1995a, 1998), the landmark attraction effect (Hubbard & Ruppel, 2000), the Fröhlich effect (Hubbard & Ruppel, 2013), or the kappa effect (Masuda et al., 2011). Based on this, it could be predicted that representational gravity might combine with other types of spatial biases whose potential influence on or interaction with representational gravity has not yet been examined. Even so, representational gravity appears to be generally weaker than many other spatial biases (e.g., the magnitude of representational gravity is considerably smaller than the magnitude of representational momentum for nonzero velocities that have been studied, and displacement attributed to representational gravity can be eliminated or reversed by landmark attraction effects). Nonetheless, the presence of gravity is ubiquitous for organisms living on Earth, and so representational gravity reflects an important environmental invariant (cf. “strong prior”; Jörges & López-Moliner, 2017). Such combinations of representational gravity with other spatial biases, as well as cross-modal effects in representational gravity, are consistent with the idea that representational gravity involves a relatively higher or more central level at which inputs from different dimensions or modalities can be combined rather than a relatively lower or more peripheral level at which stimuli are processed in dimension-specific or modality-specific receptors that cannot be combined. Alternatively, top-down effects from a relatively higher level could modulate a representation that occurs at a relatively lower level (cf. representational momentum; Hubbard, 2005b, 2006a).

Catching and intercepting

In order to successfully catch or intercept a moving target, the location of that target (in real time) must be accurately represented, and this has been studied using time-to-contact measures (for review, see Zago & Lacquaniti, 2005). McIntyre et al. (2001) compared catching responses to balls launched at different velocities for astronauts in 0 g and 1 g environments. Motor responses in 1 g were well synchronized with the arrival of the ball, but motor responses in 0 g generally occurred too early, leading McIntyre et al. to conclude that an internal model of gravity was being used in generation of catching responses. Additional studies using nonastronaut participants and that presented computer-animated descending targets that moved according to 0 g or 1 g parameters suggested that participants could not “switch off” a 1 g internal model, but could adapt such a model to reflect 0 g kinematics by shifting the timing of motor activation (Zago et al., 2004, 2005). However, other researchers suggested that an internal model of gravity is not necessary and that continuous updating of action information is sufficient (e.g., Baurès et al., 2007), although such a suggestion fails to consider effects of sensorimotor delays on behavior, the ability of internal models to overcome such delays, and that updating of action information and use of an internal model are not necessarily mutually exclusive (Zago et al., 2008). As noted earlier, the vestibular cortex is activated by visual motion coherent with natural gravity, but not by other types of visual motion (Indovina et al., 2005); this suggests that information related to effects of physical gravity on catching or intercepting an object, as well as representational gravity or any internal model of gravity, might involve the vestibular cortex.

La Scaleia, Lacquaniti, and Zago (2014; see also La Scaleia, Lacquaniti, & Zago, 2019) presented participants with a virtual display in which the target was a ball that appeared to roll down an elevated inclined plane toward the participant, and the starting position of the ball and the slope of the inclined plane could vary. Participants had to intercept the ball as it fell off the end of the inclined plane, extrapolate a trajectory for a ball that stopped prior to falling off the end of the inclined plane and intercept the trajectory at the time the ball would have fallen off the inclined plane had motion continued, or indicate with a hand motion the trajectory after the end of the inclined plane for a ball that stopped before falling off the end of the inclined plane if that ball had not stopped. Participants were generally able to extrapolate the global characteristics of target motion in all conditions, and La Scaleia et al. (2014) suggested participants used both visual information and an internal model of target motion that contained information regarding effects of gravity. La Scaleia et al. (2015) presented a ball moving down an elevated inclined plane in the picture plane, and the initial part of the trajectory after the ball fell off the end of the inclined plane was occluded. Participants had to intercept the ball as it emerged from behind the occluder. Participants were generally able to successfully intercept the ball, and La Scaleia et al. (2015) suggested an internal model of target motion that contained information regarding effects of gravity was used to extrapolate the trajectory behind the occluder, and La Scaleia et al. (2019) further suggested that such an internal model incorporated information from visual, vestibular, and somatosensory sources.

An internal model of gravity

If representational gravity is a component of an internal model of gravity, then any such model would incorporate information regarding the relationship between physical gravity and other variables, as well as provide anticipations of target locations in the near future. This anticipation would occur rapidly (within hundreds of milliseconds) and automatically, rather than occur only after explicit deliberation. Such a distinction between an automatic anticipation and an explicit deliberative prediction is consistent with findings for another type of displacement attributed to an environmentally invariant physical principle, representational momentum, as participants who explicitly predicted the next location of a target that exhibited implied motion did not exhibit forward displacement in predictions of the next location (had implied motion continued), but did exhibit forward displacement in judgments of remembered final location (e.g., Munger & Minchew, 2002). The suggestion of Zago et al. (2004, 2005) that an internal model of gravity cannot be “switched off,” but can be adapted by experience, is consistent with the idea that consequences of invariant physical principles (such as gravity) are incorporated into functional architecture of representation but can also be modulated by task-specific or stimulus-specific information (Hubbard, 2005b, 2006a). As demonstrated in studies of catching and intercepting, representational gravity appears to reflect a compensation for delays due to neural processing times and adjusts the representation of target location to reflect where a target would be in real time. Such a compensation is similar to that proposed for representational momentum, and so might reflect a more general property of the representation of invariant physical principles.

Whether representational gravity results in creation of an internal model, or whether an internal model results in creation of representational gravity, is not clear. One possibility is that representational gravity might reflect a two-stage process similar to that proposed for representational momentum; the first stage would involve an automatic default extrapolation consistent with physical principles, and the second stage would involve a potential modulation of this default extrapolation by relevant task-specific or stimulus-specific information. The first stage would overlap with an internal model of gravity (perhaps in the form of the functional architecture), but the second stage would involve other knowledge, beliefs, or expectations of the observer that were related to the target, but not necessarily related to gravity per se. The first stage would involve modular (cognitively impenetrable) processing, and the second stage would involve nonmodular (cognitively penetrable) processing. Such an approach is consistent with findings that just as representational momentum can be modulated by expectations regarding future motion or other information (e.g., expected change in direction, target identity) regarding a specific target (Hubbard, 2005b, 2014, 2018b), so too can representational gravity be modulated by learning (Zago et al., 2005). Importantly, the key determinant of the validity or usefulness of such an approach is not whether it reflects Newton’s laws or a more naïve or heuristic understanding of gravity, but whether it can consistently provide predictions of target behavior that are sufficiently close to accurate or correct to be selected for and not be selected against (cf. Zago et al., 2008).

Naïve physics

De sá Teixeira and Hecht (2014a) discussed three types of internalization that might account for displacement in remembered target location, and these involve geometric kinematics, Newton’s laws, and naïve impetus. They suggest their data (forward displacement followed by a downward displacement) were most consistent with an internalization of naïve impetus (for discussion of naïve impetus and representational momentum, see Hubbard, 2004, 2013a; Hubbard, Blessum, & Ruppel, 2001; Kozhevnikov & Hegarty, 2001); nonetheless, de sá Teixeira and Hecht did not completely accept an impetus-based account as an explanation for the entire target trajectory, because oculomotor behavior can influence forward displacement for some types of targets. However, as oculomotor behavior does not influence representational gravity, they appeared to accept an internalized model of gravity that is influenced by a belief in impetus. The existence of an internalized model of gravity, as well as the possibility of a gravity heuristic (see below), is consistent with Reiner, Slotta, Chi, and Resnick’s (2000) notion of a substance schema in which one of the properties of any substance is that it is “gravity sensitive” (i.e., it falls downward when dropped). Importantly, such a schema does not have to accurately reflect Newton’s laws, but could reflect a naïve (incorrect) understanding of physical laws (e.g., a force such as gravity is considered to be a property of one object rather an interaction between two objects; such a notion is more consistent with pre-Newtonian ideas of impetus than with Newtonian physics).

Another example involves the naïve physics belief that more massive objects fall faster than do less massive objects. The mass of a human or of an object that humans would typically interact with is negligible compared with the far greater mass of the Earth, and so trajectories for objects near the surface of Earth can be adequately calculated using just the mass of the Earth (see Eq. 2). Newton’s laws suggest that all objects near the surface of Earth fall at the same velocity (ignoring effects of resistance, friction, etc.), and so larger downward displacement for larger targets could reflect a dissociation of explicit knowledge of Newton’s laws and representational gravity that is similar to dissociations of explicit knowledge of Newton’s laws and representational momentum previously reported (e.g., Freyd & Jones, 1994; Kozhevnikov & Hegarty, 2001). More broadly, the pairing of “downward” and “direction of gravitational attraction” can also be considered as naïve physics, as the presumably invariant experience of gravitational attraction as being toward the Earth led humans to treat “downward” and “in the direction of gravitational attraction” as synonymous and to develop heuristics based on this invariance (cf. attribution of gravitational attraction along the idiotropic vector; de sá Teixeira, 2014). Given this, it is interesting that de sá Teixeira et al. (2019) suggested that representational gravity reflects a compromise between knowledge about gravity, instantiation of that knowledge in perceptual mechanisms, and task demands; such a compromise suggests that representational gravity might function as a heuristic and that an internal model of gravity could be used to predict the most likely path of an object, but would not necessarily incorporate Newton’s laws (cf. Zago et al., 2008).

A gravity heuristic

Findings in the naïve physics literature suggest that untutored observers often have incorrect beliefs regarding the functioning of physical systems. One such incorrect belief that was noted earlier involves impetus. Just as a belief in impetus suggests that a single object possesses a force that can be transferred to another stimulus or dissipated (e.g., McCloskey, 1983), so too might naïve beliefs about gravity suggest that gravity is a force possessed by a single object and that can be transferred to another object (cf. Reiner et al., 2000). Just as an impetus heuristic can result in predictions that are approximately correct, but require less understanding of correct physical principles and less effort (e.g., a stationary object given a sufficiently strong push will move a short distance and then stop), so too might a gravity heuristic result in predictions that are approximately correct (or if incorrect, would not be sufficiently damaging to have been selected against), but require less understanding of physical principles and less effort. One such heuristic is the Galileo bias, which is involved in judgments regarding the velocity at which physical objects fall (Oberle et al., 2005). It is possible that representational gravity is consistent with one or more heuristics regarding physical gravity rather than reflecting an accurate internalization of Newtonian principles of gravity (cf. gravity as a “strong prior” in a Bayesian framework in Jörges & López-Moliner, 2017). Indeed, the potential strength and ubiquity of such a heuristic is reflected in the observation that representational gravity is exhibited in the perception of computer-animated stimuli even though such stimuli do not actually possess mass (or weight) and so would not experience effects of physical gravity.

Art and aesthetics

Representational gravity could contribute to art and aesthetics. Balance in an artwork is said to be related to the “visual weight” of different elements or components of that artwork, and representational gravity might contribute to perception of visual weight; indeed, visual weight has been suggested to involve elements such as size, directionality, and dynamics (for review, see Hubbard, 2018a) that are salient for representational gravity. Perception regarding visual weight or balance occurs very rapidly, perhaps even within a single fixation (Gershoni & Hochstein, 2011; Locher & Stappers, 2002), and this is consistent with findings that representational gravity occurs within hundreds of milliseconds. Winner, Dion, Rosenblatt, and Gardner (1987) suggested that an object appears to weigh more when presented higher in the picture plane; thus, if two equal masses were in the top and bottom of the picture plane, the picture would look unbalanced, whereas if a larger mass was in the bottom of the picture plane, the picture would look more balanced. Similarly, pictures are judged as more balanced if the distribution of visual weight along the horizontal axis is centered near the physical center of the picture (McManus et al., 1985). Also, positively valanced stimuli in artworks are remembered as being slightly higher in the picture plane than are negatively valanced stimuli at the same objective height (Crawford et al., 2006); this suggests that negative valence increases representational gravity (e.g., feeling “weighted down”) and positive valence decreases representational gravity (e.g., feeling “able to rise above”).

Representational gravity could also contribute to art and aesthetics if effects of gravity or weight were captured in depictions of lifting or supporting a heavy object. Indeed, observers are able to correctly judge the weight of an object lifted by a point-light walker in a static or dynamic display (e.g., Runeson & Frykholm, 1981; Valenti & Costall, 1997). A static display would be analogous to a painting, photograph, or sculpture, and if the dynamics of lifting or supporting are depicted in such artwork, then sensitivity to those dynamics might occur in an observer's representation of whatever is doing the lifting or supporting (e.g., person, column). Sensitivity to the effort and strength required for such lifting or supporting might result in empathy and contribute to an aesthetic response (Arnheim, 1974, 1988; Freyd, 1993). The possibility of sensitivity to such dynamics has implications for broader theories involving kinematic specification of dynamics (e.g., Bingham, 1987; Runeson & Frykholm, 1983) and embodied approaches to cognition (e.g., Would inspection by an observer of a statue that depicted a person lifting or supporting a heavy object elicit activity in mirror neurons that would fire if that observer lifted or supported a similar object?). Relatedly, an emphasis on visual weight and balance in art and aesthetics is consistent with the earlier suggestion that representational gravity is related to the effects of perceived weight. Also, the idea of musical gravity (i.e., the tendency of a musical note above a stable reference or platform to descend; Larson, 2012) is consistent with findings that representational gravity occurs with auditory stimuli and might contribute to auditory art and aesthetics.

Ecological or representational theories

The effect of gravity is one of the most important invariants in the environment. In addition to contributing to localization of objects, the effects of gravity are implicated in determination of the subjective visual vertical, determination of the canonical upright (useful in object recognition), and judgment of object stability (for reviews, see Jörges & López-Moliner, 2017; Zago, 2018). In short, knowledge of the effects of gravity would be useful in a wide range of tasks. Given this, it would be useful if information regarding the effects of gravity on specific stimuli or in the environment more generally were available to perceptual and cognitive mechanisms and processes. Given gravity’s status as an invariant physical principle in the experience of organisms on Earth, it could be proposed that a sensitivity to effects of gravity is adaptive and has been selected for during evolution. Furthermore, if information regarding effects of gravitational attraction is to be maximally useful to an organism, then that information should be available without increasing the demand on perceptual and cognitive mechanisms or processes; in other words, the effects of such information should be applied automatically to the representation of the stimulus. One way to achieve this would be to incorporate effects of gravity into the functional architecture of mental representation, that is, the representation of an object would automatically contain information about the probable effects of gravity on that object in the near future. This would add a dynamic aspect to an otherwise static representation and is consistent with previous suggestions that effects of other environmentally invariant physical principles are dynamic and are incorporated into the functional architecture of mental representation (e.g., Hubbard, 1999, 2005b, 2006a, 2019).

If information regarding effects of gravity were incorporated into functional architecture of mental representation, then that would offer a potential bridge between Gibsonian and representational approaches to perception (cf. momentum-like effects as a similar bridge; Hubbard, 2005b, 2015, 2019). More specifically, there would be sensitivity to invariant information within the stimulus, and this would lead to an initial displacement in representation consistent with the effects of gravity. As this sensitivity and resulting displacement would be reflected within the functional architecture of the representational system and thus occur automatically, no additional perceptual or cognitive resources would be necessary for extrapolating the effect of gravity. However, this initial default displacement could subsequently be modulated by knowledge, beliefs, or expectations specific to the particular task or stimulus. Such a two-stage model is consistent with an analogous two-stage model proposed for representational momentum (Hubbard, 2006a). Furthermore, relative weightings of the initial default and the subsequent modulation are potentially consistent with McBeath’s (2018) proposal of a continuum of environmental regularities including universal invariants (e.g., effects of physical principles), intermediate biological motion or features (e.g., axis-aligned motion, forward-facing motion), and weak culturally learned regularities (e.g., driving on the right side of the road); the first and second stages would be weighted strongest and weakest for the universal invariants, respectively, and would be weighted weakest and strongest for culturally learned regularities, respectively.

Part III: Summary and conclusions

The effects of gravity reflect an ubiquitous aspect of the experience of organisms on Earth, and organisms that could anticipate the effects of gravity on their bodies or on stimuli in their perceivable environment could have a selective advantage. In other words, being able to anticipate the effects of gravity could be a useful adaptation. Given the lag in perceptual awareness of stimuli due to neural processing times, such an anticipation would be helpful in interacting with moving objects in real time (e.g., catching or intercepting a target) and in interacting with stationary objects (e.g., anticipating an object would fall if a support was removed). Representational gravity, that is, displacement of the judged location of a target in the direction of gravitational attraction, appears to offer such an anticipation and adaptation. Representational gravity might contribute to or be an aspect of a broader internal model of gravity (cf. Jörges & López-Moliner, 2017; La Scaleia et al., 2019; Lacquaniti et al., 2015; McIntyre et al., 2001; Zago, 2018; Zago & Lacquaniti, 2005), but the relationship between such an internal model of gravity and representational gravity is not yet clear. Although representational gravity might reflect one component of a larger or more extensive internal model of gravity, it can also be influenced by task-specific or stimulus-specific information that is not related to gravity per se. Given this, it is not surprising that influences of different variables on a target produce effects on representational gravity (or perhaps on an internal model of gravity) that are not completely consistent with influences of those variables on physical gravity experienced by a physical object.

Several different variables have been examined for potential influences on representational gravity. Variables related to the target include size/mass (larger targets exhibit larger downward displacement), velocity (faster descending targets exhibit larger downward displacement), distance traveled (longer distances result in decreased or increased representational gravity for ascending or descending targets, respectively), orientation (vertical-facing direction of a human silhouette does not influence downward displacement), and target modality (representational gravity occurs in visual, auditory, and cross-modal stimuli). Variables related to the display include retention interval (representational gravity increases during the retention interval), response measure (probe judgment reveals representational gravity earlier than does cursor positioning), and height in the picture plane (vertically moving targets higher in the picture plane exhibit decreased representational gravity). Variables related to the context include nontarget intramodal stimuli (visual representational gravity is influenced by proximity to and number of visual nontarget objects and by landmark attraction) and cross-modal stimuli (descending auditory pitch motion increases representational gravity for a horizontally moving visual target). Variables related to the observer include oculomotor behavior (eye movements do not influence representational gravity), body orientation (representational gravity is influenced by body tilt and posture, whether representational gravity is influenced more by the body axis or environmental gravity axis is not yet clear), and psychopathology (representational gravity is found in patients diagnosed with schizophrenia). Additionally, the wider literature on displacement in judged location attributed to invariant physical principles suggests numerous other variables that could be predicted to influence representational gravity, but have not yet been investigated.

Representational gravity is potentially related to numerous perceptual and cognitive tasks or processes. The extent to which judgments of the subjective visual vertical are influenced by representational gravity is not entirely clear, although it is possible that both the subjective visual vertical and representational gravity reflect a more abstract internal model of gravity. Representational gravity combines with other spatial biases in determining overall judged location, and evidence consistent with combinations of representational gravity with representational momentum, representational friction, landmark attraction effect, Fröhlich effect, or the kappa effect has been reported. Such combinations can potentially occur intramodally and cross-modally. To the extent that representational gravity can be modulated by variables or other biases that are not related to physical gravity, then representational gravity would not reflect an accurate internalization of Newton’s laws. However, an accurate internalization is not required, as even an incorrect understanding of physical principles might provide predictions that are sufficiently close to accurate or correct to be useful. Even so, a comparison of representational gravity with physical gravity suggests a few similarities (e.g., effects of height in the picture plane on vertically moving targets), but also a few differences (e.g., effects of body orientation relative to environmental axes). Several previous investigators hypothesized the existence of an internal model of gravity used in representing object location or in interacting with an object, but it is not yet clear whether such a model results from or produces representational gravity.

Representational gravity might be involved in catching and intercepting a moving target, as representational gravity potentially allows extrapolation of location that (at least partially) compensates for delays in subjective awareness of a target due to neural processing times. This type of compensation has been suggested for other types of displacement attributed to effects of environmentally invariant physical principles (e.g., representational momentum; Hubbard, 2005b, 2006a), and it is likely that representational gravity can contribute to such compensation. Representational gravity might contribute to biases regarding the effects of physical gravity in naïve physics and contribute to use of a gravity heuristic that is evoked when trying to intercept or avoid a falling object. Such a heuristic need not accurately incorporate or reflect Newton’s laws (e.g., by separating effects of weight and mass), but might reflect simpler but incorrect notions regarding the effects of gravity (e.g., heavy objects fall faster); as long as such incorrect notions resulted in predictions that allowed satisfactory interactions with stimuli in the environment and continued survival, they could be selected for and would not necessarily be selected against. Representational gravity could contribute to art and aesthetics by providing a mechanism for judgments of visual weight, as such judgments are related to balance and aesthetics of artwork. Additionally, representational gravity could be involved in perception of the dynamics depicted in an artwork, and to the extent that representational gravity matched those dynamics (e.g., signs of strain in lifting or supporting a heavy object), an aesthetic response might be more likely.

Effects consistent with the operation of physical gravity appear to be incorporated into our representations of stationary and moving targets. The presence of such effects suggests constraints for future theories of how environmental dynamics influence our perceptual, cognitive, and motor representations. As demonstrated by the numerous suggestions in Part I regarding potential variables that could be hypothesized to influence representational gravity, but which have not yet been investigated, the notion of representational gravity, as well as the broader notion of an internal model of gravity, has considerable heuristic value. A consideration of representational gravity is also relevant for several areas of more general interest, including perception and action (e.g., catching or intercepting objects), naïve physics, use of heuristics, and art and aesthetics. The ubiquity of representational gravity (and of effects attributable to other environmentally invariant physical principles) suggests such effects might provide a fundamental method of adapting to the environment by aiding observers in responding to environmental stimuli. Importantly, such effects do not necessarily have to reflect an accurate understanding of Newtonian physics, but can include processes and information that produce predictions and expectations that are only approximately correct, as long as such predictions and expectations are generally adaptive and are not selected against. Accurate spatial localization is necessary for many tasks and for successful interactions with stimuli in the environment, and a consideration of displacements such as representational gravity is relevant to spatial localization and can provide a key component of theories of perception, cognition, and action.