It is commonly observed that perceptual experiences, and especially visual experiences, have an egocentric character. In other words, the perceived entities are experienced as being positioned in egocentric relations regarding directions from the perceiving subject. Intuitively, it may seem that if experiences have such an egocentric character, then their content must include both objects and a ‘subject-element,’ i.e., an element to which perceived objects stand in egocentric relations. For example, if a subject S sees a red object to the left and a green object to the right, then it can be proposed that the content of such an experience is: red object in left-relationship to S, and green object in right-relationship to S.

However, whether such an intuitive characterization of perspectival content is correct is highly controversial. First, many authors argue that the subject-element does not figure in the content of visual perspectival experiences at all (e.g., Campbell, 1994, 119–121; Evans, 1982, 232–233; Perry, 1986; Schellenberg, 2007). In other words, it is proposed that visual experiences can represent egocentrically positioned objects without also representing the element to which they are egocentrically related. Second, even among those authors who believe that there is a subject-element in perspectival content, there is no agreement about what the subject-element is and what its properties are. For example, it has been debated whether the subject-element is the ‘self,’ the ‘bodily self,’ or the ‘subject’s location’ (see Schwenkler, 2014; Mitchell, 2021; Peacock, 1992, 30–40; Smith, 2002, p. 145; Zahavi, 1999, 90–110).

The aim of this paper is to present a new argument for the thesis that there is a complex, bodily, subject-element that figures in the content of visual perspectival experiences. More specifically, I argue that (a) visual perspectival experiences represent a subject-element, i.e., an element to which the perceived objects stand in egocentric relations, and (b) the subject-element is represented as a complex bodily structure, i.e., a body composed of parts. In other words, I argue for the following Complex Subject thesis:

(Complex Subject) Visual perspectival experiences represent objects as egocentrically related to a complex bodily subject-element.

By endorsing the Complex Subject thesis, I reject theories, according to which, the subject-element does not figure in the content of visual experience at all, as well as those theories that postulate the presence of a subject-element but do not characterize it as complex and bodily. On the other hand, the Complex Subject thesis is neutral with respect to the specific structure of the perspectival content. For example, it may be the case that experiences represent objects, a complex bodily subject-element, and the egocentric relations between them. However, it may also be the case that experiences represent objects as having complex relational properties that characterize relations to parts of the subject-element. In this case, the subject-element is a component of a complex property possessed by an object, rather than a separate element standing in relation to an object.

Some variants of the Complex Subject thesis have been argued for by showing how perspectival experiences arise through the integration of information from different body-centered frames of reference (e.g., Briscoe, 2009; Grush, 2007), or by postulating that objects are perceived as subjects of possible actions (e.g., Alsmith, 2017; Brewer, 1992). Further, I show that these approaches have certain limitations, and develop a novel argument in favor of the Complex Subject thesis. This argument consists of four steps. First, I argue that visual experiences represent objects as having directional qualities, such as ‘being on the left.’ Second, I show that there are only two plausible interpretations of such directional qualities: (a) representing directional qualities consists in representing objects as having intrinsic, non-relational properties, or (b) representing directional qualities consists in representing objects as related to parts of the bodily structure. Third, I argue that directional qualities play an important role in the selection of actions directed at visually perceived objects. Finally, I show that this role cannot plausibly be played by intrinsic directional qualities but can be played by directional qualities concerning relations to bodily structure. Consequently, the most plausible interpretation of directional qualities is that in representing objects as being to the left or right, visual experiences represent objects as being related to parts of the bodily structure. Thus, in accordance with the Complex Subject thesis, objects in perspectival experiences are represented as egocentrically related to a complex bodily subject-element.

The paper begins with a brief description of current theories of perspectival content (Sect. 1). I then introduce the notion of directional qualities (Sect. 2) and argue, in Sect. 3, that there are two plausible accounts of directional qualities: intrinsic theories, and subject externalism. While intrinsic theories are neutral with respect to the Complex Subject thesis, this thesis is entailed by subject externalism. Finally, in Sects. 4 and 5, by referring to the role of directional qualities in action selection, I argue that subject externalism is more plausible than intrinsic theories.

1 Theories of Perspectival Content

While it may be intuitive to characterize the content of perspectival experiences in terms of objects related to a subject-element, it is quite popular to deny that a subject-element figures in the representational content. One of the main motivations for such a view is phenomenological. While other elements that figure in the representational content of visual experiences ‒ such as objects, their properties, and spatial relations ‒ are associated with a phenomenal character, this does not seem to be the case with the subject-element. This intuition has been formulated in various ways; for example, by claiming that the subject-element is not directly perceived (Schellenberg, 2007), that a subject is not in its own visual field (Perry, 1986), that the content of visual experience leaves the perceiver out of the picture (Brogaard, 2010), or that there is nothing corresponding to ‘I’ in a perceptual experience (Evans, 1982, 232–233).

Authors who deny the presence of a subject-element in the content of visual experiences, usually characterize perspectival content in terms of monadic relations such as ‘A is to the left,’ which specify the object and the egocentric relation without referring to the subject-element. For example, if one perceives a red triangle to the right and a green circle to the left, then the content of such an experience may be a red triangular object to the left and a green circular object to the right, without reference to any subject-element.

Of course, denying that a subject-element figures in the content of visual experiences is not the only option. There are many theories, according to which, visual experiences represent objects in relation to ‘I,’ ‘me,’ ‘self,’ or to the subject’s location (see Schwenkler, 2014; Mitchell, 2021; Peacock, 1992, 30–40; Smith, 2002, p. 145; Zahavi, 1999, 90–110). Such theories are often called ‘self-location theories’ because they postulate that a visual experience locates the subject in relation to objects. Proponents of such theories may claim that even if the subject-element is not associated with a phenomenal character, it can still figure in content. Indeed, it is not necessary to postulate that all experiential content is phenomenal content, and such non-phenomenal content may, for example, allow for the justification of perceptual beliefs about the relations between objects and the subject (see Peacocke, 2001, 215–220; Recanati, 2012). Furthermore, it can be proposed that there are phenomenal, visual differences that are best explained by including the subject-element in the experiential content. In particular, the phenomenal difference between experiencing moving objects around a stationary subject, and experiencing self-motion between stationary objects, may require postulating that visual experiences represent both objects and the self (Schwenkler, 2014, but see Richardson, 2017 for a proposal that this difference may be explained by the structural properties of the visual field).

Nevertheless, accepting some version of self-location theory does not automatically entail accepting the Complex Subject thesis. This is because postulating the existence of the subject-element need not be accompanied by postulating that such a subject-element is bodily and complex. For example, it may be the case that in order to account for the phenomenal difference between object-movement and self-motion, it is sufficient to postulate, as part of the representational content, an atomic, non-bodily subject-element. Indeed, minimally, the subject-element can be understood simply as a geometric point from which directions in the visual field are initiated (see Alsmith, 2017; de Vignemont, 2021 for a discussion).

Theories that explicitly support some version of the Complex Subject thesis, typically adopt two general lines of argument, often in combination. The first approach concerns the way in which perspectival experiences are formed by information-processing mechanisms. In particular, it is argued that the formation of perspectival experiences requires combining information from many reference frames that are embedded in different parts of the body (e.g., Alsmith, 2017; Briscoe, 2009). For example, according to Alsmith (2017), the presence of such frames allows objects to be represented as standing in egocentric relations to various bodily parts that form a single subject-element, i.e., an entire bodily structure. Moreover, it is proposed that egocentric spatial relations, such as determining whether something is to the right or to the left, are not merely visual but multimodal experiential elements represented by combined activities of vision and bodily senses (e.g., Avila, 2012; Briscoe, 2021).

The second approach focuses on the relations between perception and action. For example, it has been proposed that objects in visual experiences are represented as objects of possible actions, and that representing them as such requires representing them as standing in relation to a complex bodily subject-element (see Alsmith, 2017). Similarly, it has been argued that a complex bodily subject-element may be required for vision-based coordination of actions directed at perceived objects (see Brewer, 1992). It has also been proposed that the representation of egocentric relations is a matter of representing certain actions available to a subject (see Grush, 2000, 2007). For example, visually representing the direction ‘to the left’ may consist in representing a certain set of available actions that is different from a set of actions that constitute the relation ‘to the right.’

I believe that focusing on bodily information, and connections between action and perception, is generally a correct way to argue for the Complex Subject thesis. However, the usual ways of arguing for this thesis have certain limitations. First, processing and integrating information related to different reference frames is largely a subpersonal process, and there is no automatic transition between subpersonal information and experiential content. For example, it seems possible that the integration of information from certain body-centered reference frames leads to the creation of a unified visual experience whose content, however, does not contain a subject-element, but only monadic egocentric relations. Second, the postulate that objects are represented as objects of possible actions presupposes a particular view of visual content, according to which, action-related properties are genuine perceptual properties. As evidenced by the ongoing and hard-to-resolve debate about high- and low-level perceptual content, such an assumption is potentially controversial (e.g., Nanay, 2011; Prinz, 2013; Reiland, 2014). Third, it is not obvious that action-related aspects of perception cannot be adequately captured without postulating content containing a subject-element. For example, one can postulate that representing an object as standing in an egocentric relation, such as ‘to the left,’ consists in representing this object as available for actions A1 and A2 without representing these actions as actions of a certain bodily subject. Schellenberg (2007), in particular, has developed an account of perspectival experiences in which objects are represented as both objects of action and objects of perception, simply by being represented as standing in monadic egocentric relations.

My goal is to formulate a new argument in favor of the Complex Subject thesis that is free from the above limitations. First, it relies on certain properties of phenomenal visual content rather than properties of subpersonal processes. Second, it does not assume that objects are represented as having action-related properties, but merely that egocentric visual content plays an important role in action selection.

2 Directional Structure

In ordinary visual experiences, each perceived object is experienced as being positioned in an egocentric direction. Such a direction is specified by two aspects. The first is the ‘relational aspect,’ which characterizes the position of a given direction within the spatial structure of the visual field. For example, an object may be perceived as being positioned in a direction that is mid-way between the center and the edge of the visual field. Overall, egocentric visual directions form a topologically connected structure, such that between any two directions Dx and Dy, there is a chain of directions where Dx is the first element of the chain, Dy is the last element, and each element of the chain is spatially connected to the previous one. The topological arrangement of egocentric directions is a structural aspect of visual experience in the sense that the arrangement of directions, determined by connectedness relations, remains the same no matter what objects are perceived.

The second aspect is the ‘qualitative aspect,’ which characterizes whether an object is to the left, right, up, down, or has another directional quality that has no well-established name in natural language. Later, for simplicity, I will focus on ‘left’ and ‘right’ qualities, since ‘up’ and ‘down’ qualities are ambiguous, as they can mean, not only directions in the structure of the visual field, but also the direction determined by the sense of gravity.

The arrangement of directional qualities is partly determined by the topological arrangement of visual directions. For example, a quality of being maximally ‘to the right’ can only characterize a direction that is at the edge of the visual field. However, it is not the case that such a determination is complete. Let us imagine a horizontal line running through the center of the visual field from one edge to the other. On such a line, there are two points in the directions Dx and Dy, which are located mid-way between the center and the edge of the visual field, in such a way that Dx is to the right, and Dy is to the left. However, the relational positions of Dx and Dy within the topological arrangement of egocentric directions does not determine which is to the left or the right. This is because the relations in which Dx and Dy stand to other directions within the visual field are symmetrical. For example, both Dx and Dy are at distance P from their near edge and from the center, and at greater distance 2P from their far edge. Consequently, there is no relational difference that would determine whether Dx or Dy should be qualified as left or right.

The attribution of directional qualities is also independent of the properties of the objects experienced. It is not the case that only objects with certain properties, such as certain colors or shapes, can be experienced as being to the left. On the contrary, virtually any perceptual object can be experienced as being to the left or right. In fact, the arrangement of directional qualities, at least in normal visual experiences, is independent of the scene being perceived. For example, in any ordinary experience, there is something that is maximally to the left, minimally to the left, and halfway to the left, between the center of the visual field and its edge.

In further considerations, I assume that directional qualities are perceptually represented, and not, for example, represented by virtue of some postperceptual state. I believe that there are good prima facie reasons for treating directional qualities as perceptual properties. First, there seems to be a perceptual, phenomenal difference between seeing something as being to the left and seeing it as being to the right. For example, a visual scene consisting of a single point of light on a black background near the left edge of the visual field seems to look different from an analogous scene in which the light is near the right edge. Second, the perception of directional properties seems to be independent from background knowledge. For example, if one sees something as being to the left, one is likely to continue to see it as being to the left, even after receiving reliable propositional information that this is merely an illusion created by an arrangement of mirrors. Third, the perception of directional properties is likely to arise from interactions between visual and bodily sensory mechanisms, rather than from cognitive mechanisms (see Briscoe, 2009, 2021; Grush, 2000, 2007).

It should be noted that assuming that directional qualities are perceptual properties does not mean postulating that different directional qualities are intrinsically different, as, intuitively, the perceptual property ‘red’ is intrinsically different from the perceptual property ‘green.’ In fact, I will later, explicitly argue against such a view (see also Chalmers, 2019; Simon, 2021 for other arguments, referring to the possibility of right/left reversed worlds, against intrinsic differences between directional qualities).

Furthermore, I assume that directional qualities are not only perceptual properties but are also represented in conscious perceptual experiences. One might doubt this by postulating that since directional qualities constitute the egocentric aspect of experiences, they are represented unconsciously by activities of the dorsal, rather than the ventral, visual stream. The main role of the dorsal stream is to guide actions based on egocentric information, while the main role of the ventral stream is to represent allocentric arrangements of objects to allow conscious identification of perceived entities. However, there are strong arguments against the strict division between (a) dorsal, unconscious, and egocentric aspects of perception, and (b) ventral, conscious, and allocentric aspects.

First, perceptual constancy phenomena suggest that we can consciously perceive both the view-invariant properties of objects, as well as their perspectival properties, based on the processing of egocentric information (Brogaard, 2012; Wu, 2014). Thus, egocentric information does not appear to be restricted to dorsal, unconscious processing. Second, while the processing of egocentric information for fine-grained action guidance may not require the formation of conscious representations, conscious egocentric representations are likely to be used in the selection of a type and general course of action (Briscoe & Schwenkler, 2011; Kozuch, 2015). Third, the presence of certain illusions − in the context of ventral processing, but not in the context of dorsal processing − is interpreted as evidence that both streams process egocentric information, but the ventral stream also integrates egocentric data with other types of spatial information (Briscoe, 2009). Overall, it is likely that egocentric information is processed by both visual streams, and that there are conscious perceptual states that represent egocentric properties.

3 Theories of Left and Right

Philosophical theories of properties such as ‘left’ and ‘right’ have focused mainly on properties of spatial objects, such as gloves or screws, rather than on directions in visual space. However, as I show below, they can be adapted to considerations on perspectival perception. Three of these positions have received the most attention: internalism, holism, and externalism (see van Cleve, 1987; Cutter, 2001; Lee, 2006 for discussion). According to internalism, the difference between a left glove and a right glove consists in a different pattern of relations between the parts of each glove. The second position, holism, claims that each glove instantiates a distinct, intrinsic (i.e., non-relational) property: ‘rightness’ or ‘leftness.’ Finally, externalism claims that the ‘rightness’ or ‘leftness’ of a glove consists in its relations to other entities.

In the context of perceptual directional qualities, the internalist position would mean that to represent entities as having such qualities is to represent them as having a certain internal structure. For example, certain arrangements of the parts of an object would constitute a ‘left’ quality, and other arrangements would constitute a ‘right’ quality. However, internalism is not plausible in the perceptual context. In particular, visual objects can be experienced as being to the left or as being to the right, regardless of their internal structure. Similarly, it is not the case that fragments of the visual field located to the right have a different internal structure than fragments located to the left. For example, both to the left and to the right there are circular visual field fragments with diameter D.

The perceptual version of holism postulates that to represent an object or place as being ‘to the left’ is to ascribe to it a monadic, intrinsic property of ‘leftness,’ which is distinct from other intrinsic properties such as ‘rightness.’ Although I will later argue against holism, I believe that unlike internalism, holism ‒ as an account of directional qualities − cannot be rejected out of hand. Indeed, within the analytic tradition, it has been endorsed by philosophers such as Russell (1948), Goodman (1977), and Casullo (1986).

The last option ‒ externalism ‒ can have several variants. The first is ‘object externalism,’ according to which, representing something as having directional qualities consists in representing it as standing in certain relations to other entities presented in the visual field. However, this version of externalism is not very plausible. As characterized in the previous section, the distribution of directional qualities within the visual field does not depend on the arrangement of perceived objects. In particular, no matter what objects are seen, things are ‘to the left’ in one half of the visual field, and ‘to the right’ in the other half.

The second version is structural externalism, which postulates that representing something as having directional qualities consists in representing it as standing in certain relations to the structural elements of the visual field, such as its boundaries or directions constituting it. The problems with this position have been analyzed in detail by Casullo (1986), who argued that structural externalism has, in fact, to assume a non-relational, intrinsic characterization of directional qualities. To observe this problem, let us imagine a horizontal line passing through the center of the visual field from one boundary to another. Given this, we can consider a direction M1, which is located mid-way between one boundary and the center of the visual field, and a direction M2, which is located mid-way between the center of the visual field and the opposite boundary. Let us assume that M1 is experienced as being on the left, while M2 is experienced as being on the right. If, as structural externalism assumes, possessing directional qualities consists in standing in relation to structural elements of the visual field, then such relations should be different in the case of M1 and M2. However, this does not seem to be the case, since M1 and M2 stand in the same pattern of structural relations. In particular, each of these directions stands at a distance D from the center of the visual field, at a distance D from one boundary, and at a distance 2D from another boundary.

A solution to this problem may be to postulate that M1 is at distance D from the ‘left’ boundary and at distance 2D from the ‘right’ boundary, while M2 is at D from the ‘right’ boundary and at 2D from the ‘left’ boundary. Nevertheless, this obviously means accepting a form of holism by attributing intrinsic directional qualities to directions positioned on the visual boundaries. An alternative is to differentiate M1 and M2 by introducing asymmetric relations (see Casullo, 1989; Falkenstein, 1989 for a discussion). For example, M1 may stand in an asymmetric relation of being in direction A from the center, while M2 stands in another asymmetric relation of being in direction B from the center. However, this also requires the introduction of some intrinsic directional properties, this time attributed to relations between visual directions, which distinguish the relation of being in direction A from the relation of being in direction B and determine that, for example, standing in the relation of being in direction A is equivalent to being to the left. To sum up, the above proposals, together with the standard holism characterized earlier, share the common postulate that accounting for directional qualities requires attributing some intrinsic property to visual directions or to relations between visual directions. For this reason, I will refer to such approaches as ‘intrinsic theories.’

There is, however, another variant of externalism that is not a form of intrinsic theory. This is ‘subject externalism,’ according to which, representing something as having directional qualities consists in representing it as standing in relations to the subject. Of course, such an externalism should not postulate that the subject is a simple, partless entity, since in that case, there would be no resources for explaining how the directional quality of ‘being to the left’ differs from the directional quality of ‘being to the right.’ A more promising version of subject externalism is to propose that representing directional qualities consists in representing relations to parts of the bodily structure. For example, the fact that M1 is experienced as being on the left may mean that M1 is represented as being closer to one of the hands, while the fact that M2 is experienced as being on the right may mean that M2 is represented as being closer to the other hand.

An initial concern might be that due to the symmetrical structure of the human body, such a version of subject structuralism requires the introduction of intrinsic directional qualities that would determine which hand is ‘left’ and which is ‘right’ (see Cutter, 2001). In particular, it seems that both M1 and M2 represent the same pattern of relations, since each of these elements is at some distance D from one hand, and also at some greater distance P from the other hand. To distinguish these patterns of relations, it may be necessary to say that M1 is at distance D from the ‘left’ hand, while M2 is at distance D from the ‘right’ hand.

However, I believe that this is not the only possibility. A common claim in the literature on bodily perception is that the body is represented as divided into parts based on proprioceptive and kinesthetic information provided by joints (e.g., Bermúdez, 1998; O’Shaughnessy, 1998; de Vignemont, 2014). From this perspective, two hands may differ, not because they possess intrinsic properties such as ‘left’ and ‘right,’ but because of differences in the joints that designate them. For example, hand A can be designated by joints J1 and J2 (corresponding to what we would call the left wrist and left shoulder), and hand B can be designated by joints J3 and J4 (corresponding to what we would call the right wrist and right shoulder). From this perspective, representing M1 as being on the left consists in representing M1 as being closer to the hand designated by J1 and J2, and farther from the hand designated by joints J3 and J4 (analogous to M2 being on the right). Ultimately, such a proposal may require the introduction of some intrinsic properties − in particular, distinguishing between particular joints − but it does not require postulating intrinsic directional qualities, and does not require attributing intrinsic properties to visual directions or the relations between them. Consequently, this version of bodily externalism is not an intrinsic theory.

Of course, the proposal that directional qualities, such as ‘left’ and ‘right’, concern relations with hands is a simplification. A more realistic version of subject externalism might propose that the representation of directional qualities consists in representing a complex set of relations to different parts of the body, or that it consists in representing relations to some higher-order bodily structures such as the left or right sides of the body. But these details have no bearing on my argument, and I will characterize bodily externalism more generally as a position, according to which, representing directional qualities consists in representing relations to parts of the bodily structure.

However, one additional clarification is needed. An important distinction, in the literature on bodily representations, is that between ‘online’ and ‘offline’ bodily representations (see Carruthers, 2008; O’Shaughnessy, 1998; Proske & Gandevia, 2012). Online representations, based on current proprioceptive and kinesthetic input, represent the arrangement of body parts at a given moment. Offline representations concern the stable relationships between body parts, such as the hand being connected to the torso, which are not affected by the current posture. In contrast to online representations, they are less dependent on sensory input at a given moment and may be shaped in important ways by innate and early developmental factors. It seems more plausible to develop subject externalism in such a way that postulated relations to bodily structure are relations to structure as characterized by offline representations, since changes in posture ‒ such as crossing hands ‒ do not seem to rearrange the distribution of directional qualities. Furthermore, there is no evidence to show that people cannot visually discriminate what is left from what is right when inputs from the body are unavailable. Such observations would be difficult to explain if directional qualities concerned relations to online body structure but are not a problem in the case of an offline structure that does not depend on current input. I am not suggesting that online bodily representations and current input from bodily senses have no effect on directional qualities (see Peacock, 1992, p. 62). However, despite such possible influences, if one accepts subject externalism, relations to offline structures should be treated as a major factor shaping directional qualities.

In addition, it should be noted that if subject externalism is true, then the perspectival content of visual experience is a form of multimodal content because it is generated, in part, by relying on information processed by the bodily senses and stored in long-term bodily representations. However, this conclusion is plausible given the current state of psychological knowledge, as there is considerable evidence that egocentric visual perception requires the contribution of bodily senses (see Alsmith, 2017; Briscoe, 2009, 2021 for a review).

The distinction between intrinsic theories and subject externalism has a straightforward consequence for the Complex Subject thesis. If subject externalism is true, then to represent something as ‘to the right’ or ‘to the left’ is to represent that something as standing in a certain pattern of relations to the parts of the bodily structure. Consequently, complex subject figures in the content of perspectival experiences, and so the Complex Subject thesis is true. On the other hand, the adoption of an intrinsic theory has no such consequences. In particular, if representing that something is to the left consists in representing it as having ‘left’ intrinsic property, then such a situation says nothing about the presence or absence of a subject in the content of experiences. In the next section, I will argue that subject structuralism has an important advantage over the intrinsic theories, and therefore, the Complex Subject thesis is plausible.

4 Action Selection

Some actions based on visual information ‒ such as dodging a ball thrown at us ‒ may occur automatically without conscious control. However, there are common situations in which an action is first consciously selected based on visual experience, and then executed. It seems very plausible that directional qualities play an important role in the selection of actions based on visual experiences. While the fine-grained control of visually guided actions may rely mainly on unconscious dorsal content, the selection of actions seems to be determined, to an important extent, by conscious content, including directional qualities. For example, based on the perception that something is ‘to the left,’ a person may decide that the object is easier to grasp with the left hand, or that to approach the object, a person must turn the whole body to the left.

Of course, this is not to say that directional qualities are sufficient for visual action selection. Other relevant factors include knowledge of the general capabilities of the body, the perception of the allocentric arrangement of objects in the visual field, and a sense of current posture of the body. Furthermore, I do not postulate that directional qualities are necessary for all cases of vision-based action selection. However, I do believe that it is very likely that in many common cases, information provided by directional qualities is used to successfully select appropriate actions. More specifically, directional qualities seem to be particularly useful in determining the part of the body that should be used to perform an action.

For example, suppose there is an object in my field of vision that I want to grasp. I may have information, provided by various bodily representations, that my hands have a certain maximum reach and that they are not currently crossed. I may also have information, from visual experience, that the object is at a certain allocentric distance from the center of the visual field and at a certain egocentric distance from me. However, even with all this information, I still cannot decide whether it is easier to grasp the object with my left hand or with my right hand. It seems that in many ordinary situations, this information gap is filled by the information provided by the directional qualities. Furthermore, if in many cases the information provided by directional qualities helps to select a body-part to engage in an action, then it is likely that this information concerns the relationship between a perceived object and the structure of the body. Such information, e.g., that an object is closer to the right hand, might plausibly allow the selection of an action involving that hand.

Overall, directional qualities seem to provide information about the spatial relationships between perceived objects and the subject’s body parts which, together with other types of information, is commonly used in the conscious selection of actions. Based on these considerations, I believe that the following Action Selection thesis can be plausibly accepted:

(Action Selection) There are common cases in which directional qualities contribute to the conscious selection of actions based on visual experiences by providing information regarding the spatial relationship between perceived objects and parts of the subject’s body.

Bodily externalism can easily account for the Action Selection thesis. According to this approach, directional qualities provide information useful for action selection simply because representing something as having directional qualities is representing something as standing in spatial relations to parts of the bodily structure. On the other hand, it is not obvious how the Action Selection thesis can be true if an intrinsic theory is adopted. If, as holism postulates, representing something as having directional qualities is representing something as having some intrinsic properties, then it is difficult to explain why such properties can inform us that, for example, an object is more easily grasped by a particular hand. In this case, attributing a directional property to an object only means that it is characterized as having a certain intrinsic property ‘left,’ which is different from other directional properties such as ‘right.’ There is no more reason to postulate that such an intrinsic directional quality can help in action selection than, for example, an intrinsic quality of ‘red’ or ‘green.’

The analogous problem arises when another variant of intrinsic theory is adopted; for instance, by postulating that characterizing an object as being to the left consists in representing it as being closer to a boundary with a ‘left’ intrinsic property, or in representing it as standing in an asymmetric relation with a ‘left’ intrinsic property. These intrinsic qualities do not inform about the relation to the body structure, and therefore, do not help in choosing actions. In the following, I will focus on holism, i.e. an intrinsic theory, according to which, directional qualities simply are intrinsic properties. However, the same problems can be reformulated in terms of different variants of the intrinsic theory.

It should be noted that the above difficulty in accounting for the Action Selection thesis occurs even if intrinsic directional qualities are represented by processing body-related information, e.g., by integrating information from different body-related frames of reference. If such body-related information is processed subpersonally and contributes causally to the representation of directional qualities ‒ but is not present at the level of experiential content, as in intrinsic theories ‒ then it cannot contribute to conscious action selection. Conscious action selection is a rational activity that is not simply caused by a visual experience, but uses information provided by experiential content to choose one action over possible alternatives. If relevant information is not present in experiential content but is only processed subpersonally, then it cannot serve as evidence for choosing a particular action.

A proponent of intrinsic theory may attempt to explain the Action Selection thesis by proposing that while intrinsic directional qualities themselves do not help in the selection of actions, they are able to do so because they are systematically associated with some additional content. For example, it may be the case that when an intrinsic directional quality L is attributed to an object A, the content has the following form: object A has L and A is closer to body part P1 than to body part P2. While the directional quality L is an intrinsic property that does not concern relations to the bodily structure, there is also an additional content that occurs together with the representation of L ‒ let us call it the ‘interpretation of L’ ‒ which specifies that the object having L is closer to the bodily part P1 than to the bodily part P2. The Action Selection thesis is satisfied because when an intrinsic directional quality is represented, it is represented together with its interpretation, which provides content relevant for action selection.

Of course, the crucial question concerns the nature of such interpretations of intrinsic directional qualities. It should be noted that proponents of intrinsic theories cannot simply postulate that the content relevant to action selection concerning relations to bodily structure is provided partly by intrinsic properties ‒ such as directional qualities ‒ and partly by their interpretations. Since they are obliged to claim that directional qualities have no body-related content, all body-related content must be the content of interpretations. Consequently, a plausible theory of such interpretations must satisfy four conditions. First, the postulated interpretations must be such that they represent relations between visually perceived entities and the structure of the body. Second, the content of the interpretations must be accessible to a subject so that it can inform a rational choice of action. Third, if for some reason interpretations are unavailable, it should result in serious disturbances in the abilities for action selection. This is because without interpretations, intrinsic directional qualities do not provide information about the relations of objects to bodily structure. Fourth, changes in associations between interpretations and directional qualities should lead to changes in selected actions. Again, this is because, according to intrinsic theories, directional qualities are not associated with any content regarding relations to bodily structure, and their relevance for action selection depends entirely on the interpretations associated with them. Below, I will consider several possible accounts of interpretations and argue that none of them plausibly satisfies all the above requirements.

5 Types of Interpretations

5.1 Perceptual Interpretations

First, the content of interpretation can be perceptual content. However, if contents of interpretations are contents of visual experiences, then bodily structure figures in the contents of visual experiences, and thus a version of the Complex Subject thesis is true. Consequently, I will not discuss this possibility further.

Alternatively, it can be proposed that the contents of interpretations are perceptual, but are not contents of visual experiences. The most plausible idea is to propose that these contents are contents of bodily proprioceptive, kinesthetic, and tactile experiences since these experiences are likely to have content regarding bodily structure. However, this immediately raises a problem, because while bodily experiences may represent the structure of the body, it is unlikely that they can have contents that characterize the relations between the body and visually perceived objects. As argued in Sect. 3, there are good reasons to accept that such relational content is multimodal content, produced jointly by visual and bodily mechanisms, and included in the content of ordinary visual, perspectival experiences. However, the view that visual experiences have multimodal perspectival content that characterize relations to the subject’s body entails the Complex Subject thesis.

5.2 Cognitive Interpretations

The above problem is not present if interpretations of intrinsic directional qualities are cognitive, since there is no reason why beliefs cannot have contents that characterize relations between visually perceived objects and bodily parts. Nevertheless, this option is not particularly plausible either, since it does not seem that in selecting actions that depend on whether something is perceived to be ‘to the left’ or ‘to the right,’ we need to formulate a belief about the relations between an object and, for example, our left and right hands. This is not to deny that there may be cases ‒ especially when actions are quite complex ‒ in which we must engage in some reasoning about how to move our body. However, in many ordinary situations, it seems that to decide, for example, whether to grasp an object with the left or the right hand, we simply rely on directional qualities without representing the relation to the body structure in a belief. Consequently, it seems that even when cognitive interpretations are not used, action selection based on directional qualities can often be successful.

Furthermore, there is reason to believe that changes in cognitive interpretations do not significantly affect action selection. This can be illustrated by cases involving the use of optical rearrangement devices. Consider a possible device that produces a mirror image of the visual field so that objects that are closer to the left hand are now experienced as being ‘to the right,’ and vice versa. It seems plausible ‒ and confirmed by reports of the use of similar devices (see Hurley, 1998, 285–287, 346–349 for a review) ‒ that when people begin to experience such reversals they become confused and often choose actions inappropriately. This happens even if a person knows how the device works and is thus able to form an appropriate cognitive interpretation of the directional qualities, e.g., that the directional quality of being ‘to the left’ is associated with being closer to the right hand. However, if the role of directional qualities in action selection is only to be associated with cognitive interpretation, then it is not clear why the use of rearranging devices is initially so confusing since correct beliefs are available. It is rather the case, as suggested by subject externalism, that directional qualities themselves provide information that is confusing in the case of rearrangement devices and cannot be easily countered by having correct beliefs.

Similar problems arise when trying to characterize interpretations in terms of imagery or memory. It does not seem that every case of action selection that relies on directional qualities requires imagining the relation between an object and the body or recalling such a relation from memory. Furthermore, if associating imagery- or memory-contents with directional qualities were required for successful action selection, then the ability to select actions should rely on the proper functioning of these mental faculties. However, there is no evidence that aphantasia ‒ the lack of the ability to form conscious imagery, or disturbances in the functioning of memory ‒ has consequences for the selection of bodily actions based on perspectival experiences (see Blomkvist, 2022 for a review of empirical results).

5.3 Motor Interpretations

Finally, it can be proposed that interpretations are motor representations. Such representations are often characterized in terms of a ‘forward model’ and an ‘inverse model’ (see Kawato, 1995; Wolpert & Ghahramani, 2000). The function of the forward model is to determine how motor commands should be executed to successfully complete the planned action. The inverse model specifies the expected outcome of motor commands characterized by the forward model (e.g., proprioceptive input after movement). Based on this distinction, it can be argued that the contents of interpretations that are systematically associated with directional qualities specify the movement patterns of certain body parts and their states at the end of the movement. In this sense, motor interpretations represent how a body part is related to a spatial position in which a visually perceived object is located. Nevertheless, the motor characterization of interpretations has problems in satisfying the remaining three conditions required for a plausible theory of interpretations of intrinsic directional qualities.

Firstly, forward and inverse models are representations used primarily for fine-grained guidance and control of actions. Consequently, they are likely to be associated with dorsal processing of spatial information and the majority of their content may not be consciously available. In fact, it does not appear that we have conscious access to details about the proper execution of movement when selecting actions. Therefore, it is not clear whether motor content enters conscious experience in a way that can be used by a subject to select actions based on directional qualities.

Second, in the case of severe proprioceptive disturbances resulting in deafferentation, the processing of motor information required to formulate forward and inverse models is severely disrupted. However, while this obviously affects the ability to perform actions, descriptions of clinical cases of deafferentation do not suggest the presence of problems with conscious action selection based on visual experience (see Gallagher, 2004; Gallagher & Cole, 1995). The lack of difficulties with action selection suggests that information about relations to bodily structure provided by motor representations is not required for visual-based action selection, and thus, the role of directional properties in action selection cannot be fully explained by their association with motor interpretations.

Finally, there are reasons, provided by detailed studies of prism adaptation, to believe that changes in motor interpretations do not fully determine changes in action selection. Prism adaptation is probably the most studied phenomenon related to optical rearrangement devices (see McIntosh et al., 2019; Petitet et al., 2018; Redding & Wallace, 2006 for reviews). For example, when a person wears prism glasses that shift the retinal image to the left, that person initially makes errors in tasks, such as pointing to an object. For example, if she tries to point to an object that is visibly straight ahead, she will miss it because it is actually to the right (in the case of the prismatic shift to the left). However, with practice, behavioral responses become accurate again, but when prism glasses are no longer used, aftereffects often occur. For example, when trying to point to an object that is straight ahead, a person without glasses will point slightly to the right, opposite the direction to the earlier prismatic shift.

Two main types of aftereffects are reported. The first are sensorimotor aftereffects related to the performed motion pattern. They can be tested by a task that requires pointing straight ahead (preferably in the dark, to avoid interference from the spatial content of the visual experience). Second, are perceptual aftereffects, often tested by ‘line bisection tasks’: a person viewing a bisected line is asked to judge whether the line is bisected at the center or to the left or right of its center. For example, after the earlier prismatic shift to the left, a person may judge that a line bisected in the middle is actually bisected to the right of its midpoint.

Perceptual aftereffects seem to be closely related to the ability to choose actions based on directional qualities. In particular, if one is able to perceive that a line is bisected to the right of its midpoint, then it is very likely that one is also able to select a certain action based on that perceptual information, e.g., to point to the bisection point with the right hand. Therefore, I assume that perceptual aftereffects are a reliable indicator of the effects of prism adaptation on action selection based on directional qualities.

On the other hand, motor aftereffects are likely to arise from motor representations that guide bodily actions in egocentric space. Such representations are plausible candidates for motor interpretations of directional qualities. When a person uses prismatic glasses, the associations between motor representations and directional qualities change. For example, prior to a prismatic shift, a directional quality Q is associated with the ‘center’ motor pattern, and so, when prism glasses are first used, initial actions are inaccurate because of this association. However, later, in the case of prismatic shift to the left, the quality Q starts to be associated with the ‘right’ motor pattern, and thus, actions become accurate. After removing the glasses, this change leads to motor aftereffects: due to the remaining association of Q with the ‘right’ pattern, pointing movements aim to the right of the actual object position.

As noted above, changes in motor representations caused by prism glasses are likely to cause motor aftereffects. Furthermore, if directional qualities are intrinsic, and their body-related content relevant to action selection is provided by motor interpretations, then changes in motor representations due to prism glasses should affect action selection based on directional qualities. As argued above, perceptual aftereffects are a reliable indicator of changes in action selection. Consequently, if the body-related content associated with directional qualities is provided by motor interpretations, then changes in the motor representation should determine both motor and perceptual aftereffects.

Furthermore, changes in motor representation should affect both types of aftereffects in similar ways, such that motor and perceptual aftereffects are highly correlated. For instance, if a motor representation associated with directional quality Q changes from a representation specifying a ‘center’ motor pattern to a representation specifying a ‘right’ motor pattern, then when one attempts to point to an object having Q, one will point to the right and not straight ahead (i.e., a motor aftereffect occurs). Similarly, if motor representations provide information relevant for conscious selection of actions, then changes from a ‘center’ motor representation to a ‘right’ motor representation should mean that when one wants to point to an object having quality Q, one chooses a ‘point to the right’ action, and not a ‘point to the center’ action. This change should be reflected in perceptual aftereffects, i.e. if a person chooses a ‘point to the right’ action when an object has Q, then it is plausible that the person judges, based on this directional quality, that the object is ‘to the right.’ Otherwise, a situation would occur in which a person recognizes, by relying on a directional quality, that an object is straight ahead, but nevertheless chooses a ‘point to the right’ action. The presence of such situations is not intuitively plausible, and their occurrence is not suggested by the results on prism adaptation.

However, one of the main findings of prime-adaptation studies is that motor and perceptual aftereffects are not highly correlated (see Colent et al., 2000; Fortis et al., 2011; McIntosh et al., 2019; Michel et al., 2003). In particular, motor aftereffects are symmetric, i.e., they occur in a similar manner regardless of whether the prismatic shift is ‘to the left’ or ‘to the right.’ Perceptual aftereffects, on the other hand, are generally weaker than motor aftereffects and are asymmetric: they occur mainly when the prism is shifted to the left, but not when it is shifted to the right. Although the reasons for these differences are not yet fully understood (see Striemer & Danckert, 2010), the lack of correlation between motor and perceptual aftereffects suggests that motor representations do not fully determine the role of directional qualities in action selection. For example, even if the motor representation associated with directional quality Q has changed after the prismatic shift, the lack of perceptual aftereffects suggests that the role of Q in action selection has remained largely the same. Consequently, there seems to be some information, relevant for action selection, that is provided by directional qualities but not by the associated motor representations.

Overall, the above considerations suggest that the role of directional qualities in action selection is unlikely to be fully explained by postulating interpretations associated with intrinsic directional qualities. This is because directional qualities can play a role in action selection even when such interpretations are not available, and changes in the associations between directional qualities and interpretations do not fully determine the role of directional qualities in action selection. Moreover, perceptual interpretations that could account for the role of directional qualities in action selection would themselves lead to some form of the Complex Subject thesis.

6 The Role of Directional Qualities

The above considerations show that intrinsic theories cannot easily account for the Action Selection thesis, even if directional qualities are systematically associated with certain interpretations. Moreover, these problems are not easily resolved by postulating that the role of intrinsic directional qualities in action selection can be explained by a combination of several interpretations. For example, let us assume that conscious action selection is determined by a combination of cognitive and motor interpretations. If such a combination involves a conjunction of cognitive and motor content, then the absence of cognitive or motor interpretations should cause a serious disruption to action selection based on directional qualities, since a significant part of the relevant content would be unavailable. However, as argued earlier, conscious action selection can occur in the absence of cognitive or motor interpretations. Alternatively, it can be postulated that the considered combination does not have the form of a conjunction, but is a synthesis of cognitive and motor content resulting in new content. In this case, however, the question arises as to the nature of this new synthetic interpretation. If it is perceptual, cognitive, imagery, memory, or motor, then this new interpretation faces the problems characterized earlier. If, on the other hand, it is a new form of interpretation, then it is unclear exactly what that form is, because when we select actions based on visual experience, it does not seem that we are also engaging some additional, novel, mental faculty.

This is not to say that cognitive, motor, or other interpretations do not provide content that influences how actions are selected on the basis of directional qualities. However, there does seem to be content regarding relations to bodily structure that is not provided by such interpretations. If the role of directional qualities in action selection is not plausibly accounted for by intrinsic directional qualities associated with interpretations, then it is likely that directional qualities are not intrinsic, but that they themselves characterize some relations between visible objects and body structure. To accept this is to accept a form of subject externalism, and consequently, to accept a form of the Complex Subject thesis.