1 Introduction

Virtual worlds raise several important philosophical questions, and their increasing sophistication, accessibility, and prominence in society will only make these questions more pressing and complex. What kind of reality do virtual worlds have? What is the moral status of actions that occur in them? Could there be a virtual world indistinguishable from the real one? If so, how can we know we’re not in a virtual world right now? One of the most notable recent contributions to this area of philosophical thought is David Chalmers’ 2022 book Reality+: Virtual Worlds and the Problems of Philosophy. Through the lens of virtual reality and virtual worlds, Chalmers gives a fascinating, wide-ranging, eminently readable introduction to and discussion of many of philosophy’s perennial questions. But it is intended to be more than an introductory textbook and should be taken seriously as a statement of Chalmers’ philosophy. This paper critically responds to Chalmers’ analysis of virtual reality and virtual worlds by focussing on two important components of it: the concept of immersion, and his distinction between virtual reality and video games. Significant to Chalmers’ analysis is the idea that what is distinctive about virtual reality (VR) is that its virtual worlds are immersive, while the virtual worlds of two-dimensional, non-VR video games are not immersive. What is decisive for Chalmers is that VR is perceptually immersive – we perceive the virtual worlds of VR as three-dimensional, surrounding us, with ourselves at the centre (2022, 193). Video games can be psychologically immersive because they can intensely occupy our attention, but they can never be properly immersive as VR is because they are not three-dimensionally perceptually immersive.

The idea that the virtual worlds of 2D, non-VR video games are not immersive is problematic, and in saying this, Chalmers treats them unfairly – despite his clear appreciation of them. Chalmers is clear (and correct) about the fact that there are different kinds of immersion and they come in degrees. But immersion is a much more complicated phenomenon than he seems to allow for, and multiple kinds of immersion are present in and caused by video games to some degree – including perceptual immersion. For the sake of making this article easier to read, I will sometimes refer to two-dimensionally represented, non-VR video games simply as ‘video games’. But to be clear, the type of video games Chalmers has in mind in his distinction of VR from video games is 2D, non-VR video games. Not all video games are two-dimensional and non-VR, but these are what he has in mind when he makes this distinction.

Ultimately, I will argue that Chalmers does not capture what is distinctive about VR, does not distinguish it adequately from non-VR video games, and that his general analysis in Reality + suffers as a result. Even though Chalmers uses the word ‘immersive’ in a particular sense, video games absolutely deserve to be described as immersive. To say otherwise is misleading, and obscures several aspects of what makes them such a compelling artform, ultimately failing to do them justice. But beyond this, I will argue that considering the problems with Chalmers’ account can lead us to a more productive way of conceiving immersive experience, and our cognitive engagement with video games and virtual worlds more broadly. I will begin by outlining the relevant aspects of Chalmers’ arguments about VR and virtual worlds. I will then move towards a discussion of video games and immersion which adopts the four-way distinction between participatory, affective, narrative, and representational immersion presented in Magdelena and Brendan Balcerak Jackson’s Immersive Experience and Virtual Reality. I discuss briefly how video games manifest the first three types of immersion before focussing in on representational immersion, which the Balcerak Jacksons acknowledge has both perceptual and non-perceptual elements.

Perceptual immersion is the kind of immersion Chalmers has in mind as distinctive of VR. But here, I will develop the idea of non-perceptual representational immersion as a way of criticising Chalmers’ analysis. Non-perceptual representational immersion includes the non-perceptual elements of the virtual world’s representation, including non-perceptual information about the world’s structure, systems, and behaviour, the cognitive states generated in the user, such as thought, reasoning, knowing, or understanding, and the immersion which comes from our constitutively cognitive, active engagement with the virtual world. This type of immersion is crucial to achieving the feeling of being present in a virtual world, which Chalmers misses out in his characterization, and which many 2D, non-VR video games are rich in. I illustrate this by discussing a type of video game commonly referred to as ‘immersive simulations’, such as the games of the Deus Ex series.

2 David Chalmers’ Reality + and the Stakes of This Study

I am offering a critique of some of the views expressed in Reality+, but there is a lot to like about it. It works well as a wide-ranging introduction to philosophy which connects many of philosophy’s most important problems to timely societal concerns and is written in a remarkably engaging manner. It is also, as Lassiter/Kagan note, to be commended “for venturing beyond the typical figures of the Western canon” and being “an object lesson towards greater inclusivity in contemporary philosophy and exposing wider audiences to traditions that are less well-known” (Lassiter/Kagan 2022, 1). Reality + has an ambitious scope, advancing several serious philosophical theses and arguments about the nature of virtual worlds and virtual reality. Chalmers states in the introduction that “this book is as full of my own philosophical theses and arguments as anything I’ve ever written” (2022, 16). Given that Chalmers “has helped shape the current landscape in the philosophy of mind and consciousness”, as Crane (2022) puts it, Reality + is worth taking seriously beyond its merits as an introductory philosophy textbook, as a serious statement of Chalmers’ philosophy.

The central thesis of Reality + is that, despite pervasive sci-fi depictions of virtual reality as ‘not real’, despite intuitions we may have that virtual reality is ‘fake’ or ‘illusory’, it is rather the case that “virtual reality is genuine reality” (Chalmers, 2022, 11). The ontological status of virtual reality, as well as the objects and events in it, is that they are genuinely real, but have a different underlying structure to the physical world. VR worlds do not have physical reality but virtual reality, and objects in them are “not illusions but perfectly real objects: they are digital objects that are made of bits” (Chalmers, 2022, 14). For Chalmers, virtual worlds and their objects have the characteristics of existence, mind-independence, causal power, genuineness, and being non-illusory – all characteristics of physical reality. So physical reality and virtual reality are both instances of reality. Chalmers calls this view “a sort of virtual realism” (2022, 114) and discusses it in connection to several distinct philosophical topics, including epistemology, metaphysics, value theory, philosophy of mind, and theology.

But Chalmers’ text is not just a book of abstract philosophical considerations - it reflects seriously on the future of society, and the role of virtual realities within it. He speculates that “within a century we will have virtual realities that are indistinguishable from the nonvirtual world” (Chalmers, 2022, 8), achieving total bodily immersion such that “no trace of the ordinary physical environment remains” (Chalmers, 2022, 193). We would become effectively disembodied, disentangled from our embodied perspective within the physical, nonvirtual world and able to take on a new body, with new possibilities, in virtual worlds. We would be able to manipulate the human body and its experiences to our will and create our own universes, putting people in the position of “simulator-god […] a local creator of our universe, with a good deal of local knowledge and power and perhaps not much goodness” (Chalmers, 2022, 136). As the nonvirtual world becomes overcrowded and harder to live in, with less economic opportunity and resources, people may emigrate to these human-made universes, presumably overseen by local simulator-gods and the corporations that employ them. Some would see all this as a dystopian nightmare, but Chalmers thinks that, in the future, emigrating to a virtual world might be a totally reasonable decision we shouldn’t be afraid of, and that our lives could be just as meaningful in virtual worlds (2022, 8, 33).

All of this is interwoven into a book-length philosophical discussion driven by its central ontological thesis, that virtual reality is just as real as physical reality. If Chalmers’ ‘virtual realism’ is true, the idea of becoming totally disembodied and moving to a virtual reality would seem less discomforting because it would be no less real than physical reality. If virtual realism is true, this is relevant information for thinking about the moral status of virtual actions because virtual actions are just as real as non-virtual actions. On virtual realism, virtual property is just as real as non-virtual property, virtual objects just as real as non-virtual objects. All of this will influence how we think about regulating virtual spaces which will certainly continue to feature, probably more prominently, in human coexistence.

There are monumental decisions that will have to be made in the decades ahead about the future direction of human society. Is allowing the mass disembodied movement of human beings into virtual worlds morally sound? Who would oversee and control the virtual worlds that future humanity would live in? How would they be regulated? Who gets to decide? These are far from trivial concerns, and I’m not saying that Chalmers has not thought about these questions, but they are very much open questions, our answers to which may very well determine the future course of human community. The thesis that virtual reality is just as real as physical reality, if true, would have knock-on effects for how we reflect on all these issues - so we had better be sure if it’s true or not.

In what follows, I will criticise Chalmers’ virtual realism by criticising his conceptualization of his key terms. This would in turn give reasons for doubting the validity of what they underpin, including his futuristic speculations. Especially problematic here are his definitions of ‘virtual reality’ and ‘immersion’, which he defines explicitly in distinction to two-dimensional (2D), non-VR video games. Ultimately, when defining VR and immersion, Chalmers holds immersion to be a distinctive quality of the perceptual representation of VR, while denying immersion to 2D, non-VR video games. Immersion is conceived of as distinctive of perceptual representation, a conception much too narrow because it leaves out various phenomena which deserve to be discussed under this label, all of which are manifested to some degree in 2D, non-VR video games. By discussing the type of video games Chalmers definitively holds to be non-immersive, I will argue that immersion goes beyond perception, that part of the immersive experience in video gaming is engendered through the experience of non-perceptual phenomena, especially cognitive states like reasoning, analysis, thinking, knowing, believing, deducing, strategizing, and cognitively exploring possibilities for action. It is partially through the representation of non-perceptual information, and the generation of non-perceptual cognitive states in players that 2D, non-VR video games can be immersive.

Building on the conceptual survey of immersion given in Brendan and Magdalena Balcerak Jackson’s Immersive Experience and Virtual Reality, I will call this type of immersion non-perceptual representational immersion. The representations of virtual worlds in VR and video games cannot be reduced only to their perceptual aspects, like visuals and sound. Part of the virtual world’s representation includes the non-perceptual information presented to the participant about the content of the world, its structure, it behaviour, the behaviour of its objects, and the various possibilities available within the world. This information appeals to our cognitive faculties, and in 2D, non-VR video games the player must apprehend and manipulate this information for success. I will argue this by discussing a type of game often referred to as ‘immersive simulations’, because they are particularly rich in non-perceptual information and particularly adept at engendering these kinds of cognitive states in players. Based on this non-perceptual information in the game’s representation, cognitive states are produced in the player which are constitutive for their engagement with the game, and this cognitive engagement, I submit, is partially constitutive of immersive experience, an important part of the way immersive experience is engendered. Immersion, therefore, cannot be reduced to perceptual representation, and can be found in 2D, non-VR video games.

This results in a conception of immersion very different from Chalmers’, for whom immersion is a dividing line between 2D, non-VR video games and VR, something which a virtual experience either has or does not have depending on whether it has 3D perceptual representation. The following considerations, however, will suggest the idea that immersion in virtual worlds is a phenomenon which takes place on a spectrum which includes both VR and non-VR experiences, and can happen in both 2D and 3D virtual worlds. Immersion comes in degrees, as do the different types of it. (Indeed, this is something Chalmers concedes.) 2D video games have some degree of perceptual immersion, in the form of graphics and sound, but are not as intensely perceptually immersive as VR. However, as I will argue, the non-perceptual aspects of a virtual world’s representation and our cognitive engagement with them are important components of digitally immersive experiences. It is partially through the representation of information about virtual worlds’ structures and behaviour, and partially through the often-intense cognitive engagement we have with virtual worlds, that we come to experience being immersed in them. Importantly for us, and for Chalmers’ argument, this is something which can happen in both VR and 2D, non-VR video games. Rather than being excluded from immersive experience, 2D, non-VR video games are positioned on the non-perceptual, or less-perceptual, part of the spectrum of digital immersion.

3 Chalmers on Virtual Reality and Video Games

Reality + begins with Chalmers talking with clear fondness about his experiences with video games, defining VR in direct distinction to them.

Like ordinary video games from Pong to Fortnite, virtual reality (or VR) involves a virtual world: an interactive, computer-generated space. What’s distinctive about VR is that its virtual worlds are immersive. Instead of showing you a two-dimensional screen, VR immerses you in a three-dimensional world you can see and hear as if you existed within it. Virtual reality involves an immersive, interactive, computer-generated space. (Chalmers, 2022, 7)

We can break this up as follows:

  1. 1)

    A virtual world is an interactive, computer-generated space.

  2. 2)

    Video games and VR both have virtual worlds.

  3. 3)

    VR is distinct because it involves immersive virtual worlds – immersive, interactive, computer-generated spaces.Footnote 1

  4. 4)

    Video games have nonimmersive virtual worlds – nonimmersive, interactive, computer-generated spaces.

  5. 5)

    Immersion hinges on whether the virtual world is represented two-dimensionally in front of you on a screen, or three-dimensionally, as a world you can see and hear all around you, ‘as if you existed within it’.

There is a lot to unpack here, and my argument will concern whether the use of the terms ‘immersive’ and ‘nonimmersive’ in this way is justified. I contend that it is not justified, given the complex, multifaceted meanings that ‘immersion’ has regarding games and virtual worlds. In what follows, I will show that 2D, non-VR video games deserve to be called ‘immersive’. They can be immersive in numerous ways, and these kinds of immersion come in degrees.

The constituent terms of Chalmers’ definition of virtual worlds – interactive, computer-generated spaces - are relatively intuitive, if not uncontroversial. They will not be my primary focus here but should still be clarified.

Interactive means that there’s two-way interaction between users and the environment, and among objects in the environment. The environment affects users; users affect the environment. Objects in the environment affect one another.

Computer-generated means that the environment is based in a computer: that is, a computer is generating the signals that are sent to our sensory systems. This contrasts with such non-computer-generated environments as the theater, movies, and TV—and with ordinary physical reality.

[…] I won’t try to define “space” for now, but I mean it in the broad and intuitive sense that includes virtual space as well as physical space. […] to exclude partial spaces (one room in a larger virtual world) and disconnected spaces (two separate virtual worlds taken together), I require that these spaces are complete and interconnected. (Chalmers, 2022, 194–195)

According to Chalmers, the virtual worlds of ordinary (non-VR) video games possess only these characteristics, being interactive, computer-generated spaces.Footnote 2 Importantly for us, however, is what Chalmers takes to be distinctive about VR, namely that it involves immersive virtual worlds.

4 Chalmers on Immersion

‘Immersion’ does a lot of work in Chalmers’ definition of virtual reality, being presented as the distinguishing feature of VR in contradistinction to video games. From his discussions of the term, the kind of immersion he has in mind when it comes to VR is perceptual immersion, whereas video games can only have psychological immersion.

Immersive means that we experience the environment as a world all around us, with ourselves present at the center. There are many degrees of immersion. An ordinary video game on a computer screen can be psychologically immersive, occupying all of our attention in a sort of flow state, but it’s not perceptually immersive because we don’t perceive the world as a three-dimensional world surrounding us.Footnote 3

For genuine virtual reality, we require perceptual immersiveness. Perceptual immersion comes in degrees. Current VR headsets achieve audiovisual immersion; the environments look and sound as if you’re immersed in them. They don’t achieve bodily immersion, in which you experience your whole body as part of the world.

The holy grail for VR is full immersion. In the Japanese Sword Art Online novels, written by Reki Kawahara from 2002 to 2008 and later adapted into a successful animé series, this was called full-dive VR. A fully immersive or full- dive VR is one that users apprehend with all their senses, as if they’re physically inhabiting the environment, and where no trace of the ordinary physical environment remains. (Chalmers, 2022, 193)

There are four crucial things here. Firstly, there are multiple types of immersion, at least two: perceptual and psychological. Secondly, immersion comes in degrees. Thirdly, perceptual immersion has different aspects which themselves come in degrees. And lastly, perceptual immersion seems to depend on a virtual world’s representation being three-dimensional rather than two-dimensional. To be perceptually immersed in a virtual world requires that we experience our being in it three-dimensionally, all around us, with ourselves at the centre.

Perceptual immersion is a multifaceted phenomenon which could potentially involve all our senses, although current VR only fully immerses sight and hearing. Chalmers argues that VR will immerse our other senses in the future, and while we do not currently have full bodily immersion, we can move in VR, look around where we want to, manipulate objects, and generally have a two-way interaction with VR’s virtual worlds while being fully audio-visually immersed in 3D. We experience the world as fully surrounding us, and only hear the sounds of the virtual world. For Chalmers, 3D audio-visual immersion is enough to distinguish VR as immersive, and lacking it is enough for video games not to be immersive. This is a very narrow understanding of immersion, and I will argue the application of immersion to 2D, non-VR video games, on a broader understanding of it, is fully justified. Chalmers could have been more specific and said what makes VR distinctive is ‘perceptual immersion’ or ‘3D audio-visual immersion’, but he is clear that what distinguishes VR is the quality of being immersive, a quality he equates with perceptual immersion. 2D, non-VR video games involve “nonimmersive” (2022, 194, 195) virtual worlds. I believe this characterization is mistaken, and clarifying why will allow us to understand the myriad different ways in which we can be immersed in virtual worlds.

Some of these ways can be captured under the label of psychological immersion, which Chalmers justifiably connects with the psychologist Mihaly Csikszentmihalyi’s famous idea of flow, a dynamic psychological state colloquially referred to as being ‘in the zone’. Flow is characterized by a person’s complete, intense occupation with and enjoyment of an activity, a “state in which people are so involved in an activity that nothing else seems to matter” (Csikszentmihalyi, 2008, p. 4). Flow is a “holistic sensation that people feel when they act with total involvement” (Csikszentmihalyi, 1975, p. 36), such that their experience of time is transformed and other needs – such as food, water, and sleep – are often disregarded. Flow is described by Csikszentmihalyi as being so enjoyable and rewarding that it is sought out for its own sake, often discussing flow using the language of ‘immersion’. Someone experiencing flow is “immersed in the activity” in the sense of being “deeply involved” (Csikszentmihalyi, 2008, p. 210) with it.

For Chalmers, when we are playing two-dimensional, non-VR video games, we are not perceptually immersed in a virtual world but psychologically immersed in an activity. A clear, recent example of this type of immersion can be found in the poker-inspired game Balatro (LocalThunk/Playstack, 2024), an immensely engaging, simple-to-learn-difficult-to-master experience which contains no narrative world, no characters, and no story. Except for a joker card who guides you through the short tutorial and pops up to praise or make fun of you depending on whether you win or lose, Balatro is fundamentally a card game in which your targets are numbers, not characters, and there is no attempt to build a world beyond the scope of the actual game. Balatro affects a concertedly minimalist style which focusses the player’s entire attention on the game, its mechanics, and strategizing to achieve success. The immersion one has in Balatro comes not from feeling as though one is present in a virtual world, but from being immersed in the compelling nature of the game.

However, what Chalmers has in mind when he speaks of being immersed in a virtual world is feeling as though we are actually present inside a virtual world itself and experiencing our presence in it from a three-dimensional perspective. For all its merits, you certainly don’t have this in Balatro. VR headsets accomplish this by removing the sound of the physical world from our experiential field, replacing it with the sound of the virtual world, and placing us in a 3D visual representation of the virtual world such that it surrounds us. We feel as though we have a perspective within it, and “the sense of really being present at that perspective” (Chalmers, 2017). This would contrast with the perspective of the external viewer playing a 2D game on a screen. In VR, through 3D audio-visual perceptual immersion, we feel as though we are actually there.

Although the two can presumably coincide in VR, there is a distinction between ‘feeling as though you are really in a virtual world’ on the one hand, and ‘being intensely and completely absorbed in an activity’ on the other. The latter is certainly not unique to VR or video games. You can be psychologically immersed and achieve flow in many activities that do not involve virtual worlds, like sport and non-video games, creative activities, work, cooking, hobbies, etc. It is justifiable to distinguish them, and perhaps it is justifiable to speak of both phenomena as types of immersion. But this is still not the whole story. The crucial thing here is that Chalmers is mistaken to place all his focus on 3D audio-visual perceptual representation as the defining factor of immersion in the sense of making you feel as though you are really in a virtual world. As he explains, perceptual immersion comes in degrees – and video games have degrees of perceptual immersion. They also have the possibility of full audio immersion: you can play video games with headphones on, removing the sounds of the ‘real’ world completely from your perceptual field and replacing them entirely with the sounds of a virtual world. I recently did this while replaying one of the most detailed, realistic, sophisticated virtual worlds ever realized in a video game: Rockstar’s epic depiction of the decline of the wild west, Red Dead Redemption 2 (2018). Playing this through headphones emphasized its incredible sound design, making my experience considerably more immersive than I expected. You experience the sound as being all around you, as though you were actually there. If something is making a noise behind you or to your left, it sounds like it’s behind you, or to your left. In every second, there are many overlapping sounds which paint an intricate picture of the details of the environment, contributing greatly to the immersion in its world one feels when playing it.

So, really, the only distinctive type of perceptual immersion in VR is (currently) complete 3D audiovisual immersion. But perceptual immersion is not the only way video games can make you feel as though you are really present in a virtual world. Video games have many ways of achieving this, which include but extend beyond their perceptual elements. The sophisticated graphics and sound design of today’s video games are much more immersive than they used to be, but there are other techniques and phenomena in video gaming that deserve to be discussed as immersive in the sense that they make you feel as though you are part of the virtual world, even though you are not three-dimensionally perceptually present in it as you are in VR. To illustrate this, let us turn in more detail to the concept of immersion as it has been analysed in academic literature.

5 Magdelena and Brendan Balcerak Jackson’s Immersive Experience and Virtual Reality

As Andersson and Strömsholm suggest, “immersion can be considered as an essential part in digital games and developers are constantly challenged when trying to create immersive game experiences to an ever-growing demand. However […] immersion is not an easy concept to grasp” (2018). Immersion has been used to refer to various aspects of players’ engagement with games, and is discussed in many contexts ranging from design, aesthetics, psychology, and others. Likewise in philosophy, immersion has been discussed in several ways in several contexts, with a broad range of meanings and applications.

Even something like Heidegger’s account of being-in-the-world could be framed as an existential-phenomenological analysis of the human being’s immersion in its world in the sense of being present in it and able to apprehend it and the things in it in terms of their meaning (1962). Something Heidegger notoriously neglected here was embodiment, a problem taken up by Maurice Merleau-Ponty in Phenomenology of Perception (2002). Merleau-Ponty details the role of the body and its perceptive capabilities in our existence, the crucial role they play in structuring the experience Heidegger called being-in-the-world. The problem of the body was also taken up by Sartre in Being and Nothingness, a text directly influenced by Heidegger which also tried to clarify the nature of the way in which we exist in the world. (1943) All these existential-phenomenological thinkers, in some form or another, try to come to terms with the nature and structure of human being-in-the-world, the way in which we are immersed in our being on an everyday basis. This is important because when game and VR developers try and create immersive experiences, it is this basic existential predicament that they are trying to represent, replicate, and manipulate.

But what exactly is immersion, and how is it understood in the literature? A full survey of the research community’s engagement with immersion is clearly beyond the scope of this paper. However, if we want to understand immersion in line with the various academic applications this term has had, an excellent guide can be found in Magdelena and Brendan Balcerak Jackson’s article Immersive Experience and Virtual Reality (2024), which outlines a distinction between four ‘types or aspects’ of immersive experience in VR. Now, being an article specifically about immersive experience in VR, this might seem an odd choice of research to guide the analysis of an article about how both 2D, non-VR video games and VR can generate immersive experience. But this move is justifiable and can perhaps lend additional credence to my argument.

The four types of immersion discussed by the Balcerak Jacksons are discussed insofar as they manifest in VR experiences but, as I will show, they all manifest to some degree in 2D, non-VR video games too. Focussing on an analysis of types of immersion which occur in VR and demonstrating that they also occur in 2D, non-VR video games lends weight to the idea that immersion in virtual worlds takes place on a spectrum which includes both VR and non-VR experiences. This already problematizes Chalmers’ account of VR and immersion, and the virtual realism which is guided by his conceptualization of these terms. But the Balcerak Jacksons’ account will aid my analysis further because I will be building particularly on their concept of ‘representational immersion’. They define this as the sense of “being in a three-dimensional world you can see and hear”, something partially facilitated by perceptual representation but which, crucially, “involves much more than perceptual experience. Just as important for representational immersion are the numerous sorts of non-perceptual, cognitive states that are generated within VR” (Balcerak Jackson/Balcerak Jackson, 2024, 4, 6).

Representational immersion involves much more than perceptual representations and our perceptual faculties, and this extra dimension is non-perceptual and cognitive. Part of the way we become immersed in virtual worlds is through the non-perceptual cognitive states they produce in us, the non-perceptual information they present to us, and our cognitive engagement with them. Following the Balcerak Jacksons’ logic on this, I distinguish between perceptual representational immersion and non-perceptual representational immersion. VR is particularly rich in the former, whereas the latter often finds expression in 2D, non-VR video games. (I will discuss this in detail later with reference to ‘immersive simulation’ video games.)

I therefore plan to follow the Balcerak Jacksons’ approach to immersion, even though it is a paper exclusively about VR. They argue that VR

discussions often take it for granted that immersive experience is a single unified phenomenon. Against this, we argue that there are four distinct types or aspects of immersive experience that should be distinguished: representational immersion, which corresponds roughly to what is sometimes called “psychological presence”; participatory immersion, which is related to the interactive aspects of VR experience; affective immersion, which has to do with the subject’s emotional relation to the experience; and narrative immersion, which captures the phenomenon of being caught up in the flow of events experienced. We argue that this four-way distinction helps us understand the powerfully immersive character of VR, while also recognizing its continuity with the immersion we experience with media such as novels, films, and music. […] the four elements distinguished here [are] different dimensions that combine to yield an overall evaluation of the degree to which a given experience is immersive. While these four types clearly interact with and influence each other, we discuss them here individually to help make their different roles in immersive experience clear. (Balcerak Jackson/Balcerak Jackson, 2024, 3, 4)

There are some crucial distinctions between this understanding of immersion and that of Chalmers’. The Balcerak Jacksons’ is more flexible, more open, and arguably more appropriate for engaging with a concept with so many meanings and applications as immersion, and lacks the problematic claim that VR is distinguished by its being immersive but is rather more in line with the idea that the immersion present in VR is continuous with the immersion of other aesthetic mediums. (Other mediums have narrative immersion, affective immersion, perhaps degrees of participatory immersion, etc.) Briefly and without detailed argument, they take Chalmers as one example among others of a problematic way of thinking about VR, saying that

their utility is limited when it comes to thinking systematically about VR and our engagement with it. They are in danger of running together several distinct elements that we think it would be useful to distinguish, and at the same time they miss other elements that, in our view, are important for understanding the immersive character of VR experience. (Balcerak Jackson/Balcerak Jackson, 2024, 2)

I agree with their assessment of Chalmers’ view and others like it, but I would add that the Balcerak Jacksons’ critique of it applies just as validly to the immersive character of video games, as does most (if not all) of their analysis of the different types of immersion. Video games have the capacity for every type, in varying degrees. Chalmers’ account of VR and immersion, therefore, misses much about the different ways in which one can be immersed in a virtual world, even in the sense he means this idea: as feeling as though you are present within the virtual world and have a perspective within it. I will turn to the details of their fourfold distinction now, beginning with participatory, affective, and narrative immersion.

6 Participatory, Affective, and Narrative Immersion

Participatory Immersion is akin to what Chalmers calls the ‘interactive’ aspect of being in a virtual world: a two-way, mutually-effecting relationship between the participant and the virtual world. The Balcerak Jacksons phrase it as “the sense of being an active participant, an agent whose actions help determine what that virtual world is like, how events unfold in it, and how she experiences it” (2024, 7). However, participatory immersion for them is not identical with interactivity, since there are VR experiences which are essentially three-dimensional films the user experiences from a perspective within the narrative. While they can do certain things, like decide where to look or position themselves from moment to moment, they cannot affect the narrative’s events. This experience would not have high levels of interactivity, but it would count as participatory immersion. I take it to be an obvious truth that 2D, non-VR video games often have very high degrees of participatory immersion and interactivity. Both Chalmers and the Balcerak Jacksons concede this, and my later discussion of ‘immersive simulation’ games will exemplify this, so I will not go into this further here.

Affective Immersion refers to our emotional engagement with virtual worlds, which parallels certain aspects of the way we engage emotionally with the actual world. Ordinarily,

we are excited or scared, happy or sad, amused or worried, impatient or content, and joyful or angry about the things that are happening around us and to us. This is a matter of what we call affective immersion, the experience of being emotionally or affectively engaged with what is happening in VR. Affective immersion is a powerful contributing factor to the sense one has of being drawn into another world (Balcerak Jackson/Balcerak Jackson, 2024, 10).

They cite a VR experience in which you balance on a narrow wooden plank high above ground, a realistic experience not only because of its perceptual representation, but “because the subject often cannot help but feel at least some of the fear and anxiety that goes along with being in that situation” (Balcerak Jackson/Balcerak Jackson, 2024, 10). The emotional aspect of the user’s engagement with this experience is partially what makes it so compelling, and is representative of a significant contributing factor to immersive experience which extends beyond fear and anxiety into the broad palette of human emotion. Non-VR games also have the capacity to stimulate affective immersion in their experiences. Sticking with fear and anxiety, we can note that what makes horror games so compelling is because they also stimulate these emotions, and this is a significant factor their drawing us into their worlds. The early Silent Hill and Resident Evil games are paradigmatic examples of this, affecting a near-constant sense of fear and anxiety as you unravel their mysteries – not to mention the additional emotional engagement one has with the characters and narrative.

Narrative Immersion is related to and often the cause of affective immersion.

A VR experience is narratively immersive when and to the extent that the subject’s experience in VR has a coherent narrative structure and flow […] in very many cases of immersive VR experience, the experience of being carried away into another world is as much a matter of getting drawn into the story of the world as it is a matter of being given a convincing representation of it. (Balcerak Jackson/Balcerak Jackson, 2024, 12, 13)

This would also be true of video games, the narratives of which have become increasingly sophisticated over time. Video game narratives now often rival those of cinema, TV, and literature in their complexity, excitement, drama, development of character, analysis of the human condition, and sociocultural commentary. Going into more detail about this here would distract too much from my main argument, and this point has become somewhat trivial. It is obvious from a cursory look at key examples that video games sometimes have strong, deep, complex, affecting narratives. Games like Silent Hill 2, Red Dead Redemption 2, Fallout: New Vegas, Disco Elysium, The Last of Us, and dozens of other examples, speak for themselves on this point.Footnote 4

What is interesting regarding video games, narrative and immersion, however, is that narrative is often discussed as being a particular site at which immersion can be broken, in what is sometimes called ‘ludonarrative dissonance’. This term was coined by game designer Clint Hocking, and refers to cases where the narrative and gameplay elements of a game do not cohere, “when the narrative forces me to do something out of character” (2007) or is otherwise inconsistent with the player’s actions. The Uncharted series “has often been mocked for being about a supposedly likable rogue who just so happens to recklessly slaughter hundreds of people”Footnote 5 (Matulef, 2016). For some players, this type of inconsistency can break their immersion in the game and undermine their investment in its narrative. This is because something runs counter to their expectations and makes them stop identifying with their character. One of the most important ways video games can make players feel as though they are present and have a perspective within a virtual world is by playing as and identifying with a character. By being put in a character’s position and afforded the possibility of carrying out actions as that character, players feel as though they are acting from a perspective within the virtual world, through their identification with the character.

I have discussed participatory, affective, and narrative immersion together, and somewhat cursorily, because it seems to me that the point that 2D, non-VR video games can manifest these types of immersion is neither a complex nor difficult point to make, and I do not want to waste the reader’s time by discussing it in any more detail than necessary. My main target of criticism is Chalmers’ treatment of immersion, (2D, non-VR) video games, and VR. From what he does say about video games, it is plausible that Chalmers would concede that they are participatory, that they contain engaging narratives, and that they can generate emotional responses. But he does not consider these as phenomena of immersion in the sense of feeling as though one is present in a virtual world. But this is a misguided way to think about immersion because a virtual world’s participatory, affective, and narrative elements can all contribute to your sense of being pulled into another world and feeling as though you are present within it. The rebuttal is open to Chalmers that when you play a video game, you feel like you are participating in the world from a perspective outside it, that you are feeling certain emotions, but feeling them from outside the world, and you are experiencing a narrative, but from a perspective outside the narrative world. The type of immersion in which you really feel as though you are present in the virtual world itself, he argues, is reserved for perceptual immersion, which the Balcerak Jacksons discuss as being part of representational immersion.

7 Representational Immersion and Video Games

Representational Immersion is defined by the Balcerak Jacksons as “being in a three-dimensional world you can see and hear” (2024, 4). The type of immersion Chalmers is capturing is representational immersion but, along with the Balcerak Jacksons, we should add the important caveat that “representational immersion often involves more than just visual and auditory perception” (2024, 4). They cite haptics in the hand controllers of VR systems, which can generate tactile sensations correspond to virtually represented tactile experiences in a VR world. Non-VR video games have this too in the form of vibrating controllers, which vibrate when explosions happen, for example, in a video game. As I mentioned previously, non-VR video games also have degrees of audio, visual, and tactile perceptual immersion. What Chalmers has in mind as being distinctive of immersive experience seems to be perceptual representational immersion, but the Balcerak Jacksons argue that there is more to representational immersion than this.

a VR experience is representationally immersive when and to the extent that it involves a rich and coherent network of […] conscious mental states that represent the subject’s surroundings as being a certain way – for example, as including certain objects and events – and that represent the subject herself as bearing certain relations to those objects and events. […] representational immersion, as we understand it, involves much more than perceptual experience. Just as important for representational immersion are the numerous sorts of non-perceptual, cognitive states that are generated within VR […] [in which] the user acquires various kinds of information about the virtual world she is in. (Balcerak Jackson/Balcerak Jackson, 2024, 4, 5)

Since I have characterized the type of immersion Chalmers holds to be distinctive of VR as perceptual representational immersion, perhaps we can call this extra aspect non-perceptual representational immersion. Like the Balcerak Jacksons, I understand this as referring to the non-perceptual cognitive states generated by virtual experiences, the non-perceptual information also constitutive of the virtual world’s representation, and our cognitive engagement with virtual worlds generally. These are just as crucial to immersive experience, feeling as though one is in a virtual world, as its perceptual representation.

Being in a virtual world, whether in VR or a non-VR video game, involves more much than just perceiving and acting within it. It involves understanding it, knowing about things in it, thinking, deducing strategies for success, interpreting the significance of the things around you, and generally acquiring and manipulating information about your environment and its contents. When an object is represented to me in a virtual world, I get perceptual information about it – its colour, shape, etc. But I also get information about its capabilities, uses, power, etc. This kind of information, and the thought and understanding it generates, extends far beyond perceptual information into the realm of the cognitive, into thought, understanding, reasoning, and so on. In discussing cognition, I am guided by some of the insights of a family of theories known as 4E cognition, especially theories of ‘enactive’ cognition, one of these four Es. Generally,

The proponents of 4E cognition have argued against the assumption that cognition is an isolated and abstract, quasi-Cartesian affair in a central processing unit in a brain. […] the cognitive phenomena that are studied by modern cognitive science, such as spatial navigation, action, perception, and understanding others’ emotions, are in some sense all dependent on the morphological, biological, and physiological details of an agent’s body, an appropriately structured natural, technological, or social environment, and the agent’s active and embodied interaction with this environment. (Newen et al., 2018)

If we conceive of cognition in the broad sense of involving a host of mental processes facilitating the acquisition, manipulation, production, and storing of information about our environment, and of these processes as structurally involving embodied, embedded, and enactive elements, then it is clear not only that cognition is an important part of our engaging with virtual worlds, but also that it involves more than sense-perception.

The perceptual representation of a virtual world plays an important role in convincing us that we are in one, but so does the non-perceptual representation of how the world behaves, what it contains, what its rules are, the information and possibilities available in it, and the non-perceptual cognitive states generated by the information in the virtual world which position ourselves in certain relations to it and the things and events in it. VR and video games do not just represent perceptual sense-data, they represent things which are interconnected in their meanings, give the player information about them, and generate cognitive states which cannot be reduced to perceptual categories. A virtual world’s perceptual representation, therefore, need not even be ‘realistic’ to be immersive. A cartoon, comic-book style virtual world can be just as immersive as a photorealistic one because it can behave and be structured in a way that makes sense, operate according to consistent rules, be believable as a representation of that world, generate non-perceptual cognitive states about the information represented in that world, and about the relations in which they stand to the things in that world.

Non-perceptual representational immersion is one of the most important contributing factors of immersive experience, and goes hand in hand with the perceptual immersion Chalmers accords distinctive importance when it comes to VR. In doing so, he makes immersion seem a much more simplistic phenomenon than it actually is, and makes the immersive experience in VR more distinctive than it actually is. I have already argued that, because Chalmers is right that perceptual immersion comes in degrees, and that video games have degrees of perceptual immersion. I will now show by discussing a particular type of game that video games are also rich in non-perceptual representational immersion. This justifies the use of the term immersive to describe video games, and accounts for a large part of their immersive character.

8 Non-Perceptual Representational Immersion in ‘Immersive Sim’ Video Games

The ‘immersive sim’ (or ‘im sim’, short for ‘immersive simulation’) is a historically significant and influential type of game which, while underperforming in sales, has often produced critically revered releases. Ion Storm’s Deus Ex (2000) is a prominent example of an immersive sim generally recognized as a particularly important and influential title in video game history. PC Gamer, for example, describes Deus Ex as “a PC classic. The sheer breadth of its systems and the sheer complexity of its level design are unmatched, even today” (Kelly, 2020), and Hans-Joachim Backe justifiably speaks of “a widespread perception of Deus Ex as a paradigm-changer” (Backe, 2022). Other immersive sims, like Prey and Dishonored (Arkane, 2017, 2012) are recognised as some of the best games of their generation. I have chosen immersive sims to examine because they represent a type of game rich in non-perceptual representational immersion. They offer detailed, believable representations of virtual worlds which are structured and behave in ways which make rational sense and give players a tremendous amount of freedom within them without holding their hand. This results in deeply strategic and immersive experiences which make the player feel as though they are really acting in and have a perspective within those virtual worlds because these worlds are believable and react believably and consistently to the player’s actions. As such, they are deeply immersive, not only perceptually, but because they are rich in non-perceptual representational immersion.

But what is an immersive sim? Given that this label takes its name from a word that is itself hard to define, perhaps it is not surprising that the meaning of ‘immersive sim’ has also been the subject of debate. The journal Paidia had a special issue on immersive sims, with its call for papers questioning “are these games a genre, a style, a school, or a mode?” (2021), and there are arguments to be made for each. But I do not want to take a position on this issue here and will refer to immersive sims as a ‘type’ or ‘genre’ of game only for simplicity’s sake. When discussing immersive sims, it is arguably more productive to discuss concrete examples of games that tend to be unquestioningly accepted as being immersive sims, analysing their most important gameplay elements, and the most important aspects of their underlying design strategy. The games I will be most inspired by in the following discussion represent some of the most historically significant examples of this type of game, and all belong to a shared lineage of influence.

  • Ultima Underworld: The Stygian Abyss, System Shock, System Shock 2, and Thief: The Dark Project (Looking Glass Studios)

  • Dishonored, Dishonored 2, Prey, and Deathloop (Arkane Studios)

  • Deus Ex (Ion Storm), Deus Ex: Human Revolution and Deus Ex: Mankind Divided (Eidos-Montreal)

  • Bioshock (2 K Games)

Without offering an answer to what kind of label ‘immersive sim’ actually is, these examples are usually accepted without question as being immersive sims in gaming discourse. Despite being very different games, they are united by a certain design strategy. Crucial to this strategy, as laid out convincingly by the game design YouTube channel Game Maker’s Toolkit, are the interrelated concepts of agency, systems, emergence, consistency, and reactivity. The focus on and implementation of these concepts produces a high level of immersion which is not reducible to perceptual representational immersion, or participatory, affective, or narrative immersion. Arguably, the most significant immersive aspect of immersive sims is better captured in terms of non-perceptual representational immersion, in terms of cognitive states in which the player obtains, deduces, and manipulates information about the game world, what they come to know, understand, and think about it.

As the developer of Deus Ex, Warren Spector, said in a ‘postmortem’ blog post about the game’s development, the first recorded use of this sense of the term ‘immersive simulation’,

It’s an immersive simulation game in that you are made to feel you’re actually in the game world with as little as possible getting in the way of the experience of “being there.” Ideally, nothing reminds you that you’re just playing a game - not interface, not your character’s back-story or capabilities, not game systems, nothing. It’s all about how you interact with a relatively complex environment in ways that you find interesting (rather than in ways the developers think are interesting), and in ways that move you closer to accomplishing your goals (not the developers’ goals). (Spector, 2000)

Immersive sims are immersive because they simulate a detailed, believable world which operates according to realistic, consistent rules which can be strategically exploited by the player, even in ways the designers did not necessarily expect. Within these worlds, the player is afforded a high level of freedom to pursue their objectives as they please, without being forced into playing a certain way. All of which contributes to a sense that they are really part of that world and acting within it – immersed in it.

As Game Maker’s Toolkit explains, one of the most important aspects of immersive sims is that they have “high levels of agency, which means you can achieve goals in multiple ways, and pick your own routes, tactics, and gameplay style. The developers tell you what to do […] but don’t tell you how to do it” (Game Maker's Toolkit, 2016). The way you accomplish goals in video games is often highly pre-determined. Grand Theft Auto V might offer a huge, detailed, believable representation of a world and an engaging story, but when it comes to choosing the way you accomplish your objectives, you are afforded very little agency. Almost all goals are accomplished by driving or shooting – you can usually choose which car you drive and which weapons you use, but the option of finding creative, elegant solutions to problems beyond driving and shooting is not available. This option, however, is crucial to immersive sims, and one of the reasons they are so adept at making you feel as though you are really in their virtual worlds.

Let’s consider an example from Deus Ex: Mankind Divided, where you must get past a bank guard to access a security-restricted elevator which takes you to a corporate vault, so you can steal information about the shady dealings of a shady company. You can shoot him – perfectly viable option. But if you do, you’ll raise an alarm, alerting all the other security guards in the area as well as some strong security robots. You can take all these down with guns and explosives – also a perfectly viable option. But it’s difficult, you might die, and if you don’t, your cover will be blown. Thankfully, there are many other options. You can sneak around, perhaps by crawling through the extensive air ventilation system. You can hack the security turrets and robots to fight the guards for you. You can hack a security computer in a nearby room to turn off the cameras, then tranquilize the guard, incapacitating them non-lethally and quietly. In this case, bribery is not an option, although it is a common one in many similar situations in immersive sims. But you can throw an object near the guard to distract him, making him abandon his post and investigate the noise while you sneak past. Or you could grab a vending machine from another part of the building, place it between you and the guard to obscure his line of sight, and sneak by unnoticed. And this is far from an exhaustive list.

This level of agency is an important reason why immersive sims are so immersive. The means by which you can proceed are widely open, and your solutions are your solutions which you think about and decide for yourself, not pre-determined solutions the developers force on you. When you succeed, it’s not just that you accomplished a goal, but that you acted as you would have really had to if you were in that situation. You investigated your surroundings, took stock of your situation, your resources, thought about how to approach the problem, decided how to do it, and did it – just as you would have had to if you were really in that world. All of which involves a variety of non-perceptual cognitive states in which you acquire, manipulate, and deduce information about the game world, allowing for a much higher level of immersion than just following a pre-planned solution to a pre-planned objective.

The reason the number of possible approaches to objectives in immersive sims is so high is because they are:

highly systemic […] elements have globally defined characteristics, which means that every alarm post works the same way, every torch can be extinguished, and while different doors might have different properties, they’re all based off the same generic door mould. There are also endless rules that the world follows: enemies can find you based on sight and sound, and will then run off and trigger an alarm, footsteps sound louder on tiled floors, turrets shoot who you tell them to shoot, and objects fall when pushed. (Game Maker’s Toolkit, 2016)

Many video games are scripted, meaning that their worlds and the things in them are built to do particular things at particular moments, and nothing else. Once something has served its purpose, it can cease to function as an interactive part of the game’s world entirely. A visual example Game Maker’s Toolkit use here is of a wooden plank in The Last of Us, which the player character places over a gap between two buildings so they can cross it. Once the plank has been picked up and placed, it ceases to function as a manipulable object, and the player is unable able to engage with it again. This is because the plank is not programmed systemically, as part of a wider object-manipulation system that behaves consistently across the board. These games are designed to deliver linear narrative experiences, guiding the player towards predetermined outcomes, so once the plank has served its initial purpose, it becomes surplus to requirement (Game Maker’s Toolkit, 2016).

Immersive sims, however, simulate worlds that behave in more realistic ways, according to systems which behave consistently.

Immersive sims are also consistent. They try to avoid special cases and one-offs, and there are rarely any failure states for anything other than getting killed. You won’t find these games telling you to return to the mission area, or making you restart the level because an ally was killed in duty. The simulation just continues. (Game Maker’s Toolkit, 2016)

This consistency manifests throughout the simulation, even when it comes to something as simple as the system for manipulating objects. Objects that the player is strong enough to carry will always be able to be picked up, moved, thrown, etc. If you pick up a plank, lay it across a gap, and walk across it, you would be able to pick it up again and use it for something else. Immersive sims systemically simulate a world, and in doing so effect simulations of many different systems within that world, and convey information about how these systems work, and interact, to the player. There will be a system for manipulating objects, a system for guard and non-player character behaviour, a lighting system, a door system, a trap system, an alarm system, and so on – and these systems all interact with and respond to each other.

For instance, if you attach a trap to an alarm and deliberately get yourself noticed by a guard, they might run to alert the alarm and meet a deadly surprise (Game Maker’s Toolkit, 2016). This would be an example of different systems consistently interacting with each other such that they can be strategically exploited, and of the reactivity of immersive sims.

Immersive sims are reactive. The plot doesn’t have to drastically change to reflect your choices, but characters will act in different ways and say different things to reflect on your decisions. Even if you thought no one would notice. The games judge your actions during gameplay, not in clearly designated choice sections. (Game Maker’s Toolkit, 2016)

Rather than simulate a world with one very particular path through it, immersive sims systemically simulate places, and the interacting behaviours of the objects and people within those places. The player is then dropped in and forced to figure out how to pursue objectives within this complex, consistent, believable environment – which accounts for a large aspect of their immersive character. The worlds they simulate usually behave like actual worlds, according to systems that accurately reflect the things from the actual world they are simulating - like alarm systems, people’s behaviour, or manipulating objects. Acting in the world of an immersive sim feels like you are acting in an actual world.

A particularly interesting aspect of immersive sims is that, because they simulate so many interlocking systems and they behave consistently, these systems can interact with each other in ways the developers of the game might not have even expected.

Immersive sims can be emergent. When two systems talk to each other, interesting new behaviours can emerge. These interlinking systems give the player opportunities to come up with smart, intentional strategies that exploit the game’s rules. Put a gas grenade on an alarm and then let yourself be spotted to create a nasty trap. Coax a self-detonating enemy near a fragile door and then kill it to open a passageway. Emergent gameplay can also lead to a mad chain reaction that you could not possibly predict, and puzzle solution that the developer simply did not foresee. (Game Maker’s Toolkit, 2016)

One of the classic examples of this unexpected emergent behaviour exploited the Lightweight Attack Munition (LAM) mines in Deus Ex, which could either be thrown as a grenade, or attached to any surface, and would explode if an enemy came close enough to it but would not react to you. Deus Ex players quickly realised that if you attached a LAM mine to a wall, they were big enough that you could actually stand on them. You could then attach another one above you, jump to that one, remove the previous one, attach it above the one you were now standing on, jump to that one (and so on), effectively creating an infinite ladder you could use to climb any wall. Doing this would allow you to access areas of the map you otherwise could not at that point, and perhaps accomplish your objectives more quickly and easily. The possibility of emergent gameplay of this kind only adds to the feeling of being present in a virtual world and having a perspective within that world. So much so that you can deduce information about it that is not explicitly given you to even by the developers. Emergent gameplay shows how complex, consistent, and believable immersive simulation games are: they’re so detailed that even the people who made them cannot foresee everything capable within them, and it is possible that players can come up with genuinely unexpected solutions to problems. Players can potentially come up with unique solutions that are genuinely theirs, which exploit the rules of a systemic simulation from within. This is reflective of the fact that it is possible to find unexpected, creative solutions to problems in the actual world, and to use objects for purposes for which they were not intended, like putting a beer mat under the leg of a wonky table.

Deus Ex director Warren Spector said that

the tools that Looking Glass and Ion Storm used to make games that felt immersive, not with photorealistic graphics […] but by letting go of the player’s hand. Deus Ex, for example, ‘was designed, from the start, as a game about player expression […] The game was conceived with the idea that we’d accept players as our collaborators, that we’d put power back in their hands, ask them to make choices, and let them deal with the consequences of those choices.’ (Warren Spector, quoted in Game Maker’s Toolkit, 2016).

Immersive sims simulate 2D virtual worlds that are not only often realistically perceptually represented, but make rational sense, convey a rich amount of information about their systems, objects, and behaviours, and give the player a high degree of freedom to think, choose, experiment, and decide what to do with that information. There is a highly cerebral aspect to immersive sims, such that you are constantly cognitively experimenting with creative solutions to problems, largely unguided by the developer. Immersive sims are therefore rich in what, following the Balcerak Jacksons, I have called non-perceptual representational immersion. Immersive sims make the player feel as though they are part of the world by putting the onus on them to think about how they act in it much more than most linear, story-driven games do.

9 Participatory Immersion and Non-Perceptual Representational Immersion

Before closing, I will consider an objection which will allow me to clarify certain points of my position. It could be objected that my discussion of immersive sim games has not succeeded in showing non-perceptual representational immersion as a distinct category of immersion, and that the phenomena that I have shown in this discussion ultimately collapse into participatory immersion. In the previous remarks, I argued that non-perceptual representational immersion involves non-perceptual information, and the generation of non-perceptual cognitive states, such as belief, understanding, etc. This partially accounts for how we become immersed in virtual worlds in the sense of feeling as though we are present in them. But it might be argued that this is just an instance of us having an effect on the virtual world and the virtual world having an effect on us, the exact two-way dynamic of participatory immersion. This would still be a kind of immersion, but not the kind I have argued for.

I claim that non-perceptual representational immersion goes beyond passively receiving information and having cognitive states generated in us. Following the Balcerak Jacksons, I argued that it involves conscious mental states that represent our surroundings to us as being a certain way, and which represent us as subjects as bearing certain relations to our surroundings and its contents (2024, 4). Crucially, these mental states and processes cannot be divorced completely from, or understood without reference to, our acting in the world and actively discovering information about it. Non-perceptual representational immersion involves the passive reception of non-perceptual information, but also consists in our carrying out an active, transactional engagement with our environment – an engagement which is itself cognitive in nature.

In saying this, I am conceiving cognition broadly in line with what has been labelled ‘enactive cognition’ in academic literature, ‘enactive’ being one of the four ‘Es’ of ‘4E cognition’, along with embodied, extended, and embedded. As the introduction to The Oxford Handbook of 4E Cognition explains, enactive cognition argues that “cognition is enacted in the sense that it involves an active engagement in and with an agent’s environment” (Newen et al., 2018). Cognition is not something that only happens inside one’s skull and while it is often passive, it is not entirely passive. Rather, if we conceive cognition in the broad sense it often is – in terms of a subject’s acquiring and processing information about their world – we should think of cognition as involving a dynamic, transactional relationship between the subject and their world, and virtual worlds would involve a virtual simulation of this process. As Mark Rowlands points out in a phenomenologically-informed account of 4E cognition,

  1. 1.

    Some (not all, by any means, but some) cognitive processes are partly (not completely, obviously) made up of processes whereby an individual operates on (typically, manipulates, transforms, and/or exploits) structures in its environment.

  2. 2.

    The structures carry information that is relevant to the cognitive task in which the individual is engaged.

  3. 3.

    The processes are ones whose function is to make information available, either to the subject of those processes or to further processing operations. (Rowlands, 2018)

One of the implications of these claims, all of which I take to be valid, is that cognition is inherently tied to action, even if not all cognitive processes are always active. Part of the way we cognitively acquire information about our environment is by acting in it, and carrying out particular actions which allow us to deduce, interpret, generally acquire and manipulate information about it. As Rowlands explains using the example of a jigsaw puzzle, this type of action is

action whose function is to change the nature of the cognitive task to be accomplished. Two jigsaw pieces will carry information about their relative fit with each other. This information is present in the pieces, but is unlikely to be available without further action on our part. The further action in question involves physically manipulating the pieces, bringing them into proximity, turning them relative to each other, etc. This action transforms the information contained in the pieces from information that is merely present to information that is available to the subject: the pieces can now be judged to fit, or not fit, with each other, and the basis of this judgment is information that was always there but has now been made available through these actions. […] this making available of information is the essence of cognition: what cognition is really all about. (Rowlands, 2018)

Immersive sim games, as well as many other types of 2D, non-VR video games, involve this type of cognitive action in spades. Just as the jigsaw puzzler manipulates pieces to deduce the information latently present in them and make it available for their consciousness so they can act on it, immersive sim players must constantly engage in this type of activity with the objects around them in the virtual world. You will constantly be picking up things, moving them, reading about them, manipulating them, experimenting with them (etc.) to obtain information about them which is present but not directly available to you, to then act upon. This is (partially) what it is to cognitively engage with one’s environment, and (partially) how we become immersed in any activity, as well as in virtual worlds. Not all the information we need to move about or be successful in a video game is immediately, directly available to us – but it is there, waiting to be cognitively discovered. We just need to act in the world in such a way that we can make it available. Immersive sims stimulate this type of information-discovering action in players particularly well, even to the extent that by engaging inquisitively with their virtual environments, players can discover things about them even the developers were not aware of, by actively disclosing information about them and making it available for their consciousness. In carrying out this type of action, we cognitively connect with the environment around us, and thereby feel immersed in it, as though we are a part of it.

By reducing immersion to 3D perceptual immersion, Chalmers’ account of VR excludes this cognitive element of the experience of feeling as though one is in a virtual world. Certainly, this experience can be partially engendered through sophisticated perceptual representation, but it can also be partially engendered by putting the player in a space in which you can cognitively inquire into, discover, manipulate, and act upon the virtual world’s information. I would argue that this is just as fundamental to immersive experience as perceptual representation. Rather than conceive of immersion in the manner he does, as something a medium either has or does not have depending on whether it is 3D perceptually represented, immersion in virtual worlds is a phenomenon which falls somewhere upon a spectrum which includes both 2D and 3D experiences, and involves perceptual as well as non-perceptual representation.

10 Conclusion: Non-Perceptual Representational Immersion in Video Games

I set out to critically respond to Chalmers’ account of VR, video games, and immersion in Reality+, where he claims that the distinctive characteristic of VR is that its virtual worlds are immersive, in the sense of being three-dimensionally audio-visually immersive. Video games, on the other hand, involve nonimmersive virtual worlds. This distinction underpins the central ontological thesis of the book – that virtual reality is genuine reality – which itself underpins much of the futuristic speculations and judgements about the future of human society Chalmers makes. I proposed to undercut Chalmers’ account by problematizing the way he conceptualizes some of the key terms of his analysis, especially VR and immersion, by discussing video games and developing an account of non-perceptual representational immersion.

I have argued that Chalmers mischaracterizes immersion, that he treats 2D, non-VR video games unfairly, and that his account of immersion and VR suffers from conceptual problems which can be illuminated by discussing video games. A particularly important conceptual problem is Chalmers’ admission that perceptual immersion, as well as other types of immersion, come in degrees. This, it seems to me, is true. But if it is true, it renders Chalmers’ basing his distinction between video games and VR on perceptual immersion problematic. Video games have degrees of perceptual immersion, even if they are not represented in 3D. Furthermore, there are other distinct phenomena that deserve to be spoken of in terms of immersion, such that video games, contra Chalmers, absolutely deserve to be called immersive. To illustrate these points, I adopted the helpful four-way distinction, in the work of Brendan and Magdelena Balcerak Jackson, between representational, participatory, affective, and narrative immersion. I concentrated mostly on representational immersion, since it is a lot more obvious (and so arguably less important to demonstrate) that video games possess the capacity to stimulate participatory, affective, and narrative immersion. The type of immersion Chalmers has in mind as distinctive of VR is perceptual immersion, which is captured by the Balcerak Jacksons under the label ‘representational immersion’. I followed their claim that there is more to representational immersion than perceptual immersion, and so distinguished between perceptual representational immersion and non-perceptual representational immersion. The latter involves ‘mental states which represent a subject’s environment as being a certain way, and represent the subject themselves as bearing certain relations to it and the objects and events in it.’ Non-perceptual representational immersion involves and accounts for the aspects of the representation of a virtual world that cannot be reduced to perceptual information. This includes the non-perceptual information about the world provided to you by the game beyond that information which is directly related to the narrative, and the non-perceptual cognitive states induced in the player by the game in their engagement with it, such as those involved in thinking, knowing, understanding. Non-perceptual representational immersion is brought about and partially constituted by the player’s active cognitive engagement with the virtual world, which I explicitly conceive of in line with ‘enactive cognition’, which conceives of cognition as partially constituted by a subject’s dynamic, transactional engagement with their environment in which they disclose and manipulate information about it.

I then moved to a discussion of an important type of video game which I claimed is particularly rich in and stimulative of non-perceptual representational immersion, relying heavily on it in the detailed simulations of worlds they effect. ‘Immersive sims’ construct complex, reactive worlds which behave consistently and react to the player’s actions according to discernible rules. The player is then charged to pursue objectives in these worlds, but with a remarkably high degree of freedom, with no explicit direction from the developers of the game beyond being shown what the rules of the game are. The simulations and their systems are so detailed and interactive that it is possible for players to devise solutions to problems that even the developers of the game did not foresee. Through their non-perceptual representational elements and the cognitive engagement they engender, immersive sims deliver highly immersive experiences even though they have a relatively low degree of perceptual immersion compared to VR. Immersive sims make their worlds believable by structuring them and making them behave like actual worlds and making them react consistently and believably to player actions, allowing them to experiment with multiple creative solutions to their objectives. The player feels like they are really in that world because the simulation is consistent, detailed, and complex, and consistently behaves like a real world in response to player action. The player is placed in a space in which they must interpret, manipulate, discover and act upon the information latent in the game world, cognitively disclosing it such that it is available for consciousness, and act upon it. As such, these games mirror the way we cognitively engage with actual world, and this is one of the most important mechanisms by which video games make us feel like we are a part of, immersed in, their virtual worlds.

Although all video games involve non-perceptual representational phenomena to some degree, immersive sims demonstrate the crucial role of non-perceptual representational immersion in making one feel as though one is really in a virtual world. This is what Chalmers misses out by claiming that VR is distinct because it is three-dimensionally perceptually immersive. There are many ways in which one can be immersed in a virtual world, many interlocking parts at play which reach beyond perception into human cognition. Immersion should not be thought of as either belonging or not to a particular medium based on one defining factor, like 3D perceptual representation. Immersion is much better captured by thinking of it as taking place on a complex spectrum which includes both 2D and 3D-represented experiences, with its position on the spectrum being determined by multiple related factors.