Seeing the Landscape Through Textual and Graphical Media Products

Abstract

Space is a central element in human communication. In this chapter, the authors compare representations of landscapes in different media types, including texts, maps, and virtual reality to show how they express spatial conceptions. Such a comparison is necessary to understand the mechanisms behind combinations and transformations among different media; hence, it forms a useful basis for analysing media interacting with spaces more generally. Taking a step back, the authors then discuss how the materiality and concreteness of space interact with the abstract conceptual level of models. This makes it possible to study traditional two-dimensional maps and texts as well as three-dimensional modelling and virtual reality systems. Finally, the authors introduce virtual reality as a new media form in the light of empirical experiments conducted at the University of Cologne.

1 Introduction: Being in the World

All humans have experiences from being in an environment. For most of us, this is based on movements within some sort of space,¹ and the experiences can include cycling in the woods, walking a city street, or riding through mountains. Exceedingly, over the last two centuries, this movement has been supported by engine driven devices, such as cars, trains, and airplanes; in earlier days, oars, wind, and animal energy were used for transport. The experience of existing in the world involves interaction with a landscape where other beings also dwell and machines are used. An environment where humans and other animals coexist can pose risks, support, and cooperation amongst other types of impact for the individual. It provides, in the sense established by Gibson (1986), affordances. The affordances go in different directions, forming networks of connections between beings based on a rich set of possible forms of communication (Ingold 2000).

One key element of communication is the landscape itself. Many fundamental aspects of communication, both in general and of human languages specifically, are connected to spatial organisation and movement (Levinson 2003; Tversky 2019). This is the case for all types of media used in human communication. Spatiotemporal modes are active in all media products, however different they may be in different types of expressions. The decoding and understanding of these modes as part of the spatiotemporal modality of a media product is always, in one way or another, connected to the recipient’s embodied memories of experiences in an environment.

Such memories can be specifically linked to a concrete reference. In a novel, where the events take place in Prague, the reference to Náměstí Republiky is recognisable as a concrete place by a reader who knows the place. References can also be type based. The island in which Robinson Crusoe was shipwrecked cannot be experienced in the same way as the square in Prague because it does not exist. However, the landscape described is still recognisable by readers based on general knowledge of landscapes, including beaches and forests. The embodiment is also linked to the situatedness of the reading process, as pointed out in Have and Pedersen (2020: 200).

Space, communication about space, and media, including transformations between them, are all interwoven. In this chapter, we will focus on expressions describing landscapes in different media. In order to better understand how different media are used to communicate about and convey understanding of landscapes, one central aim of this chapter is to include a focus on the transformative processes connected to the expression of spatial understanding. We begin by discussing traditional ways of expressing knowledge about landscapes: written and oral texts and maps, and their combination in geocommunication systems. Furthermore, we will use the four media modalities explained in Elleström (2020) as a system for understanding and comparing these media forms as they are used for spatial expressions. By focusing on transformation processes, we will aim to attain a better understanding of the limitations and possibilities within this approach. Moving beyond graphical two-dimensional expressions and writtentexts describing landscapes into virtual three-dimensional ‘landscapes’ will enable us to better comprehend the spatial media characteristics influencing media products.

The relationship between the spaces in a virtual world and the spaces we know as embodied animals is complex and many-faceted. This establishes virtual reality (hereafter: VR) both as mediated expressions (VR is a type of communication device) and as an experienced space, linked to an equally complex set of connections between different forms of embodiment and different sets of affordances.

These claims will be further explored through discussing landscapes in a number of different senses. We will start off with the real environment we move around in as embodied creatures. We will focus on how this has been mediated through centuries and millennia of human culture. We will then move over to the landscapes we move around in through avatars, that is, VR. That enables us to discuss in a more abstract way connections to modelling and modelling processes in general. This will include studying concrete types of spatial expressions through new media, such as VR applications. These are less affected by convention and have a much shorter tradition of developing their ways of using the available affordances, due to being new media in a historical sense. The technical development also changes the affordances themselves.

2 Descriptions of Landscapes

The human mind-body system has the capability of linking media products, expressed in a wide range of different media, to embodied experiences. This is used when creating media products based on pre-knowledge on the side of the creator, from oral, written, and pictorial media and dance to films, computer games, and VR. We will begin this chapter by focusing on traditional ways of making descriptions: texts, written and oral, and maps.² Furthermore, with two-dimensional static documents as our starting point, we will use the media modalities as a system for understanding these specific media forms in a structured way. This will show us that they are similar in both material and sensorial modalities, as they are both flat documents approached mainly through seeing.³ The differences between documents carried by computer screens versus paper and similar technical interfaces will not be discussed at this stage.

One main connector between texts and maps is place names, which are used by both media types to reference something external to the text and map. The links from documents to this externality can have different levels of realism. The reference function of place names is fairly similar in text and maps, but their documentary contexts are fundamentally different. This links to the spatiotemporal as well as the semiotic modalities, as we will see below.

2.1 Written Texts

Textual documents using the writing systems we know from languages such as English are generally based on a basic level of sequentiality, where letter follows letter and word follows word. This is the case even when writing directions vary (for instance, left-right, right-left, top-down) and some writing systems have a less strict sequentiality. The sequentiality is expressed in a spatial form and it is never absolute—there is always a possibility of moving back and forth in texts. Some forms of writing, such as certain poetic forms and footnotes, are designed to break the sequential pattern of reading; however, this becomes an addition to the low-level sequentiality. The order of the letters making up a word and the words making up a sentence are fundamental for the semiotic systems of symbols that languages constitute even if this order can be broken and played with. The sequential conventions are elementary parts of the code of writing systems. Still, these sometimes differ and are not as ‘absolute’ as the space of a figure, such as a map.

In this sequential structure, a place name is a word (expressed as a number of characters) which is related to other words around it (Firth 1957: 11). Syntactically, the primary relationships are within the sentence. When the same page of text includes several place names, the relationships between the targets of their denotations are not based on the actual locations on the page, but rather on the meaning which can be found in the text. To competent readers, the locations described are decodable through the relationships expressed within the text, in context of their own pre-knowledge. The space of the landscape referred to by the text is a reconstructed space and the semiotic system used in the text, also to relate to the landscape, mostly relies on symbols. The reconstruction of the referenced space happens when the reader interprets the meaning of these symbols, and understands the links between strings in the text and places in the landscape. This reconstruction is not dependent on whether the landscape is understood as a real place by the reader, but rather on whether it is recognisable or imaginable. This is partly based on the embodied experiences discussed above, partly on impressions from media products, and partly on other inputs. This implies that the distinction between VR and other media is not essential for our ability to reconstruct spaces.

The spatial relationship between places can be established in two different ways. It can be expressed directly by relational language constructions such as ‘north of’ or ‘down the river.’⁴ In this case, landscape descriptions can be understood even if they are not connected to any landscape pre-known to the reader. The language constructions establishing relationships between places are sufficient to get a basic understanding of the landscape described, but they are usually strongly under-specified (Eide 2015: 120–125).

Spatial relationships can also be assumed based on the knowledge of the specific landscape being described. In this case, the mere mentioning of two place names will establish a spatial relationship between them if they are both known to the reader, as long as ‘known’ implies that the locations of both places are known. No other language mechanisms than the place names are necessary in order to express such relationships. In neither case is it necessary that the referred places are real. The relationships between places in J.R.R. Tolkien’s Middle Earth are known to many readers, even if there is no physical landscape to reference. However, in the case of fictitious worlds, the two places in the last example must belong to the same ‘world’.

In usual reading, these two ways of establishing relationships are utilised together. This includes relational expressions such as ‘three miles down the river from Prague’ or ‘the hills between Prague and Wrocław.’ References to places can be expressed in a number of different ways in different languages, and are often mixed with other references, to places as well as to other things. In German news, the capital is often used to refer to a country. ‘Moscow claims that …’ is a typical reference to the government of Russia, where the capital acts as a reference to the city of Moscow, where the government is located. ‘The Municipality of Oslo’ can refer to a geographical area, an actor, a community, a group of people, and more. Another variable factor of establishing these relationships is the distinctions between different languages used. Indo-European languages have well-developed preposition systems for expressing spatial relationships, whereas Finno-Ugric languages usually rely on grammatical cases.

2.2 Map Documents

Similarly to the aforementioned reference system for written text, the way we reference places on maps includes the use of place names, which can establish links from the map to a location through pre-knowledge in the mind of the reader.⁵ However, the only understanding needed in order to read the precise spatial relationship between two places claimed by the map is to grasp the geometrical reference system of the space of the map. Once the scale and direction of the map is understood, each pair of places can be precisely related to each other by measuring the distance and the direction between them on the map. Furthermore, unnamed places are related to all other places in the same way. A black dot on the map can, given a certain map schema, be understood as a boulder, without any name needed in order to express the claim that this is a specific boulder in the external landscape referred to by the map (MacEachren 2004).

The links are not just denotational, they are also connotational. The black dot on the map also links to the concept of what a boulder is (given it is known to the map reader), in addition to linking to the reader’s knowledge of a specific boulder in a real or fictitious landscape. Place names on maps are not needed in order to refer to particular places, but they help the map reader establish the connections to specific places. A red dot can be understood as a town, and letters next to the red dot will usually establish the claim that this town has the name expressed by the letters.

Nothing can be put on a map without simultaneously making claims about how the mark on the map relates to other marks on the same map. Relationships can be fuzzy, but they are there; borders of the fuzzy area indicated can be read from the map, establishing a span of the possible direction and distance between two places based on the scale and orientation of the map.⁶ Even unmarked places on the map (the white areas) are related spatially to other parts of the map, in their expressing ‘nothing relevant enough to be put on the map exist in this part of the space referred to by the map.’ An interesting difference between maps as they are discussed here and architecture is pointed out by Miriam Vieira: maps describe/model space, whereas the role of architecture is to create space in the sense of altering or organising it (Vieira 2020: 60).

2.3 Oral Texts

We will now widen the scope beyond texts and maps expressed on flat documents, and proceed to discuss spoken words. Up until the spread of recording equipment in the first half of the twentieth century, spoken words were linked to presence in time—the message could only be heard at the time it was uttered.⁷ The message was created, transferred, and received within milliseconds. Additional understandings of the message could follow much later, in the same way as when one only understands the full meaning of a warning when what is warned against actually takes place. In other cases, the creation of the message could be a reproduction of an expression that had existed for a long time, as in the re-performance of oral tradition.

While oral communication often included visual presence, this was not necessary in the same way as time based presence: it could be dark, foggy, or there could be physical hindrances between the communication parties (as in the case of catholic confession). Since the arrival of recording equipment, the time presence is no longer obligatory for the oral communication to carry through. We now have the capabilities to communicate not only over long distances (as with TV and radio) but even across time. This secondary orality (Ong 2002) is relevant to communication about space due to the disappearance of the present human body from the communication situation. Until around 1900, only the human body in itself was available as a combined technical medium for and producer of oral text. This meant that gestures and facial expressions were usually available in most communication situations, as was the possibility to comment and ask questions. This is no longer the case, which has led to new forms of oral communication—new oral qualifiedmedia types (cf. Elleström 2020: 54–60). We will now move into a more detailed analysis of the consequences and influences of this for oral communication about space.

One example of oral communication about space is the explanation of a travel route from one person to another, discussed by Vieira (2020: 71) as a mental tour. This interaction involves a series of place references within a real landscape, in the form of place names either in a strict sense or as other spoken words and/or non-verbal sounds. Oral communication is based on the technical interface of sound, where hearing is the sense of reception. This means that there is time manifested in the material interface, but no space—almost. There are cases where the sounds of the pronunciation of words represent space in an onomatopoeic sense (‘the road is straight for a lo-o-o-ong time’). The musical aspects of chanting about places, where, for instance, heavily marked high-pitched tones can be used to represent mountain peaks, can describe space through auditory iconicity (Tirén 1942: 47; cf. Graff 2004). However, in the material we have worked on, these examples are not frequent, leaving symbolicrepresentations establishing virtual spaces as the main way to represent spatiality through the sound of oral communication. Elleström (2020: 78) points out that, in the case of pop music, there is a mix between auditory text and auditory image. This mix is always there in oral texts, but the balance between the aspects varies.

If we consider the situation of face-to-face oral communication, the parties have access to a number of additional affordances. One can ask questions and request clarification, but one can also use gestures and other bodily movements to indicate spatial aspects, even in an indexical sense: ‘We went to that mountain [gesturing the direction towards a visible mountain peak].’ Furthermore, oral spatial communication is a common way of using maps (Hyttfors and Tirén 2011: 5), as the basis for a series of gestures indicating past or future travels. These gestures may also include leaving traces with a pen or a pencil (Wood 1993). In this sense, it can include traces of human bodily acts in the material interface.

Oral maps do exist as a concept, but they are quite insignificant compared to texts and map documents. They make up neither a technical nor a qualified medium. The concept is used mostly for descriptive oral texts. As opposed to maps as they were defined above, they do not have space manifested in the technical interface and they are approached mostly through hearing, even if some gesture can be involved when the communication happens in real time between people present at the same place and seeing each other. What is called oral maps represents a subclass of (and is therefore classified with) oral texts, with an especially strong focus on space.

3 Geocommunication

All the media described above can be referred to as ‘maps’ in ordinary language, and this use of language also stretches far into academic communication in several disciplines (Eide 2021). Elleström (2020: 56) claims that everyday language only covers a few basic media types. This is true. It is also the case that the media types that are covered are not based on clear definitions of the terms. Furthermore, as pointed out by Simonson (2020), media borders are constantly being pushed and will change over time, due to the active engagement with and across the borders of media, in line with Elleström’s description of qualified media (2020: 54–66).

Additionally, the word ‘map’ is extended further to include, for instance, mental construction in human minds (cognitive maps).⁸ While this use of language is common also historically, it is inconvenient in scholarly work. Hence the more limited definition of ‘map’ given above. It is nevertheless useful to have a wider concept for the media-based methods humans and other animals use to communicate about landscapes. In this chapter, we will follow Brodersen (2008) in using the word ‘geocommunication’ for this purpose.⁹

All the media forms described above fall into the broader concept and can be described as geocommunication. That also includes other forms such as braille text and maps, smell, sign language, and dance. These other forms, common among a number of animal species, will not be discussed any further in this chapter. Vieira (2020) shows how an equally complex space/time situation is found also in architecture, linked to the negotiations between the persons involved in practical work. All situations where the qualified media listed above are used also have a potential for including other aspects of geocommunication.

4 References Between Maps and Texts

When studying different media and media products, it is important to also focus on the links between them. The reference functions of maps and texts describing space do not only link media products to landscapes, real or fictional. Additionally, they link media products, internally and with other media products, through the existence of co-reference (Eide 2009). Co-reference is a basic feature of human communication. In linguistics, it is used in the resolution of anaphors: ‘We went to Rome. We had a good time there.’ In this example, ‘Rome’ and ‘there’ co-refer; they both have the city of Rome as their denotation. Co-reference in text is fundamental in the establishment of spatial meaning, way beyond anaphor resolution. Most texts include a series of references to locations, in which co-reference is used to establish connections between such references. In editions of texts, such as historical source material, it is common to include a gazetteer, a list of place names mentioned in the text with references to where they occur in the text as well as information about their assumed real or fictitious location. The place name occurrences grouped under one headword in a gazetteer are grouping together a set of co-referring place names.¹⁰ This can also be done when the denotation is uncertain or not known at all, or when it is assumed to be a non-existing place. The criterion is common denotation, not the nature of what is denoted.

This form for co-reference does not exist in any single map image, as one place in the referred landscape can only be represented as one visual item on the map image. Only in multi-image maps, such as printed maps with a zoom function or with a series of maps depicting the same area, does the understanding of the map include a reliance on co-reference. The co-references link locations on the main map image and the zoomed map image, or in multiple map images denoting different aspects of the same area. Co-reference is also used in rectification of digitised maps, where points on the scanned map are linked to points on a digitally georeferenced map. The scanned map is morphed to fit the digitally georeferenced map and is thus fixated to a standard projection. The rationale for connecting two points in rectification is that they refer to the same place in the space referred to by the two maps.

Connections between documents in the form of texts and maps are established by a competent reader in a comparable way. When using a map to trace the movements expressed in a travel narrative, places in the text are linked to places on the map through denotational co-reference. Place names in both are important; however, identity in names is neither a necessary nor a sufficient condition for co-reference. Relational expressions, as discussed above, can be used. Furthermore, the names have to be interpreted correctly. Someone reading a travel narrative from Canada may know that there is a much larger ‘London’ in England, but may still choose London, Ontario, as the place on the map she assumes ‘London’ in the text co-refers with. On the other hand, reading about ‘Königsberg’ in a book on the history of mathematics, it is of no avail to look for the identical name on most modern maps; Kaliningrad or Калининград will be the two most likely series of letters to be found referring to the same city as the one the historical name ‘Königsberg’ refers to.

Automatic mapping of texts, which is more popular than ever in the age of widespread digital maps, is done in a similar way. A computer programme is used to identify all place references, usually focusing on place names. Then these names are run through another programme to geo-refer them, that is, to find geographical coordinates for each of the place names. Various types of cleaning processes may be added in order to find the most likely candidate for several different places with the same name. These can include both digital decision-making systems and human intervention. The next step is using the coordinates to position the place names on the map, possibly with links to frequency information, the context around their occurrences in the text, and other relevant information. The quality of such systems varies depending on the nature of the source texts, and for historical sources, it is hard to reach acceptable quality without human intervention.

So where do we find co-reference here? The assumption in the creation and use of such tools is that the geocoding can identify a place with coordinates based on occurrences of names of the place, so that the coordinates refer to the same place in an external landscape as the names in the text. Ideally, this process is supported by additional criteria and can identify places across different names, while distinguishing between different places with the same name. Without the co-reference assumption, the process would not make any sense.

Mapping of written texts is a much more common procedure than mapping of oral texts. When working with recorded oral material, such as historical sources, the normal procedure would be to first transcribe the oral recording into a written text, and then identify and map the place names in the transcript. There are cases, however, where oral storytelling is mapped directly. When other phenomena than the mere existence of a place reference in the oral text is to be mapped, an interactive situation is needed in order to capture the richness of the knowledge possessed by the persons being interviewed. The mapping also includes additional qualitative aspects. In the tradition of mapping indigenous historical use and occupation of land based on oral testimony, gestures and other embodied movements complement oral texts in the telling of historical spatial stories, which are then transferred to a static paper map, a so-called use-and-occupancy-map (Tobias 2009).

These maps document indigenous use of land, based on interviews with witnesses about their memories. The stories told come into being in interaction between the witness and the interviewer. In this interactive performance, time is manifested in the material interface. Parts of the information exchanged in the interaction are transferred to the map where it is recorded as a timeless trace—it is moved from time to space. Gestures make up an important part of the performance, and some of these lead to traces on the map. While being expressed on a ‘timeless’ document, these traces can include cues which can be used to re-establish some of the time aspects from the interviews. Combined with other pieces of evidence, the document is used to establish a coherent understanding of the often overlapping historical use of the land. This is then used as documentation in future work on land rights issues, often including legal processes.

Creating texts based on maps is not common. Classical ekphrasis, where poetry describes works of plastic art, is not common in texts with maps as sources, even if it happens in architecture (Vieira 2020). Map-based texts, however, are well known from other situations. In fictionliterature, especially in novels or series of novels where the site of action is a fantasy world, it is common to draw maps of the spaces before the text is written.¹¹ While the text in those cases is not a description of the map, they are still based on the understanding of a fictitious landscape as it is expressed on the map. The map works as a geometric structure for the text—as a figure used to organise the spatial aspects of the meaning to be expressed in the text.

Textual descriptions of maps are found in scholarly works, for instance on the history of cartography.¹² Facsimiles of maps discussed in detail tend to be included. In these cases, the description is a guide for the eyes of the beholder, rather than a text meant to replace the map. Additionally, in such cases, the denotational function of the map rarely plays a significant role in the textual description.

5 On the Realism of Landscapes

Until now, we have described landscapes referred to from texts or maps as either real or unreal. This is a simplification of how texts and maps work. While there are clear rhetorical cases of claims for existing landscapes, such as in historical texts about The Second World War, and claims for non-existing landscapes, such as Narnia, these are only two specific ways of referring media products to landscapes. In works of fiction, the landscape described in the text is often an adjusted version of the real world. For instance, in Robinson Crusoe, the world described is the one we know, with the exception of the island where Crusoe spent his 21 years after one of the shipwrecks in the book. Map examples include Soviet propaganda maps from the cold war where places were systematically displaced (Monmonier 2018: 125–129).

Any linking of a place in fiction to any map is part of an interpretative act, including the assumption that the fictitious text can meaningfully be mapped onto a real world map. But what is a fictitious text and what is not? For some people, texts describing the Apollo 11 expedition are fictitious, and the truth/fiction nature of religious texts is permanently contested. While the fact that the links to real world maps is an act of interpretation is important in scholarly projects creating maps of literary works, it is questionable how relevant it is for a general reader. Reading is organised according to the principle of minimal departure (Ryan 1980) in the understanding expressed by Elleström (2018: 27): the extra-communicational domain, which is the prior knowledge of the reader, is key to establish an understanding of any work of art. As pointed out already by Ryan (1980), deviations from prior knowledge are only made when unavoidable.

When an assumed non-fiction text is read as a presentation of actual history or geography, the principle of minimal departure is also applied. Deviations from what the reader knows from beforehand will be negotiated. However, whether the results of the negotiations include changes to the world view of the reader in her extra-communicational domain will be influenced by the reader’s assumptions as to whether the text is fictitious or not. If the work is assumed to be fictitious, the reader will create an adjusted world view for that work of fiction and others based on it (sequels, adaptations, fan fiction, etc.). The assumed world of Robinson Crusoe includes the island where he shipwrecked, but the assumed real world does not, even after reading the book. If the text is assumed to be non-fiction, the change to the world view stands a much higher chance to be general to the world view of the reader, such as a new atlas correcting the locations and forms of the misplaced Soviet places from the cold war. This is a prototypical distinction which glosses over many cases not clearly located in either category. Historical fiction is often assumed to have a true historical core, so that a reader might adjust her general extra-communicational domain for at least some of the adjustments she makes during reading. As the level of persuasiveness is not directly linked to the validity of the underlying claims, as pointed out by Tseng (2020), the adjustments can only be understood in relationship to the openness for the persuasions from the reader’s perspective.

Further complexities are added by the fact that maps, being models, always simplify what is mapped. There are a number of systematic deviances on maps. One example is a traditional paper based 1:50,000 scale map depicting a road and a house. The road is drawn as a black line, the house as a black square, and the real world distance between the middle of the road and the middle of the house is 20 metres. Each map symbol exaggerates the size to avoid them being too small to be identified. The road might be depicted as 1 mm wide, and the house a rectangle of 1 times 2 mm. Given the scale of the map, 1 mm represents 50 metres. Thus, if the location of the two symbols were precise they would overlap and, being the same colour, the house would normally be read as a small parking lot or some other part of the road. In such cases, not only the size of the symbols but also the locations are adjusted. The house is typically moved at least 1 mm away from the road, possibly more. Based on the fact that this is the smallest distance between two such symbols used on this series of maps, the house will be read by a competent map reader as being next to the road, without any other assumption about distance than it being less than around 100 metres.

All these types of situations, of which many more exist, show that neither maps nor texts are either true or false, and understanding the complexities in how they relate to facts and fiction is dependent on the readers competence. These complexities similarly exist in new media, such as computer games and VR. What sets these apart is the way the environment is experienced in new media, which creates different situations, as we will see below. A competent reader will have well-founded assumptions about the realism of texts and maps, which will often work well. However, in some cases, they will be imprecise or even downright wrong, as when a fiction book is read as if it were non-fiction. Going forward, we will therefore not assume that fictitious and real maps are two different categories in real media use. We will instead reason in terms of degrees of realism in the way a landscape is depicted by maps and texts. Realism must here be taken both as intention by the author or cartographer and as testable empirical facts.

6 Maps Are Not Texts, But Neither Are Texts

When reading a written text, listening to an oral text, or reading a map, one interacts with two different types of material interfaces using two different senses. The written text and the map are both flat documents with space manifested in the material interface, whereas the oral text is made up by vibrations in the air with time manifested in the material interface. The latter is often complemented by a human body using arms, fingers, and other body parts, as well as external objects, to indicate directions and make references. These three are also often combined, in either pairs or all three together. This can also be the case with other media types such as VR, as we will see below. Humans possess a wide area of mixed geocommunication methods. In this section, however, we will see each of these analytically isolated as a specific media type with its specific modalities.

What our eyes see and what our ears hear lead to meaning. This transfer process has been studied extensively over millennia. In the following description, the basis is Elleström’s four media modalities (2020). These modalities do not indicate any essential division (the material interface has no clear border to the spatiotemporal modality); the distinctions are analytical. Furthermore, there is no obligatory development in time from one modality to the other. We do not first meet a material interface, then use our senses, then go through spatiotemporality so that we, in the end, can apply semiotics. The different parts of the process are all mixed together.

As one may expect for media with a special link to landscapes: the spatiotemporal modality offers a certain amount of complexity when we study maps, written texts, and oral text (Elleström 2018: 19). What is established as senses interacting with physical material (in our cases sight and documents or hearing and sound waves in the air) goes through the spatiotemporal tunnel into a world of meaning. Here the signs seen and heard make sense in interaction with the pre-knowledge of the specific human mind.

In the sensorial and material modalities, the two types of spatial documents—written texts and maps—are similar to each other, but quite different from oral text, which is one separate track of sounds in the air. In the semiotic modality, on the other hand, the map with its geometrically based spatial reference system is different from oral and written text, where we see a language-based spatial reference system. Written text loses its closeness to maps and gains a new closeness to oral text. This is not to deny virtual time in maps and virtual space in texts. However, these are established at another level, based on the meaning of each expression. The space of the map and the time of the text is a primary level in the semiotic process of making meaning in the context of landscapes and embodied human experience.

This implies that transmediation (Elleström 2020: 81–83) between oral and written texts, on the one hand, and maps, on the other, is slowed down by the differences between how the two media forms refer to external landscapes in their representational systems. Between oral and written texts, so different in their material interface and sense interaction, transmediation can be done with much less effort at the semiotic level. When it comes to integrating different media products in the same material interface, on the other hand, written text and maps integrate so well that it looks almost seamless, when maps illustrate textual documents as well as when texts add information to a perimap. Integrating oral text with either maps or written texts was much harder and drew more attention to the hybridity of the resulting media product, at least until the introduction of computer carried recorded sound.

Transmediation between both is complex and will often be based on going back to the original story. This was clearly expressed for plastic arts and poetry already in 1766, when Lessing clarified the background for his suggestion that painting should be based on quite different principles from those underlying poetry: painting is about objects in space, and poetry about events in time (Lessing 1893 [1766]). The relationship between map/text studies and interart studies generally and Lessing specifically is discussed further in Eide (2015), also indicating which of Lessing’s observations are still relevant and which are based on dated and now irrelevant aesthetic principles.

7 Interpretative Journeys

There is an important additional aspect of spatial reasoning as it is expressed in texts and maps. The spaces we experience in landscapes, and are reminded of in media products, also form basic aspects of human thinking outside the actual navigation of landscapes. Two examples from classical rhetorics are the art of memory and the linearisation problem. We will also look into as if-travel in physics.

The art of memory traces its roots back to Simonides (see Yates 1966: 223) as a memory technique used in the preparation of speeches. Of special interest to us here is that its operational mode consists of linking a landscape to a textual narrative, in this case oral. It is not about remembering every word exactly,¹³ but rather about remembering the right order of the topics to be covered in the speech. One assumes, for instance, a garden and a walk from place to place. At each place, the speaker mentally stores objects reminding her of the upcoming topic. This has been shown to work as a memory technique across cultures and is still part of commercially available mind training packages.

The starting point for the art of memory technique is an already created speech, and the spatial configuration is used to remember the main points in this speech. Thus, an oral or written text is transferred into a mental space, which is remembered and later used to re-create the original speech. Spatial memory is used as a mental storage technique.

The linearisation problem is also known from classical rhetorics. The problem here is to create an oral or written text in the first place. How can one establish a linear text from a number of different topics which all need to be included? The solution here is to create a space where the topics are spread out, and then navigate this space like a walk. This walk is assumed to be linear in the same way as a speech (Levelt 1981) and it is important that the route passes through all of the topoi,¹⁴ that is, all the topics. In this case, it is not about remembering a text but creating it, out of a mass of relevant topics.

In the development of thinking towards scientific hypotheses and eventually theories that can be formalised in propositional language, active use of drawings is an important part of the building of mental models (Nersessian 2008). This has been documented as video recordings during social interaction among groups of physicians at different career levels (Ochs et al. 1994). The video footage shows a figure being drawn on the blackboard by one participant, then others adding their own marks to the drawing. These can either be unclear lines created by moving a finger over the blackboard drawing, or additional drawings with chalk. Furthermore, in the model space used to discuss particle phenomena, movement is described. Not just particles moving, but particles and other parts of nature are even addressed with words such as ‘me’ and ‘you.’ Thus, in the process of making sense of physical reality through the models used to talk about it, metaphorical travels through the model space—and presumably indirectly through micro-space—is done by humans in the room where the discussion takes place. This happens both through oral language—including gestures—and with the use of figures.

All these examples show the fundamental power and potential for meaning-making in thinking about space in a figurative way. This happens either as a space standing in for another space, however different it may be, or as a space standing in for a more abstract structure. Meaning is then expressed through stories about travelling through that space. This shows how the differences between figures, such as maps, on the one hand, and texts, oral and spoken, on the other, can be used creatively as a tool in developing, expressing, and negotiating meaning. As Simonson (2020), we see also here how media borders are pushed. Model-based thinking is fundamentally tied to the complexities in the relationship between texts and maps, as it passes the tunnel from senses and physical interfaces through space and time, real and represented, to a semiotics-based establishment of meaning.

8 Modelling and Media Transformations

The previous examples showed how visual forms are used in individual and social negotiations of scientific meaning in modelling processes. Indeed, what has emerged over the last years in digital humanities is a model concept which is in line with the concept of ‘media product’ in Elleström (2018: 11): “Since being a media product should be understood as a function rather than an essential property, virtually any material existence can be used as one, including not only solid objects but all kinds of physical phenomena that can be perceived by the human senses.” The functional aspect of ‘model’ is made explicit through the focus on the verbalised form ‘modelling’ rather than on the noun ‘model’. A model defines anything which is used by someone (a person, a group) in modelling. The physical item is important, but cannot determine if the process is a process of modelling, in line with the criteria for something being a technical medium of display (Elleström 2020: 33–40).

A map is a media product insofar it is understood as a human-made expression intended to convey some sort of meaning to other humans.¹⁵ This leads to a question archaeologists struggle with, not only for maps but also for other forms of expression. How can one decide if lines on the wall of a cave are parts of a media product or not? The first attempt to clarify the question would be to identify the species behind the marks. If the marks are made by the claws of an animal they will not be seen as establishing a media product. But even if they are assumed to be human made, they might be the results of sharpening a tool, and not intended to represent rivers and hills. If it is decided that the marks are made by a human and do indeed represent a landscape, then it is seen as a media product, and commonly described as a map. This map is a model of the landscape.

This also applies to scholarly and scientific modelling—the same scratches could make up a model also in this sense. Imagine, for instance, two scientists in the late nineteenth century trying to understand the hydrological structures in the landscape around. Seeking refuge from the hot sun in the cave, they start discussing different hypotheses and eventually, as they have left their writing utensils at the camp, use the wall and sharp stones to express different models, in line with the example from physics above. Scholarly and scientific modelling is an activity defined by who takes part in the activity and with which purposes. Thus, a media product is a model insofar as it is used in modelling, and it is a scientific or scholarly model insofar as it is used in scientific or scholarly modelling.

The maps therefore represent models when they are used as models of a landscape. This happens in most intended use of maps, but not all. Maps are also decorative objects which can be used for other purposes than modelling. The modelling aspects are based on their representational function.

But what is a map a model of? This question involves some additional complexities. From the outset, the map is a model of a landscape with some degree of realism. Some maps, however, are clearly presented as maps of a fictitious world, and are assumed to be so by most readers. A map of Tolkien’s world is assumed to represent the same world as in the books written by Tolkien, and maybe also as in the films, the fan fiction, and other derived works. Such maps will be assumed to be more or less correct. Thus, there exists something that the truthfulness of a map of a fictitious landscape can be tested against. This is a shared conceptualisation between the users of the text, testable against documents describing this world—in the case of Tolkien’s cosmos, first and foremost his own work. A fictitious world can have an assumed internal truth system, following some sort of spatial principles. It does not have to be coherent,¹⁶ but it is assumed to be truthful within the context of the fictitious work. The establishment of the intracommunicational domain (Elleström 2018) follows certain rules.

A virtual world, as it is known through VR systems, adds another level to the experience of space. Independently from whether that virtual world is mediating a fictitious space or not, it does not only have to follow some sort of spatial principles, but also the principles of physical sensation made by experiencing the known space. There is a specific type of embodiment present in this experience. As pointed out above: from the outset, the map is a model of a landscape with some degree of realism. VR might, at first glance, appear to present a model of space with a higher degree of realism, but what happens if a virtual space is not tested against the real physically experienced world, but rather against another media product? What if the virtual world is established through a media transformation (Elleström 2020: 79–83) from, for instance, a text to a VR system?

Going forward, we will focus on space as it is expressed and experienced in VR. On the one hand, space is crucial or even existential for media products which are experienced as VR. On the other hand, immersion is often seen as an inherent part of VR. Through analysing experimental media transformations, we will show that a transmission of the perception of space—which is as realistic as possible—only creates an illusion of a multi-dimensional space. This experience is therefore more connected to the traditional ways of expressing knowledge about landscapes (as it was described above) than one might expect. The linking between a media product that is conceptually creating space and the connection to be experienced (also through traditionally mediated space as it was outlined above) is widening the discussion in new ways.

9 Landscape, Space, Reality, and the Virtual

We will now extend the discussion to three-dimensional modelling as a process of creating virtual worlds and accordingly VR systems. In such systems, spaces are expressed by computer screens, mobile phones, head mounted displays, or other technical media, enabling a virtual world. The way spatiality is expressed in these systems makes the representation of spatial relationships and landscapes, also morphing landscapes, accessible in ways not available through other technical media. Investigating VR as a medium for conveying experiences of landscapes opens up new perspectives on mediated space and time. Furthermore, examining how scholarly and scientific models are expressed in multi-dimensional space allows analysing the spatial understanding, not only of the creator but also of the user. This has significant impact on how models can be used in scholarly and scientific work.

The purpose of creating a 3D model is dependent on its intended use, but also connected to the medium to be used to visualise the model. Displaying 3D models through VR devices and thus representing an interactive and explorable three-dimensional computer generated environment (a virtual world) is usually associated with expectations deriving from a general conceptual understanding of VR as a specific type of reality emulation. It is more challenging to establish a conceptual definition of VR. It must be based on an understanding of the complexity of immersion, in line with Virginás’ discussion of the video quality of reproduced recordings in Lynch’sLost Highway (2020: 160–161).

This complexity is linked to the special nature of embodiment in the use of VR equipment, and links to performance studies (Vieira 2020: 8, 10). Maybe the experience of travelling can be extended to the context of architecture, beyond the possibility to see and move in non-existing buildings. This begs the question, however, of the vital role of embodiment in architecture and how far this can be re-created in VR. Can, in fact, other media forms experienced in VR, for instance, theatre, be seen as experiences of the represented media forms, given the differences in embodiment (Elleström 2020: 81–83)?

Experiencing a virtual world is conceptually connected to the idea of being immersed within an artificial environment. It implies the possibility of strongly realistic user interaction with the virtual surroundings simulating an experience, which gives the user an impression of being part of an illusionary perfect other reality. This other reality can mediate, for example, past events and distant places, but also entirely different, unprecedented worlds. However, at this level, it is questionable how new VR is. The idea of realistically modelling an immersive representation is not only expressed in current development and implementations in VR but also expressed much earlier in paintings, optical illusions, and other forms of pre-digital media products.

The dream of VR as it arises from the conceptual model presented above has never been fulfilled completely or even satisfactory for most users. There are limitations both technically and in terms of which graphical representations are possible. In order to feign a fully realistic experience of the unrealistic, a VR system would have to trigger all the sense and perception systems of the user. Existing VR implementations and devices nevertheless have a potential for representation which can be unlocked without insisting on a specific level of performance in the VR system.

Setting aside the conceptual expectations and considering existing implementation of VR within the technical possibilities available today, one can explore in practice the actual possibilities for representation. VR can be used for modelling highly realistic yet completely different realities—completely different in the sense of different to any known place but still realised through representations of known objects and known physical characteristics. One example is SpaceVR, a project that attempts to make the universe explorable through VR. Such attempts apply analogously to many unattainable spaces in the world known to us. Accordingly, the attempt to make known, but physically inaccessible places explorable through VR is a frequently observable phenomenon.¹⁷ Photorealism is one important goal in VR, but other ways of representation are also important, including highly abstract forms; there are, for instance, several implementations in VR representing paintings by Hieronymus Bosch.

Vieira (2020) established the role of the architect as a sort of a theatrical producer. Also in the analysis of VR, it is highly useful to include practical experiences from the creation of VR systems. This balances the focus between conceptual ideas and possibilities on the one hand and the actual possibilities of implementation on the other. While it is important to plan VR implementations within the framework of existing technical possibilities, the creation of three-dimensional worlds with the help of computers additionally reflects the perception of known reality. As pointed out for other media above, the users will assume similarities with their known environments and only change their expectations when they have to.

For the experience of VR, a virtual environment must be created. This environment consists entirely of three-dimensional objects. None of the visual characteristics from our known reality is self-evident. Everything, including reflecting light distinguishing between different materials (for instance, wood or plastic), shadows, and the appearance of highlights in white, is calculated by algorithms and can be influenced by the creator of the virtual world. Only what the user can see needs to be calculated, and when something can only be seen at a distance, it can appear realistic even if in lower quality. Consequently, there is no need to adapt the representation to the user’s world of experience, at least not when considering the possibilities for implementation. As in theatre and film sets, the world can be set up by invented backdrops. The artistic process of creation, in this case the modelling of virtual surroundings, always has artificial aspects which open up for new interpretations.

The ability to experience space and to move in the virtual world is usually, but not always, linked to different user interactions. Ultimately, almost every decision in the modelling process can be made by the modeller. Metaphors and experiences from known reality are often used; however, the way we orient ourselves can only partially be transferred. Orientation in an unknown space is always of key importance and will be of immediate concern to a new user. As we saw above, everybody has the experience of being in an environment, even if the experienced environments vary significantly. The creation of VR systems is connected to a desire to share experiences, but also to create a world which cannot be experienced in reality based on the experiences the user already has. Such worlds can still be expressed through mediation, and, as pointed out by Simonson (2020), this also happens in other media than VR. The experience of movement in a virtual environment establishes a direct reference to a reality understandable by humans. The ability for abstraction we saw through maps and text has its counterparts which can be analysed in the implementation of a VR.

A commonly used case for VR is games, which are similar to video games known from other technical media. Even VR applications not meant to be games often use interfaces and elements known from video games. In the context of games, users interact with a so-called cyberspace, a completely virtual surrounding in which gamification aspects are not essential. The experiences of how space is represented in video games are used in a comparable way in VR. Maps are often important elements in this context, being used to discover the virtual world. The abstractions used in these maps are similar to the maps described above. The same language is used. Modelling a virtual world is often combined with designing maps, which is of essential importance to user interaction, communicating the space to the user. Besides this communication, characters can also talk to each other about places that are not covered by maps.

The use of both maps and texts in video games is well established for communicating information about the spatial structure of the virtual world, and for telling stories to the user.¹⁸Maps mostly function as navigation tools: they show where you are and sometimes they also help users navigate, by showing them where they can go. Pathfinding and navigation are often parts of the users own decisions in the game. The map covers all places that are available, implying also that some places are not of interest or not reachable. One can design virtual worlds where the user can get lost. The double nature of maps as tools for overview and control and as tools for navigation (Wood 2010) is often present in the use of game maps.

Places visualised on a map establish the expectation that they will in fact be reachable. The appearance of a map often fits the design of the virtual world it is describing. The places covered by the map are only references and cannot in themselves establish a story around the places being referred to, or cover their meaning. Compared to descriptions of places in texts, the reference will be less likely to create emotions connected to the place (Eide 2015: 152–153). The experience of actually visiting the virtual place can overcome this.

Video games and maps have a quite special relationship, as shown by an endless number of examples from the last decades. Similar to board games, maps can become a central element of the game also through functions that go beyond pure orientation.¹⁹ As a result, positions of players on the map are elementary in the sense that each location and spatial relationship on the map is important. At the same time, the respective positions must be visible and perceivable either by an opposing team or by other players. A live mapping of all objects of interest is possible in a virtual environment, where also moving objects such as cars can be shown. The virtual world and its functions are established through maps. The map can function as an interface, independent from the user. Sometimes the map itself is responsible for offering challenges.

Groups of media computer science students at the University of Cologne have examined modelling processes of media transformations, partly through VR implementations and partly through attempts to model maps in VR (Eide et al. 2020). Analysing the results of the students’ work allowed us to make some observations of the modalities of VR applications. Given that specific media products in VR differ from each other, and the rapid technological development, the results are bound to be preliminary.

We identified three general questions that pinpoint specific decisions made through the modelling. The questions and the students’ answers are shown in Figs. 7.1, 7.2, and 7.3. Our observations for VR as a non-bodily media product can be summarised in the context of media modalities (Elleström 2020). The material modality as a latent corporeal interface of the medium, and how its different modes are expressed, depend partly on which type of VR device is used. The bodily movements of a user can in some cases be fully recognised through motion detection. The movement of the head is essential to control the point of view. The materiality of a controller, gloves, and a VR headset is normally established as demarcated. Less clearly demarcated material manifestations are expressed by artificial sounds and lights added to the virtual world. The perception of the interface of VR happens through the different senses, as modes in the sensorial modality. Feeling, tasting, and smelling are usually not included; however, seeing and hearing are attended through the VR headset with earphones.

Fig. 7.1

How do you point your users to ‘interesting’ aspects of your VR-environment? Do you explicitly point your users to aspects relevant for your narration?

Fig. 7.2

Do the objects in your VR-environment present your users a functional (symbolic) or a realistic view?

Fig. 7.3

Would you say your VR-environment is more a game or a narration?

Space manifested in the material interface can be implemented through a map. Cognitive space is explorable through looking around in the virtual world. Virtual space is present in the virtual world. Time manifested in the material interface can be influenced by interaction to the extent it is implemented in the system. Perceptual time is established through interaction or animated camera movement. Virtual time is given through animations.

VR uses a complex and many-faceted semiotic language. The basis for the photorealistic expressions one sees is iconic. In the image, however, there is always a set of symbolic elements. These can be graphical as well as in the form of text, or as texts used as elements of interaction. Some parts of VR are close to map semiotics (MacEachren 2004; Brodersen 2008), whereas other aspects trigger the indexical complexity of digital photography and video (Lister 2007). Being highly dependent on concrete implementations, it is hard at this stage of our research to generalise the semiotics of VR. Further research is needed to establish how different the various technical implementations used for VR are semiotically, and whether it is possible to establish a general core understanding of VR semiotics.

10 Being in a Virtual World, Experiencing Space Real and Unreal

By putting on a theatre mask, the actress transforms into the role character.²⁰ In other theatre forms, the masks are invisible, but still metaphorically present. In all embodied, actor-based art forms, the actor meets the role character in a shared body, whose trajectory is both that of the role character and of the actor, as discussed also by Simonson (2020). In the 1930s, the mask of a gangster was gradually glued to the face of Humphrey Bogart, opening up for a new type of hero when the actor who had impersonated several criminals became the doubtful but inherently good guy in film noir of the 1940s.

When putting on a VR mask (a head mounted display), only the eyes are fully covered. This is necessary in order for the targets of light being emitted from two small screens, creating an illusion of a three-dimensional room. The earphones provide some replica of surrounding sound, while the sound of the physical space around the user of the VR system can be heard as distant, sometimes even unreal or dreamlike, dependent on the specific equipment used. This might be compared to the architecture of the theatre building (Crossley 2020: 96–97). Clarifying the relationship between the consequences of the different types of equipment and settings is beyond the scope of this chapter, but Crossley’s discussion of theatre as a hypermedium could be extended by also looking at the theatrical text as basis for a VR application, in addition to VR within a theatrical event.

As Alice experienced behind the looking glass, everything is the same yet everything might be twisted. In the end, the room on the other side turned out to be merely a dream. This is in line with literary functions, such as metalepsis, as pointed out by Lutas (2020), also for its counterpart in film. The idea of the computer game as life (Gaskill 2009) is a mirage, no more real than the dream of jazz music simplified to one single tone (Skagen 1993). Yet, it is a powerful argument ad absurdum, pointing towards the sciencefiction nightmare of the virtual that can kill you, in our century unified through the popularity of the Matrix trilogy. What can kill you is real and not just virtual.

What we see in our VR worlds is both real and virtual. The lights and sounds are real. The 3D and surround effects are virtual. The avatars we meet can represent artificial intelligences on which our acts have no impact. They can also represent our best friend or partner, in which case mistakes done in the VR system can alter or even destroy lives. Like any other technical medium, from parchment and paper to Instagram and Twitter, it can be used to send messages to the ones who do not care. It can be used for fully fictitious games, where all is virtual and nothing has consequences. It can also be used to communicate with others who rely on our words, images, and acts, in the realness of letters and images establishing or breaking family bonds and starting or ending wars. The novelty of VR is not the fact that it is virtual. The virtual has always had a potential for linking to reality. The novelty of VR is its way to relate senses to a mix of real and fictitious environments. It hides the realness in a secluded space by being in a media artefact, and hides its virtualness though photorealistic immersion.

VR is not harmless. Books are not harmless. Whether we say that books kill, or we say that books are but tools used by humans in order to initiate acts that lead to killings, is a matter of where we stand. Nevertheless, what we say about books we can also say about VR. Behind all the differences between media types—behind their different ways of representing something beyond themselves and behind the differences in time and space and mixes of them manifested in the material interface—they are all tools used to make the unreal real and the real unreal. The various media types are used to communicate across the abyss between human minds. As shown in Elleström (2020), the basis for all the discussions in this volume is communication.

In spite of all the differences presented in some detail in this chapter, all these media forms can represent landscapes and events taking place there. None of them are the landscapes and events they represent. The differences between their own landscape and soundscape are what establish them as different media, qualified and technical. The links to the landscape come about based on our knowledge of landscapes. The only thing the media can do is to remind us of the landscapes we know, reshuffling elements in them. The rest is up to us.

11 Conclusion

This chapter focuses on how one can see the landscape through textual and graphical media products. This is important in the context of the publication Beyond Media Borders: Intermedial Relations among Multimodal Media because the landscape itself and spatial thinking is a basis for human thinking and communication (Tversky 2019). The recipient’s embodied memories of experiences in an environment (real or virtual) are always connected to the decoding and understanding of spatiotemporal modes which are active in all media products.

We have analysed different media expressions related to landscape, with a focus on spatial representations in different forms and on how transformations can deepen our understanding. We started with written and oral texts and maps, and their combination in geocommunication systems, which are traditional ways of expressing knowledge about landscapes. By studying them as different systems for communicating spatial understanding, we clarified how each system works. The sequentiality of textual documents is expressed in a spatial form, but is still recognisable based on the code of writing systems, which is different from the other visual media we study.

Interestingly, the spatial relationship between two places is established differently in texts—oral as well as written—compared to map documents. By focusing on place names as reference system, the different forms of spatial communication were highlighted: the geometrical reference system of the space of the map versus the primary syntactic relationships between words within a sentence. Opening up the discussion to spoken words also highlighted the importance of the human body as part of communication. Different media products are moreover concretely linked. The reference functions of maps and texts describing space do not just link media products to landscapes, real or fictional; they also link media products, internally or with other media products, through co-reference.

Mental space, mental models, movement, and memories are basic building blocks in journeys, physical and mediated, with different levels of realism. Scientific modelling and fiction are different, yet connected (Suarez 2009). Thus, we were able to expand our research and discuss in a more abstract way how all this relates to modelling and modelling processes in general. By applying the concept of modelling from digital humanities and showing the connections with Elleström’s concept of media product (2020), we could see that both are functions defined by the purposes of those who take part in the modelling and communicative activities. Thus, a media product is a model insofar as it is used in modelling, and it is a scientific or scholarly model insofar as it is used in scientific or scholarly modelling.

This analysis was then used to study concrete types of spatial expressions through new media types such as VR applications. Being new media, in a historical sense, means that they are less affected by conventions and have a much shorter tradition of developing their ways of using the available affordances. Indeed, the affordances are still in formation and the technical development changes the affordances themselves. Moving beyond graphical two-dimensional expressions and writtentexts describing landscapes into virtual three-dimensional ‘landscapes’ enables us to understand better the spatial media characteristics influencing media products. An abstract versatility results from the existential basic property of VR spaces as they are used to create other spaces for users being in a virtual world, experiencing space real and unreal. This reflects on how space can be mediated and formed through experiences, memories, and illusions. Thus, landscapes in a number of different senses were discussed. We started with the real environment that we move around in as embodies creatures and how it has been mediated over centuries and millennia of human culture, before shifting our attention to the landscapes we move around in through avatars, that is, VR.

Our claim is that the relationship between the spaces in a virtual world and the spaces we know as embodied animals is complex and many-faceted, establishing VR both as mediated expressions (VR is a type of communication device) and as experienced spaces. This is linked to an equally complex set of connections between different forms of embodiment and different sets of affordances. We need more research and more practice in this area to be able to clarify how all these parts interact in order to establish human mediated experiences in communication.