Having presented the map and calibrated the compass, I will now sketch the contours of a conceptual framework for communication. Narration does not exist by itself; it happens when we communicate with each other. Consequently, narratives and stories are not something that we find floating around independently but something that is communicated by minds. Before I define narration in Chap. 3 and then scrutinize transmedial narration, this chapter will present certain general concepts that I find helpful for modeling communication and the work of communicating minds. Without such a background, it is difficult to model narration and the work of narrating minds; the part cannot be properly understood without access to the whole.

Communication, Mediation, Representation

I start by postulating that communication should be understood here not only as communication among minds in general, but more specifically among human minds. This is simply because the capacities of human minds partly differ from the capacities of the minds of other animals. While recent research has revealed amazing cognitive abilities among several mammals and birds, there are still large differences. Although these differences can be bridged, making communication among different species possible and even widespread, a discussion of that subject is beyond the limits of this study.

To communicate is to share ideas, thoughts, notions, and understandings. I call these shared entities cognitive import. At least two minds must be involved in communication and, somehow, a transfer of cognitive import occurs between them. My suggestion is to speak and write about the producer’s mind and the perceiver’s mind to refer to the mental places in which cognitive import appears. First, there are certain mental configurations in the producer’s mind; then, following the communicative transfer, there are mental configurations in the perceiver’s mind that are at least remotely similar to those in the producer’s mind. Clearly, an intermediate entity is required to make such a transfer possible. I suggest calling this intermediate entity a media product. A media product enables the complex transfer of cognitive import from one (or several) producer’s mind(s) to one (or several) perceiver’s mind(s). I have elaborated on all these notions elsewhere (Elleström 2018a, b). It may be noted, for instance, that in certain media types, such as those called interactive, one and the same mind may have the separate functions of being both co-producer and perceiver of the evolving media product.

The media product must be somehow material, although not necessarily solid or even palpable. It needs to be a physical entity or process that has the capacity to trigger mental reactions through semiosis, meaning that it prompts the creation of cognitive import in the perceiver’s mind; it acquires the function of a sign or a collection of signs. As the transfer of cognitive import among minds involves both material and mental aspects, I find it helpful to distinguish between two profoundly interrelated but discernible basic facets of the communicative process: mediation and representation (Elleström 2014a: 11–20). Mediation is the display of sensory configurations that are perceived by human sense receptors within a communicative situation. It is a presemiotic phenomenon and should be understood as the physical realization of entities (with material, sensorial, and spatiotemporal qualities, and semiotic potential). Consider the example of a person hearing certain sounds. Representation is a semiotic phenomenon and should be understood as the heart of signification (which is delimited here to how humans create cognitive import in communication). When a human agent makes sense of the mediated sensory configurations, sign functions are activated and representation is at work. Using the example, the listener may interpret the sounds she has heard as a voice uttering meaningful words.

My current emphasis is on the idea that both a presemiotic and a semiotic side exist to basic encounters with media. Whereas the concept of mediation highlights the material realization of the medium, the concept of representation highlights the semiotic conception of the medium. Although mediation and representation are clearly entangled in complex ways, upholding a theoretical distinction between them is helpful in analyzing complex relations and processes. In practice, however, mediation and representation are deeply interrelated. Every representation is based on the distinctiveness of a specific mediation. Furthermore, some types of mediation facilitate certain types of representation and render other types of representation impossible. As a case in point, vibrating air emerging from the vocal chords and lips that is perceived as sound, but not as words, is well suited for the iconic representation of bird song, whereas such sounds cannot possibly form a detailed, three-dimensional iconic representation of a cathedral. However, distinctive differences among mediations are frequently subtler and less easily spotted without close and systematic examination.

Thus, representation in communication is the creation of cognitive import through perception and cognition. To say that a media product represents something is to say that it triggers a certain type of interpretation. This interpretation may be more or less hardwired in the media product and the manner in which one perceives it with one’s senses, but it never exists independently of the cognitive activity of the recipient—there are no signs unless there is a mind to activate sign functions. When something represents, it calls forth something else; the representing entity makes something else—the represented—present to the mind. This is to say, in terms of Charles Sanders Peirce’s foundational semiotic concepts, that a sign or representamen stands for an object; Peirce’s third sign constituent, the interpretant, may be understood as the mental result of the representamen–object relation (see for instance, 1932: CP 2.228–229 [c. 1897]). My concept of cognitive import created in the perceiver’s mind in communication is an example of Peirce’s concept of interpretant. However, the entire triad of sign constituents is actually part of a mental process, although both representamens and objects may be connected to external material elements or phenomena (see Elleström 2014b).

Whereas representation—the very essence of the semiotic—constantly occurs in our minds when we think without having to be prompted by sensory perceptions, it is also triggered by external stimuli. In this context, it is appropriate to focus on external stimuli resulting from mediation. In other words, representation also occurs in pure thinking and in the perception of things and phenomena that are not part of mediation, but the account of representation in this research is mainly limited to the creation of cognitive import on the basis of mediated sensory configurations—stimuli picked up by our sense receptors in communicative situations. The contention is that all media products represent in various ways as soon as sense is attributed to them. Hence, the media product can be understood as an assemblage of representamens that, due to their physical qualities, represent certain objects (that are available to the perceiver), thus creating interpretants (cognitive import) in the perceiver’s mind.

I find these concepts indispensable for a methodical modeling of meaning-making processes in communication, and they will serve as a firm spine for delineating the concept of narration and, in particular, the transmedial aspects of narration. As the following chapters will demonstrate, these concepts make it possible to discern the fundamental similarities among media and still pinpoint where the essential dissimilarities are to be found. However, they should be complemented with some concepts that highlight the importance of background knowledge for meaning-making.

Background Knowledge, Virtual Spheres

As no mind is a static and isolated entity that is dependent only on its inherited characters, the concept of communicated cognitive import in the producer’s and the perceiver’s mind must be examined also with an emphasis on how minds are molded by surrounding factors. In addition to its innate basic capacity to perceive and interpret mediated qualities (discussed in more detail later), the mind is inclined to form cognitive import on the basis of acquired knowledge, experiences, beliefs, expectations, preferences, and values—preconceptions that are largely shaped by individual experiences of culture, society, geography, and history. It is clear that all this is immensely important for the outcome of communication. The perceiver’s mind acts upon the perceived media product on the basis of both its hardwired cognitive capacities and its attained predispositions; evidently, the cognitive import that is stored in the mind before the media product is perceived has a significant effect—to various degrees—on the new cognitive import formed by communication.

This is a recognized phenomenon that has been extensively theorized in various ways and minutely scrutinized within theory of interpretation and other research areas. What I offer here is a complementary semiotic way of modeling how cognitive import in communication is formed by private and public environments. Although I focus on the perceiver’s mind, the basic suggested principles are also relevant to the formation of cognitive import in the producer’s mind.

I have already established that the representamens that initiate semiosis in communication come from sensory perception of media products. One perceives configurations of sound, vision, touch, and so forth that are created or brought out by someone and understood to signify something. They make objects (in the Peircean sense) present to one’s mind—and eventually result in interpretants based on the representamen–object relation: it is these interpretants that constitute the cognitive import being communicated. But where do the objects come from? They clearly do not emerge out of nothing; they are drawn forth from earlier percepts, sensations, and notions that are stored in the perceiver’s mind, either in long-term or short-term memory that may also cover ongoing communication. ‘Earlier’ could be a century ago or a fraction of a second ago.

In semiotic terms, the stored mental entities may be direct percepts from outside of communication, interpretants from semiosis outside of communication, interpretants from semiosis in earlier communication, or interpretants from semiosis in ongoing communication. This is to say that objects of semiosis always require “collateral experience” (Peirce 1958: CP 8.177–185 [1909]; cf. Bergman 2009) that may derive both from within and without ongoing communication. In other words, collateral experience may both be formed by semiosis inside the spatiotemporal frame of the communicative act and stem from other, earlier involvements with the world, including former communication as well as direct experience of the surrounding existence.

In line with this twofold origin of collateral experience, I distinguish between two intertwined but distinct areas in the mind of the perceiver of media products: the intracommunicational and the extracommunicational domains. In doing so, I emphasize a difference between the forming of cognitive import in ongoing communication and what precedes and surrounds it (Elleström 2018a, Forthcoming; related but divergent distinctions in cognitive psychology have been proposed by Brewer 1987: 187). I also find it appropriate to make a corresponding distinction between intracommunicational and extracommunicational objects, both of which are formed by collateral experience from their respective domains.

From a broad temporal perspective, the extracommunicational domain is clearly prior to each new intracommunicational domain created and should therefore be understood as the background area in the mind of the perceiver of media products. This comprises everything one is already familiar with. As it is a mental domain, it does not consist of the world as such, but rather of what one believes and knows through perception and semiosis. In other words, one’s stored experiences not only consist of percepts as such but also of percepts that have been contemplated and processed by the mind through semiosis. This involves estimations and evaluations of encounters with people, societies, and cultures that are consciously or unconsciously accepted, put in doubt, or rejected. The extracommunicational domain includes experiences of both what one presumes to be more objective state of affairs (e.g., dogs, universities, music, and statistical relations), what one presumes to be more subjective state of affairs (states of mind related to individual experiences), and everything in between. Thus, it is actually formed in one’s mind not only through semiosis and immediate external perception but also through interoception, proprioception, and mental introspection. Hence, the extracommunicational/intracommunicational domain distinction is very different from exterior/interior to the mind, world/individual, material/mental, and objective/subjective.

It is imperative to note that vital parts of the extracommunicational domain are constituted by perception and interpretation of media products. Thus, former communication is very much part of what precedes and surrounds ongoing communication. Together, non-communicative and communicative prior experiences form “a horizon of possibilities”, to borrow an expression from Marie-Laure Ryan (1984: 127); the extracommunicational domain is the reservoir from which entities are collected to form new constellations of objects in the intracommunicational domain.

In contrast to the extracommunicational domain, the intracommunicational domain is at the foreground of the mind of the perceiver of media products. It is formed by one’s perception and interpretation of the media products that are present in the ongoing act of communication. It is based on both extracommunicational objects (emanating from the extracommunicational domain) and intracommunicational objects (arising in the intracommunicational domain) that together result in interpretants making up a salient cognitive import in the perceiver’s mind. However, the intracommunicational domain is largely mapped upon the extracommunicational domain. Rehashing Ryan’s “principle of minimal departure” (1980: 406), I argue that one construes the intracommunicational domain as being the closest possible to the extracommunicational domain and allows for deviations only when they cannot be avoided. In other words, one does not question familiar ideas and experiences until such questioning is called for.

As the intracommunicational domain is formed by communicative semiosis, it may be called a virtual sphere. The virtual should not be understood in opposition to the actual but as something that has potential. Hence, I define the virtual as a mental sphere, created by communicative semiosis, that has the potential to have real connections to the extracommunicational. In other words, a virtual sphere may possibly represent extracommunicational objects indexically (indices being signs based on real connections); it may be truthful to the extracommunicational domain (this notion of truthfulness is further elaborated in Elleström (Forthcoming) and will be conferred in some detail in Chap. 9 in this treatise).

A virtual sphere can consist of many kinds of cognitive import. This could be anything from a brief thought triggered by a few spoken words, a gesture, or a quick glance at an advertisement, to a complex narrative or a scientific theory formed by hours of watching television or reading books. Depending on the degree of attention to the media products, the borders of a virtual sphere need not be clearly defined. As communication is generally anything but flawless, a virtual sphere may be very incomplete or even fragmentary. It may also include clashing ideas or inconsistent notions. As virtual spheres consist of cognitive import resulting from communication, they are, by definition, shareable among minds to some extent.

The coexistence of intracommunicational and extracommunicational objects results in a possible double view on virtual spheres. From one point of view, they form self-ruled spheres with a certain degree of experienced autonomy; from another point of view, they are always heavily dependent on the extracommunicational domain. The crucial point is that intracommunicational objects cannot be created ex nihilo; in effect, they are completely derived from extracommunicational objects. This is because nothing can actually be grasped in communication without the resource of extracommunicational objects. Even the most fanciful narratives require recognizable objects in order to make sense (cf. Bergman 2009: 261). To be more precise: intracommunicational objects are always, in some way, parts, combinations, or blends of extracommunicational objects. Even more precisely, intracommunicational objects are parts, combinations, or blends of interpretants resulting from representation of extracommunicational objects. For example, it is possible to represent a griffin (which, to the best of our knowledge, exists only in virtual spheres) because we are acquainted with extracommunicational material objects such as lions and eagles that can easily be combined. A virtual sphere may even include notions such as a round square, consisting of two mutually exclusive extracommunicational objects that together form an odd intracommunicational object. Literary characters such as Lily Briscoe in Virginia Woolf’s novel To the Lighthouse are composite intracommunicational objects consisting of extracommunicational material and mental objects that stem from the world as one knows it. One cannot imagine Lily Briscoe unless one is fairly familiar with notions such as walking, talking, and eating; what it means to refer to persons with certain names; what women and men, adults and children are; what it means to love and to be bored; and what artistic creation is. Also, more purely mental extracommunicational objects may be modified or united into new mental intracommunicational objects. Objects such as familiar emotions may be combined into novel intracommunicational objects consisting of conflicts between or blends of emotions that are perceived as unique, although one is already acquainted with the components. For instance, one may already be familiar with the separate emotions of feeling affection and disgust and then, through communication, have these clashing sensations merged into what one perceives as a new intracommunicational object.

The question that then arises is, if all intracommunicational objects are ultimately derived from extracommunicational objects, how come virtual spheres, narratives and others, are often experienced as having a certain degree of autonomy? This is because they, either in part or in whole, may be perceived as new gestalts that disrupt the connection to the extracommunicational domain. This happens when one does not immediately recognize the new composites of extracommunicational objects. The reason why they are not re-cognized is that they have not earlier been cognized in the particular constellation or merger in which they appear in the virtual sphere. Several such disruptions lead to greater perceived intracommunicational domain autonomy. Even though intracommunicational objects are entirely dependent on extracommunicational objects, one could say that they emerge within the intracommunicational domain.

The relation between extracommunicational and intracommunicational objects may be even more complex than hitherto indicated. Intracommunicational objects that are perceived as new gestalts might, in turn, be part of more embracing gestalts that are recognized from the extracommunicational domain. A virtual sphere can contain representations of intracommunicational objects such as living trains formed by familiar extracommunicational objects such as ‘man-made machines for transportation’ and ‘the quality of being animate and conscious’ that together form a new gestalt. However, when living trains quarrel or fall in love with each other, they interact in a way that is directly recognized from the extracommunicational domain. The conclusion is that intracommunicational objects may be interspersed among extracommunicational objects in numerous, complicated ways.

In brief, then, virtual spheres are made of clusters of objects represented by media products; these clusters form cognitive import with a certain degree of internal coherence. This may be described as intracommunicational indexicality , indices being signs based on real connections; in this case, connections within a virtual sphere (the notion of internal coherence is developed in Elleström (Forthcoming) and will be dealt with in Chap. 8 in this study). I submit that narratives should be understood as virtual spheres with certain features, to be defined in the next chapter. As virtual spheres, by definition, may be communicated, it follows that they are intersubjective to a certain extent; they may be shared among several minds.

The advantage of such a conceptualization is that narratives can be neatly compared with other forms of communicated cognitive import; they are given a theoretical as well as pragmatic framework that enables methodical investigations of both the peculiarities and the commonalities of narratives. While narration is a communicative form that is specific and important enough to deserve special attention, it is, at the same time, only a variation of, and sometimes not at all clearly delimited from, producing virtual spheres in general. Furthermore, the concept of virtual sphere offers an instrument to relate narratives and other clusters of objects represented by media products to what we perceive to be the actual world and to the background knowledge of the perceivers in a nuanced way (see also Chap. 9).

It should also be noted that the concept of virtual sphere is compatible with the common narratological concept of storyworld . However, although the latter concept is defined in a variety of ways by different authors, it is clear that the idea of a storyworld is narrower than the idea of a virtual sphere: it mainly refers to the kinds of virtual spheres that may be evoked by certain kinds of artistic media types such as literature, motion pictures, and comics, characterized by the prevalence of represented humans that act and interact in clearly perceptible spatiotemporal settings (see, for instance, Ryan and Thon 2014).

Perception, Gestalts, Image Schemes

I have suggested that virtual spheres are utterly dependent on background knowledge, which in more technical semiotic terms may be called extracommunicational objects. I have also suggested that, even though virtual spheres are composed of already known objects, narratives and other virtual spheres are generally experienced as having a certain degree of autonomy and coherence. Both of these interconnected conceptualizations are compatible with influential psychological and cognitive theories. Gestalt psychology has long taught us that, whether we want to or not, we constantly structure sensory perceptions to make them coherent and intelligible. Early on, the leading gestalt psychologist Wolfgang Köhler established that “stimulation, as such, is completely unorganized”. The result of the operation of the rules of “sensory organization”, which aim to put related things together in gestalts, is very often “a kind of reconstruction of those aspects of the objective physical situation which are temporarily lost on the way between the objects and the sense organ”; therefore, this process is in no way absolutely reliable (Köhler 1929: 177).

Modern cognitive and neurological research also confirms what many philosophers have suspected: that our perception is always an interpretation of the external world. The stimuli that reach our senses are not in themselves systematically arranged patterns that mirror actual reality but are instead a collection of more-or-less separate stimuli that the brain, on the basis of inherited skills and acquired experience, puts together into a comprehensible unity; they become meaningful by receiving form. Some information is selected and some is neglected. In fact, perception “may have evolved exclusively for extracting statistical regularities from the natural world” (Ramachandran and Hirstein 1997: 453). Although it increases our chances of survival to believe that our sensations are immediate effects of perceived external matters, and it is indeed the external world that causes our sensations, it is not the perceived external world, the world we see and feel, that causes our sensations. The perceived item (not the item in itself, of course) is actually caused by our perception of that item. Thus, as Norman N. Holland concluded in his enlightening article on neurological research from the point of view of literary reader-response criticism, the item is a projection of our sensations (Holland 2002: 29).

Neurological research has also established the idea that separate pieces of information are given meaning when perceived as coherent form, as forcefully demonstrated by gestalt psychology. This idea can actually be related to the notion that background knowledge shapes all semiosis, including the formation of virtual spheres. Perhaps the most foundational sort of background knowledge consists of our experiences of being living bodies moving within and interacting with the surrounding world, including other living bodies. Mark Johnson famously demonstrated that, as a result of these profound experiences, our minds are embodied. His concept of image schemas, understood as “preconceptual gestalt structures” that are formed by bodily experiences and various sorts of perception, gives an account of how this particular kind of background experience actually permeates the ways we think and communicate (Johnson 1987: 74). The influential psychologist Jean Mandler, who more generally emphasized the importance of perception of the outside world, stated that “children become able to think, that is, to go beyond perceptual categorization to form concepts” because “the attributes of adult concepts can be derived from the primitives of infants”. Perceptual categorization involves elements of “conceptual activity”, which means that all of our earliest concepts are based on sensory experience. Mandler proposed that “perceptual analysis results in redescriptions of spatial structure in the form of image-schemas” (1992: 587). Thus, both Johnson and Mandler stressed the close connection between bodily experiences and thinking, and both used the term ‘image scheme’ to denote the idea that bodily experiences and perception deeply affect thinking and conceptualization.

My conclusion is that the concept of image scheme also involves an idea of coherence: our thinking strives toward coherence partly because it is, to a large extent, derived from bodily perception, which our minds strive to give meaningful coherent form because it is beneficial for our survival. Importantly, sensory perceptions are not the only phenomena structured by our brains. Cognitive formations, such as those in virtual spheres, are also structured to make sense—to fit schemes. Therefore, gestalt psychological principles should also be valid for stimuli that have already been cognitively processed into conceptions. Whether we want them to or not, our minds also form thoughts, ideas, and notions into meaningful, somehow coherent gestalts. We crave structure and sense and virtual spheres are offspring of such organizing mental activities. Thus, narration is an important example of our need and inclination to represent and understand the world around us as meaningful gestalts.