Remembering events and representing time

Abstract

Episodic memory—memory for personally experienced past events—seems to afford a distinctive kind of cognitive contact with the past. This makes it natural to think that episodic memory is centrally involved in our understanding of what it is for something to be in the past, or to be located in time—that it is either necessary or sufficient for such understanding. If this were the case, it would suggest certain straightforward evidential connections between temporal cognition and episodic memory in nonhuman animals. In this paper, I argue that matters are more complicated than this. Episodic memory is memory for events and not for the times they occupy. As such, it is dissociable from temporal understanding. This is not to say that episodic memory and temporal cognition are unrelated, but that the relationship between them cannot be straightforwardly captured by claims about necessity and sufficiency. This should inform our theoretical predictions about the manifestations of episodic memory in nonhuman behaviour.

Introduction

Episodic memory—memory for personally experienced past events—seems to afford a kind of cognitive contact with the past not provided for by semantic memory. Episodic recollection characteristically involves ‘mentally reliving’ past events: when I remember falling over as a child and skinning my knee there is a sense of being ‘back there’, seeing what I saw and feeling what I felt. As Matthew Soteriou puts it, in episodic memory, the past is ‘made present’ (2018). Semantic memory, by contrast, is typically construed as a store of decontextualized general knowledge. When I recall that Ap Lei Chau is the third most densely populated island in the world, I’m recalling something learned in the past, but the past doesn’t figure in the memory. My remembering does not mentally transport me to another time.

Given that episodic memory affords this cognitive contact with the past, it’s tempting to think that episodic memory is centrally involved in our understanding of what it is for something to be in the past—or more generally, what it is for something to be located in time. Bertrand Russell expresses a version of this thought when he writes, ‘but for the fact of memory in this sense we should not know that there ever was a past at all, nor should we be able to understand the word “past”, any more than a man born blind can understand the word “light”’ (2001, p. 66). Similarly, Ludwig Wittgenstein claims that ‘a person learns the concept of the past by remembering’ (2009, pt. II §370).¹

These claims express a pair of distinct but related ideas. The first is that episodic memory is necessary for possessing a concept of the past. This claim has been labelled the ‘dependency thesis’ (Hoerl 2017); I discuss it briefly in Sect. 2. But my central interest is in the second claim: that it is through remembering that we acquire a concept of the past. A strong reading of this claim might be labelled the ‘sufficiency thesis’: it would say that episodic memory is sufficient for possessing a concept of the past. When this is married with the idea that episodic memory is a particular application of a more general faculty for mental time travel, it becomes tempting to think that episodic memory is what provides for temporal representation in general—an understanding not only of the past, but of time as a dimension stretching into both the past and the future. Thomas Suddendorf and Michael Corballis suggest as much when they write that ‘the mental reconstruction of past events […] may have been responsible for the concept of time itself’ (2007, p. 301).

This thought underpins a close association between episodic memory and temporal cognition in comparative psychological research. Investigations into whether animals have episodic memory are often framed in terms of whether they are ‘stuck in time’ (e.g. Clayton et al. 2009; Zentall 2005). Frequently, this work also features discussion of the Bischof-Köhler hypothesis, according to which humans are unique in their capacity to anticipate future desires or needs—a hypothesis which does not, on its face, concern episodic memory (Bischof-Köhler 1985). The underlying thought seems to be this: if animals lack episodic memory, they lack any capacity to represent time—and conversely, if they have episodic memory, they can represent time. So, on this view there is a straightforward evidential relationship between episodic memory and temporal cognition: evidence for temporal cognition in animals would support the hypothesis that they have episodic memory, and a lack of evidence of nonhuman temporal cognition would undermine it. As Suddendorf (2013, p. 110) writes, for instance, ‘the best evidence for episodic memory should come from signs of episodic foresight […] yet animals do not overtly express any of the obvious manifestations of such a capacity’.

My goal in this paper is to explore the relationship between episodic memory and temporal cognition, with a view to determining whether we should expect that a creature with episodic memory will be able to represent time. I argue that we should not. If we begin with an account of episodic memory which is adequate with respect to the theoretical role episodic memory is supposed to occupy, we can see that there might be creatures with episodic memory which were nevertheless incapable of temporal thought. If this is right, failure to find convincing evidence of temporal cognition would not count against the hypothesis that animals had episodic memory.² This is not to say that there are no interesting connections between episodic memory and temporal cognition. On the contrary, I suggest that there are: episodic memory may play a role in the development of temporal concepts, and temporal concepts may expand the roles that episodic memory can play in a creature’s mental life. These complex relationships should inform our predictions about how episodic memory would manifest in nonverbal behaviour.

The dependency thesis

Discussions of the relationship between episodic memory and temporal cognition have focussed largely on the dependency thesis, according to which episodic memory is necessary for possessing temporal concepts. The thesis can be fleshed out in different ways.

First, the temporal concepts under discussion may vary. Some discussion has focussed on the concept of the past, where representing something as having occurred in the past involves appreciating that it cannot be undone, and that it has an irreversible causal significance for what happens now, and perhaps in the future (Hoerl 1999). More recently, Christoph Hoerl has defended a version of the thesis according to which episodic memory is necessary for possessing a ‘realist conception of time’—representing time as a continuous domain in which all times are temporally related to one another, and which exists independently of the particular events occupying it (2017).

Second, there are developmental and constitutive versions of the thesis (Hoerl 2017). The developmental dependency thesis claims that one must have episodic memory in order to develop temporal concepts. On this view, once temporal concepts are established they can remain intact even if episodic memory is abolished. The constitutive dependency thesis makes the stronger claim that episodic memory is partly constitutive of temporal understanding—meaning that a loss of episodic memory also results in a loss of temporal concepts.

Amnesia provides reasons for doubting the dependency thesis, at least in its constitutive forms. Carl Craver and colleagues conducted interviews with the amnesiac known as K.C. to determine the extent of his temporal understanding. They found that K.C. understood that the past is ‘events that have already happened’; that ‘once an event is in the past, it will always stay in the past, and that it is not possible for someone to undo a murder at some time after the murder has occurred’ (Craver 2012, p. 465). So, it appears that K.C. retained his concept of the past. It also seems that he retained a realist conception of time. He understood time ‘as a series of events ordered earlier or later than one another, irrespective of any reference to the present’ (Craver et al. 2014, p. 192)—suggesting an understanding of time as an internally connected linear domain. He also appeared to understand the semantics of regret, saying that to regret something is to ‘wish that you hadn’t done’ it, and his behaviour suggested sensitivity to anticipated regret (Craver et al. 2014, p. 194). This is significant since regret is typically construed as a counterfactual emotion. To regret an event is to treat the time at which it occurred as a time at which something else could have happened, suggesting an event-independent conception of time—the second component of a realist conception.

Against this, Hoerl (2017) has recently argued that the interviews with K.C. suggest an impaired understanding of regret, since although K.C. was able to say something about what regret means, he was unable to think of a single thing that he himself regretted. This is surprising, given the many regrettable events in his life—including the head injury that gave rise to his amnesia—and given he knew these events had occurred. So, it is at least surprising that he did not claim to regret them. He was also unable to name anything that his mother or Richard Nixon might regret (Craver et al. 2014, p. 194). Hoerl proposes that this is indicative of a failure to appreciate the event-independence of time—a failure to realise (for instance) that the time at which his head injury occurred, something else might have happened instead (2017, p. 215).

Whilst I doubt that Hoerl intends K.C.’s remarks to be more than suggestive evidence, there are limits to the conclusions we can draw from cases of amnesia. If somebody with amnesia has a certain capacity, then it is safe to assume that episodic memory is not necessary for that capacity. But if someone with episodic memory lacks a capacity, it is not safe to conclude that episodic memory is necessary for that capacity. First, brain damage is rarely so localised as to be limited to the brain areas responsible for episodic memory, often coming along with damage to other unrelated brain areas. If K.C. suffered damage to other parts of his brain, this may explain his lack of regrets. Second, brain areas often perform more than one function. The hippocampus, central to episodic recollection, also plays a central role in the construction of imagined scenarios (Hassabis et al. 2007). It may be that K.C. struggles to name things that he regrets not because he lacks episodic memory, and perhaps not even because he lacks an event-independent understanding of time, but because he is unable to imaginatively construct counterfactual scenarios that would not have been regrettable, making the possibility of regret less salient. So, it is not clear that there’s any particularly compelling reason to think that K.C. lacks a realist conception of time, or—if he does—that this is as a result of his episodic memory deficits.

All this casts doubt on the dependency thesis, at least in its constitutive form—but one might note that since K.C. was an adult-onset amnesiac, his case leaves the developmental dependency thesis untouched. There is little published work that bears directly on this, meaning there is little positive reason to accept it beside its intuitive plausibility. Moreover, it’s reasonable to think that if there were systematic deficiencies in temporal understanding in developmental amnesia, this would have been noted and reported. Yet, to my knowledge, no studies of developmental amnesia do report this. In fact, one widely known account of three individuals with developmental amnesia found that although they displayed the usual impairments with respect to memory, navigation, keeping appointments and so on, they achieved strikingly average performance in mainstream education and on a range of cognitive tests (Vargha-Khadem et al. 1997). Their language skills were normal, and they understood and replied to questions in the past tense. So, there is at least some reason to doubt the developmental dependency thesis, too.

Of course, this is by no means a conclusive refutation of the dependency thesis. And of course, even if it turned out that episodic memory was not necessary for the development and maintenance of temporal understanding, it might nevertheless play a role in the development and maintenance of such understanding in healthy individuals. It might even be that episodic memory is sufficient for such understanding. But this all leaves open a substantial question: what is it about episodic memory that is supposed to underpin its connection to temporal cognition? How, precisely, might episodic memory be involved in furnishing a (healthy) individual with an understanding of time? As I argue in what follows, it is just not clear that episodic memory is the kind of thing that could furnish one with temporal understanding.

What is episodic memory?

Saying anything about the relationship between episodic memory and the representation of time requires arriving at a view about what episodic memory is. This is not an easy question, and what I say about it in this section will be unavoidably brief. In particular, I shall largely set aside an ongoing debate in the metaphysics of memory between causal and simulationist accounts of memory, on which the following should be neutral (see Michaelian and Robins 2018). My concern here is with what features should be taken to characterise episodic memory for the purposes of psychological research. I begin by introducing Endel Tulving’s account of episodic memory, before arguing that a more inclusive account in which episodic memory is more sparsely characterised better satisfies the theoretical ends for which the term ‘episodic memory’ was introduced.

The term ‘episodic memory’ was introduced by Tulving in (1972). He wrote that ‘most terms referring to different kinds of memory serve the function of dividing some larger domain of memory […] into smaller areas within which empirical observations and theoretical propositions are thought to be generalisable. Such divisions frequently take the form of a dichotomy: two complementary categories exhaust a superordinate category’ (1972, pp. 382–383). It was with this in view that he proposed the term ‘episodic memory’, to pick out that part of declarative memory which is not semantic memory. At root, the distinction was between the sort of memory involved in remembering the meaning of the word, and that involved in remembering an occasion on which the word was presented. As Tulving sometimes points out, the distinction is somewhat captured by the contrast between ordinary uses of the words ‘remember’ and ‘know’.

Tulving went on to offer and develop a theoretical account of episodic memory. In early work, he characterised episodic memory as a store of information about ‘temporally dated episodes or events, and temporal-spatial relations among these events’ (1972, p. 385); the content of an episodic memory on this view is of the form ‘I did such and such, in such and such a place, at such and such a time’ (1972, p. 389). This was later distilled into the ‘what-where-when’ criterion for episodic memory, which was the basis for the pioneering study on nonhuman episodic memory in scrub-jays (Clayton and Dickinson 1998). However, Tulving came to doubt that the distinctive features of episodic memory could be captured in terms of this kind of representational content. One part of the worry is that semantic memories can have what-where-when content. Another is that many episodic memories seem to lack one or more of these kinds of content. In later work, he focussed on episodic memory’s phenomenological features. In particular, he proposed that episodic memory characteristically involves a distinctive form of self-consciousness he calls ‘autonoesis’, as well as a special awareness of time he labels ‘chronesthesia’ (2005, p. 14).

It has become more or less conventional to defer to Tulving on the question of what episodic memory is. One reason that this is problematic is that Tulving offers little theoretical clarity about what autonoesis or chronesthesia, which he takes to be its central components, are supposed to be. He characterises chronesthesia as an awareness of ‘subjective time’, which is ‘related to but not identical with physical time’ (2005, p. 16). Subjective time is described as the time ‘in which we exist’ (2002, p. 311), the time ‘through which we mentally travel’ (2002, p. 311) and ‘the thought about time in which one’s personal experiences take place’ (2002, p. 313). These ideas are obscure and as I’ve argued elsewhere (Boyle 2019b, p. 5), non-equivalent. Autonoesis, in turn, is characterised as ‘the kind of conscious awareness that characterises conscious recollection’ (2005, p. 15), and as the possession of a ‘projectable, or time-travelling, or remembering self’ (2005, p. 29). The former is not explanatory; the latter is difficult to disentangle from chronesthesia. Tulving also offers a number of non-equivalent claims about the relationship between episodic memory and autonoesis—characterising the latter variously as something that ‘accompanies’ episodic memory (2005, p. 11), an ‘integral component’ of episodic memory (2005, p. 31), a ‘medium’ through which episodic memory occurs (2005, p. 9), and something on which episodic memory ‘critically depend[s]’ (2005, p. 30).

Moreover, both autonoesis and chronesthesia are treated as distinctive forms of awareness, which can’t be reduced to or captured in terms of the representational content of memory. And yet both are described in richly conceptual terms—leading some to interpret them as requiring a high degree of cognitive sophistication. Kourken Michaelian, for instance, writes that ‘in light of the cognitive sophistication presupposed by the phenomenology of mental time travel, which involves both a sense of self and a sense of subjective time, it seems unlikely that nonhuman species are capable of undergoing the relevant phenomenology’ (2016, p. 208). It isn’t entirely clear how to reconcile the idea that these features of episodic memory are forms of experience with the idea that they require this cognitive or conceptual sophistication.

Quite aside from these interpretative issues, there is another reason it is problematic to treat Tulving’s accounts of episodic memory as authoritative on the question of what episodic memory is—namely, that these accounts are highly speculative. Tulving acknowledges this: for instance, he offers chronesthesia up as an idea for consideration, but grants that there are legitimate questions about whether it exists (2002, p. 321). He does offer some evidence in support of his theoretical claims—but this evidence, primarily drawn from cases of amnesia, seems to bear more on the existence of a distinction between semantic and episodic memory than on more speculative claims about unique forms of awareness characteristic of the latter. For instance, Tulving claims that K.C. is deficient in autonoetic awareness. But the evidence offered in support of this claim is simply that K.C. is ‘densely’ episodically amnesic (2005, p. 26)—and it is unclear that this motivates any further claim about his sense of self.

How, then, ought we to characterise episodic memory? Here, I think it’s helpful to return to the purposes for which the term ‘episodic memory’ was introduced. First, the term was introduced to mark an exhaustive division in declarative memory, between the kind of memory involved in knowing things and that involved in recollecting—where ‘episodic memory’ picks out the latter. Second, the term is supposed to pick out a class ‘within which empirical observations and theoretical propositions are thought to be generalisable’. To a philosopher, it’s natural to paraphrase this in terms of natural kinds: the distinction between episodic and semantic memory is supposed to mark a ‘joint’ in nature, and ‘episodic memory’ is intended to pick out a psychological natural kind falling on one side of this joint. So, if there is no psychologically real division in declarative memory, the term ‘episodic memory’ will fail to refer.

A compelling reason for thinking that there is a psychologically real division in declarative memory is provided by the existence of functionally different forms of amnesia with different causal histories. Damage to the neocortex results in semantic memory loss, or (less question-beggingly) an inability to retrieve previously acquired general knowledge, but does not result in an inability to recollect past events. Damage to the medial-temporal lobe (MTL)—such as that suffered by K.C.—results in a loss of memory for past experiences, but does not damage existing stores of general knowledge. The dissociations between the two types of memory aren’t clean, since general knowledge and memory for events seem to interact in various ways. Most dramatically, individuals with MTL amnesia are also severely impaired in their ability to acquire new general knowledge, even whilst their existing stores of general knowledge remain intact (Elward and Vargha-Khadem 2018; Manns et al. 2003). Nevertheless, this pattern of dissociability suggests that there is a psychologically real distinction to be drawn—that within declarative memory there are two less inclusive kinds of memory about which theoretical and empirical claims might generalise.

This makes it reasonable to treat ‘episodic memory’ as picking out the kind of declarative memory that isn’t semantic, and which is lost in MTL amnesia. David Rubin and Sharda Umanath (2015) propose that this is a very inclusive kind—more inclusive than Tulving’s accounts of episodic memory would suggest.³ They argue that ‘the fundamental natural kind that is an opposition to knowledge (i.e., semantic memory)’ (p. 2) is what they term ‘event memory’, where an event memory is one involving ‘a mental construction of a scene recalled as a single occurrence’ (p. 1)—that is, involving a ‘single spatially organised scene’ (p. 7). On their account, ‘the scene can be experienced as happening to the person recalling it or imagined as happening to another person’—that is, in either field or observer perspective (p. 1). It may, but need not, represent actions, actors or emotions (p. 1), and it might result from the encoding of a single event, or be a merging together of several encodings of similar events (p. 2). In the latter case, Rubin and Umanath suggest, the remembering subject might take the memory to represent a single event or to represent a ‘prototype’ event (p. 7). Their claim that event memories are ‘recalled as a single occurrence’ should consequently not be taken to indicate that an event memory must represent, or be taken by the subject to represent, a particular event that occurred at a particular point in time. Rather, it indicates that the retrieved event details are integrated into a single spatial representation as of a single scene or event.

Rubin and Umanath argue that this inclusive category is what’s required if the goal is to exhaustively partition declarative memory. The episodic/semantic memory distinction suggested by Tulving’s account, they argue, ‘provides no clear classification for memories that are recalled as single events but that are imagined from the perspective of a protagonist who is not the person recalling the event, or that are the merging of more than one similar occurrences, or that are recalled without a sense of reliving’ (p. 2).

The point is not that Tulving’s account leaves us with marginal cases in which it’s unclear how a given memory should be classified. The natural world is messy enough that we should expect marginal cases to arise for any plausible account of episodic memory. Nor is it that, for instance, autonoesis isn’t essential to episodic memory, since there are some cases of memories that would naturally be described as episodic, in which we remember from another’s perspective. Again, messiness in the natural world makes it unreasonable to expect an account of episodic memory to define its ‘essence’ or to give necessary conditions for its occurrence. Prominent accounts of natural kinds allow for precisely these kinds of messiness (e.g. Boyd 1989, 1991).

Rather, the point is that the class of declarative memories that are not semantic is far more diverse than Tulving’s account of episodic memory would suggest—it includes many memories that don’t involve any rich sense of self-awareness or temporal extension, memories constructed from third-personal perspectives and memories that aren’t for unique events, as well as memories that lack what-where-when content. An adequate account of this class of memories need not provide necessary and sufficient conditions for class membership, but ought to capture what, in general, members of the class have in common and what underpins those commonalities. Rubin and Umanath argue that what unites these memories is that they involve spatially structured scene constructions, and that this similarity is underpinned by the involvement of ‘neural structures in the medial temporal lobes and the visual ventral stream that are needed for the construction and experiencing of scenes’ (2015, p. 8).

At this point one might worry that event memory, as described here, scarcely qualifies as a form of memory at all. We deploy the capacity to construct integrated, spatially structured representations of scenes when we invent scenarios out of whole cloth. On the assumption that these imaginings are not memories, there must either be more to event memory than has been said here, or event memory is not a form of memory at all.⁴ The answer is that there is more to event memory than the construction of spatial scenes. Rubin and Umanath are primarily concerned with drawing a distinction between two forms of memory, rather than between memory and imagination. On the account they offer, event memories are those memories which involve the construction of a single spatially organised scene, and can be contrasted with semantic memories, which do not. This takes the mnemonic status of event memories for granted, leaving open the question of precisely what this mnemonic status consists in, and how event memories differ from mere imaginings of events.

What makes something a memory, and in particular what distinguishes memories from mere imaginings, is the subject of a rich and ongoing debate. There are a wide range of views on this question, but speaking very broadly, we can characterise this as a debate between causal and simulationist theories of memory (De Brigard 2017; Michaelian and Robins 2018). Causal theorists argue that a memory must stand in an ‘appropriate causal connection’ to the remembered event (Bernecker 2010; Martin and Deutscher 1966). Simulationists, on the other hand, have argued that memories are scene constructions produced by systems or processes that tend reliably to produce accurate representations of past events (Michaelian 2016), whether or not this involves a causal connection of the sort invoked by causal theorists. Rubin and Umanath characterise event memory in the language of encoding and retrieval, perhaps suggesting an affinity for the causal view: event memories are constructed out of information encoded during the event(s) being represented. But their account is compatible with either position on the metaphysics of memory, and since nothing in what follows turns on the choice between them, the reader may continue with their preferred account in mind.⁵

Summarising, Rubin and Umanath argue that the contrast class for semantic memory is event memory, where event memories are those memories involving the construction of a single spatially organised scene. In offering this account, Rubin and Umanath do not explicitly take themselves to be offering a revisionary account of episodic memory—they seem happy to surrender the term ‘episodic memory’ to those who, like Tulving, want to build more phenomenological and conceptual richness into the phenomenon. So, they use ‘episodic memory’ to pick out a special case of event memory. It’s worth acknowledging that others have also used the terms ‘episodic memory’ and ‘event memory’ to draw a contrast between more and less complex forms of memory for events. For instance, Johannes Mahr and Gergely Csibra (2018) reserve the term ‘episodic memory’ for event memories embedded under a metarepresentational attitude of remembering. And Nazim Keven (2016) argues for a distinction between event memories which are perceptual snapshots of earlier events and episodic memories, which also have temporal and causal structure and relate remembered events to the subject’s goals.

However, I have argued that there are good reasons for taking ‘episodic memory’ to pick out the type of declarative memory that is not semantic: marking this fundamental division in declarative memory is the central theoretical work the episodic/semantic distinction is supposed to do. And Rubin and Umanath argue that the basic division in declarative memory is not between episodic memory, in any of these narrower senses, and semantic memory, but between event memory, in their sense, and semantic memory. So, I suggest that we should take their account to be an account of episodic memory. That is, we should take episodic memory to be event memory, in Rubin and Umanath’s sense: to have an episodic memory is to have a memory involving a single spatially organised mental construction of a scene.⁶ In what follows, I shall explore the relationship between episodic memory and representing time with this inclusive conception of episodic memory in mind. I return to narrower conceptions of episodic memory towards the end of Sect. 4.

Representing time

When Russell claims that episodic memory is at the heart of our understanding of the past, he writes that this is because episodic memory gives us ‘immediate knowledge’ of the past (2001, p. 26). Similarly, Hoerl (1999, p. 246) argues that episodic memory is central to possessing a concept of the past because episodic memories make one ‘directly aware of […] the fact that past events impose certain constraints on the present’. These claims of ‘direct’ or ‘immediate awareness’ suggest a view of episodic memory according to which having episodic memory is sufficient for possessing a concept of the past. But the account of episodic memory as event memory, offered in Sect. 3, makes this view difficult to sustain. On that view, it is not clear why one ought to think that an episodic memory of an event will wear its causal or temporal connection to the present on its sleeve, such that the temporal or causal relationship a remembered event stands into the present will be transparent to the remembering subject.

The issue is not with the idea that episodic memory involves ‘immediate’ or ‘direct’ awareness of a remembered event. Let’s take ‘immediate’ or ‘direct’ to mean something like ‘not mediated by inference’, and let’s grant for the sake of argument that episodic memory provides immediate or direct awareness to remembered events. What is not clear is why we should think that episodic memories provide immediate or direct awareness of precisely this feature of a remembered event: that it occurred at some (particular) past time and constrains the present. This is not obvious because episodic memories are spatial scene constructions, which need not contain temporal information. Of course, many episodic memories do have what we might call an ‘internal’ temporal structure, which is to say that they may carry information about the remembered event’s duration or the order in which its constituent parts occurred (see Boyle 2019b; Cheng and Werning 2016). So, insofar as they provide direct access to remembered events, it’s reasonable to take them to provide direct access to these internal temporal properties. It is quite another thing to suppose that, in remembering, one is directly aware of what we might call an event’s external temporal properties—that is, where in time the remembered event is situated. This just isn’t central to what episodic memories are, on the account I’ve offered.

One might be tempted to treat this as an oversight in that account: perhaps we should add that episodic memories provide for a direct awareness of events as having occurred at some (particular) past time. But in fact, there are good reasons for not taking direct awareness of external temporal properties to be a characteristic feature of episodic memory. First, there are clearly a great many cases in which we have episodic memories for events we are unable to temporally locate. I have a certain memory of taking a trip to the beach, but could not tell you when this happened. Second, insofar as we are able to locate remembered events in time, this is unlikely to be because memories make one directly aware of a remembered event’s external temporal properties. Rather, it is because we are able to exploit our general knowledge about temporal patterns and about our life history, and use contextual information provided by the memory to infer its location in time (Friedman 1993, p. 58). For instance, I know that when I fell over and skinned my knee, I must have been between the ages of two and seven, since this memory unfolds in the garden of a house I know I only lived in between those ages. But my memory of the beach trip is sparse, and provides me with no contextual cues I can use to locate it in time.

Of course, one might point out that I am still at least able to say that this beach trip happened at some point in the past—I know that my representation of this event is not the product of imagination. This is true—but it’s again unclear that this is to be explained in terms of an immediate or direct awareness of the pastness of the event, such that any subject who had this memory would necessarily recognise the remembered event as belonging to the past. Whilst my memory may have a phenomenal quality that my imaginings typically lack, as a result of which I am inclined to believe it represents a past event, we should not be too quick to assume that this felt quality amounts to a direct awareness of the pastness of events.

The ability to distinguish between memories and imaginings comprises two problems: the source problem, and the process problem (Michaelian 2016, Chapter 9). The source problem is the problem of determining whether information currently being remembered has its origins in experience, testimony, my previous imaginings, or some other source. The process problem is the prior problem of determining whether the cognitive process in which I am currently engaged is one of remembering or imagining. According to the ‘source monitoring’ framework, the source problem is solved by metacognitive processes which ‘infer’ the likely source of a memory on the basis of features of the memory itself—such as the amount and type of perceptual detail it provides, and its fit with background knowledge and other memories (Johnson et al. 1993). The underlying thought is that there are average differences with respect to these features between memories with different sources. Michaelian (2016, pp. 194–198) argues that the process problem is solved in a similar way, by a metacognitive process that infers the likely cognitive process based on features of the representation it outputs, including affective valence and intensity, level of detail, and familiarity.

Whilst these inferences about the source of a representation may sometimes be drawn consciously on the basis of deliberate reflection, in many cases they occur below the level of awareness: ‘heuristic source attributions take place constantly without notice, and are relatively automatic or effortless’ (Johnson and Raye 2000). In the latter case, these ‘inferences’ cannot plausibly be attributed to the agent, but are better understood as ‘occurring at the level of the system’ (Michaelian 2016, p. 165). In these cases, these processes give rise to phenomenal differences between memories and imaginations, in the form of ‘metacognitive feelings’ (Michaelian 2016, p. 196). These feelings may ground a willingness to assign memories episodic epistemic significance that is not to be assigned to imaginings, without the agent being aware of the reasons for their willingness.

Given what we—adult humans—know about the relationship between memory and the past, the presence of metacognitive feelings characteristic of memory gives us a reason to think that the remembered event belongs in the ‘personal past’ (Johnson et al. 1993, p. 21). As a result, we may be inclined to call this feeling a ‘feeling of pastness’. Nevertheless, the feeling is not a direct awareness of the pastness of the event, but the qualitative outcome of a metacognitive monitoring process. It might just as well be designated a feeling of reality, authenticity, epistemic significance or anything else: it is simply a feeling that prompts subjects to assign the memory an epistemic significance not to be assigned to imaginings. In a creature lacking temporal concepts, similar metacognitive feelings could ground a capacity to discriminate memories from imaginings, and to assign epistemic significance to the former but not the latter, without giving rise to a belief that remembered events occurred at some point in the past.

The point, for my purposes, is that if episodic memory and locating remembered events in the past are separable phenomena, then there could be creatures which have episodic memory and which are nevertheless unable to locate remembered events in the past. And I’ve argued that these are separable phenomena. Having episodic memories, in the sense I’ve outlined, does not entail appreciating the pastness of those events, and nor does the capacity to discriminate episodic memories from imaginings. If that’s right, then we ought not to expect (for instance) that creatures with episodic memory will treat remembered events as past in Hoerl’s sense of causally constraining the present, or that they will possess any other kind of temporal understanding.

But, one might reasonably ask, what is it to assign a memory epistemic significance, if it is not to treat it as representing a past event—one that happened at a particular (perhaps unknown) point in the past? And moreover, what good can episodic memory do, if it does not involve locating remembered events in time? It is one thing to say that remembering events and locating them in time are separable phenomena in theory, but quite another to say what cognitive work ‘temporally unanchored’ episodic memories might do. If they could play no useful or adaptive role in a creature’s cognitive economy, it’s unclear why we should take seriously the possibility that any creatures have them. Hoerl and McCormack (2019, p. 4) express this kind of concern, noting that it is unclear how such temporally unanchored event representations could systematically or purposively guide action.

Broadly speaking, I suggest that assigning epistemic significance to a memory means taking it to accurately represent some feature of the world, being inclined to form beliefs about the world on this basis and treat those beliefs as rational constraints on decision making. Standardly, we take episodic memories to have epistemic significance with respect to what happened in the past (Debus 2010). But as I’ll explore in more detail below, there are at least two other ways in which one might take an episodic memory to be epistemically significant, besides by taking it to provide information about the past. First, one might take a memory to accurately represent how things are in the world now. For instance, suppose I have a memory in which my path is blocked by a fallen tree. Even if I do not recognise that this memory represents a past event, I might take it to have epistemic significance with respect to how the world is now: the path is blocked by a fallen tree. Second, one might take a memory to provide information about kinds of events, indicating what tends to happen in certain circumstances. For instance, suppose I have a memory of being bitten by the neighbour’s dog in her yard. Again, without recognising that this memory represents a past event, I might take it to have epistemic significance with respect to the sort of event that’s likely to occur if I go into the neighbour’s yard. Taking episodic memories to have epistemic significance in either of these senses provides for several roles episodic memories might adaptively play in the cognitive economies of creatures lacking any temporal concepts.

First, as Hoerl and McCormack (2019) argue, creatures which do not represent time may nevertheless have what they call a ‘temporal updating system’—that is, a system for keeping track of the current state of the world. One way to do this would be to have a single map-like representation of one’s environment, and update it as one goes about the world to reflect the arrangement of objects one finds. But this is not the only way to keep track of the state of things. One might also store an episodic memory relating to one’s most recent experience at various places and consult this as a record of how things are arranged in the world. So, for instance, suppose I am wondering where my keys are, and a recent memory occurs to me in which my keys have fallen behind a piece of furniture. Even if I lack a concept of the past, I might take this memory to provide information about where the keys are now, which might prompt me to look for them behind the furniture. A creature who used episodic memories only in this way—as a source of information about how things are—would have no need for locating remembered events in time.

One might object that using episodic memories as part of a temporal updating system in this way could not possibly be adaptive.⁷ If subjects do not believe that the events being remembered in fact occurred, it would be irrational for them to use these memories to inform their model of the present state of the world. But whether it is rational for subjects to form beliefs in this way is a distinct question from whether it is adaptive. It would be maladaptive if subjects were disposed to form beliefs on the basis of events which did not happen but were only imagined—since this would tend to result in their having false beliefs about the world. As noted above, however, metacognitive source monitoring processes enable subjects to discriminate between memories and imaginings, even without access to what grounds this discrimination. Sufficiently sensitive source monitoring mechanisms could ensure that an agent reliably forms beliefs on the basis of memories and not on the basis of imaginings, avoiding the maladaptive consequences of the latter, regardless of whether the agent has any beliefs about the temporal status of remembered events.

Second, I’ve argued previously (Boyle 2019a) that storing episodic memories of events increases opportunities for explicit semantic learning—something which may be helpful in situations where it is not obvious what the ‘learning points’ are. Sometimes we are confronted with events which are difficult to interpret, meaning that we can’t at the time draw out any useful information about how to anticipate or respond to these situations. The capacity to store detailed memories of events enables us to revisit them later on, reinterpreting them in the light of more recently acquired knowledge, and extracting relevant information. Again, being able to locate remembered events in time is not important for this purpose. One need not construe the remembered event as something which happened in the past—one might simply view it as an epistemically significant representation of a certain kind of event.⁸

For instance, suppose I am a young chimpanzee learning to fish for termites, and I notice that one of my conspecifics is having much better luck than me. But let’s suppose that I cannot work out what the relevant difference is between me and my conspecific. If I lack any understanding of time, but retain a memory of the event, I might treat it as representative of a certain kind of thing that happens: my conspecific getting lots of termites, using a certain kind of tool, on a certain kind of day. On a later occasion when this or another conspecific is harvesting more termites than me, I might revisit this memory. The commonalities between the current situation and the kind of event I remember might suggest something: perhaps in both cases, my conspecific uses a tool modified with a ‘brush’ tip, whilst I use an unmodified plant stem. This might prompt me to try this technique, and to learn that the brush tool is more effective.⁹ As before, for this use of episodic memories to be adaptive, it is not necessary that I construe remembered events as located at some point in the past, so long as the representations I use in this way are in fact of events located at some point in the past—that is, so long as I reliably use memories in this way, and not mere imaginings. Once again, this discrimination between memories and imaginings can be achieved through source monitoring processes which do not require agent-level judgments about the location of events in time.

Third, episodic memories also seem to play a role in supporting more implicit semantic learning. There’s good evidence in humans that episodic memory plays a role in facilitating the rapid acquisition of semantic information. As noted above, individuals with episodic amnesia—both developmental and adult-onset—are impaired when it comes to the acquisition of semantic information. One explanation of this is that information is initially encoded in episodic memory and then ‘replayed’ for the benefit of semantic learning mechanisms, which can gradually extract information from them. Without episodic memory, many more ‘real’ exposures to information are required. In this way, episodic and semantic memory act as ‘complementary learning systems’ (McClelland et al. 1995). Again, it does not seem important for episodic memory to play this role that subjects be able to locate remembered events in time: treating these memories as epistemically significant representations of events of certain kinds is sufficient. Of course, for this process to be adaptive, semantic memory must reliably extract information from memories of events that did in fact occur, and not from mere imaginings. But, again, the subject need not be capable of temporal thought for this discrimination between memories and imaginings to be achieved.

The above considerations all concern episodic memory’s relationship to locating remembered events in the past. This leaves open the relationship between episodically remembering and having a realist conception of time. Of course, if a creature did not even locate remembered events in the past, it’s unlikely that she would possess a realist conception of time. But suppose that a creature does locate events in the past, in Hoerl’s sense—that is, she treats certain events as being irrevocable, and as causally constraining the present. Should we thereby expect her to have a realist conception of time—that is, to represent time as a domain in which all points are temporally related to all other points, and which exists independently of the events occupying it?

There seems little reason to think so. First, to represent something as past, in this sense, is to represent it as something which did happen. It is unclear how having such a representation would give rise to a judgment to the effect that something else might have happened at that time instead, and thereby to a conception of time as event-independent. Second, to represent something as past in this sense is at most to represent its temporal relationship to the present: because it has happened, it makes an irrevocable difference to how things are now. But this is to say nothing about the temporal relationships it (or the time it occupies) stands into other times—and so it falls short of conceiving of time as a continuous domain. Teresa McCormack and Christoph Hoerl (2017) propose that children may develop this minimal understanding of the difference between past and future events between 2 and 3 years old. But this capacity to discriminate those situations which can still be altered from those which, having happened, cannot be altered differs considerably from having ‘one unified model of the world within which time itself is represented’ (Hoerl and McCormack 2019, p. 11).

Locating a remembered event in the past, even in this minimal sense, increases the cognitive work that one’s memory of that event can do. For one thing, recognising that an event has happened and cannot be undone might lead one to realise that there is no point dwelling on it (McCormack and Hoerl 2017, p. 310). More concretely, if one treats the remembered event as constraining the present, one might draw certain inferences about what must be true now, or what I ought to do now, given that the event occurred. For instance, remembering an occasion on which I saw a sapling in a certain spot, I might infer that there is likely to be a mature tree there now. Similarly, remembering an interaction in which I was unduly harsh toward someone might lead me to adopt a more conciliatory attitude when I meet them now. But treating events as past in this sense does not provide for a sense of the temporal relationships between remembered events, which limits the rational uses to which memories can be put. For instance, suppose that I remember seeing Tina in the garden, and that I also have a distinct memory of finding a book in the garden. If I represent these events as past but can’t conceive of any temporal relations between them, certain inferences—such as that Tina left her book in the garden—will be unavailable to me.

On the view I’ve proposed, episodic memory is memory for events, not memory for the times they occupy: one can remember events without representing the times they occupy, and without representing those times as event-independent. Episodic memories which fail to locate events in the past or in event-independent time may nevertheless play a number of productive roles in a creature’s cognitive economy. So, we should take seriously the possibility that some creatures—perhaps nonhuman animals and young children—have episodic memories, and yet lack any kind of competence with temporal concepts. If this is right, then the straightforward evidential link I considered above does not obtain: we should not infer from a lack of evidence for temporal cognition in a creature that the creature lacks episodic memory.

In this section, I have assumed a relatively inclusive picture of episodic memory, according to which episodic memories are those memories involving a single spatially organised construction of a scene. As I noted above, there exist a number of more restrictive accounts of episodic memory, which view episodic memories as a proper subset of the memories I have been discussing, characterised by the possession of further distinguishing features. So, one might worry that perhaps when philosophers have found it plausible to claim that episodic memory furnishes us with our concepts of the past or of time, they had one of these less inclusive conceptions of episodic memory in mind.¹⁰

Since it is difficult to rule this out, my argument can be viewed as limited in scope. I’ve argued that there are good reasons to treat episodic memories as memories involving spatially organised constructions of scenes; certainly, this idea is at the heart of any plausible account of episodic memory, even where it is taken to have other distinguishing features. I’ve argued that this kind of memory does not provide for an understanding of the past: remembering events does not entail representing time. Of course, this does not rule out that on some narrower construal of episodic memory, it does furnish us with a concept of the past, in virtue of other features by which it is characterised. Nevertheless, I think there are at least two reasons for scepticism.

First, the prima facie reason for thinking that episodic memory furnishes us with temporal understanding is that it provides for direct awareness of the pastness of events. But I’ve argued that whilst episodic memory may provide for direct awareness of past events, there is no reason to think that it provides for direct awareness of their pastness. In particular, the most plausible candidate feature of episodic memory for providing a direct awareness of pastness is the so-called ‘feeling of pastness’. But, I’ve argued, this is not a direct awareness of the pastness of events, but the felt outcome of a metacognitive process enabling subjects to discriminate memories from other cognitive phenomena. Since there is nothing intrinsically ‘past-ish’ about this feeling, it is difficult to see how it could furnish a creature with temporal concepts. Rather, I’ve suggested, we may call it a feeling of pastness because we possess temporal concepts. This undercuts a central motivation for thinking that episodic memory furnishes us with temporal understanding.

Second, one might suspect that on at least some narrower construals of episodic memory, it depends developmentally on the possession of temporal concepts—and if so, it could not be true that it furnishes us with them. For instance, take Mahr and Csibra’s construal of episodic memories as those event memories agents explicitly take themselves to remember. Since understanding what it is to remember something entails knowing what it is for something to be in the past, remembering in this sense is clearly sufficient for temporal understanding. But this tells us nothing about the development of temporal understanding, unless we have an account of our capacity to judge that we remember—and one might reasonably suspect that this turns on the acquisition of temporal concepts. Similarly, take Keven’s account, according to which episodic memories are characterised by the possession of causal, temporal and teleological structure, and not merely spatial structure. Keven argues that in humans, the capacity for narrative storytelling is key to the development of this more sophisticated form of memory—and one might think that developing the capacity for narrative storytelling involves acquiring at least some facility with temporal concepts. If that’s right, then episodic memory in Keven’s sense is unlikely to be what furnishes us with temporal concepts. More generally, making the case that episodic memory (on some narrower construal) furnishes us with temporal concepts requires giving an account of how episodic memory (in this narrower sense) might develop in the absence of temporal understanding. But there is at least some reason to suspect that matters are more complicated than this: that more sophisticated forms of episodic memory are developmentally entangled with an emerging understanding of time.

Of course, it may be that even if episodic memory does not furnish us with a concept of the past on these narrower construals, it nevertheless stands in interesting evidential relationships to temporal cognition. For instance, clearly, if a creature has episodic memory in Mahr and Csibra’s sense, then it must have some understanding of what it is for an event to be past; this much is true by definition. But these evidential links are unlikely to be entirely straightforward. We could not assume, for instance, that a creature possessing a concept of the past would have episodic memory in Mahr and Csibra’s sense since, aside from anything else, this also requires a capacity for explicit metacognitive judgment. Similarly, there is no obvious reason to expect that a creature possessing episodic memory in Mahr and Csibra’s sense would have a realist conception of time, or any concept of the future. So, whilst evidential connections may exist between narrower construals of episodic memory and temporal cognition, determining the precise nature of those connections requires careful attention to the details of the relevant construal of episodic memory, and to the varieties of temporal cognition.

Conclusions

In Sect. 2, I argued that the evidence of amnesia casts some doubt on the dependency thesis, according to which episodic memory is necessary for possessing the concept of the past, or a realist conception of time. In Sects. 3 and 4, I defended an account of episodic memories according to which they are event memories—memories involving spatially structured scene constructions of events—and argued that on this account, episodic memory is not sufficient for possessing temporal concepts. There could be creatures with episodic memory which lack any capacity to represent time. None of this, however, shows that episodic memory and temporal cognition are unrelated. Quite the opposite: it suggests that the relationship between episodic memory and temporal understanding may be more complex than the dependency and sufficiency theses indicate.

First, to say that episodic memory is neither necessary nor sufficient for temporal understanding does not rule out that it plays a role in the development of temporal understanding in healthy individuals. In fact, it seems likely that the two develop in parallel. Linguistic instruction may be sufficient for developing temporal understanding even in the absence of episodic memory. But where temporally unanchored episodic memories are present, they provide a resource for introducing temporal notions through something like ostension. Utterances like ‘remember the time when we…?’ might prompt the retrieval of a temporally unanchored memory, telling children something about the significance of temporal terms and tense. These collective acts of reminiscing, as well as providing children with a foothold on temporal properties and relations, also provide their memories with an organisational structure, locating them within an increasingly complex temporal framework (Nelson and Fivush 2004), and perhaps facilitate the development of more complex forms of memory. Consistently with this, cross-cultural data suggest a correlation between the frequency and elaborateness with which parents reminisce, and both the age at which children report memories, and the level of detail in their memory reports (Van Bergen and Sutton 2019). As Penny Van Bergen and John Sutton (2019) argue, this provides for a sense in which autobiographical memory, understood to be a compound capacity involving episodic memory alongside other components, perhaps including an understanding of time, could be what Cecilia Heyes (2018) calls a ‘cognitive gadget’—a complex cognitive capacity which is culturally transmitted, but which builds on a simpler genetically inherited capacity. If anything like this view is correct, then it is not that remembering and representing time are unrelated, but that their relationship cannot be straightforwardly captured in terms of necessity and sufficiency—and this is so whether we are talking about episodic memory, in the sense offered here, or a more complex, compound capacity like autobiographical memory.

Second, although I have been arguing against a straightforward evidential connection between temporal cognition and episodic memory in nonhumans, there are nevertheless theoretically important relationships between these two capacities. It should be clear from the foregoing that there are roles that episodic memory plays in the mental lives of humans only because we are able to locate remembered events in time. Lacking the temporal concepts this requires would correspondingly limit the cognitive work a creature’s episodic memories could do. If a creature remembers an event but does not represent it as past, then she will not appreciate the way in which the remembered event constrains the present. For instance, as noted above, she will not be able to infer that there is likely to be a mature tree in a given spot now given that she saw a seedling there earlier—since she does not locate her memory of the seedling in the past. So, whilst her memory may play a number of adaptive roles in her cognitive economy, there will nevertheless be limits to the uses she can put it to. Moreover, if she represents an event as past but not as occupying event-independent time, she will not understand certain things about how it temporally and causally relates to other remembered events, or about how things might have been different. This, again, will limit the role that her memory can play in her rational thought and action. So, if the evidence suggests that animals lack temporal concepts, we should not conclude that they lack episodic memory—but we should revise our theoretical predictions about how they ought to behave if they do.