Recollection, perception, imagination

Two Questions

Remembering a cat sleeping (specifically, recollecting the way the cat looked), perceiving (specifically, seeing) a cat sleeping, and imagining (specifically, visualizing) a cat sleeping are of course importantly different. Nonetheless, from the first-person perspective they are palpably alike. Our first question is:

Q1. What are these similarities (and differences)?

The question can equally well be asked about other modalities—a purring cat could be substituted for a sleeping one, for example. But the chief examples will be visual.

Despite the similarities between remembering, seeing, and imagining, one can readily distinguish them in one’s own case. When one sees a cat sleeping, one is often in a position to know that one sees a cat sleeping. (Or, at least, to know that one sees something, which perhaps unbeknownst to one is a cat sleeping.) In ordinary cases, there is no danger of confusing vision with recollection or imagination. Although confusion between recollection and imagination is more likely, it is not pervasive: usually there is little difficulty in knowing that one is recalling a cat sleeping, not (merely) visualizing a cat sleeping.¹ Our second question concerns this latter fact:

Q2. How does one tell that one is recalling (and so not perceiving or imagining)?

“By introspection” is no doubt a correct answer, if ‘introspection’ is taken as a label for that special first-person method—whatever it is—that delivers knowledge of one’s mental life. Correct, but hardly illuminating. Of course, there are versions of Q2 for perceiving and imagining, with equally unobvious substantive answers. The version of Q2 for perception will be addressed en route to answering the version for recollection.

Distinctions

Vision and visualizing are clear enough to be going on with. The kind of remembering at issue, though, needs some explanation.

Long term memory is often divided into two basic kinds, “declarative” (or “explicit”) and “non-declarative” (or “implicit”), with declarative memory given a gloss along the lines of “conscious recollections of facts and events” (Squire 1992, p. 232). Declarative memory itself is divided into “semantic” and “episodic”; non-declarative memory covers a grab-bag of other learning abilities, but the central category is memory of skills or habits, or “procedural” memory.²

Although in psychology “[t]he recognition that there are multiple form of memory developed beginning in the 1980s” (Squire 1992, p. 232), ordinary language was there first.³ Compare:

• (1) Fred remembers that eggs have yellow yolks/that he saw an egg.

• (2) Fred remembers seeing an egg/eating a egg/being hit by an egg.

• (3) Fred remembers how to boil an egg/how to imagine an egg breaking.

Sentences like (1), where ‘remembers’ takes a sentential complement, are typically (but not exclusively) used as reports of semantic memory. (2), with a gerundival complement, is used as a report of episodic memory, and (3), with an infinitival complement, as a report of procedural memory.

The kind of memory at issue in Q1 and Q2 is episodic. The linguistic facts just mentioned may reassure us that this is a category worth distinguishing, but what exactly is episodic memory, and how does it differ from semantic memory? The neuroscientist Endel Tulving (who introduced the ‘episodic’ label in his 1972) informally puts this way: episodic memory is “memory for personally experienced events” or “remembering what happened where and when” (2001, p. 1506). More poetically, episodic memory “makes possible mental time travel through subjective time, from the present to the past, thus allowing one to re-experience…one’s own previous experiences” (Tulving 2002, p. 5).⁴ Semantic memory, on the other hand, is mundane by comparison, being “memory for general facts” (Tulving 2001, p. 1506), or “general knowledge of the world” (Baddeley 1997, p. 4).⁵

However, these explanations are more suggestive than accurate. Suppose I was so drunk at the party that I cannot recall dancing with a lampshade on my head. The next day I learn of this mortifying episode; later I remember what I learned, that I was dancing at the party in inappropriate headgear. I remember “a personally experienced event”, or “what happened where and when”, but this is semantic memory, not episodic. Contrariwise, suppose I have seen many skunks, and on that basis can recall what skunks look like. When I recall what skunks look like, I visualize a prototypical skunk, a perceptual amalgam of the various skunks I have encountered. Such a memory is best classified (at least initially) with paradigmatic episodic memories—recalling seeing a skunk in my garden this morning, for instance. Yet it is not a memory of a “personally experienced event”.

For a final clarificatory distinction, consider the following two pairs:

• (a) Bertie remembers that the Pope is German

• (b) Bertie is remembering the Pope’s nationality

• (a′) Bertie remembers meeting the Pope

• (b′) Bertie is remembering meeting the Pope

The a-sentences might be used to truly describe Bertie when he is sound asleep, with (a) ascribing semantic memory and (a′) episodic. The b-sentences, on the other hand, apply only when Bertie is undergoing a process of recollection. It is the process, rather than the state, of episodic recollection, that is palpably similar to perceiving and visualizing.

Similarities and differences (Q1)

Before turning to Q1, a working assumption that will guide the proposed answer needs to be put on the table, if not defended. One central function of perception is to allow the organism to acquire information (more exactly, knowledge) about its environment. To borrow a title from Dretske, perception is in the business of knowledge and the flow of information. The different senses make different contributions to this information flow: vision supplies information (or misinformation) about color, texture, motion, position, illumination, and so on; the other senses supply information about a different set of features.⁶ The representational content of visual perception is this distinctive package of visual information (or misinformation) that characterizes the (ostensible) scene before the eyes.

This very abstract picture of perception leads naturally (but not inevitably) to our working assumption: the phenomenal character of perceptual experience can be explained in terms of the representational content of perception. This “intentionalist” or “representationalist” view is often endorsed (for example, Tye 2000), but is also controversial (for notable dissent see Block 2003).

The same content thesis

In the Humean tradition, the basic similarity between seeing, visually recalling, and visualizing consists in the similarity of the images or picture-like items immediately present to the mind. In Hume’s terminology, the “impression” produced in the mind by a strawberry, and the “idea” produced in the mind when one visually recalls the strawberry, or visualizes a strawberry, “differ only in degree, not in nature”. Specifically, the idea is a “faint image” of the impression (Hume 1740/1978, I.i.1). So Hume endorses:

Same Image: the images present in visualizing and visual recollection are of the same kind as those present in visual perception—albeit degraded and transformed in various ways.

Which he supports by noting that:

…in sleep, in a fever, in madness, or in any very violent emotions of the soul, our ideas may approach to our impressions, as, on the other hand, it sometimes happens that our impressions are so faint and low that we cannot distinguish them from our ideas. (I.i.I)

The second part of the last sentence anticipates the “Perky effect”—the phenomenon of mistaking perception for imagination. And—granting the existence of “impressions” and “ideas”—the Perky effect is just one of many pieces of evidence for Hume’s view.⁷

Despite its undeniable attractions, however, Hume’s view faces a serious objection. If ‘faintness’ is taken literally (and it is unclear how else to take it), Hume’s view apparently predicts that visualizing a strawberry is more similar to seeing a strawberry in dim light than it is to seeing a strawberry in sunshine, but that is surely incorrect.⁸

The problem is fundamental to Hume’s conception of ideas and impressions as picture-like. The only available models for understanding these alleged entities are physical pictures: ideas or impressions of a strawberry are much like physical pictures of a strawberry (at any rate as the naïve folk think of such things), except that they “arise in the soul”. Hence, for any episode of visualizing or recalling, it should be in principle possible to create a physical picture of a strawberry such that viewing the picture in certain conditions exactly reproduces the felt quality of visualizing or recalling. And this is what seems wrong: any way of degrading the picture, such as blurring, desaturating, dimming, and so on, just yields another perceptual experience, plainly discernable from visualizing or recalling.

The contemporary representationalist version of Hume’s thesis is the following:

Same Content: the content of visualizing and visual recollection is the same kind as the content of visual perception—albeit degraded and transformed in various ways.

Importantly, this is not vulnerable to the objection just rehearsed. Consider a picture of a bright scarlet strawberry, analagous to an “impression” of a strawberry. The objection was, in effect, that visualizing a strawberry is evidently different from looking at a degraded copy of the picture. But that does not impugn Same Content, because a degraded copy of visual information need not be the content of a possible visual perceptual experience—yet it may still be information of a distinctively visual kind. For example, it is not possible to paint a picture that depicts a strawberry as simply red (that is, of no particular shade, brightness, or saturation)—“simply red” is not found in any paint catalog. On the other hand, “simply red” information (or misinformation), for instance the proposition that the strawberry is red, can be bought off the shelf. This is a toy illustration of how the content of perception could be degraded while remaining distinctively visual.

Since Same Content has all of the explanatory virtues of Same Image, while evading its most serious drawback, this adds up to a powerful case. Visual perception, visual recollection and visualizing, we may provisionally conclude, all share the same distinctive kind of representational content. The content of recollection and visualizing is (as one might expect) a degraded and transformed version of visual content. (Just how it is degraded and transformed is a difficult and complex issue that is beyond the scope of this paper.) The content of recollection and the content of visualizing, on the other hand, are often very similar, if not identical. A quick way of motivating this last claim is to note that episodic memory can be seamlessly integrated into a visualizing project, for instance visualizing how one’s living room couch might look in the bedroom.

Memory as preservation

What about differences? As just mentioned, content does not seem to separate recollection and imagination, so the need to find some distinguishing mark here is particularly pressing.

Recollection is (as it might be put) episode-entailing: if one recalls a party then there is (was) a party that one is recalling. (Similarly if one sees a party.) Recollection is also factive, in this sense: if one recalls being drunk at the party then one was drunk at the party. And of course imagination is neither episode-entailing nor factive.

But these differences are not fundamental. One may imagine the party occurring slightly differently—one imagines oneself sober, engaging in witty repartee. One is imagining a party, and moreover that party did in fact actually occur (although not as one is imagining it). We can call this E-imagining—imagining event e with the restriction that e exists. E-imagining is trivially episode-entailing, but is not a kind of recollection. Similarly, one may not know that one was drunk at the party; one veridically imagines being drunk at the party, which we can call factive-imagining. Factive-imagining is not a kind of recollection either. The significant difference between recalling and imagining must lie elsewhere.

A natural thought is that episodic memory, unlike imagination, preserves something supplied by previous perception. Semantic memory preserves knowledge, in this sense: if one remembers p then one knows p and (at least typically⁹) one knew p, with one’s current knowledge being explained, in part, by one’s past knowledge. What might episodic memory preserve?

Could it preserve knowledge also? According to Martin, no: “we cannot conceive of such remembering as the preservation of knowledge, for one cannot have known what one now recalls in episodic memory” (2001, p. 264). In support, he points out that the gerundival complement of ‘remembers’ or ‘recalls’ (a sign of episodic memory; see Sect. 2) cannot grammatically be the complement of ‘knows’. Thus while

• (1) Mary remembers John falling asleep in the talk

  is grammatical,

• *(2) Mary knows/knew John fall asleep in the talk

  “is simply not English” (p. 264).

The problem with this line of argument is that it is not true in general that if expression A cannot be substituted, preserving grammaticality, for expression B, where A and B are of different syntactic categories, then A and B differ in semantic value.¹⁰ Thus, for all Martin’s argument shows, what one now recalls might be what one knew.

Still, Martin’s conclusion is correct. Suppose one seems to recall being sober at the party, wearing a pink tie, talking to McX, and so on. In fact, one was drunk, tieless, and McX never turned up. It is consistent with misrembering all these details that one is genuinely recalling the party. One may misremember practically everything (including misremembering the party as a faculty meeting), and yet still be recalling the party. We may further suppose that (despite the inebriation) one had plenty of knowledge of the party when it occurred; this knowledge has vanished when one later recalls the party.

In this case one’s knowledge, of the party, that it occurred, is preserved. But even this is not necessary:

Suppose that someone asks a painter to paint an imaginary scene. The painter agrees to do this and, taking himself to be painting some purely imaginary scene, paints a detailed picture of a farmyard…. His parents then recognize the picture as a very accurate representation of a scene which the painter saw just once in his childhood…. Although the painter sincerely believes that his work is purely imaginary, and represents no real scene, the amazed observers have all the evidence needed to establish that in fact he is remembering a scene from childhood. (Martin and Deutscher 1966, pp. 167–168)

Martin and Deutscher take their example, surely correctly, to show that one may “remember X…and not believe that X happened” (p. 168); by the same token, one may remember X and not know that X happened.

If episodic memory does not preserve knowledge, what is left? Martin suggests episodic memory retains the “past apprehension of the event now recalled” (2001, p. 267), where “apprehension” is explained as follows: “We apprehend events through either perceiving the events or through being their conscious agent”, for instance hearing “John falling asleep in the talk”, or “being inoculated for smallpox” (pp. 264–265).

However, this proposal is less substantial that it might first appear. In the case of semantic memory, one “retains one’s past knowledge of p” in the sense that one now knows p. But what does it mean to “retain one’s past apprehension of event e”? If this were entirely parallel to semantic memory, it would mean that one now apprehends e, which would of course be incorrect, and is certainly not what Martin intends.¹¹ Recalling the party is not now to perceive the party or attend it again. Absent another explanation, talk of “retaining one’s past apprehension of e” seems to be a longhand way of saying that one remembers e, in which case Martin’s proposal is true but trivially so, and hence merely rephrases what was supposed to be explained.

What’s left? Apart from knowledge, what does perception supply that is worth preserving? As is often emphasized (for example, by Campbell 2002), perception enables thought and talk about individual objects and events. In a resonant phrase, perception supplies cognitive contact with objects and events (and, it could be added, properties).¹² The notion of cognitive contact will be left at an impressionistic level; even without further explanation, it is initially plausible that cognitive contact is what episodic memory preserves. In recollecting the party, one may not now know anything one used to know about the party, but recollection puts one in a position to talk and think about that very event.

Another example: one may see the red car hit the lamppost; at the time one knows that the car is red. Later one may recall the collision while misremembering the car as a blue, etc. What is preserved is not knowledge of the car, but cognitive contact with it: episodic recall enables one to talk and think about that particular object, to wonder whether it was blue, or a Prius, and so on.

Here, then, is the summary answer to Q1. The palpable similarity between perception, recollection and imagination consists in a distinctive kind of representational content. The content of recollection and the content of imagination are degraded and transformed versions of the content of perception. Cognitive contact is the point of overlap between perception and recollection: the latter preserves the cognitive contact supplied by the former. Although imagination can involve cognitive contact—as when one visualizes the living room couch in the bedroom—it is not itself preservative: without memory, the couch would be unavailable to imagination.

It will help to set the answer to Q1 in the context of the “empiricist theory of memory”, as critically examined in R. F. Holland’s classic eponymous paper. That will also provide a segue to Q2.

Interlude: the empiricist theory of memory

The empiricist theory of memory, as Holland understands it, is, “…a large and variously developed body of thinking…within the august tradition of Locke, Berkeley, and Hume” (1954, p. 464). That large body is principally distinguished by the claim that recollection is significantly similar to perception—so much so, that recollecting may aptly be glossed as perceiving the past. And since the dominant theory of perception in the “august tradition” is the sense-datum theory (or the “representative theory of perception”), the dominant version of the empiricist theory also involves sense-data, specifically “memory-images”. One perceives a hopping kangaroo by being aware of an image (or perhaps a succession of images) of a hopping kangaroo; one recollects that past episode of hopping by being aware of a similar albeit etiolated image.

The paradigm case of the empiricist theory is Russell’s account in The Analysis of Mind. Lecture 11 of that book is devoted to episodic memory (and, in particular, episodes of recalling rather than states of remembering). In fact, Russell does not recognise semantic memory as a distinct kind of memory at all. Following Bergson in Matter and Memory, he does distinguish episodic memory from “habit memory” (procedural memory, more or less) but, again following Bergson, he singles out episodic memory for the tendentious title of “true memory” (Russell 1921/1995, p. 138; Bergson 1911, p. 195). As Holland notes, non-episodic memory “tends to be dismissed [by “the Empiricist”] as philosophically uninteresting and unproblematic” (1954, p. 479).

The basic outline of Russell’s account is quite simple: “Memory demands (a) an image, (b) a belief in past existence. The belief may be expressed in the words ‘this existed’” (1921/1995, p. 155).

As one might expect, the belief that this event existed (or occurred) is not supposed to be unconnected to the image of it. Intuitively, the belief is explained by the image, which is indeed Russell’s view: one believes that event e occurred because one is aware of an image of e. But this connection raises a puzzling question. Clearly Russell thinks that the images of imagination may be equally “dim or vague or faint” (p. 128) as memory images. So why does the memory image tend to produce the belief that e occurred?

That question Russell divides into two: why does the image intimate that e exists (at some time), and why does it suggest that that time was (to a greater or lesser degree) in the past? Russell’s answer to the first question is that the memory-image is accompanied by a “feeling of familiarity”, “the characteristic by which we distinguish the images we trust” (p. 134). His (more tentative) answer to the second is that (one) of the “various factors that concur in giving us the feeling of greater or less remoteness in some remembered event” is “a specific feeling which could be called the feeling of ‘pastness’” (p. 134).

One oddity about Russell’s answer to these two questions is that the “feeling of familiarity” which, as he points out, is uncontroversially a feature of ordinary experience, already carries the sense of pastness with it, thus going beyond its official job as a trustworthiness certifier. If someone seems familiar, this is a sign that one has seen that person before. (Admittedly, although the feeling of familiarity comes in degrees, a high degree of familiarity does not correspond to the more remote past, unlike the degrees of the supposed “feeling of pastness”.) Another oddity is that while the “feeling of familiarity” is, well, familiar, surely the “feeling of pastness” is not. Indeed, Russell gives the impression of being reluctantly forced to postulate the latter feeling in order to save his account.

Since Russell’s “feeling of familiarity” is not likewise a dubious invention, could it shoulder at least part of the burden? Perhaps the “feeling of familiarity” attending an image of e could account for the belief that e is past, if not for the belief that e is (say) recently past. Let us grant that one explanation of the familiarity of the image might be that one has apprehended (in Martin’s terminology) e before. As Holland points out, though, it would be an unwarranted leap to conclude that this is the explanation—and so that e is past. For there is another hypothesis: an image might be familiar because “you have amused yourself by creating some such fanciful image as this on many occasions in the past” (1954, p. 468). Naturally one often knows which one of these hypotheses is correct, but evidently the feeling of familarity cannot be the explanation.

For present purposes the important point is that Same Content (see Sect. 3.1), is an updated version of Russell’s empiricist theory, with visual images traded for visual information (and misinformation), and with clause (b) dropped, along with the ambition of providing an “analysis” of memory. As a result, although recognizably in the empiricist tradition, Same Content points towards an answer to the second question discussed in the previous two paragraphs, moreover one that does not involve any mysterious “feeling of pastness”.¹³

An image, thought of as something like a picture or photograph, is atemporal, in the sense that the time of the depicted events is not itself depicted.¹⁴ Photographs taken with contemporary cameras often have time stamps on them, recording the date the photograph was taken. This information is not depicted—not represented pictorially—but rather represented linguistically. (The intuitive distinction between pictorial and linguistic representation is notoriously hard to spell out precisely; it will not be here.) Since “memory-images” are plainly not time-stamped like photographs, there is pressure to postulate a “feeling of pastness” to play that role.

However, if images are jettisoned in favor of representational content—visual information and misinformation—the source of the belief that the remembered event is past is no mystery. That an event was in the past is just another piece of information, and so can be part of the content of the recollection.¹⁵

Still, there is another role for which a distinctive feeling might be required, namely as a sign or mark that enables one to know that one is recalling, and so not imagining. After all, if temporal information can be part of the content of memory, presumably it can also be part of the content of imagination, so this does not distinguish between the two. And this brings us to Q2.

Epistemology (Q2)

According to Hume:

A man may indulge his fancy in feigning any past scene of adventures; nor wou’d there be any possibility of distinguishing this from a remembrance of a like kind, were not the ideas of the imagination fainter and more obscure. (1740/1978, I.iii.5; see also Appendix, pp. 627–628)

Thus, if one is aware of an image of a cat sleeping that is “somewhat intermediate betwixt an impression and an idea” (I.i.3), then probably one is recollecting a cat sleeping, or at least seeming to. Even granted Hume’s apparatus of ideas and impressions, that is not very plausible (Holland 1954, pp. 465–466). The representationalist counterpart of Hume’s suggestion is that memory and imagination may be distinguished (that is, told apart) by their content. But this time the representationalist counterpart is no clear improvement.

The “feeling of familiarity” mentioned in the previous section might appear a much better candidate for “the Memory-Indicator, the allegedly distinctive experience which is supposed to distinguish memory from imagination” (Holland 1954, p. 471). On meeting McX, he might strike me as familiar, which is a good sign that I remember him, or perhaps someone who looks very much like him. However, my well-remembered kitchen (for instance) does not likewise produce “feelings of familiarity”, as Holland observes (p. 467). Certainly my kitchen is familiar, but that is just another way of saying that I remember it well; it is not to hint at how I know that I remember it.¹⁶

Given the general empiricist tenor of the account so far, with the stressed kinship between perception and recollection, perhaps the epistemology of perception will provide a clue. One may know that a cat is sleeping by using one’s eyes. How does one know that one sees a cat sleeping?

Here, in outline, is an account in the spirit of Evans 1982 (see especially pp. 224–228).¹⁷ Start with vision itself, not with our knowledge of vision. Vision supplies information of a distinctive kind—visual information, or visual facts. When one visually perceives, the content of one’s visual experience, if true, is a visual fact. The visual world is the totality of visual facts: it is the world as potentially revealed by vision. Since there are no sounds in the visual world (for example), the visual world is not the perceptual world, the totality of all perceptual facts. (Likewise, the perceptual world is not the world, the totality of all facts.) When one discovers that a cat is sleeping by using one’s eyes, one finds a sleeping cat in the visual world: vision presents a certain black-and-white furry object which one identifies as a sleeping cat. Importantly, to discern that the sleeping cat is an object in the visual world does not require any knowledge of one’s mental states—one only needs to develop a sensitivity to the visual aspects of one’s environment.

Now it is a short step to (a sketch of) the epistemology of vision. One comes to know that one sees a sleeping cat by an inference from the visual world to the mind. One uses one’s eyes to investigate the visual world, discovers that it contains a sleeping cat, and concludes from this premise that one sees a sleeping cat. This method will usually produce knowledge of the conclusion, because one would not know the premise unless one were to see a sleeping cat. Hence one knows what one sees by literally directing one’s eyes “outward—upon the world” (Evans 1982, p. 225): if there is a sleeping cat there, then one may safely conclude one sees it.

Another short step will take us to (an equally sketchy account of) the epistemology of episodic memory. Vision reveals the present visual world, how things are visually now. (Supernovas exploding in the remote past illusorily appear in the present.) Visual recollection, in contrast, reveals the past visual world, how things were visually. One comes to know that one recalls a sleeping cat by an inference from the past visual world to the mind. One uses one’s memory to investigate the past visual world, discovers that it contains a sleeping cat, and concludes from this premise that one recalls a sleeping cat. As before, this method will usually produce knowledge of the conclusion, because one would not know the premise unless the conclusion were true.

A final remark about the picture of episodic memory that has emerged in the answers to our two questions. Tulving’s striking metaphor of “mental time travel” is misleading. Time travellers have no special experience of the past—when Dr. Who steps out of his TARDIS in the Silurean Period he experiences the events going on then as present.¹⁸ But in episodic recollection events appear as past, somewhat as mountains on the horizon appear as distant. An updated version of the Empiricist Theory of Memory can respect this fact.