Advertisement

Memory & Cognition

, Volume 45, Issue 8, pp 1295–1305 | Cite as

Task-relevant perceptual features can define categories in visual memory too

Article

Abstract

Although Konkle, Brady, Alvarez, and Oliva (2010, Journal of Experimental Psychology: General, 139(3), 558) claim that visual long-term memory (VLTM) is organized on underlying conceptual, not perceptual, information, visual memory results from visual search tasks are not well explained by this theory. We hypothesized that when viewing an object, any task-relevant visual information is critical to the organizational structure of VLTM. In two experiments, we examined the organization of VLTM by measuring the amount of retroactive interference created by objects possessing different combinations of task-relevant features. Based on task instructions, only the conceptual category was task relevant or both the conceptual category and a perceptual object feature were task relevant. Findings indicated that when made task relevant, perceptual object feature information, along with conceptual category information, could affect memory organization for objects in VLTM. However, when perceptual object feature information was task irrelevant, it did not contribute to memory organization; instead, memory defaulted to being organized around conceptual category information. These findings support the theory that a task-defined organizational structure is created in VLTM based on the relevance of particular object features and information.

Keywords

Visual long-term memory Retroactive interference Categorization 

Visual long-term memory (VLTM) has been shown to have an enormous capacity and fidelity (e.g., Brady, Konkle, Alvarez, & Oliva, 2008; Nickerson, 1968; Standing, 1973; Standing, Conezio, & Haber, 1970; see Brady, Konkle, & Alvarez, 2011, for a review). These capacities have been theorized to rely on either the viewer’s ability to use both pictorial and conceptual information (dual coding; Paivio & Csapo, 1973) or on the perceptual distinctiveness of images (Weldon & Coyote, 1996; see also Nelson, Reed, & Walling, 1976). In a recent study, Konkle, Brady, Alvarez, and Oliva (2010) explored the excellent memory for pictures by investigating the contributions of conceptual and perceptual information to the structure of VLTM. They used retroactive interference—a reduction in memory performance for an item resulting from the subsequent presentation of additional items—as an indication of the relative importance of conceptual and perceptual information by manipulating the number of exemplars they presented from a particular conceptual category (from one to 16 objects from each category). If increasing the number of exemplars from conceptually similar categories increases retroactive interference, then conceptual information is critical to the structure of VLTM. On the other hand, if increasing numbers of objects from categories that were rated to be perceptually similar increase retroactive interference, then VLTM would appear to be organized around perceptual information.

Konkle et al. (2010) found excellent memory for pictures with no interference (89% accuracy in a two-alternative forced-choice task). More importantly, they found increasing retroactive interference with subsequently presented similar category exemplars, supporting the importance of conceptual organization for visual memories. In contrast, Konkle et al. found no relationship between the size of the retroactive interference effects and perceptual similarity within a category—that is, objects that looked more alike did not produce more interference than objects that looked more dissimilar. Konkle et al. concluded that conceptual category information is the supporting and organizing mechanism that provides VLTM with its high capacity and fidelity, and VLTM is benefitted by having an underlying preexisting conceptual knowledge structure upon which visual details can be indexed when pictures are learned.

Although Konkle et al. (2010) demonstrated that conceptual information was critical to VLTM, studies examining VLTM for visual search objects indicate that conceptual organization alone does not appear to be sufficient to explain visual memory effects. Williams and colleagues (Thomas & Williams, 2014; Williams, 2010a, b; Williams, Henderson, & Zacks 2005) found that the role of an object in search affects VLTM. Specifically, memory for conjunction search targets was much better than memory for distractors related by category or color. Even when the distractor objects are viewed longer, distractor objects are remembered substantially worse than target objects are (Thomas & Williams, 2014; Williams, 2010b; Williams et al. 2005). In this case, the contextual information surrounding the encoding process of an object appears to be important in retrieving memories from VLTM. If conceptual category information is the sole organizing principle in VLTM, then category distractors (i.e., distractors that share the target’s category but not its color) should increase interference in memory for the target object, similar to Konkle et al. However, memory for the target of a visual search is significantly higher than memory for the same object learned under memorization instructions (Williams, 2010a), indicating that the additional category exemplars from the distractors do not interfere with the object memory when it is a search target as much as when the same object is memorized. In other words, unlike the findings of Konkle et al. (2010), seeing several other backpacks, for example, did not appear to interfere with the memory of a target backpack even though they all share a conceptual category.

Why would multiple members of a conceptual category lead to retroactive interference in the Konkle et al. (2010) study but not affect the memory of a search target in the Williams (2010a) study? Williams (2010a, b) proposed that memory for search objects is affected by the contextual information that is present during the encoding event. When a search target is encountered, it is the critical event of the search task. This event may attach a specific importance to the visual memory for that target, allowing the target to be remembered especially well (like the objects in Konkle et al.). In contrast, encountering category distractors during a search is not the event of the search, and thus these category distractors are encoded without that same information (see Janzen & van Turennout, 2004, for a similar finding regarding choice points in a maze).

The difference in how an object is encountered could alter the relative importance of its conceptual category. In the repeat detection task of Konkle et al. (2010), with no other instruction when a picture is being viewed, a participant would likely implicitly name each object in an effort to memorize it while simultaneously testing that object against others of the same category previously presented (i.e., “Have I seen this backpack before?”). Thus, every backpack would be labeled with the conceptual category and would be stored with all of the other backpacks seen during the experiment. The more backpacks seen during the experiment, the more interference there would be for retrieving a particular backpack.1

In contrast, in a visual search, when encountering a target specified by conceptual category and a perceptual feature (e.g., blue backpack), a participant has to evaluate a current object on both of these aspects (“Is this object a blue backpack?”). When one encounters a target blue backpack, it would be identified as meeting both of those criteria and thus would be identified and stored with both of those features, leading to their superior memory. However, all other backpacks, even though they are part of the target conceptual category, fail to meet both criteria. Thus, within the context of the search, these other backpacks are different than the target backpacks, and it is this difference that leads to the distractor backpacks being encoded separately. We argue that the manner in which they are encoded (i.e., “Not what I am looking for”) prevents these distractor objects from being encoded with the full advantage of the conceptual category information that Konkle et al. claimed supported VLTM.

The current study was designed to test the extent that conceptual, perceptual, and contextual information was critical to VLTM organization. We hypothesized that whatever task factors are relevant at the moment of encoding will help organize VLTM. If the organization of VLTM is limited to only conceptually defined relationships, as suggested by the findings in Konkle et al. (2010), all conceptually related objects should be expected to produce retroactive interference for an earlier presented member of a conceptual category (e.g., interference should occur equally for all presented “backpacks” regardless of how they are encoded). On the other hand, if the organization of VLTM relies on the relevance of information to the encoding context, then interference effects for objects should be expected to be constrained along the task-relevant defining factors for that object (e.g., interference would be constrained to “blue backpacks” when both the color and category are important to the task).

In order to accomplish this, we needed a design that could take advantage of retroactive interference to probe the structure of VLTM (as in Konkle et al., 2010), but with the ability to manipulate the task information that was relevant at the moment an object is encoded. We used a sequential target-identification trial task (similar to Williams, 2010a) where participants are given a target prior to a sequence of pictures and are asked after the sequence if the target appeared anywhere in that sequence (see Fig. 1). By manipulating the information provided about the target object for that sequence (only the category label, backpack, or a more specific label of a color and category label, blue backpack), we can determine if providing a perceptual feature can alter the way that objects are stored in VLTM.2 In other words, we can ask if a blue backpack is encoded and remembered as just a backpack, where any backpack can interfere, or is it now a special category that is exclusively made up of blue backpacks, and only blue backpacks can interfere with each other.
Fig. 1

Sequential target-identification trial task used during encoding and interference phases in Experiment 1

Experiment 1

In Experiment 1, we attempted to replicate the effects of Konkle et al. (2010) using our sequential target-identification trial task. We argue that in the repeat detection task of Konkle et al. participants would naturally identify only the category of the object when making the repetition decision; thus, VLTM from the repeat detection task would emphasize the conceptual category in its structure. In Experiment 1, we had participants identify targets in the sequential target-identification trial task that were defined by only category label (e.g., backpack). Because the color was not critical to identifying the target in the sequence, we hypothesized that color would not be part of the VLTM structure that was created to support memory of these objects, which would replicate the findings of Konkle et al. (2010). In other words, if a blue backpack was seen initially when looking for a backpack, subsequently presented red and blue backpacks should both produce equal levels of retroactive interference.

Participants

Thirty-two participants were recruited from the Psychology Research Program at Mississippi State University and received partial credit toward an undergraduate psychology course for their participation. Participants were restricted to those with normal or corrected-to-normal visual acuity, normal color vision, who were 18 years old or older, and those who had not participated in similar visual cognition studies (i.e., naïve to the purpose of the experiment). Participants were allowed to participate in only one of the two experiments.

Design and procedure

In both experiments, the same procedures and images were used, with only the target identification label changing. The experiment had three phases: an original encoding phase, an interference phase, and a memory test phase (see Fig. 2). Critically, the design of the experiment is tied to the procedure that we used. We will discuss each phase below, but the general format follows Konkle et al. (2010): An originally encoded object was presented in the encoding phase (hereafter called the originally encoded target). Retroactive interference for the originally encoded targets was generated by presenting varying numbers of objects in the interference phase that were from the originally encoded target’s conceptual category (hereafter called interference objects). Finally, we tested the memory for the originally encoded object in the memory test phase.
Fig. 2

Representation of the procedure in this study. In all conditions, the critical target object was looked for in the encoding phase and all eight of the interference phase blocks. Top row shows the 7–1 condition, where, in the encoding phase, a blue backpack was the target of the trial (in Experiment 1, the target label was Backpack; in Experiment 2, the target label was Blue Backpack). In the interference phase, a blue backpack was presented in seven of eight blocks and a red backpack was presented in the remaining block (not shown). Middle row shows the 1–7 condition, where, in the encoding phase, a yellow leaf was the target of the trial (in Experiment 1, the target label was Leaf; in Experiment 2, the target label was Yellow Leaf). In the interference phase, a yellow leaf was presented in one of eight blocks, and a red leaf was presented in the remaining seven blocks (in this example, the yellow leaf interference object appeared in the first interference block; the block in which the matching interference object appeared was counterbalanced). Bottom row shows the 1–0 condition, where, in the encoding phase, a blue mug is the target of the trial (in Experiment 1 the target label was Mug; in Experiment 2, the target label was Blue Mug). In the interference phase, a blue mug was presented in one of eight blocks (in this example, the second block). In the remaining seven blocks, no mug (red or blue) was presented. Trial order within each block was randomized, and there was no discernable break in between the encoding phase and interference phase or between blocks in the interference phase. The memory test followed the completion of the interference phase. The memory test was a two-alternative forced-choice token-discrimination task where the participants indicated which of the two objects had been presented. The originally encoded targets from the encoding phase were tested against a nonpresented foil that matched both the color and category of the target (Color figure online)

Encoding phase

To begin the experiment, participants were provided with an informed consent document to read and complete, followed by a basic demographic questionnaire that included screening information on visual abilities. Participants were seated in front of a computer monitor and given general instructions. In both the encoding and interference phases, we used the sequential target-identification trial task (see Figs. 1 and 2), where participants were instructed to indicate if a target object was present in a sequence of seven objects. The sequence began with a target label, and the participant pressed a button on the button box to indicate that he or she knew what the target was for that sequence. After the button press, the seven-object sequence was presented, with each object being presented in the center of the screen for 750 ms with a 100 ms blank interstimulus interval (ISI). The 750-ms presentation time was longer than the average time that target objects were fixated in a visual search task using these stimuli (637 ms; Williams et al., 2005; see also Williams, 2010a). Following the presentation of the seven objects, a query screen would appear that asked the participant, “Was the search object present? Yes or No,” and the participant input his or her response using buttons labeled either “Yes” or “No” (see Fig. 2 for complete structure of the procedure). The target object, if present, was equally likely to appear in Positions 2, 3, 4, 5, 6, and 7 in the sequence. Following the response, the target description for the next trial was shown; the target description changed on each encoding trial. A practice trial preceded the encoding phase to familiarize the participant with the task.

For each participant, there were a total of 54 sequential target-identification trials in the encoding phase. Twenty-seven critical originally encoded targets from 36 different categories (see Appendix) were presented along with 18 filler trials. Because the originally encoded targets had to be present in the encoding phase (i.e., present trials), the 18 filler trials were target-absent trials. Each target-present sequence consisted of six random distractor objects, and the critical target object (target-absent sequences had seven random objects). None of the distractors or the target-absent trial categories shared the basic-level category with any of the originally encoded targets. All trials were presented for the encoding phase in a random order.

Interference phase

Upon completing the encoding phase, participants began the interference phase (there was no discernable break to the participants). In the interference phase, participants continued to perform the sequential target-identification trial task. The number of interference objects and their relationship to the originally encoded target were the critical manipulations in the experiment. Each participant saw each of three critical interference conditions. All of the critical interference objects matched the originally encoded target’s conceptual category, but the total number of interference objects (one or eight) and whether they matched the originally encoded target’s color was manipulated across conditions. The interference conditions were labeled by the number of interference objects that match and mismatch respectively the originally encoded target’s color. In the 7–1 interference condition, participants were presented seven color-category matched interference objects and only one category-only matched interference object across the eight blocks. For example, in the 7–1 interference condition, using a blue backpack as the originally encoded target object, participants would see seven blue backpack interference objects and one red backpack interference object (top row, Fig. 2). The block in which the category-only matched object appeared was counterbalanced. We decided to use 7–1 rather than an 8–0 manipulation to encourage participants in all conditions to pay attention to the target identification task’s instruction and not simply respond based on the category label (this will be more relevant in the subsequent experiment). In the 1–7 interference condition (middle row, Fig. 2), participants were presented with one color-category matched interference object (e.g., a yellow leaf) and seven category-only matched interference object (e.g., seven red leaves) across the eight interference blocks. For this condition, the single color-category matched interference object appeared equally often in each of the eight interference blocks. Critically, in both the 7–1 and 1–7 interference conditions, all eight interference objects match the originally encoded target on its conceptual category. The third condition only presented one interference object (bottom row, Fig. 2); in the 1–0 interference condition, one color-category matched interference object was presented during the interference phase (e.g., one blue mug) in one of the eight interference blocks (counterbalanced across subjects). In the remaining seven interference blocks, the category was the target of a trial, but no member of that category appeared in the sequence. A fourth condition was included in the design (a 0–1 interference condition, where one category-only matched interference object was presented) in order to balance the number of categories and colors. However, that condition is not directly comparable to the other conditions, and so we will not discuss it further. Figure 3 provides an illustration of example interference objects presented for the three critical conditions for a single originally encoded target (a blue backpack). Interference condition was counterbalanced across participants with each category appearing equally often in each condition.
Fig. 3

Interference objects that would be presented in individual sequential target-identification trial tasks during the interference phase. In this example, one blue backpack is the originally encoded target seen in the encoding phase. Each individual object pictured is an example of an object that would be presented in the interference trial block associated with the original blue backpack, within an individual trial, across the eight interference blocks for each condition. The order of presentation of color matching or mismatching objects (e.g., red or blue) was randomly distributed across the eight blocks. Following interference in one of the conditions, the original blue backpack was tested against an original color-category matched foil object. (Color figure online)

Nine of the 36 originally encoded target categories were assigned to each of the three interference conditions (the remaining nine were assigned to the undiscussed 0–1 interference condition). Thus, in the interference phase, a total of 216 critical interference trials (3 conditions × 9 categories × 8 interference blocks) along with 152 filler trials were presented. Because the target was identified only by the category in Experiment 1, any object matching the category label would yield a yes response. Thus, both red and blue backpacks appearing in the interference trials would be responded to with yes.

Following the interference phase, participants were given instructions for and completed the memory test phase. Participants were told that two objects would appear (left and right side of the screen; see Figs. 2 and 3, for examples); one of the two objects had been previously seen during the experiment. Participants were not informed about when the objects had been previously seen. Because the tested object was always an originally encoded target, the presented object in the memory test had appeared in the encoding phase of the experiment. Participants indicated which object had been presented by pressing the left or right button on a button box. The memory test was self-paced with test objects displayed until the participant made a response. Participants were instructed to take as much time as was necessary to be as accurate as they could be, and they were instructed to guess if they did not remember the object. The originally encoded target appeared equally often on the left and the right, and the foils for half of the participants served as the originally encoded targets for the other half (counterbalanced across participants).

When the memory test was complete, the participant was debriefed and dismissed. All stimuli were presented using E-Prime 1.2 experimental software (Schneider, Eschman, & Zuccolotto, 2002). The experiment took approximately 1 hour to complete.

Materials

For each category (see Appendix), two colors were selected in which the originally encoded target and the interference objects could appear. For each category-color combination, two unique tokens were chosen to serve as the originally encoded target, and the memory test foil, most of which were taken from the stimuli set of Williams (2010a). In addition to the originally encoded target and its foil, eight additional tokens were chosen for each color-category combination to serve as interference objects. Thus, for each color-category combination, 10 unique object tokens were chosen. In addition to the objects from the categories appearing in the Appendix, 546 additional objects that did not share the same category with critical categories were used as filler pictures in the sequences. Filler pictures in the interference phase were repeated three times over the entire experiment, pseudorandomly distributed across trials; objects from critical categories were presented exactly one time.

All objects presented were full color photographs of real-world stimuli and were resized so that they were 90 pixels along the longer (vertical or horizontal) dimension; the objects were displayed on a neutral gray background (RGB = 120) and subtended 3.5° of visual angle at a distance of 57 cm, though viewing distance was not strictly controlled. Each object was presented in the center of the computer screen.

Results and discussion

Experiment 1 tested memory performance and interference effects when object feature information was not relevant to the target identification task, as we hypothesize was the case in Konkle et al.’s (2010) repeat detection task. Targets were only identified by the category, and we expected that only the total number of interference objects would affect the level of retroactive interference. Thus, we predicted that objects in the 7–1 and 1–7 conditions would be remembered similarly and would be remembered worse than objects in the 1–0 condition.

Due to experimenter error, one of the 36 originally encoded targets (wallet) was contaminated with more than the planned number of exemplars from its conceptual category in the stimuli set; the data relating to that category were dropped from all analyses reported here.

In the encoding phase and the interference phase, participants were highly accurate at responding to the originally encoded targets and the interference objects. On average, participants responded correctly to the originally encoded targets 96.9% (SD = 3.8%) of time and to the interference trials containing an interference object 95.0% (SD = 3.6%) of the time. Because the retroactive interference conditions critically depend on both the successful encoding of the originally encoded target and the correct interpretation of the interference objects, we conditionalized memory test performance on successful identification of the originally encoded target (i.e., correctly saying “yes” in the encoding block) and 75% or better performance for a category during the interference phase. These eliminations removed 38 memory test trials or approximately 4.5% of the trials

The critical data for this study were the memory test performance values for the three interference conditions. Because targets were identified by only the category label in Experiment 1, we expected that the number of category exemplars, regardless of color, would be the primary determinant in retroactive interference. There was an overall effect of interference condition, F(2, 62) = 6.65, p < .01, ηp 2 = .177 (see Fig. 4). Similar to Konkle et al. (2010), we found that memory performance was better in the 1–0 condition (M = .87) compared to memory in the 7–1 condition (M = .77), t(31) = −3.61, Bonferroni corrected p < .01, d = .827, indicating that increasing the number of subsequently presented color-category matched exemplars increased retroactive interference. Importantly, memory performance in the 1–7 condition (M = .78), where most of the interference objects were color mismatched category exemplars, was also worse than memory in the 1–0 condition, t(31) = −3.15, Bonferroni corrected p = .01, d = .665. The 7–1 and 1–7 conditions did not statistically differ from each other, t(31) < 1. Because color information was not part of the target identification task in Experiment 1, any exemplar from the target category could interfere with the originally encoded target. This pattern replicates the learning conditions seen in Konkle et al. (2010), where conceptual category, but not a perceptual feature, was relevant and critical to VLTM organization. In other words, because the target was identified only by its category, retroactive interference was dependent on the number of category tokens subsequently presented, not the color of the object tokens. Therefore, when we replicate what we surmised were the task demands of Konkle et al.’s repeat detection task, using our sequential target-identification trial paradigm, we find that all category interference objects, regardless of perceptual feature, interfere with memory of the originally encoded target.
Fig. 4

Memory results for mean accuracy from Experiment 1. Chance is .50. Error bars represent standard error

Experiment 2

Experiment 1 demonstrated that we could replicate the pattern of retroactive interference that Konkle et al. (2010) found for category members. By emphasizing only category membership, any category member, regardless of perceptual relatedness, could interfere with an originally encoded target. Although the results are consistent with the conceptual-category-only VLTM structure claims of Konkle et al., we believe that if perceptual information (in this case, color) was critical to identifying the target (i.e., made task relevant), then both the color and the category would be part of the structure of VLTM. To test our alternative claim, in Experiment 2, we used the same materials and procedure as Experiment 1, but we changed the target identification label to include both the category label and color (e.g., blue backpack instead of backpack). By making the color critical to the identity of the target, we believe that the organization of VLTM will be different than it was in Experiment 1. Instead of all category members, regardless of color, being grouped together in VLTM, the importance of the color to the target label in Experiment 2 will create a new subgroup that only contains objects that match the color and the category of the target description (e.g., a new subgroup that contains only blue backpacks). If we are correct, then retroactive interference for the originally encoded target (a member of the subgroup) should be affected by the number of objects that are in that subgrouping, not the number of objects in the category in total. In other words, retroactive interference for an originally encoded blue backpack target should be limited to the number of subsequently presented blue backpacks, and it should be unaffected by the number of red backpacks. Thus, the 1–0 condition (where one interference object matches the color and category of the originally encoded target) and 1–7 condition (where one interference object matches the color and category of the originally encoded target, but seven interference objects match on only the category) should produce equivalent levels of retroactive interference (i.e., similar memory performance). In contrast, the 7–1 condition (where seven interference objects match the color and category of the originally encoded target, and only one interference object matches on only the category) should produce substantial retroactive interference and result in memory performance that is worse than the other conditions.

Method

Thirty-two new participants were recruited from the Psychology Research Program at Mississippi State University and received partial credit toward an undergraduate psychology course for their participation. Experiment 2 used the same stimuli and procedure as Experiment 1 with one exception: The target in Experiment 2 was identified in the sequential target-identification task by both the color and category of the originally encoded target, for each object category during both the encoding and interference phases. In other words, if a participant was instructed to identify a blue backpack during the encoding phase (which would be present), then in each of the interference blocks, the participant would also look for a blue backpack. Because the color and category are part of the target identification, the three interference conditions would have different numbers of matching and nonmatching interference objects compared to Experiment 1. The 1–0 condition presented one category-color matching interference object (e.g., one additional blue backpack when looking for blue backpacks). The 7–1 condition presented seven category-color matching interference objects (e.g., 7 additional blue backpacks) and one nonmatching interference object (e.g., one red backpack when looking for blue backpacks, making it a nontarget object that would be responded to with a “no” in the sequential target-identification trial task). Finally, the 1–7 condition presented one matching interference object (e.g., one blue backpack) and seven nonmatching interference objects (e.g., seven nontarget red backpacks).

Results and discussion

As in Experiment 1, performance in the sequential target-identification task was high, with participants answering 97.8% (SD = 3.7%) of the encoding trials correctly and 93.9% (SD = 8.7%) of the interference trials correctly. The larger standard deviation in the interference trials was partly caused by two participants who both performed worse than 70% in the interference trials; those participants were removed from the analyses. Without those two participants interference trial performance was 96.0% (SD = 3.0%). Consistent with Experiment 1, the results for memory accuracy for all experiments were conditionalized on individual participants’ search accuracy during the encoding and at least 75% of that object’s interference trials. In total, approximately 3.3% of data were eliminated based on these criteria.

In Experiment 2, there was an overall effect of interference condition, F(2, 58) = 3.17, p = .049, ηp 2 = .099 (see Fig. 5), with the 7–1 condition (M = .81) having significantly worse memory performance than the 1–7 condition (M = .88), t(29) = 2.55, Bonferroni-corrected p = .05, d = .588, indicating greater interference in the former case than in the latter case. In contrast, the 1–0 (M = .86) and 1–7 conditions were numerically close and were not significantly different from each other, t(29) = 1.21, Bonferroni corrected p > .30, d = .26, indicating that the addition of seven nonmatching interference objects (e.g., red backpacks) did not significantly add to the retroactive interference for the originally encoded target. Thus, even though the 1–7 interference objects matched the conceptual category, the lack of a match to the task-relevant color appeared to prevent these objects from interfering with the originally encoded targets. As expected, the 7–1 condition, with seven matched interference objects, resulted in numerically lower memory accuracy than the 1–0 condition, with one matched interference object, indicating a retroactive interference effect. However, this 5% drop in memory was not significant, t(29) = −1.30, Bonferroni-corrected p > .30, d = .32.
Fig. 5

Memory results for mean accuracy from Experiment 2. Chance is .50. Error bars represent standard error

As a whole, this pattern of results supports the hypothesis that interference objects that do not match all of the relevant aspects of the originally encoded targets do not contribute to interference in memory. The fact that memory in the 1–7 condition (M = .88), with seven additional nonmatching interference objects that shared the category label was numerically better than memory in the 1–0 condition (M = .86), shows that these category-related, but color-unrelated, objects were not interfering with memory for the originally encoded target. It appears as if within VLTM these nonmatching interference objects were stored separately from the originally encoded target, potentially protecting/insulating the memory of the originally encoded target. However, if the interference objects matched the originally encoded target on both the color and category, it appears that they were stored with the originally encoded target (i.e., part of the same subgroup that was set up in the moment), leading to retroactive interference. In other words, if the color of the object is relevant to the task in the moment of encoding an object, the resulting VLTM structure will be defined by both categorical and perceptual information.3

General discussion

The current study examined whether task-relevance shapes the organization of VLTM information. Using the same set of visual stimuli and only changing the search instructions, we could manipulate whether a specific feature was relevant or not at encoding. Experiments 1 and 2 illustrated that the relevance, or lack thereof, of an object feature—color, in this case—had ramifications for how VLTM was organized. When color was relevant, objects not matching the originally encoded targets on color did not interfere with memory for the originally encoded target. However, when color was irrelevant, the same pictures of the objects did interfere with memory for the originally encoded target. These results indicate that the structure of VLTM can be altered if different types of information are important at the moment of encoding.

The primary goal of this study was to examine the structure of VLTM as in Konkle et al. (2010) and to test and extend the claim that VLTM is organized around conceptual categorical information. Specifically, we attempted to reconcile the results of Konkle et al. with the visual memory results from visual search experiments (Castelhano & Henderson, 2005; Thomas & Williams, 2014; Williams 2010a, b; Williams et al., 2005), where the encoding context, sometimes based on specific object feature information, seems to impact VLTM organization. Based on Williams (2010a, b), we hypothesized object features that were relevant at the moment of encoding would be critical to VLTM representation and structure. Our view is that VLTM is sensitive to these task-related factors allowing different types of information to be important, or not, to the structure of VLTM.

The results from these experiments support the importance of task-relevant visual information to the organization of VLTM. Specifically, results supported our hypothesis that whatever information about a visual stimulus is relevant to the task at the moment of encoding is what will define the organization of that visual memory. A critical task feature that is almost always relevant is, as in the case of Konkle et al. (2010), the conceptual category. Knowing the category of an object is important to VLTM. However, because their repeat detection task did not emphasize the perceptual details, the perceptual details, like color, were not relevant and so likely did not contribute to the organization of VLTM even if the perceptual details are still encoded. That is, the viewer is likely to be able to recognize the object with its color or other perceptual details, but those details are not used in an organizing manner in memory when it is not relevant to the encoding task. Alternatively, if the relevant features to the task include both the perceptual and conceptual information, such as when one is defining a conjunction search target, then both of these details become important to the structure of VLTM. We theorize that this encoding event, in essence, creates a new task-defined category in VLTM. This task-defined category will include all visual stimuli that meet the criteria for that category within the experimental context (e.g., all blue backpacks), but it will exclude any visual stimuli that do not meet the relevant criteria.

The formation of task-defined structures in VLTM may be adaptive in that it creates a temporal grouping of specific stimuli that are important to the processing of a local environment or situation—a way to organize information that will be conducive to completing the task at hand. The ability to rapidly organize visual memory around factors other than purely conceptual information can be seen in the hybrid search paradigm (Wolfe, 2012). In this paradigm, participants learn a set of potential targets that they will search for in visual search tasks. Wolfe (2012) found that participants were able to rapidly encode up to 100 random objects and perform the search accurately. Interestingly, search times followed a logarithmic curve based on the memory set size, indicating that participants were activating a subset of visual long-term memory that contained the target objects rather than entirety of visual long-term memory (Wolfe, Drew, & Boettcher, 2015). Because Wolfe and colleagues used random objects as stimuli, the memorized objects did not share a conceptual category to aid in the organization of visual long-term memory. We would argue that the instruction to learn these objects as a set is the same as identifying an object as a specific target within a task. Because these objects are identified as an important set, they are organized together. Boettcher, Drew, and Wolfe (2013) found that even within the activated long-term memory set, subcategories could be formed and selectively searched. In both our study and the hybrid search task, the importance of the encoding task matters to how that memory can be searched.

Although we have argued that task relevance is critical, we are not arguing against the importance of conceptual category information. In the absence of any other instruction, it would make sense to default to the conceptual category as an organizational strategy. One likely interpretation our results is that VLTM is conceptually organized, but a task-related filter is placed over this base organization. The objects that match the filter can become part of the activated set or subgroup; objects not matching the filter remain inactivate. Only those objects in the activated set can compete/interfere with other activated objects. This interpretation would be consistent with the results of Konkle et al. (2010) and the current search results as well as the hybrid search results of Wolfe and colleagues (2012; Boettcher et al., 2013).

Alternatively, one could argue for a more extreme point that the only critical organizing factor of VLTM is task relevance. In other words, any task-related information could be the basis of VLTM organization (e.g., color, shape, use), and the previous findings that conceptual category is used is an artifact of the task itself—the task emphasizes category, and thus category is part of the organizational strategy. We cannot, based on the current results, argue for such an extreme position. Although the ability to rapidly learn and remember dozens of random objects in the hybrid search task would seem to support this interpretation, those results do not preclude an underlying conceptual organization. It is difficult to create a situation where the conceptual meaning of an object is completely irrelevant to a task. Without evidence from such an unlikely task, it seems more reasonable to assume that VLTM is based on a conceptual categorical structure, but that this structure can be selectively activated based on the task relevant information.

The filter that we are proposing can act upon VLTM organization and activation could be related to Cowan’s (1995) activated long-term memory (see Cunningham & Wolfe, 2014, for an application of this memory type to hybrid search). The activated portion of memory would be those objects that have passed the task-related filter and could be easily accessed. Thus, task relevance would be the gateway to activated memory. Only those objects that are in the activated portion of visual memory could interfere with each other; objects that are not task related/activated will not interfere. Although the sheer number of objects that would be activated stretches the definition of activated memory (thousands of objects in Konkle et al., 2010), it is consistent with the results of the current study.

In conclusion, we believe that the current results demonstrate that VLTM is sensitive to both task-relevant information and conceptual categorical information rather than only categorical information as implied by Konkle et al. (2010). Task-relevant information filters what can or cannot interfere with other memories, indicating that VLTM organization is malleable to factors in addition to the conceptual category of visual stimuli at the moment of encoding. This flexibility allows the viewer to emphasize a particular detail of an object in the moment to protect it from other categorically related objects that do not possess that feature. As such, although the supporting organizational structure of conceptual information in long-term memory may explain the strength and fidelity of VLTM, conceptual information alone does not account for all VLTM memory patterns. Task-related information critical to encoding must also play a role in the structure of VLTM.

Footnotes

  1. 1.

    The same would be true if a category had multiple subtypes that were easily identifiable. In their article, Konkle et al. (2010) described the automobile category as being conceptually distinctive in kind (their Fig. 4). If the exemplars are identified by different labels (e.g., race car, police car, SUV) by the participants without prompting, then the different labels would be the basis of organization in VLTM.

  2. 2.

    These levels of specificity, category and category + color, are the same as search identifiers found in Schmidt and Zelinsky (2009), who referred to them as precise and precise + color, respectively. Because we are manipulating the categorical relationships of the objects, we felt that using the term category rather than the term precise would be more transparent for our purposes. However, in both the present study and Schmidt and Zelinsky, the same relationships are being described.

  3. 3.

    A separate experiment was performed to test if the target or nontarget status of interference objects was the reason for the change in the memory pattern; the nonmatching interference object served as a target in the interference phase in Experiment 1 and a distractor in Experiment 2. To test this possibility, we ran a separate group of participants (N = 21) in a version of Experiment 2, where instructions were switched at the interference phase. For the originally encoded target of blue backpack, the interference task identified the nonmatching interference object (e.g., red backpack) as the target. Even though we changed the target identification in the interference phase, the data pattern was the same as in Experiment 2. Memory performance for the originally encoded targets in the 1–0 condition (M = .853, SE = .03) and the 1–7 condition (M = .852, SE = .03) was the same, t < 1, but memory performance in the 7–1 condition (M = .778, SE = .03) was numerically worse compared to the 1–0 condition, t(20) = 1.71, p = .10, and significantly worse than the 7–1 condition, t(20) = 2.24, p = .037. These results indicate that whether the interference object was or was not the target of the sequential target-identification trials during the interference phase did not affect the memory results.

Notes

Author note

This research derives from K.B.A.’s doctoral dissertation at Mississippi State University. Aspects of the data were reported at the 15th annual meeting of the Vision Sciences Society, St. Pete Beach, Florida, May 2015. We thank dissertation committee members Deborah Eakin, Gary Bradshaw, and Lesley Strawderman for their helpful comments and discussion on the dissertation

References

  1. Boettcher, S. P., Drew, T., & Wolfe, J. M. (2013). Hybrid search in context: How to search for vegetables in the produce section and cereal in the cereal aisle. Visual Cognition, 21(6), 678–682. doi: 10.1080/13506285.2013.844959 CrossRefGoogle Scholar
  2. Brady, T. F., Konkle, T., Alvarez, G. A., & Oliva, A. (2008). Visual long-term memory has a massive storage capacity for object details. PNAS Proceedings of the National Academy of Sciences of the United States of America, 105(38), 14325–14329. doi: 10.1073/pnas.0803390105 CrossRefGoogle Scholar
  3. Brady, T. F., Konkle, T., & Alvarez, G. A. (2011). A review of visual memory capacity: Beyond individual items and toward structured representations. Journal of Vision, 11(5), 1–34. doi: 10.1167/11.5.4
  4. Castelhano, M. S., & Henderson, J. M. (2005). Incidental memory for objects in scenes. Visual Cognition, 12, 10171040. doi: 10.1080/13506280444000634 CrossRefGoogle Scholar
  5. Cowan, N. (1995). Attention and memory: An integrated framework. New York, NY, US: Oxford University Press.Google Scholar
  6. Cunningham, C. A., & Wolfe, J. M. (2014). The role of object categories in hybrid visual and memory search. Journal of Experimental Psychology: General, 143(4), 1585–1599. doi: 10.1037/a0036313
  7. Janzen, G., & van Turennout, M. (2004). Selective neural representation of objects relevant for navigation. Nature Neuroscience, 7(6), 673–677. doi: 10.1038/nn1257 CrossRefPubMedGoogle Scholar
  8. Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010). Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. Journal of Experimental Psychology: General, 139(3), 558. doi: 10.1037/a0019165 CrossRefGoogle Scholar
  9. Nelson, D. L., Reed, V. S., & Walling, J. R. (1976). Pictorial superiority effect. Journal of Experimental Psychology: Human Learning and Memory, 2(5), 523. doi: 10.1037/0278-7393.2.5.523
  10. Nickerson, R. S. (1968). A note on long-term recognition memory for pictorial material. Psychonomic Science, 11, 58. doi: 10.3758/BF03330991 CrossRefGoogle Scholar
  11. Paivio, A., & Csapo, K. (1973). Picture superiority in free recall: Imagery or dual coding? Cognitive Psychology, 5(2), 176–206. doi: 10.1016/0010-0285(73)90032-7 CrossRefGoogle Scholar
  12. Schneider, W., Eschman, A., & Zuccolotto, A. (2002). E-Prime user’s guide. Pittsburgh, PA: Psychology Software Tools.Google Scholar
  13. Schmidt, J. & Zelinsky, G. J. (2009). Search guidance is proportional to the categorical specificity of a target cue. The Quarterly Journal of Experimental Psychology, 62(10), 1904-1914. doi: 10.1080/17470210902853530Q
  14. Standing, L. (1973). Learning 10,000 pictures. The Quarterly Journal of Experimental Psychology, 25, 207–222. doi: 10.1080/14640747308400340 CrossRefPubMedGoogle Scholar
  15. Standing, L., Conezio, J., & Haber, R. N. (1970). Perception and memory for pictures: Single-trial learning of 2,500 visual stimuli. Psychonomic Science, 19, 73–74. doi: 10.3758/BF03337426 CrossRefGoogle Scholar
  16. Thomas, M. D., & Williams, C. C. (2014). The target effect: Visual memory for unnamed search targets. Quarterly Journal of Experimental Psychology, 67(11), 2090–2104. doi: 10.1080/17470218.2014.905611 CrossRefGoogle Scholar
  17. Weldon, M. S., & Coyote, K. C. (1996). Failure to find the picture superiority effect in implicit conceptual memory tests. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(3), 670. doi: 10.1037/0278-7393.22.3.670
  18. Williams, C. C. (2010a). Incidental and intentional visual memory: What memories are and are not affected by encoding task? Visual Cognition, 18, 1348–1367. doi: 10.1080/13506285.2010.486280 CrossRefGoogle Scholar
  19. Williams, C. C. (2010b). Not all visual memories are created equal. Visual Cognition, 18, 201–228. doi: 10.1080/13506280802664482 CrossRefGoogle Scholar
  20. Williams, C. C., Henderson, J. M., & Zacks, R. T. (2005). Incidental visual memory for targets and distractors in visual search. Perception & Psychophysics, 67, 816–827. doi: 10.3758/BF03193535 CrossRefGoogle Scholar
  21. Wolfe, J. M. (2012). Saved by a log: How do humans perform hybrid visual and memory search? Psychological Science, 23(7), 698–703. doi: 10.1177/0956797612443968 CrossRefPubMedPubMedCentralGoogle Scholar
  22. Wolfe, J. M., Drew, T., & Boettcher, S. P. (2015). Hybrid search: Picking up a thread from Schneider and Shiffrin (1977). In J. W. Raaijmakers, A. H. Criss, R. L. Goldstone, R. M. Nosofsky, M. Steyvers, J. W. Raaijmakers, . . . M. Steyvers (Eds.) , Cognitive modeling in perception and memory: A festschrift for Richard M. Shiffrin (pp. 63–77). New York, NY: Psychology PressGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2017

Authors and Affiliations

  1. 1.Department of PsychologyMississippi State UniversityMississippi StateUSA
  2. 2.NRTC on Blindness and Low VisionMississippi StateUSA
  3. 3.Psychology DepartmentCalifornia State University San MarcosSan MarcosUSA

Personalised recommendations