Coherence and recurrency: maintenance, control and integration in working memory
- First Online:
- Cite this article as:
- Wolters, G. & Raffone, A. Cogn Process (2008) 9: 1. doi:10.1007/s10339-007-0185-8
Working memory (WM), including a ‘central executive’, is used to guide behavior by internal goals or intentions. We suggest that WM is best described as a set of three interdependent functions which are implemented in the prefrontal cortex (PFC). These functions are maintenance, control of attention and integration. A model for the maintenance function is presented, and we will argue that this model can be extended to incorporate the other functions as well. Maintenance is the capacity to briefly maintain information in the absence of corresponding input, and even in the face of distracting information. We will argue that maintenance is based on recurrent loops between PFC and posterior parts of the brain, and probably within PFC as well. In these loops information can be held temporarily in an active form. We show that a model based on these structural ideas is capable of maintaining a limited number of neural patterns. Not the size, but the coherence of patterns (i.e., a chunking principle based on synchronous firing of interconnected cell assemblies) determines the maintenance capacity. A mechanism that optimizes coherent pattern segregation, also poses a limit to the number of assemblies (about four) that can concurrently reverberate. Top-down attentional control (in perception, action and memory retrieval) can be modelled by the modulation and re-entry of top-down information to posterior parts of the brain. Hierarchically organized modules in PFC create the possibility for information integration. We argue that large-scale multimodal integration of information creates an ‘episodic buffer’, and may even suffice for implementing a central executive.
KeywordsWorking memoryMaintenanceControl of attentionIntegrationPrefrontal cortexRecurrent networksSynchronization
When we return home from work we sometimes find ourselves deeply engaged in thinking about an unsolved problem. Yet, at the same time we manage to find our way home and avoid accidents. This indicates that apparently we are able to automatically control our behavior without needing conscious attention. If suddenly an obstacle appears, however, we are also able to switch attention in a split second. Given the right cues, we may also remember that in the morning we planned to do some shopping on the way home. That too would put solving the problem to a halt and bring us to figure out the best route to the shop. This shows that control may be taken over by retrieving a plan, and that in order to carry it out we need access to a large database, including a route planner. If later on we resume thinking about the problem, we may figure out a solution by combining known elements in a novel configuration, and then start thinking how to use it in the future.
The description given above is an example of the coordinated and goal-directed thoughts and actions that are generated in our brains all the time. Clearly, such coordinated behavior requires a versatile system of cognitive control for which as yet there does not seem to exist an adequate concept. Some of the contours of such a control system, however, are discernable in the example given above, like the ability to maintain representations, to direct attention and to combine previously unrelated information.
An early and influential framework for the study of control and coordination of cognitive behavior is the multi-component model of working memory (Baddeley and Hitch 1974). Working memory (WM) refers to a limited capacity system for temporary holding and manipulating information, that is required for performing a wide range of cognitive tasks such as comprehension, learning and reasoning. Originally the model distinguished three-components. A general control system, or ‘central executive’, and two subsidiary slave systems, a phonological loop and a visuo-spatial sketch pad, each capable of maintaining a limited amount of information. More recently (Baddeley 2003), an episodic buffer was added to connect working memory with long-term memory and to allow binding together information from different sources into integrated episodes. The central executive is the most important but least understood component of WM, and it looks conspicuously like a homunculus. Baddeley is well aware of this criticism, but he defends the central executive concept by pointing out that it defines a problem area for which the processes have to be specified.
An important conceptual model was suggested by Norman and Shallice (1986). They proposed a ‘data base system’ for automatic habitual action routines, containing well-learned stimulus-response associations and cognitive and behavioral skills, and a ‘supervisory attentional system’ capable of controlling behavior by selectively biasing and reconfiguring the available skills and schemas. Automatic control develops gradually through practice as learning processes create associative pathways between perception and action. The conscious form of control is applied when we are confronted by novel and unexpected stimuli, and when we have the intention to attain specific goals. In this case, automatic actions to stimulus patterns in the environment have to be suppressed and replaced by novel task- and goal-directed actions.
The model of Norman and Shallice aimed to explain deficits of executive control that are often observed in patients with damage in prefrontal cortical areas. It is now generally accepted that the prefrontal cortex (PFC) is of crucial importance when behavior must be guided and controlled by internal states and intentions, when automatic responses have to be suppressed, and when tasks require the establishment of new or rapidly changing mappings between perception and action (e.g., Goldman-Rakic 1996; Miller and Cohen 2001; Smith and Jonides 1999; Wood and Grafman 2003). Anatomically, the PFC is well positioned to coordinate processing in the rest of the brain. It consists of a number of strongly interconnected areas that collectively have reciprocal connections with virtually all other neocortical and subcortical structures. It is also an area that shows late development, both phylogenetically and ontogenetically, reaching maturity only in adolescence (Fuster 2001).
Based on these anatomical considerations, we have suggested (Phaf and Wolters 1997) that working memory or, more general, executive functions, may have emerged as a consequence of an evolutionary development of the PFC. This development may have created the possibility for neural processes that could run (partly) independent of present input and output, but that also could modulate, and thus control, perception-action relations in the rest of the cortex. In this view, the role of PFC in controlling behavior is modulatory rather than transmissive, which is similar to the role assigned to PFC in several other models (e.g., Norman and Shallice 1986; Miller and Cohen 2001; O’Reilly et al. 1999). Whereas simple adaptive behavior rests on a cycle of perception, action, and perception-of-action results, the added PFC would allow an internalization of this loop, freeing the organism of the restriction of being aware of, or acting upon, physically present objects or situations only. In fact, it would implement a medium for creating a virtual world, i.e., manipulating internal representations that are independent of the present environment (for a similar idea, describing thinking as simulated behavior and perception, see Hesslow 2002).
More specifically, we suggested that the development of PFC created the possibility to maintain physically absent information in an active state by recurrent connections (loops) between PFC and the rest of the cortex. The recurrent activity in these loops may affect subsequent perceptual and motor processing, i.e., it can redirect attention and control actions by activating or inhibiting particular motor programs. By assuming interactions and integration between loops, more complex forms of representation and control may develop. For instance, recurrent connections with memory systems would allow access to all stored information, and mechanisms for combining information in the loops would allow the formation and updating of future goal states, and ways to achieve them.
A taxonomy of working memory functions
Although there is no generally accepted taxonomy of executive functions, we may speculate which functions would be required. It can be argued, for instance, that to be able to exert top-down control over cognitive processes, patterns of activity representing task relevant information must be actively maintained in the PFC. Moreover, PFC must be able to generate biasing signals to guide the flow of information by selectively inhibiting or activating particular representations and pathways in other parts of the brain. This interaction between PFC and posterior parts of the brain would also be needed to retrieve any relevant information stored in memory. Finally, PFC must be able to integrate information from different sources to implement goal directed behavior over time. We suggest, therefore, that three main executive functions have to be distinguished in PFC. The first function is maintenance, i.e., holding a limited amount of currently needed information, i.e., all task-relevant information supplied by preceding events, in an active form. The second function is attentional control, i.e., top-down controlled selective activation of task-relevant stimulus representations and responses. Only selective activation is required, because in a competitive system this automatically induces selective inhibition of task-irrelevant stimuli and responses. The third function is integration. This function consists of the ability to flexibly combine and reorganize information from different sources in the service of controlling task execution. This also includes control over search in LTM, monitoring and evaluating results of actual or imagined actions, and sequencing operations that are needed for planning, decision making and problem solving. In our view, the central executive operations in PFC are strongly interdependent and continuously interacting with processing in other cortical areas (see also Duncan 2001, for emphasis on interdependence of control, working memory and attention in PFC). A somewhat similar view of PFC functions was suggested by Smith and Jonides (1999). They proposed a distinction between short-term storage and two executive processes, selective attention and task management.
In this paper we will first discuss the neuroanatomical correlates of these hypothesized interdependent working memory functions. Then we will present a neural model for binding and segregation in working memory to simulate the maintenance function. This model is based on recent ideas about synchronization of activation patterns in networks and it is able to explain capacity limitations of maintenance in WM. Next, extensions of this model will be discussed to realize the attentional control mechanism. It is suggested that modulatory effects within PFC may selectively enhance or suppress representations in posterior cortex by recurrent connections. Finally, we will present some ideas on how to realize integration and manipulation functions, given that large-scale multi-modal integration is necessary to explain coherence and coordination of behavior.
Neuroanatomical correlates of working memory in PFC
Generally, the brain shows a remarkable specialization of anatomically distinctive areas performing specific cognitive functions. Also the PFC comprises a number of cyto-architectural distinctive areas or modules that differ in patterns of connectivity with other brain regions (e.g., Miller and Cohen 2001; Wood and Grafman 2003). Much effort has been invested in finding functional specializations within the PFC, i.e., the extent that different areas or modules of PFC are selectively involved in encoding particular types of information (e.g., verbal versus visual), or in implementing various processing functions (e.g., maintenance versus attentional control).
Although this debate is still ongoing, there seems to be a growing consensus that broadly speaking, three major areas in PFC seem to be distinguishable in terms of processing specific types of information (Fuster 2001; Wood and Grafman 2003), probably with additional sub-specializations. First, ventromedial and orbitofrontal regions (BA 11/12/47) seem specifically involved in representing and processing reward and affective information, and thus participate in emotionally and motivationally driven behavior (e.g., Fuster 2001; Rolls 2000). These regions have major reciprocal connections with the temporal lobe and the amygdale complex. Second, medial regions of the PFC (in particular the anterior cingulate cortex, BA 24/32) are suggested to be involved in error monitoring, and in detecting conflicts between competing stimuli and responses (e.g., Ridderinkhof et al. 2004). Third, lateral and anterior PFC areas (BA 9/10/44/45/46; see Petrides 2005, for a review of the architectonic organization of this area) are supposedly involved in all executive functions that are needed for organized goal-directed behavior, such as the selection of goal-relevant information, the manipulation and maintenance of information, and monitoring multiple-events (e.g., Miller and Cohen 2001; Petrides 2000). This area has strong reciprocal connections with parietal and temporal association areas, the hippocampal formation and with all other PFC regions. Damage to lateral PFC typically impairs the ability to formulate and carry out plans and sequences of actions (Fuster 2001) and to control long-term memory encoding and retrieval (Blumenfeldt and Ranganath 2006; Tomita et al. 1999).
There is also increasing evidence, however, that localization of informational content is not precise, but rather a matter of degree. A review of imaging results showed that quite different cognitive demands (i.e., response conflict, task novelty, WM delay and load, and perceptual difficulty) induced very similar patterns of prefrontal recruitment, mainly involving lateral and medial PFC areas (Duncan and Owen 2000). This finding is corroborated by single cell studies that have shown a substantial adaptability of function at the level of individual neurons. Many lateral PFC neurons show highly specific activity patterns depending upon the current task and task conditions (e.g., Asaad et al. 1998, 2000; White and Wise 1999; Quintana and Fuster 1999; Wallis et al. 2001).
These results led Duncan (2001) to postulate the ‘adaptive coding’ model. According to this model, neurons in PFC adjust their function to match the requirements of particular tasks that are carried out. Apparently any given cell can be driven by many different kinds of input, both from posterior parts of the brain and from other PFC areas. The collective responsiveness pattern of all PFC cells thus reflects the particular task relevant conditions. Duncan also noted, however, that the potential of cells to process different types of information, does not exclude regional specializations. This would reconcile results indicating regional specialization, with results from single cell studies showing large scale integration and adaptation. For example, a statistical rather than an absolute specialization seems to apply for left-hemisphere PFC recruitment for verbal material and right-hemisphere PFC recruitment for non-verbal stimuli (see, e.g., Passingham et al. 2002).
A slightly different perspective on functional specialization of PFC areas is to assume that some areas are relatively specialized, whereas others serve more general integrative purposes and come into play in almost any task. Gruber and von Cramon (2003), for instance, found both modality specific and amodal areas in PFC in verbal and visuo-spatial working memory tasks. Other findings suggest that many cells in ventrolateral areas of PFC are specifically sensitive to maintaining representations of single stimuli (e.g., objects or locations), whereas cells sensitive to maintaining complex integrated stimuli and elaborative rehearsal are found more often in dorsolateral areas (Prabhakaran et al. 2000; Owen 2000; Wagner et al. 2001). Wallis and Miller (2003) presented evidence suggesting that orbitofrontal cortex, which is specifically involved in reward and affect processing, passes on this information to dorsolateral areas of PFC where it is combined with other information to control behavior. Similarly, Ridderinkhof et al. (2004) concluded from a review that detection of response conflicts and response errors elicits overlapping foci of activation in medial PFC areas, and that this activation serves as a signal that engages regulatory processes in lateral PFC areas.
Learning and working memory
Our view of working memory is that its informational content consists of the active part of long-term memory representations that is available at any point in time for controlled processing (e.g., Cowan 1999; O’Reilly et al. 1999). These representations presumably are stored in the temporal, parietal and occipital cortex, so the functions of working memory (maintenance, selection and integration) that apply to these representations are realized by an interaction between PFC and posterior cortical areas. In this view, PFC is not itself involved in storing long-term memory information. Its representational role would be limited to temporarily maintaining and integrating information stored elsewhere. This view seems to be endorsed by the fact that the neural circuits and pathways assumed to be involved in the two main learning processes (Eichenbaum and Cohen 2001), and the corresponding procedural and declarative memory systems (Squire and Zola-Morgan 1991), do not include the PFC as a representational medium.
The suggestion that PFC implements WM functions, but is not itself involved in associative learning, would fit the requirement that it is maximally flexible in the service of generating novel combinations of representations that are stored somewhere else. There are no indications, however, that the neural architecture and processes of the PFC differ fundamentally from those in posterior cortex which intrinsically generate a learning capacity. Therefore, some learning (and representational) capacity in PFC cannot be ruled out a priori.
Some evidence suggests that the information used in working memory tasks does not have to be stored in PFC. First, prefrontal patients generally perform quite well in tasks requiring the retrieval of previously learned procedural and declarative knowledge. However, they are typically impaired in episodic memory tasks requiring for instance elaboration of stimuli, ordered recall and source monitoring. So the problem with prefrontal damage does not seem an inability to store information in memory, but an inability to control encoding and retrieval processes (e.g., Shimamura 1995). Second, several recent studies have linked activity in PFC areas with promoting effective LTM formation (e.g., Blumenfeld and Ranganath 2006; Buckner 2003). They suggest, however, that the information itself is not stored in PFC. Instead, the interaction between PFC and posterior cortex controls what is selectively activated, maintained and elaborated, and what is subsequently stored as a novel representation via the medial temporal lobe and the hippocampus (see, e.g., Ranganath et al. 2005). Third, an increasing number of studies showed that the activation patterns of many single neurons or neuron ensembles in PFC quickly and flexibly adapt to represent completely arbitrary rules (Asaad et al. 2000; Wallis et al. 2001; White and Wise 1999), or to arbitrary action sequences (Averbeck et al. 2006). Although in all of these studies a number of learning trials precedes the development of task specific activation patterns, the speed and flexibility of the adaptation process seems more suggestive of an adaptive coding than of an associative learning process.
Other authors have suggested that learning in PFC does occur. O’Reilly et al. (1999), for example, suggested that the information that is maintained in WM consists of activation patterns in PFC that are sustained by strong mutual excitation of the neurons involved. To explain the flexibility in combining these representations without losing their specificity, they assumed that PFC representations had to be relatively isolated from each other. Therefore, learning in PFC was supposed to be slow, taking place over many years, and eventually producing a rich palette of independent PFC representations that enable flexible problem-solving skills. Such representations might have a hierarchical structure with an increasing level of abstraction along a posterior-anterior PFC axis, probably with the most posterior PFC representations being closely connected to detailed memory representations in posterior cortex.
A worked out version of this model by Rougier et al. (2005) showed that extensive training of their model with various tasks resulted in the development of abstract rule-like PFC representations that supported flexible generalization in novel tasks. The PFC representations developed slowly, but once learned adaptive behavior was mediated by a search for a task appropriate pattern of activity (cf. the adaptive coding mechanism of Duncan 2001), rather than the need to update connection weights. Although the simple tasks simulated in this model may also apply to more realistic and complex rules, the authors point out that it still leaves unexplained how PFC representations can be dynamically recombined and can interact with other systems (such as episodic memory, language and affect).
It remains to be seen whether the supposed PFC learning mechanism is viable. Over brief periods of time, learning probably does not play a major role, but a role of learning over extensive periods cannot be ruled out.
Evidence for three interdependent working memory functions in PFC
Neurophysiological studies have established persistent activity during delays (i.e., after stimulus offset) as the main candidate for a neural substrate of the maintenance function of WM. Single cells showing persistent firing during a delayed-matching task were first discovered in PFC, and later in other neocortical areas like inferotemporal (IT) cortex (Fuster and Alexander 1971; Goldman-Rakic 1996; Nakamura and Kubota 1995).
Different areas, especially in posterior parts of PFC, may be involved in maintaining modality specific information. Smith and Jonides (1999), for instance, suggested a relatively specialized role of left and right PFC hemispheres in maintaining verbal and visual information, respectively. Moreover, they proposed that maintenance of spatial information may involve more dorsally located regions than maintenance of object information (see also Baddeley 2003).
In principle, there are two ways to maintain activity in neurons during delays. One is to assume that input activation creates an activation pattern in a network of cells that is maintained within the network by auto-association (Amit 1995; Deco and Rolls 2003), and possibly augmented by a dopamine gating mechanism (O’Reilly et al. 1999). The other possibility is that the activation pattern oscillates in a recurrent loop between different networks of cells.
There is much evidence that supports an important role of recurrent loops in maintaining information during working memory tasks. Strongest evidence is that during delays continued activity is not only found in PFC, but also in modality specific areas in IT cortex when coding for objects (Fuster et al. 1985; Miller et al. 1993; Tomita et al. 1999; Ungerleider et al. 1998), and in parietal cortex when coding for locations (Curtis and d’Esposito 2003; Rowe et al. 2000; Sakai et al. 2002). Moreover, localized cooling of either PFC or IT cortex interferes with activity in the other area and causes behavioral deficits in a working memory task (see Fuster 2001). From a review of the literature, Ranganath (2006) concluded that information in visual WM tasks is maintained through persistent activity in visual cortical areas (e.g., inferotemporal and parietal areas) that is promoted by top-down input from PFC. So according to Ranganath, maintenance would be an interaction between PFC and modality or object specific posterior areas. An additional loop involving the medial temporal lobe (i.e., the hippocampus and related areas) may be required to maintain complex novel stimuli, and to quickly create new long-term representations (e.g., Ranganath et al. 2005).
Ranganath (2006) is somewhat unclear about the nature of the feedback from PFC in maintenance. As a further specification we would suggest a fixed connection scheme (probably shaped by long-term learning) between posterior brain areas, containing long-term memory or newly created representations of stimuli, and adjacent areas in PFC. These pathways convey to PFC what information is being maintained. This may also explain the finding that during maintenance, activation in posterior areas is more vulnerable to distraction than activation in PFC (Miller et al. 1993; O’Connor et al. 2002). Maintenance may resist such distraction because PFC activation can reinstate activation of corresponding lower level representations. Such reinstatement probably also underlies memory retrieval and mental imagery. It has been shown, for example, that in tasks requiring participants to imagine faces or buildings, category specific regions in the IT cortex became activated (e.g., Ishai et al. 2002; O’Craven and Kanwisher 2000).
As was argued by O’Reilly et al. (1999), there are two seemingly incompatible core requirements of working memory. The first is to robustly maintain goals and task instructions over time even when faced with strong interference. The second is to rapidly and flexibly update goals and tasks if circumstances change. Their solution to combine robust maintenance with flexible updating has been to assume a dopamine gating mechanism. Under tonic dopamine levels a strong mutual excitation within a cell assembly would lead to sustained activity and resistance to interference by irrelevant input. Phasic shifts of dopamine levels induced by significant events (e.g., success or failure in a task or novel stimuli predicting reward) would enhance the strength of afferent input and cause updating of the activation state, which subsequently may lead to representing novel strategies, rules, goals or task states.
Although a dopamine-gating mechanism is an interesting possibility, we believe that re-entry of activation through recurrent neuronal circuits is the main mechanism for the maintenance function of WM. It is likely that there are many of such recurrent loops for different types of information. These loops link the perceptual, memory and motor representational areas in posterior cortex to PFC. They feed information into PFC and, in turn, are activated by the recurrent activation from PFC. Modulating activity in these loops by other PFC sources would modulate the top-down recurrency. We also suggest and that these loops form a hierarchy, at the lowest level maintaining simple stimuli or features and at higher levels maintaining increasingly complex stimulus relations and rules. We will show how this idea can be implemented and explore the feasibility of such an implementation. We agree, however, that especially for the highest and most global or integrative levels of maintaining information, a dopamine gating mechanism for auto-associative maintenance cannot be ruled out.
Attentional control by executive functions in PFC requires top-down effects on local information processing in posterior brain areas (e.g., Desimone and Duncan 1995; Miller and Cohen 2001; Shimamura 2000). Evidence for such a role comes, for instance, from findings showing that an important characteristic of behavioral deficits following damage to the PFC is extreme distractability (i.e., an inability to suppress interfering information) and disinhibition (i.e., an inability to suppress inappropriate responses).
A mechanism for top-down attentional control is suggested by the ‘biased competition model’ (Chelazzi et al. 1993; Desimone and Duncan 1995; Downing 2000; Reynolds et al. 2000; Gazzaley et al. 2005). According to such a model, top-down control activates corresponding representations that are then in a better position to compete with irrelevant information for perceptual awareness and control of motor behavior. Such top-down attentional modulation of neural responses, i.e., relative enhancement of neuronal responses to task-relevant stimuli and relative suppression of neuronal responses to task-irrelevant stimuli, has been shown throughout the visual system as early as the primary visual cortex (e.g., Desimone and Duncan 1995; Reynolds et al. 1999), and the exact areas that are biased depend on the task that is performed (e.g., Kastner et al. 1999).
Top-down attentional bias not only pre-activates specific target representations but also spreads activation to related representations. Distractors with visual similarities to targets also attract attention (are positively biased), as well as distractors with semantic or associative links to the target (Moores et al. 2003). Task specific activity in PFC also generates top-down signals involved in the selection of actions, and in long-tem memory retrieval (Hasegawa et al. 1998) and storage (Brewer et al. 1998; Kyd and Bilkey 2003; Blumenfeld and Ranganath 2006; Wagner et al. 1998). This perspective thus suggests a single underlying mechanism of cognitive control by the PFC, namely a top-down biasing effect on processing in specialized sensory, motor and memory systems responsible for task performance (see, e.g., Miller and Cohen 2001).
We will explore the feasibility of a mechanism for attentional control based on modulating feedback signals from PFC in the recurrent circuits that are used for maintenance. We suggest that attentional control is mediated by biasing the recurrent loops between PFC and posterior cortex. In this view, the biasing signals stemming from higher integrative areas in PFC would modulate the recurrent feedback to posterior cortical areas (see also Deco and Rolls 2003).
The importance of a large-scale integrative system for the coordination of behavior has been widely acknowledged. Such a system would provide the combination and integration of all information necessary for controlling task-relevant behavior, i.e., it would comprise all task-relevant information supplied by the present context and goals, and by knowledge retrieved from memory. Baddeley (2000), for instance, suggested an episodic buffer as an additional subsystem in his WM model which is capable of integrating and maintaining information from different sources. In a similar vein, Miller and Cohen (2001) argued for a cognitive control system in the PFC that represents goals and the means to achieve them. They suggest that this system integrates converging inputs from many sources. It exerts control by divergent feedback signals to sensory, memory and motor areas in posterior cortex which mediate directed attention, response selection, and guide retrieval from LTM.
A high level integrative control system was also proposed by Koechlin et al. (2003). They suggested a cascade model, consisting of three nested levels of cognitive control. At the top of this cascade, located in rostral (anterior) parts of the PFC, they assume an episodic control system involved in selecting task-set representations ‘according to events that previously occurred or to ongoing internal goals’ (p. 1181). The view that progressively ‘higher’ neural areas support functions that are increasingly more integrative also has been endorsed also by Fuster (2001). He argued that a cascaded control model, assuming several nested levels of control, with a highly integrative system at the top, may explain neuropsychological results showing that damage to the top system only affects tasks that require a high degree of information integration (e.g., planning and problem solving), but does not interfere with tasks that can be executed at a lower level of control.
Potentially the highest level of integration is attained in the anterior PFC (aPFC), in humans corresponding to BA10. In a recent discussion, Ramani and Owen (2004) pointed out this area as a likely candidate for being the apex of a hierarchical system of PFC modules, because it is predominantly (or even exclusively) reciprocally connected to other supramodal PFC areas. Therefore, it is in the position to integrate everything the brain is capable of representing. Moreover, the aPFC has anatomical characteristics (a high dendritic spine density combined with a low density of cell bodies) that make it likely to be involved in integration. The aPFC has been suggested to be specialized for processing internal mental states and introspective evaluation (Christoff and Gabrieli 2000), monitoring successful retrieval (Ranganath and Paller 2000), management and monitoring of goals and sub-goals (Koechlin et al. 2000; Braver and Bongliatti 2002), integration of information over time (Braver et al. 2001; Koechlin et al. 2003; Sakai et al. 2002), and manipulation of relational knowledge (Kroger et al. 2002).
A hierarchical model of integration based on the convergence of input from lower level specialized PFC modules on higher integrative levels, is consistent with claims that a distinction can be made between PFC areas involved in simple maintenance, and others involved in maintenance of complex information and executive processes (D’Esposito et al. 1999; Sakai et al. 2002). For example, several findings suggest a distinction between ventrolateral (vlPFC) and dorsolateral (dlPFC) areas in terms of the level of abstractness of information processed (e.g., Koechlin et al. 2003; O’Reilly et al. 2002). Tasks requiring processing of single words or concepts mainly seem to involve vlPFC (e.g., Wagner et al. 2001), whereas in more complex tasks, like sentence processing, enhanced involvement of dlPFC has been found (e.g., Hashimoto and Sakai 2002; Kerns et al. 2004b). A relatively high integrative role for the dlPFC (as compared to vlPFC) is also suggested by findings showing this area to be involved in tasks that go beyond simple maintenance, such as elaborative rehearsal (Blumenfeldt and Ranganath 2006), processing relations between stimuli (Kroger et al. 2002), and solving conflicts that are signalled by medial areas of PFC (Egner and Hirsch 2005; Kerns et al. 2004a).
A neurocomputational model of maintenance, control and integration
Elsewhere (Raffone and Wolters 2001), we have presented a model for the temporary holding in (visual) working memory of a limited number of neural patterns, simulating either single features or integrated objects. The model implemented a cortical mechanism of maintenance in a network of model neurons with biologically plausible parameters. Although the model implemented a visual working memory system, the principles may be applicable to any form of information or type of working memory.
In the model WM was assumed to be based on recurrent connections between IT cortex containing representations of objects or features, and corresponding neurons in PFC. The IT representations were modelled as strongly associated neural assemblies that generate synchronized firing patterns when activated by external input. The simultaneous activation of independent assemblies in IT causes competition via inhibitory interneurons. Due to the neuron characteristics, this leads to desynchronization among the activation patterns of competing assemblies resulting in a sustained phase-locked activation of multiple assemblies over time.
Maintenance in cortical circuits of visual working memory was shown to be possible in terms of oscillatory reverberations between PFC and IT modules. Firing rate oscillations induced during stimulus presentation were maintained after stimulus offset by active feedback from prefrontal areas. Neurophysiological plausible model parameters enforced a limitation of about three to four independent assemblies that could be maintained in this way. This number closely coincides with recent estimates of the maintenance capacity of WM (e.g., Cowan 2001). The same mechanism that optimizes coherent pattern segregation, also poses a limit to the number of assemblies (about four) that can concurrently reverberate. The model thus indicated that selective synchronization and desynchronization of feedback-based oscillatory reverberations creates a suitable medium for a visual working memory. Simulations showed that the model was able to explain both the existence of severe limits in the number of assemblies (stimuli) that can be held (e.g., Luck and Vogel 1997; Luck and Beach 1998), and the absence of a limit on the size of assemblies, i.e., representing either simple stimuli or complex chunks (e.g., Ericsson and Delaney 1999). We introduced the concept of ‘chunking fields’ to account for the creation of more complex neural assemblies (e.g., higher order information units or chunks) through previous Hebbian learning (e.g., Hummel and Biederman 1992; Singer 1995). The model can potentially account for different degrees of within-object feature integration (Olson and Jiang 2002) in terms of graded synchrony between neurons coding for features of the same object.
In the IT module, strong connections are implemented within and weak connections between assemblies coding different ‘stimuli’. We also implemented a global inhibition (competition) mechanism between IT assemblies. This circuitry, in which a given assembly is inhibited by the firing of the other assemblies in the IT module, mediates an active desynchronization mechanism.
Stimulus input to the IT module is given by stochastic spike trains from (not explicitly modeled) lower visual areas, during a limited onset-offset period. During stimulus presentation all stimulus specific IT neurons received a high frequent train of spikes as input, which was added to a continuous stochastic low frequency spike input to all IT neurons. The vlPFC module has a coding structure “matching” the structure of the IT module (see Fig. 2). Each IT assembly is recurrently connected with one vlPFC assembly, with a signaling delay of 15 ms in both directions. Although we implemented a monosynaptic feedback circuit, in real cortical networks this delayed feedback is likely to be mediated by multisynaptic circuitries of the ‘synfire’ type, with a stable transmission along multiple diverging/converging synaptic links (Abeles et al. 1993a, b; see also Villa and Fuster 1992). In vlPFC and dlPFC there are no inhibitory neurons and no inter-assembly connections.
Although stimuli are coded by large assemblies of neurons in real cortical networks, in the present model individual stimuli are coded by single neurons. Our simulations will show that the functional processes we have investigated previously at the level of assemblies of neurons (Raffone and Wolters 2001) also hold at the level of single neurons, thus pointing out the robustness of the observed effects. The use of single neurons instead of assemblies of tens of neurons, is also motivated by the sake of running relatively fast simulations with a smaller time-step in numerical integration of neuronal equations (higher computational accuracy) than in our previous investigations.
The model, of course, is very simplified with respect to the complexity of the real cortical networks. Feedback from the vlPFC module does no more than maintaining the oscillatory state of IT assemblies after the stimulus offset. More realistic network versions might include “closed” reverberatory circuits within prefrontal areas, making prefrontal assemblies independent from the IT assemblies in maintaining the delay activity. This would allow modality shifts by activating other prefrontal assemblies, which in turn would trigger reverberatory circuitries in lower cortical areas, in the absence of actual physical stimulation. Moreover, we used a simplified one-to-one matching between IT and vlPFC assemblies, whereas it seems more realistic to assume that prefrontal assemblies are relatively non-specific, thus being connected to multiple sets of neurons in posterior areas. Instead of the one-to-one matching of assemblies, dlPFC neurons might simply send back activation to all vlPFC neurons, and these in turn to all IT neurons from which they receive activation. However, the present simple architecture is sufficient for demonstrating the functional principles we have specified earlier.
Simulating maintenance in working memory
In this simulation, we did not replicate our previous demonstration (Raffone and Wolters 2001) that the capacity of the reverberatory circuit is independent of the size of the IT representations. As we showed there, larger cell assemblies of connected neurons with fast-signaling positive weights, quickly synchronize and behave as single units. We demonstrated that the maintenance capacity of our model is not a function of the size of a cell assembly, but of the presence and strength of associations between the units of an assembly. This probably mimics the fact that working memory capacity is not a function of the amount of information per se, but of the level of organization or chunking of the material to be maintained (Miller 1956).
Simulating attentional control in working memory
Given the severe limitation of the capacity of working memory, neural mechanisms are necessary to restrict access to it, depending on the behavioral relevance of the information. As shown by Asaad et al. (1998, 2000), prefrontal neurons of monkeys performing a ‘delayed-matching-to-sample’ task exhibited a higher firing rate when they coded for a rule-defined target-object. Such neurons were suggested to be involved in both the selection and maintenance of behaviorally relevant information.
To model a selective attention mechanism by which selection and retention is controlled by working memory cortical circuits, a top-down input to IT neurons has to be supplied by a prefrontal source involved in supervisory control rather than in mere information maintenance. A similar assumption was made in a model for attentional control by Deco and Rolls (2003). Such a mechanism implements the top-down biasing of the competition between neural assemblies in posterior cortex as suggested by Desimone and Duncan (1995). In our model this supervisory control signal could be supplied through the dlPFC and vlPFC modules.
As can be inferred from Fig. 4b, the attentional bias (attentional input for four out of eight items) in the voltage dependent condition is expressed after the stimulus-input onset, in terms of an amplification effect. Also in this simulation all four biased (and one unbiased) items were maintained after stimulus offset. We believe that voltage-dependent input to high-level areas of the visual system may enable an effective top-down control mechanism, since it produces strong amplification effects, but only in the presence of relevant stimulus-input (see also Hirsch and Gilbert 1991; Tononi et al. 1992, about voltage-dependent signaling in the visual system). This multiplicative signaling effect would prevent spurious activation of representations in the visual system in the absence of any bottom-up sensory evidence.
A series of follow-up simulations showed that the probability of item retention depends on the relative biasing top-down input for a given item. A strong attentional input to one or two items may ‘narrow’ visual working memory capacity to one or two highly focused elements, due to the higher firing rate (e.g. bursting) of the neurons coding for the focused items.
Mechanisms for integration in working memory
Any domain-specific account of functions or representations in the brain ultimately implies a binding problem, since the specialized neural representations need to be dynamically bound to enable the creation (and a conscious readout) of coherent and integrated representations of complex events. These binding processes may occur in terms of the selective synchronization of reverberating cell assemblies, as shown in our simulations. In this framework, segregated neural representations may be bound via reciprocal synchronizing connections that originate within the prefrontal cortex (for a discussion of this and other forms of binding, see Murre et al. 2006; see also Engel et al. 2001; Varela et al. 2001).
The same mechanism may allow Hebbian learning to occur between meaningful or behaviorally-salient conjunctions of independent elements (see Miltner et al. 1999). In order to account for the retention of integrated units in visual working memory, in previous simulations we defined units by assuming already established chunking fields, i.e. pre-existing associative connections between IT neurons, coding different features of the same unit. A major problem, however, is how ‘chunking fields’ or integrated representations in long-term memory are formed, i.e. how associative Hebbian learning may take place in the simulated cortical circuits of (visual) working memory. This learning process should operate against the phase-segregation tendency due to mutual desynchronizing inhibition. In the present model, the dlPFC module may perform this temporary synchronization control processes.
In a series of simulations, we used the entrainment of vlPFC and IT neurons by a top-down dlPFC signal and combined it with a timing-dependent Hebbian learning rule in IT (simulating timing-dependent plasticity of synapses, see e.g., Körding and König 2000; Song et al. 2000, see Fig. 5c, d). These simulations showed that the synchronous firing in IT, induced by a top-down recurrent signal, causes the weights between the IT neurons to increase, resulting in a ‘chunked’ representation of initially independent features. As synaptic weights increased according to the timing-dependent learning rule, the synchronization among related IT neurons becomes gradually less dependent on the feedback action of dlPFC neurons on related vlPFC neurons, ultimately leading to an “automatic” within-chunk integration within the IT module, in the absence of any control feedback from the dlPFC module (Fig. 5e, f).
Control simulations with the same model and learning parameters, showed that a relatively high number of simultaneous presentations of initially independent elements, may slowly give rise to novel chunks with a variable degree of stability, even in the absence of feedback from dlPFC to vlPFC. This slow chunk-learning process may be related to an implicit learning process, rather than the more rapid process of controlled chunk formation by feedback from dlPFC.
Miller and Cohen (2001) concluded that PFC is critical when top-down control of processing is needed for the guidance of behavior by internal states or intentions. In this paper we conjecture that such control is possible by assuming that the PFC is a brain system that is reciprocally connected to posterior parts of the brain and therefore capable of modulating direct perception-action relations. We argued that three interdependent functions are required for such control: maintenance of activation patterns even in the face of distraction, large scale integration of information from different sources, and top-down selective attention. We also showed that these functions can be simulated, at least in principle, with a biologically plausible model assuming recurrent connections and a hierarchical structure of PFC. Importantly, the functions were simulated using a single framework consisting of a hierarchy of recurrently connected modules. This framework proved capable of maintaining a limited number of mutually desynchronized patterns, of controlling selective attention by top-down modulatory signals, and of high level integration by top-down synchronization of the activity of independent patterns.
Of course, the model suggested here is extremely simplified and it does not capture many of the complexities and intricate details of the real system. For instance, we did not try to incorporate modulatory effects of neurotransmitters and we have not discussed the many possible interactions of the PFC with subcortical systems, like the basal ganglia and amygdala, and the cerebellum. It may also be noted that we discussed, but did not try to incorporate, the possibly important role of a dopamine gating system for maintaining high level integrative activation states (O’Reilly et al. 1999, 2002).
One of our aims to develop the model was to try to find out how the structural and functional characteristics of the PFC may explain the central executive component of working memory assumed by Baddeley (2003). In the latest version of this model an additional component, an episodic buffer, was proposed. This component was thought necessary to explain interactions between working memory and LTM, for example to explain how chunking may supplement the capacity for immediate serial recall. A continuous interaction between working memory and LTM is a basic assumption in our model, and we have shown that chunking is a natural consequence of this interaction.
According to Baddeley (2003), the episodic buffer allows information from different systems to be integrated, and it may be regarded as the ‘storage component of the executive’. Moreover, Baddeley suggested that the buffer is a separate temporary store in which long-term memory information is downloaded in order to be manipulated and used for creating new representations. In our view, the structure of such an episodic buffer would actually be a hierarchy of PFC modules maintaining and integrating information at successively higher levels. Information is not copied and downloaded in PFC, but selected task-relevant perceptual and memory information in posterior cortical areas is kept in an active state by recurrent loops. We assume that at the highest integration levels, presumably located in anterior and dorsolateral areas of PFC, information can be integrated over space and time. At this level, general goals, and plans to achieve them, would be generated. We suggest that this may be the implementation of an episodic buffer and even of a central executive. The creation of short-term and long-term goals at high levels of integration, allows to control behavior by top-down regulation of activation in subordinate modules and eventually by modulating perception-action systems. Current goals and tasks would have to be maintained here in order to control ongoing behavior. More distant goals, however, may be stored in LTM to be retrieved later on.
We do not imply that control is always hierarchical. Because the PFC is assumed to be a modulatory system, simple tasks may be executed automatically without PFC involvement. Only controlled processing requires modulation of automatic processes by PFC feedback. Also here, however, control does not need to be completely hierarchical. Depending on the type and complexity of a task, control may be executed by specialized modules at subordinate levels. Automatic processing and controlled processing under the guidance of subordinate levels in PFC, is a requirement to free higher integrative areas to engage in the kind of simulated actions and perceptions, and using anticipated outcomes, that are the contents of conscious thought and planning.
So there are things we can do automatically, using previously established associations between perceptual events and effective responses, and there are things we can do in a controlled way, using the possibility of PFC to modulate and thus control perception and action. The possibility of PFC to work off-line, to use and integrate all present and past knowledge for creating virtual worlds and for making plans and carry them out if conditions are appropriate, has generated a tremendously flexible potential for control. The model we propose is vastly insufficient to simulate such cognitive feats that we engage in daily. We believe, however, that the principles for a controlling system we have suggested may be a first step towards a better understanding of what is until now a cognitive Terra Incognita.