Introduction

When we return home from work we sometimes find ourselves deeply engaged in thinking about an unsolved problem. Yet, at the same time we manage to find our way home and avoid accidents. This indicates that apparently we are able to automatically control our behavior without needing conscious attention. If suddenly an obstacle appears, however, we are also able to switch attention in a split second. Given the right cues, we may also remember that in the morning we planned to do some shopping on the way home. That too would put solving the problem to a halt and bring us to figure out the best route to the shop. This shows that control may be taken over by retrieving a plan, and that in order to carry it out we need access to a large database, including a route planner. If later on we resume thinking about the problem, we may figure out a solution by combining known elements in a novel configuration, and then start thinking how to use it in the future.

The description given above is an example of the coordinated and goal-directed thoughts and actions that are generated in our brains all the time. Clearly, such coordinated behavior requires a versatile system of cognitive control for which as yet there does not seem to exist an adequate concept. Some of the contours of such a control system, however, are discernable in the example given above, like the ability to maintain representations, to direct attention and to combine previously unrelated information.

An early and influential framework for the study of control and coordination of cognitive behavior is the multi-component model of working memory (Baddeley and Hitch 1974). Working memory (WM) refers to a limited capacity system for temporary holding and manipulating information, that is required for performing a wide range of cognitive tasks such as comprehension, learning and reasoning. Originally the model distinguished three-components. A general control system, or ‘central executive’, and two subsidiary slave systems, a phonological loop and a visuo-spatial sketch pad, each capable of maintaining a limited amount of information. More recently (Baddeley 2003), an episodic buffer was added to connect working memory with long-term memory and to allow binding together information from different sources into integrated episodes. The central executive is the most important but least understood component of WM, and it looks conspicuously like a homunculus. Baddeley is well aware of this criticism, but he defends the central executive concept by pointing out that it defines a problem area for which the processes have to be specified.

An important conceptual model was suggested by Norman and Shallice (1986). They proposed a ‘data base system’ for automatic habitual action routines, containing well-learned stimulus-response associations and cognitive and behavioral skills, and a ‘supervisory attentional system’ capable of controlling behavior by selectively biasing and reconfiguring the available skills and schemas. Automatic control develops gradually through practice as learning processes create associative pathways between perception and action. The conscious form of control is applied when we are confronted by novel and unexpected stimuli, and when we have the intention to attain specific goals. In this case, automatic actions to stimulus patterns in the environment have to be suppressed and replaced by novel task- and goal-directed actions.

The model of Norman and Shallice aimed to explain deficits of executive control that are often observed in patients with damage in prefrontal cortical areas. It is now generally accepted that the prefrontal cortex (PFC) is of crucial importance when behavior must be guided and controlled by internal states and intentions, when automatic responses have to be suppressed, and when tasks require the establishment of new or rapidly changing mappings between perception and action (e.g., Goldman-Rakic 1996; Miller and Cohen 2001; Smith and Jonides 1999; Wood and Grafman 2003). Anatomically, the PFC is well positioned to coordinate processing in the rest of the brain. It consists of a number of strongly interconnected areas that collectively have reciprocal connections with virtually all other neocortical and subcortical structures. It is also an area that shows late development, both phylogenetically and ontogenetically, reaching maturity only in adolescence (Fuster 2001).

Based on these anatomical considerations, we have suggested (Phaf and Wolters 1997) that working memory or, more general, executive functions, may have emerged as a consequence of an evolutionary development of the PFC. This development may have created the possibility for neural processes that could run (partly) independent of present input and output, but that also could modulate, and thus control, perception-action relations in the rest of the cortex. In this view, the role of PFC in controlling behavior is modulatory rather than transmissive, which is similar to the role assigned to PFC in several other models (e.g., Norman and Shallice 1986; Miller and Cohen 2001; O’Reilly et al. 1999). Whereas simple adaptive behavior rests on a cycle of perception, action, and perception-of-action results, the added PFC would allow an internalization of this loop, freeing the organism of the restriction of being aware of, or acting upon, physically present objects or situations only. In fact, it would implement a medium for creating a virtual world, i.e., manipulating internal representations that are independent of the present environment (for a similar idea, describing thinking as simulated behavior and perception, see Hesslow 2002).

More specifically, we suggested that the development of PFC created the possibility to maintain physically absent information in an active state by recurrent connections (loops) between PFC and the rest of the cortex. The recurrent activity in these loops may affect subsequent perceptual and motor processing, i.e., it can redirect attention and control actions by activating or inhibiting particular motor programs. By assuming interactions and integration between loops, more complex forms of representation and control may develop. For instance, recurrent connections with memory systems would allow access to all stored information, and mechanisms for combining information in the loops would allow the formation and updating of future goal states, and ways to achieve them.

A taxonomy of working memory functions

Although there is no generally accepted taxonomy of executive functions, we may speculate which functions would be required. It can be argued, for instance, that to be able to exert top-down control over cognitive processes, patterns of activity representing task relevant information must be actively maintained in the PFC. Moreover, PFC must be able to generate biasing signals to guide the flow of information by selectively inhibiting or activating particular representations and pathways in other parts of the brain. This interaction between PFC and posterior parts of the brain would also be needed to retrieve any relevant information stored in memory. Finally, PFC must be able to integrate information from different sources to implement goal directed behavior over time. We suggest, therefore, that three main executive functions have to be distinguished in PFC. The first function is maintenance, i.e., holding a limited amount of currently needed information, i.e., all task-relevant information supplied by preceding events, in an active form. The second function is attentional control, i.e., top-down controlled selective activation of task-relevant stimulus representations and responses. Only selective activation is required, because in a competitive system this automatically induces selective inhibition of task-irrelevant stimuli and responses. The third function is integration. This function consists of the ability to flexibly combine and reorganize information from different sources in the service of controlling task execution. This also includes control over search in LTM, monitoring and evaluating results of actual or imagined actions, and sequencing operations that are needed for planning, decision making and problem solving. In our view, the central executive operations in PFC are strongly interdependent and continuously interacting with processing in other cortical areas (see also Duncan 2001, for emphasis on interdependence of control, working memory and attention in PFC). A somewhat similar view of PFC functions was suggested by Smith and Jonides (1999). They proposed a distinction between short-term storage and two executive processes, selective attention and task management.

In this paper we will first discuss the neuroanatomical correlates of these hypothesized interdependent working memory functions. Then we will present a neural model for binding and segregation in working memory to simulate the maintenance function. This model is based on recent ideas about synchronization of activation patterns in networks and it is able to explain capacity limitations of maintenance in WM. Next, extensions of this model will be discussed to realize the attentional control mechanism. It is suggested that modulatory effects within PFC may selectively enhance or suppress representations in posterior cortex by recurrent connections. Finally, we will present some ideas on how to realize integration and manipulation functions, given that large-scale multi-modal integration is necessary to explain coherence and coordination of behavior.

Neuroanatomical correlates of working memory in PFC

Generally, the brain shows a remarkable specialization of anatomically distinctive areas performing specific cognitive functions. Also the PFC comprises a number of cyto-architectural distinctive areas or modules that differ in patterns of connectivity with other brain regions (e.g., Miller and Cohen 2001; Wood and Grafman 2003). Much effort has been invested in finding functional specializations within the PFC, i.e., the extent that different areas or modules of PFC are selectively involved in encoding particular types of information (e.g., verbal versus visual), or in implementing various processing functions (e.g., maintenance versus attentional control).

Although this debate is still ongoing, there seems to be a growing consensus that broadly speaking, three major areas in PFC seem to be distinguishable in terms of processing specific types of information (Fuster 2001; Wood and Grafman 2003), probably with additional sub-specializations. First, ventromedial and orbitofrontal regions (BA 11/12/47) seem specifically involved in representing and processing reward and affective information, and thus participate in emotionally and motivationally driven behavior (e.g., Fuster 2001; Rolls 2000). These regions have major reciprocal connections with the temporal lobe and the amygdale complex. Second, medial regions of the PFC (in particular the anterior cingulate cortex, BA 24/32) are suggested to be involved in error monitoring, and in detecting conflicts between competing stimuli and responses (e.g., Ridderinkhof et al. 2004). Third, lateral and anterior PFC areas (BA 9/10/44/45/46; see Petrides 2005, for a review of the architectonic organization of this area) are supposedly involved in all executive functions that are needed for organized goal-directed behavior, such as the selection of goal-relevant information, the manipulation and maintenance of information, and monitoring multiple-events (e.g., Miller and Cohen 2001; Petrides 2000). This area has strong reciprocal connections with parietal and temporal association areas, the hippocampal formation and with all other PFC regions. Damage to lateral PFC typically impairs the ability to formulate and carry out plans and sequences of actions (Fuster 2001) and to control long-term memory encoding and retrieval (Blumenfeldt and Ranganath 2006; Tomita et al. 1999).

There is also increasing evidence, however, that localization of informational content is not precise, but rather a matter of degree. A review of imaging results showed that quite different cognitive demands (i.e., response conflict, task novelty, WM delay and load, and perceptual difficulty) induced very similar patterns of prefrontal recruitment, mainly involving lateral and medial PFC areas (Duncan and Owen 2000). This finding is corroborated by single cell studies that have shown a substantial adaptability of function at the level of individual neurons. Many lateral PFC neurons show highly specific activity patterns depending upon the current task and task conditions (e.g., Asaad et al. 1998, 2000; White and Wise 1999; Quintana and Fuster 1999; Wallis et al. 2001).

These results led Duncan (2001) to postulate the ‘adaptive coding’ model. According to this model, neurons in PFC adjust their function to match the requirements of particular tasks that are carried out. Apparently any given cell can be driven by many different kinds of input, both from posterior parts of the brain and from other PFC areas. The collective responsiveness pattern of all PFC cells thus reflects the particular task relevant conditions. Duncan also noted, however, that the potential of cells to process different types of information, does not exclude regional specializations. This would reconcile results indicating regional specialization, with results from single cell studies showing large scale integration and adaptation. For example, a statistical rather than an absolute specialization seems to apply for left-hemisphere PFC recruitment for verbal material and right-hemisphere PFC recruitment for non-verbal stimuli (see, e.g., Passingham et al. 2002).

A slightly different perspective on functional specialization of PFC areas is to assume that some areas are relatively specialized, whereas others serve more general integrative purposes and come into play in almost any task. Gruber and von Cramon (2003), for instance, found both modality specific and amodal areas in PFC in verbal and visuo-spatial working memory tasks. Other findings suggest that many cells in ventrolateral areas of PFC are specifically sensitive to maintaining representations of single stimuli (e.g., objects or locations), whereas cells sensitive to maintaining complex integrated stimuli and elaborative rehearsal are found more often in dorsolateral areas (Prabhakaran et al. 2000; Owen 2000; Wagner et al. 2001). Wallis and Miller (2003) presented evidence suggesting that orbitofrontal cortex, which is specifically involved in reward and affect processing, passes on this information to dorsolateral areas of PFC where it is combined with other information to control behavior. Similarly, Ridderinkhof et al. (2004) concluded from a review that detection of response conflicts and response errors elicits overlapping foci of activation in medial PFC areas, and that this activation serves as a signal that engages regulatory processes in lateral PFC areas.

Learning and working memory

Our view of working memory is that its informational content consists of the active part of long-term memory representations that is available at any point in time for controlled processing (e.g., Cowan 1999; O’Reilly et al. 1999). These representations presumably are stored in the temporal, parietal and occipital cortex, so the functions of working memory (maintenance, selection and integration) that apply to these representations are realized by an interaction between PFC and posterior cortical areas. In this view, PFC is not itself involved in storing long-term memory information. Its representational role would be limited to temporarily maintaining and integrating information stored elsewhere. This view seems to be endorsed by the fact that the neural circuits and pathways assumed to be involved in the two main learning processes (Eichenbaum and Cohen 2001), and the corresponding procedural and declarative memory systems (Squire and Zola-Morgan 1991), do not include the PFC as a representational medium.

The suggestion that PFC implements WM functions, but is not itself involved in associative learning, would fit the requirement that it is maximally flexible in the service of generating novel combinations of representations that are stored somewhere else. There are no indications, however, that the neural architecture and processes of the PFC differ fundamentally from those in posterior cortex which intrinsically generate a learning capacity. Therefore, some learning (and representational) capacity in PFC cannot be ruled out a priori.

Some evidence suggests that the information used in working memory tasks does not have to be stored in PFC. First, prefrontal patients generally perform quite well in tasks requiring the retrieval of previously learned procedural and declarative knowledge. However, they are typically impaired in episodic memory tasks requiring for instance elaboration of stimuli, ordered recall and source monitoring. So the problem with prefrontal damage does not seem an inability to store information in memory, but an inability to control encoding and retrieval processes (e.g., Shimamura 1995). Second, several recent studies have linked activity in PFC areas with promoting effective LTM formation (e.g., Blumenfeld and Ranganath 2006; Buckner 2003). They suggest, however, that the information itself is not stored in PFC. Instead, the interaction between PFC and posterior cortex controls what is selectively activated, maintained and elaborated, and what is subsequently stored as a novel representation via the medial temporal lobe and the hippocampus (see, e.g., Ranganath et al. 2005). Third, an increasing number of studies showed that the activation patterns of many single neurons or neuron ensembles in PFC quickly and flexibly adapt to represent completely arbitrary rules (Asaad et al. 2000; Wallis et al. 2001; White and Wise 1999), or to arbitrary action sequences (Averbeck et al. 2006). Although in all of these studies a number of learning trials precedes the development of task specific activation patterns, the speed and flexibility of the adaptation process seems more suggestive of an adaptive coding than of an associative learning process.

Other authors have suggested that learning in PFC does occur. O’Reilly et al. (1999), for example, suggested that the information that is maintained in WM consists of activation patterns in PFC that are sustained by strong mutual excitation of the neurons involved. To explain the flexibility in combining these representations without losing their specificity, they assumed that PFC representations had to be relatively isolated from each other. Therefore, learning in PFC was supposed to be slow, taking place over many years, and eventually producing a rich palette of independent PFC representations that enable flexible problem-solving skills. Such representations might have a hierarchical structure with an increasing level of abstraction along a posterior-anterior PFC axis, probably with the most posterior PFC representations being closely connected to detailed memory representations in posterior cortex.

A worked out version of this model by Rougier et al. (2005) showed that extensive training of their model with various tasks resulted in the development of abstract rule-like PFC representations that supported flexible generalization in novel tasks. The PFC representations developed slowly, but once learned adaptive behavior was mediated by a search for a task appropriate pattern of activity (cf. the adaptive coding mechanism of Duncan 2001), rather than the need to update connection weights. Although the simple tasks simulated in this model may also apply to more realistic and complex rules, the authors point out that it still leaves unexplained how PFC representations can be dynamically recombined and can interact with other systems (such as episodic memory, language and affect).

It remains to be seen whether the supposed PFC learning mechanism is viable. Over brief periods of time, learning probably does not play a major role, but a role of learning over extensive periods cannot be ruled out.

Evidence for three interdependent working memory functions in PFC

Maintenance

Neurophysiological studies have established persistent activity during delays (i.e., after stimulus offset) as the main candidate for a neural substrate of the maintenance function of WM. Single cells showing persistent firing during a delayed-matching task were first discovered in PFC, and later in other neocortical areas like inferotemporal (IT) cortex (Fuster and Alexander 1971; Goldman-Rakic 1996; Nakamura and Kubota 1995).

Different areas, especially in posterior parts of PFC, may be involved in maintaining modality specific information. Smith and Jonides (1999), for instance, suggested a relatively specialized role of left and right PFC hemispheres in maintaining verbal and visual information, respectively. Moreover, they proposed that maintenance of spatial information may involve more dorsally located regions than maintenance of object information (see also Baddeley 2003).

In principle, there are two ways to maintain activity in neurons during delays. One is to assume that input activation creates an activation pattern in a network of cells that is maintained within the network by auto-association (Amit 1995; Deco and Rolls 2003), and possibly augmented by a dopamine gating mechanism (O’Reilly et al. 1999). The other possibility is that the activation pattern oscillates in a recurrent loop between different networks of cells.

There is much evidence that supports an important role of recurrent loops in maintaining information during working memory tasks. Strongest evidence is that during delays continued activity is not only found in PFC, but also in modality specific areas in IT cortex when coding for objects (Fuster et al. 1985; Miller et al. 1993; Tomita et al. 1999; Ungerleider et al. 1998), and in parietal cortex when coding for locations (Curtis and d’Esposito 2003; Rowe et al. 2000; Sakai et al. 2002). Moreover, localized cooling of either PFC or IT cortex interferes with activity in the other area and causes behavioral deficits in a working memory task (see Fuster 2001). From a review of the literature, Ranganath (2006) concluded that information in visual WM tasks is maintained through persistent activity in visual cortical areas (e.g., inferotemporal and parietal areas) that is promoted by top-down input from PFC. So according to Ranganath, maintenance would be an interaction between PFC and modality or object specific posterior areas. An additional loop involving the medial temporal lobe (i.e., the hippocampus and related areas) may be required to maintain complex novel stimuli, and to quickly create new long-term representations (e.g., Ranganath et al. 2005).

Ranganath (2006) is somewhat unclear about the nature of the feedback from PFC in maintenance. As a further specification we would suggest a fixed connection scheme (probably shaped by long-term learning) between posterior brain areas, containing long-term memory or newly created representations of stimuli, and adjacent areas in PFC. These pathways convey to PFC what information is being maintained. This may also explain the finding that during maintenance, activation in posterior areas is more vulnerable to distraction than activation in PFC (Miller et al. 1993; O’Connor et al. 2002). Maintenance may resist such distraction because PFC activation can reinstate activation of corresponding lower level representations. Such reinstatement probably also underlies memory retrieval and mental imagery. It has been shown, for example, that in tasks requiring participants to imagine faces or buildings, category specific regions in the IT cortex became activated (e.g., Ishai et al. 2002; O’Craven and Kanwisher 2000).

As was argued by O’Reilly et al. (1999), there are two seemingly incompatible core requirements of working memory. The first is to robustly maintain goals and task instructions over time even when faced with strong interference. The second is to rapidly and flexibly update goals and tasks if circumstances change. Their solution to combine robust maintenance with flexible updating has been to assume a dopamine gating mechanism. Under tonic dopamine levels a strong mutual excitation within a cell assembly would lead to sustained activity and resistance to interference by irrelevant input. Phasic shifts of dopamine levels induced by significant events (e.g., success or failure in a task or novel stimuli predicting reward) would enhance the strength of afferent input and cause updating of the activation state, which subsequently may lead to representing novel strategies, rules, goals or task states.

Although a dopamine-gating mechanism is an interesting possibility, we believe that re-entry of activation through recurrent neuronal circuits is the main mechanism for the maintenance function of WM. It is likely that there are many of such recurrent loops for different types of information. These loops link the perceptual, memory and motor representational areas in posterior cortex to PFC. They feed information into PFC and, in turn, are activated by the recurrent activation from PFC. Modulating activity in these loops by other PFC sources would modulate the top-down recurrency. We also suggest and that these loops form a hierarchy, at the lowest level maintaining simple stimuli or features and at higher levels maintaining increasingly complex stimulus relations and rules. We will show how this idea can be implemented and explore the feasibility of such an implementation. We agree, however, that especially for the highest and most global or integrative levels of maintaining information, a dopamine gating mechanism for auto-associative maintenance cannot be ruled out.

Attentional control

Attentional control by executive functions in PFC requires top-down effects on local information processing in posterior brain areas (e.g., Desimone and Duncan 1995; Miller and Cohen 2001; Shimamura 2000). Evidence for such a role comes, for instance, from findings showing that an important characteristic of behavioral deficits following damage to the PFC is extreme distractability (i.e., an inability to suppress interfering information) and disinhibition (i.e., an inability to suppress inappropriate responses).

A mechanism for top-down attentional control is suggested by the ‘biased competition model’ (Chelazzi et al. 1993; Desimone and Duncan 1995; Downing 2000; Reynolds et al. 2000; Gazzaley et al. 2005). According to such a model, top-down control activates corresponding representations that are then in a better position to compete with irrelevant information for perceptual awareness and control of motor behavior. Such top-down attentional modulation of neural responses, i.e., relative enhancement of neuronal responses to task-relevant stimuli and relative suppression of neuronal responses to task-irrelevant stimuli, has been shown throughout the visual system as early as the primary visual cortex (e.g., Desimone and Duncan 1995; Reynolds et al. 1999), and the exact areas that are biased depend on the task that is performed (e.g., Kastner et al. 1999).

Top-down attentional bias not only pre-activates specific target representations but also spreads activation to related representations. Distractors with visual similarities to targets also attract attention (are positively biased), as well as distractors with semantic or associative links to the target (Moores et al. 2003). Task specific activity in PFC also generates top-down signals involved in the selection of actions, and in long-tem memory retrieval (Hasegawa et al. 1998) and storage (Brewer et al. 1998; Kyd and Bilkey 2003; Blumenfeld and Ranganath 2006; Wagner et al. 1998). This perspective thus suggests a single underlying mechanism of cognitive control by the PFC, namely a top-down biasing effect on processing in specialized sensory, motor and memory systems responsible for task performance (see, e.g., Miller and Cohen 2001).

We will explore the feasibility of a mechanism for attentional control based on modulating feedback signals from PFC in the recurrent circuits that are used for maintenance. We suggest that attentional control is mediated by biasing the recurrent loops between PFC and posterior cortex. In this view, the biasing signals stemming from higher integrative areas in PFC would modulate the recurrent feedback to posterior cortical areas (see also Deco and Rolls 2003).

Integration

The importance of a large-scale integrative system for the coordination of behavior has been widely acknowledged. Such a system would provide the combination and integration of all information necessary for controlling task-relevant behavior, i.e., it would comprise all task-relevant information supplied by the present context and goals, and by knowledge retrieved from memory. Baddeley (2000), for instance, suggested an episodic buffer as an additional subsystem in his WM model which is capable of integrating and maintaining information from different sources. In a similar vein, Miller and Cohen (2001) argued for a cognitive control system in the PFC that represents goals and the means to achieve them. They suggest that this system integrates converging inputs from many sources. It exerts control by divergent feedback signals to sensory, memory and motor areas in posterior cortex which mediate directed attention, response selection, and guide retrieval from LTM.

A high level integrative control system was also proposed by Koechlin et al. (2003). They suggested a cascade model, consisting of three nested levels of cognitive control. At the top of this cascade, located in rostral (anterior) parts of the PFC, they assume an episodic control system involved in selecting task-set representations ‘according to events that previously occurred or to ongoing internal goals’ (p. 1181). The view that progressively ‘higher’ neural areas support functions that are increasingly more integrative also has been endorsed also by Fuster (2001). He argued that a cascaded control model, assuming several nested levels of control, with a highly integrative system at the top, may explain neuropsychological results showing that damage to the top system only affects tasks that require a high degree of information integration (e.g., planning and problem solving), but does not interfere with tasks that can be executed at a lower level of control.

Potentially the highest level of integration is attained in the anterior PFC (aPFC), in humans corresponding to BA10. In a recent discussion, Ramani and Owen (2004) pointed out this area as a likely candidate for being the apex of a hierarchical system of PFC modules, because it is predominantly (or even exclusively) reciprocally connected to other supramodal PFC areas. Therefore, it is in the position to integrate everything the brain is capable of representing. Moreover, the aPFC has anatomical characteristics (a high dendritic spine density combined with a low density of cell bodies) that make it likely to be involved in integration. The aPFC has been suggested to be specialized for processing internal mental states and introspective evaluation (Christoff and Gabrieli 2000), monitoring successful retrieval (Ranganath and Paller 2000), management and monitoring of goals and sub-goals (Koechlin et al. 2000; Braver and Bongliatti 2002), integration of information over time (Braver et al. 2001; Koechlin et al. 2003; Sakai et al. 2002), and manipulation of relational knowledge (Kroger et al. 2002).

A hierarchical model of integration based on the convergence of input from lower level specialized PFC modules on higher integrative levels, is consistent with claims that a distinction can be made between PFC areas involved in simple maintenance, and others involved in maintenance of complex information and executive processes (D’Esposito et al. 1999; Sakai et al. 2002). For example, several findings suggest a distinction between ventrolateral (vlPFC) and dorsolateral (dlPFC) areas in terms of the level of abstractness of information processed (e.g., Koechlin et al. 2003; O’Reilly et al. 2002). Tasks requiring processing of single words or concepts mainly seem to involve vlPFC (e.g., Wagner et al. 2001), whereas in more complex tasks, like sentence processing, enhanced involvement of dlPFC has been found (e.g., Hashimoto and Sakai 2002; Kerns et al. 2004b). A relatively high integrative role for the dlPFC (as compared to vlPFC) is also suggested by findings showing this area to be involved in tasks that go beyond simple maintenance, such as elaborative rehearsal (Blumenfeldt and Ranganath 2006), processing relations between stimuli (Kroger et al. 2002), and solving conflicts that are signalled by medial areas of PFC (Egner and Hirsch 2005; Kerns et al. 2004a).

We endorse the idea that the PFC is hierarchically organized, with subordinate modules being (relatively) specialized in processing simple aspects of tasks, and super-ordinate modules, located in dlPFC and frontopolar cortex, being specialized in large scale integration (i.e., binding) of inputs from different sources. At the apex of the supposed hierarchy, the aPFC might integrate all goal related information over space and time (see Fig. 1). Here, emotional and motivational evaluations (from orbitofrontal areas), results of memory retrieval, language and rule-like representations (from dlPFC), and anticipated action effects, response conflicts and bodily states (from medial PFC areas) converge. So the aPFC would be crucial for integrating multiple forms of information in the pursuit of general goals (e.g., Ramnani and Owen 2004), or when information from temporally dispersed events has to be integrated (e.g., Koechlin et al. 2003, 2006). Conversely, the ensuing activation patterns in aPFC set the stage for task and goal directed behavior, with control being exerted by top-down modulation of subordinate modules. In this way a hierarchy of top-down coordinated control is implemented by entraining successively lower order areas, eventually biasing perceptual processing, memory search, and action selection.

Fig. 1
figure 1

A schematic view of the hierarchical structure of PFC. Many details regarding sensory input and motor output processing in subcortical centres and cerebellum are not shown. Double arrowheads indicate recurrent connections. Connections and modules shown are not suggested to be anatomically fully complete and accurate. aPFC anterior PFC; dlPFC dorsolateral PFC; vlPFC ventrolateral PFC; oPFC orbital and ventromedial PFC; mPFC medial PFC (anterior cingulate cortex); pm cortex premotor cortex

Full size image

A neurocomputational model of maintenance, control and integration

Elsewhere (Raffone and Wolters 2001), we have presented a model for the temporary holding in (visual) working memory of a limited number of neural patterns, simulating either single features or integrated objects. The model implemented a cortical mechanism of maintenance in a network of model neurons with biologically plausible parameters. Although the model implemented a visual working memory system, the principles may be applicable to any form of information or type of working memory.

In the model WM was assumed to be based on recurrent connections between IT cortex containing representations of objects or features, and corresponding neurons in PFC. The IT representations were modelled as strongly associated neural assemblies that generate synchronized firing patterns when activated by external input. The simultaneous activation of independent assemblies in IT causes competition via inhibitory interneurons. Due to the neuron characteristics, this leads to desynchronization among the activation patterns of competing assemblies resulting in a sustained phase-locked activation of multiple assemblies over time.

Maintenance in cortical circuits of visual working memory was shown to be possible in terms of oscillatory reverberations between PFC and IT modules. Firing rate oscillations induced during stimulus presentation were maintained after stimulus offset by active feedback from prefrontal areas. Neurophysiological plausible model parameters enforced a limitation of about three to four independent assemblies that could be maintained in this way. This number closely coincides with recent estimates of the maintenance capacity of WM (e.g., Cowan 2001). The same mechanism that optimizes coherent pattern segregation, also poses a limit to the number of assemblies (about four) that can concurrently reverberate. The model thus indicated that selective synchronization and desynchronization of feedback-based oscillatory reverberations creates a suitable medium for a visual working memory. Simulations showed that the model was able to explain both the existence of severe limits in the number of assemblies (stimuli) that can be held (e.g., Luck and Vogel 1997; Luck and Beach 1998), and the absence of a limit on the size of assemblies, i.e., representing either simple stimuli or complex chunks (e.g., Ericsson and Delaney 1999). We introduced the concept of ‘chunking fields’ to account for the creation of more complex neural assemblies (e.g., higher order information units or chunks) through previous Hebbian learning (e.g., Hummel and Biederman 1992; Singer 1995). The model can potentially account for different degrees of within-object feature integration (Olson and Jiang 2002) in terms of graded synchrony between neurons coding for features of the same object.

Here, we will explore an extension of the model of Raffone and Wolters (2001), simulating not only maintenance, but also a selective attention mechanism and a particular characteristic of an integration mechanism. The network architecture presented here to model these functions, is composed of three modules, which we assume to correspond to an IT module, a ventrolateral prefrontal module (vlPFC), and a dorsolateral prefrontal module (dlPFC), respectively (see Fig. 2). We assume that visual features are coded by individual assemblies of neurons in IT, which are “matched” to one assembly in vlPFC in a recurrent circuit. Moreover, we assume that different subsets of four vlPFC assemblies coding for given visual chunks, are bi-directionally connected to dlPFC assemblies (one for each set of four vlPFC assemblies).

Fig. 2
figure 2

Scheme of the cortical network architecture. In the IT module, 20 neural assemblies code for 20 hypothetical visual features or separate representational elements. The figure shows the case with five four-feature chunks, with synchronizing connections between the assemblies coding the features of the same object (depicted as diamond-like configurations). The IT module also comprises an assembly of globally inhibitory neurons, which are implicitly modeled through inhibitory postsynaptic potentials (IPSPs). The vlPFC module is a set of 20 assemblies of neurons, with a coding structure “matching” the IT module structure. For simplicity, each IT assembly is recurrently connected with one vlPFC assembly, with a signaling delay of 15 ms in both directions. The vlPFC assemblies coding for a given chunk are reciprocally connected with a dlPFC assembly, which propagates synchronous firing before Hebbian learning takes place in the IT module

Full size image

In the IT module, strong connections are implemented within and weak connections between assemblies coding different ‘stimuli’. We also implemented a global inhibition (competition) mechanism between IT assemblies. This circuitry, in which a given assembly is inhibited by the firing of the other assemblies in the IT module, mediates an active desynchronization mechanism.

Stimulus input to the IT module is given by stochastic spike trains from (not explicitly modeled) lower visual areas, during a limited onset-offset period. During stimulus presentation all stimulus specific IT neurons received a high frequent train of spikes as input, which was added to a continuous stochastic low frequency spike input to all IT neurons. The vlPFC module has a coding structure “matching” the structure of the IT module (see Fig. 2). Each IT assembly is recurrently connected with one vlPFC assembly, with a signaling delay of 15 ms in both directions. Although we implemented a monosynaptic feedback circuit, in real cortical networks this delayed feedback is likely to be mediated by multisynaptic circuitries of the ‘synfire’ type, with a stable transmission along multiple diverging/converging synaptic links (Abeles et al. 1993a, b; see also Villa and Fuster 1992). In vlPFC and dlPFC there are no inhibitory neurons and no inter-assembly connections.

Although stimuli are coded by large assemblies of neurons in real cortical networks, in the present model individual stimuli are coded by single neurons. Our simulations will show that the functional processes we have investigated previously at the level of assemblies of neurons (Raffone and Wolters 2001) also hold at the level of single neurons, thus pointing out the robustness of the observed effects. The use of single neurons instead of assemblies of tens of neurons, is also motivated by the sake of running relatively fast simulations with a smaller time-step in numerical integration of neuronal equations (higher computational accuracy) than in our previous investigations.

The model, of course, is very simplified with respect to the complexity of the real cortical networks. Feedback from the vlPFC module does no more than maintaining the oscillatory state of IT assemblies after the stimulus offset. More realistic network versions might include “closed” reverberatory circuits within prefrontal areas, making prefrontal assemblies independent from the IT assemblies in maintaining the delay activity. This would allow modality shifts by activating other prefrontal assemblies, which in turn would trigger reverberatory circuitries in lower cortical areas, in the absence of actual physical stimulation. Moreover, we used a simplified one-to-one matching between IT and vlPFC assemblies, whereas it seems more realistic to assume that prefrontal assemblies are relatively non-specific, thus being connected to multiple sets of neurons in posterior areas. Instead of the one-to-one matching of assemblies, dlPFC neurons might simply send back activation to all vlPFC neurons, and these in turn to all IT neurons from which they receive activation. However, the present simple architecture is sufficient for demonstrating the functional principles we have specified earlier.

Simulations

Simulating maintenance in working memory

We first replicated our earlier results (Raffone and Wolters 2001) with stimulus features coded by individual neurons, rather than by sets of neurons, as well as with a higher temporal resolution of the simulations (computational accuracy) and a more realistic global inhibition mechanism. In this simulation the weights of the synchronizing inter-neuron connections in IT were set to zero, so all neurons code for different independent features. As shown in Fig. 3, about four elements are retained in the network after offset of the stimuli, which is in accordance with actual capacity limitations (e.g., Cowan 2001; Luck and Vogel 1997). The frequency of the reverberatory oscillations is approximately 30 Hz. Figure 3a and b show the retention of three out of four retained items, Fig. 3c and d the retention of four out of four, and Fig. 3e and f the retention of four out of eight items. Note the automatic phase-segregation due to the mutual inhibition desynchronizing effects (see Raffone and Wolters 2001, for a systematic investigation of retention capacity with different network parameters). The missed retention of some items (Figs. 3a, b, e, f) is due to the transient inhibition propagated by the firing of other competing neurons in IT. This inhibition “counteracts” the feedback input from the matched vlPFC neuron, thus interrupting the reverberatory cycle after stimulus offset. Such an interruption is more likely to occur with multiple reverberatory activities and stronger inhibition, as in that case the probability of the simultaneous arrival of strong inhibition and feedback signals increases.

Fig. 3
figure 3

Limited retention capacity related to between-item segregation. Due to mutual inhibitory activity, the neuronal action potentials (spikes) become spaced in the oscillatory phase, thus allowing a markedly discriminative oscillatory reverberation and retention of the coded items. Often not all reverberations survived the stimulus offset. The panels show the fluctuating membrane potentials of neurons, as well as their spikes by short vertical bars above the membrane potentials. The panels a, c, and e shows the dynamic behavior of IT neurons, and the panels b, d and f the responses of respectively matched vlPFC neurons. In (a, b), three out of the four neurons remained active. In (c, d), all the reverberations remained active, with all the four items being retained. In (e, f), four out of eight neurons maintained their sustained oscillation. Note that the oscillatory reverberations tend to be optimally spaced in the phase-lag, depending on the number of reverberating neurons

Full size image

In this simulation, we did not replicate our previous demonstration (Raffone and Wolters 2001) that the capacity of the reverberatory circuit is independent of the size of the IT representations. As we showed there, larger cell assemblies of connected neurons with fast-signaling positive weights, quickly synchronize and behave as single units. We demonstrated that the maintenance capacity of our model is not a function of the size of a cell assembly, but of the presence and strength of associations between the units of an assembly. This probably mimics the fact that working memory capacity is not a function of the amount of information per se, but of the level of organization or chunking of the material to be maintained (Miller 1956).

Simulating attentional control in working memory

Given the severe limitation of the capacity of working memory, neural mechanisms are necessary to restrict access to it, depending on the behavioral relevance of the information. As shown by Asaad et al. (1998, 2000), prefrontal neurons of monkeys performing a ‘delayed-matching-to-sample’ task exhibited a higher firing rate when they coded for a rule-defined target-object. Such neurons were suggested to be involved in both the selection and maintenance of behaviorally relevant information.

To model a selective attention mechanism by which selection and retention is controlled by working memory cortical circuits, a top-down input to IT neurons has to be supplied by a prefrontal source involved in supervisory control rather than in mere information maintenance. A similar assumption was made in a model for attentional control by Deco and Rolls (2003). Such a mechanism implements the top-down biasing of the competition between neural assemblies in posterior cortex as suggested by Desimone and Duncan (1995). In our model this supervisory control signal could be supplied through the dlPFC and vlPFC modules.

We first considered an additive (voltage-independent) input to a subset of four (the bottom four in Fig. 4a) out of eight independent IT neurons all receiving external input. This additional input was modeled in terms of additional spikes with an excitatory postsynaptic potential (EPSP) amplitude equal to the external input signals and a spiking probability equal to 0.04. As can be seen in Fig. 4a, the four items receiving top-down input exhibited a higher (subthreshold) membrane potential before the stimulus onset. This top-down bias was crucial in selecting the items to be retained in terms of reverberatory oscillations. All four biased (and one unbiased) item were maintained after stimulus offset.

Fig. 4
figure 4

Attentional modulation of reverberatory maintenance. Four out of the five maintained activities are of neurons (the first four neurons from below in the panels) the activities of which are selectively enhanced either by additive (voltage-independent) modulation (a), or by voltage-dependent (multiplicative) signals (b). Note that with additive modulation the membrane potentials of the four ‘focused’ neurons are higher before the stimulus onset (a), whereas the amplification mostly takes place after stimulus onset in the voltage-dependent modulation case (b)

Full size image

We also implemented a voltage-dependent modulatory input to IT neurons. A voltage-dependent gain effect simulates the possible role of NMDA receptors (i.e., receptors that activate a neuron only if a signal arrives at an already depolarized synapse). NMDA-based gain effects are suggested to play a crucial role in cognitive coordination and control (Phillips and Silverstein 2003, see also Raffone et al. 2003), and such gain effects are often modelled as a multiplication of input signals. This multiplicative effect was modeled in terms of the product of the voltage-independent external input (VI) and voltage-dependent top-down signals (VD) with amplitude equal to 0.5/5, and spiking probability equal to 0.25, according to the following equations

$$ {\text{Net}}_{i} = {\text{VI}}_{i} {\left( {1 + {\text{VD}}_{i} ^{ * } } \right)} $$
(1)
$$ {\text{VD}}_{i} ^{ * } = {\text{VD}}_{i} {\left( {u_{i} - {\text{VT}}} \right)}\quad {\text{if}}\;u_{i} \ge {\text{VT}}\quad {\text{else}}\;{\text{VD}}_{i} ^{ * } = 0 $$
(2)

Following suggestions of Tononi et al. (1992), the product of equation (2) was set equal to 0 when a membrane potential (u i ) was less than a voltage threshold VT (set equal to 0.5). In that case, only a voltage-independent stimulus-related input determines the net input to the neuron, without any amplification.

As can be inferred from Fig. 4b, the attentional bias (attentional input for four out of eight items) in the voltage dependent condition is expressed after the stimulus-input onset, in terms of an amplification effect. Also in this simulation all four biased (and one unbiased) items were maintained after stimulus offset. We believe that voltage-dependent input to high-level areas of the visual system may enable an effective top-down control mechanism, since it produces strong amplification effects, but only in the presence of relevant stimulus-input (see also Hirsch and Gilbert 1991; Tononi et al. 1992, about voltage-dependent signaling in the visual system). This multiplicative signaling effect would prevent spurious activation of representations in the visual system in the absence of any bottom-up sensory evidence.

A series of follow-up simulations showed that the probability of item retention depends on the relative biasing top-down input for a given item. A strong attentional input to one or two items may ‘narrow’ visual working memory capacity to one or two highly focused elements, due to the higher firing rate (e.g. bursting) of the neurons coding for the focused items.

Mechanisms for integration in working memory

Any domain-specific account of functions or representations in the brain ultimately implies a binding problem, since the specialized neural representations need to be dynamically bound to enable the creation (and a conscious readout) of coherent and integrated representations of complex events. These binding processes may occur in terms of the selective synchronization of reverberating cell assemblies, as shown in our simulations. In this framework, segregated neural representations may be bound via reciprocal synchronizing connections that originate within the prefrontal cortex (for a discussion of this and other forms of binding, see Murre et al. 2006; see also Engel et al. 2001; Varela et al. 2001).

In our model, dlPFC can play a crucial role in what we call “associative control”, i.e. in selecting meaningful or currently salient conjunctions of otherwise separated representational elements held in a temporarily active state within vlPFC-IT circuits. We have shown that reciprocal fast signaling between a dlPF neuron and its corresponding vlPFC neurons may “entrain” all these vlPFC neurons coding for different features or units to fire in a nearly synchronous manner. This synchronous firing is propagated to IT neurons, which in turn fire almost coincidently before the action of the mutual inhibitory signals within the IT module (Fig. 5a, b). We suggest that such a top-down controlled selective synchronization of independent representations is a likely candidate of the process underlying integration in PFC. This mechanism ensures the simultaneous and coherent activation of independent representations that may be widely dispersed over various brain areas. It may cause, for instance, the large-scale task-dependent synchronization in the gamma band, as observed by Rodriguez et al. (1999).

Fig. 5
figure 5

Associative control on vlPFC neuronal activities by dlPFC neurons. The panels a, c, and e shows the dynamic behavior of IT neurons, and the panels b, d and f the responses of respectively matched vlPFC neurons. As shown in panels a, b, inter-neuron synchronization of the four vlPFC neurons is induced by a dlPFC neuron (activity not shown), and is then propagated to the matched IT neurons. As shown in panels c, d, the joint effect of dlPFC feedback and Hebbian associative signals in IT may induce a higher firing rate of neurons in IT and vlPFC. Panels e, f show that a stable synchronization of IT and vlPFC neurons is observed after Hebbian strengthening of IT chunking synapses, with feedback from the dlPFC neuron being switched-off

Full size image

The same mechanism may allow Hebbian learning to occur between meaningful or behaviorally-salient conjunctions of independent elements (see Miltner et al. 1999). In order to account for the retention of integrated units in visual working memory, in previous simulations we defined units by assuming already established chunking fields, i.e. pre-existing associative connections between IT neurons, coding different features of the same unit. A major problem, however, is how ‘chunking fields’ or integrated representations in long-term memory are formed, i.e. how associative Hebbian learning may take place in the simulated cortical circuits of (visual) working memory. This learning process should operate against the phase-segregation tendency due to mutual desynchronizing inhibition. In the present model, the dlPFC module may perform this temporary synchronization control processes.

In a series of simulations, we used the entrainment of vlPFC and IT neurons by a top-down dlPFC signal and combined it with a timing-dependent Hebbian learning rule in IT (simulating timing-dependent plasticity of synapses, see e.g., Körding and König 2000; Song et al. 2000, see Fig. 5c, d). These simulations showed that the synchronous firing in IT, induced by a top-down recurrent signal, causes the weights between the IT neurons to increase, resulting in a ‘chunked’ representation of initially independent features. As synaptic weights increased according to the timing-dependent learning rule, the synchronization among related IT neurons becomes gradually less dependent on the feedback action of dlPFC neurons on related vlPFC neurons, ultimately leading to an “automatic” within-chunk integration within the IT module, in the absence of any control feedback from the dlPFC module (Fig. 5e, f).

Control simulations with the same model and learning parameters, showed that a relatively high number of simultaneous presentations of initially independent elements, may slowly give rise to novel chunks with a variable degree of stability, even in the absence of feedback from dlPFC to vlPFC. This slow chunk-learning process may be related to an implicit learning process, rather than the more rapid process of controlled chunk formation by feedback from dlPFC.

Discussion

Miller and Cohen (2001) concluded that PFC is critical when top-down control of processing is needed for the guidance of behavior by internal states or intentions. In this paper we conjecture that such control is possible by assuming that the PFC is a brain system that is reciprocally connected to posterior parts of the brain and therefore capable of modulating direct perception-action relations. We argued that three interdependent functions are required for such control: maintenance of activation patterns even in the face of distraction, large scale integration of information from different sources, and top-down selective attention. We also showed that these functions can be simulated, at least in principle, with a biologically plausible model assuming recurrent connections and a hierarchical structure of PFC. Importantly, the functions were simulated using a single framework consisting of a hierarchy of recurrently connected modules. This framework proved capable of maintaining a limited number of mutually desynchronized patterns, of controlling selective attention by top-down modulatory signals, and of high level integration by top-down synchronization of the activity of independent patterns.

Of course, the model suggested here is extremely simplified and it does not capture many of the complexities and intricate details of the real system. For instance, we did not try to incorporate modulatory effects of neurotransmitters and we have not discussed the many possible interactions of the PFC with subcortical systems, like the basal ganglia and amygdala, and the cerebellum. It may also be noted that we discussed, but did not try to incorporate, the possibly important role of a dopamine gating system for maintaining high level integrative activation states (O’Reilly et al. 1999, 2002).

One of our aims to develop the model was to try to find out how the structural and functional characteristics of the PFC may explain the central executive component of working memory assumed by Baddeley (2003). In the latest version of this model an additional component, an episodic buffer, was proposed. This component was thought necessary to explain interactions between working memory and LTM, for example to explain how chunking may supplement the capacity for immediate serial recall. A continuous interaction between working memory and LTM is a basic assumption in our model, and we have shown that chunking is a natural consequence of this interaction.

According to Baddeley (2003), the episodic buffer allows information from different systems to be integrated, and it may be regarded as the ‘storage component of the executive’. Moreover, Baddeley suggested that the buffer is a separate temporary store in which long-term memory information is downloaded in order to be manipulated and used for creating new representations. In our view, the structure of such an episodic buffer would actually be a hierarchy of PFC modules maintaining and integrating information at successively higher levels. Information is not copied and downloaded in PFC, but selected task-relevant perceptual and memory information in posterior cortical areas is kept in an active state by recurrent loops. We assume that at the highest integration levels, presumably located in anterior and dorsolateral areas of PFC, information can be integrated over space and time. At this level, general goals, and plans to achieve them, would be generated. We suggest that this may be the implementation of an episodic buffer and even of a central executive. The creation of short-term and long-term goals at high levels of integration, allows to control behavior by top-down regulation of activation in subordinate modules and eventually by modulating perception-action systems. Current goals and tasks would have to be maintained here in order to control ongoing behavior. More distant goals, however, may be stored in LTM to be retrieved later on.

We do not imply that control is always hierarchical. Because the PFC is assumed to be a modulatory system, simple tasks may be executed automatically without PFC involvement. Only controlled processing requires modulation of automatic processes by PFC feedback. Also here, however, control does not need to be completely hierarchical. Depending on the type and complexity of a task, control may be executed by specialized modules at subordinate levels. Automatic processing and controlled processing under the guidance of subordinate levels in PFC, is a requirement to free higher integrative areas to engage in the kind of simulated actions and perceptions, and using anticipated outcomes, that are the contents of conscious thought and planning.

So there are things we can do automatically, using previously established associations between perceptual events and effective responses, and there are things we can do in a controlled way, using the possibility of PFC to modulate and thus control perception and action. The possibility of PFC to work off-line, to use and integrate all present and past knowledge for creating virtual worlds and for making plans and carry them out if conditions are appropriate, has generated a tremendously flexible potential for control. The model we propose is vastly insufficient to simulate such cognitive feats that we engage in daily. We believe, however, that the principles for a controlling system we have suggested may be a first step towards a better understanding of what is until now a cognitive Terra Incognita.