Lateral specialization in unilateral spatial neglect: a cognitive robotics model
In this paper, we present the experimental results of an embodied cognitive robotic approach for modelling the human cognitive deficit known as unilateral spatial neglect (USN). To this end, we introduce an artificial neural network architecture designed and trained to control the spatial attentional focus of the iCub robotic platform. Like the human brain, the architecture is divided into two hemispheres and it incorporates bio-inspired plasticity mechanisms, which allow the development of the phenomenon of the specialization of the right hemisphere for spatial attention. In this study, we validate the model by replicating a previous experiment with human patients affected by the USN and numerical results show that the robot mimics the behaviours previously exhibited by humans. We also simulated recovery after the damage to compare the performance of each of the two hemispheres as additional validation of the model. Finally, we highlight some possible advantages of modelling cognitive dysfunctions of the human brain by means of robotic platforms, which can supplement traditional approaches for studying spatial impairments in humans.
KeywordsUnilateral spatial neglect Embodied cognition Cognitive robotics Hemisphere specialization Neuropsychology
Unilateral Spatial Neglect (USN) comprises a collection of behavioural symptoms in which patients appear to be incapable to perceive stimuli in spatial locations contralateral to the damaged cerebral hemisphere, e.g. after stroke (Heilman et al. 1994; Karnath et al. 2002). USN is also referred to as “visual neglect”, “hemispatial neglect” or “hemineglect”, and it is typically associated with damage to the Posterior Parietal Cortex (PPC), although, in many patients, lesions can be more extensive and involve also the premotor cortex (PMC). USN is a pathological condition that is more frequent, longer lasting, and more severe following lesions to the right hemisphere, RH, than to the left hemisphere, LH (De Renzi 1982; Plummer et al. 2003). As result, patients with neglect commonly ignore objects on the left side of space, fail to eat from the left side of the plate, and may dress the right side of the body only.
Most contemporary views of the neglect syndrome consider it to be a heterogeneous condition consistent with the heterogeneous nature of the associated lesion sites. The neglect emerges as a result of a combination of component cognitive deficits that may vary across patients and need not be neglect specific (Parton et al. 2004). Pouget and Driver (2000) theorized that USN is a selective loss of neurons representing particular locations in space for particular functions.
Indeed, USN is far from a unitary phenomenon and has been shown to fractionate into a number of dissociable components in terms of sensory modality, spatial domain, response laterality, motor output, and stimulus content (Barbieri and De Renzi 1989; Robertson and Halligan 1999). Furthermore, different USN disorders may exist, which may require type-specific rehabilitation approaches. This may have implications for epidemiological studies and for the development of new treatments. Theoretically driven epidemiological studies are required before adequately powered randomized controlled trials of rehabilitation can be conducted (Bowen et al. 1999). Given the complexity of the disease, various tools are needed to be able to diagnose the presence and the relative degree of impairment of the areas involved. This is crucial also for the patient’s rehabilitation. The most direct approaches to explore spatial impairments are neurophysiological studies in animals (e.g. Gottlieb et al. 1998), neuroimaging and lesion studies in humans (e.g. Parasuraman and Yantis 1998). Computer simulations can supplement these methods by testing hypotheses about the normal and disordered function of attentional processes. Computational models enable experimenters to make explicit assumptions and hypotheses, and to implement only the portions of the brain that need more focus. Moreover, the analysis of results can be conducted at a level of detail which would be difficult to achieve in other domains of cognitive neuroscience. This “ecological”, or Artificial Life approach adds further power to the connectionist modelling by means of simulating not only the brain and the nervous system, but also the body and the environment of artificial organisms (Langton 1995; Parisi et al. 1990).
Previous research (e.g. Cohen et al. 1994; Mozer et al. 1997) has shown that computational models of neglect can reveal emergent behaviours that are beyond the typical scope of speculating with non-computational models. Mozer and Behrmann (1990) “lesioned” an existing computational model of visual perception and selective attention called MORSEL (Mozer 1991) in accordance with the damage that was hypothesized to occur in the brains of neglect patients. The damaged model was then used to simulate some puzzling aspects of the performance of patients with neglect dyslexia (a reading disorder associated with neglect). Similarly, Lanyon and Denham (2010) examined the effects of a parietal lesion in their model of visual attention and search that is based on neurobiological evidence from monkey electrophysiology (Lanyon and Denham 2004).
Theoretical models of visual neglect can be usually divided into approaches based on an attentional or a representational account of the syndrome. An attentional account (e.g., Chatterjee 2003) considers neglect as a deficit in orienting visual attention to the affected hemispace, whereas a representational account interprets neglect as the result of impairment of one side of a particular spatial representation. Deco and Zihl (2004) presented an attentional model that was based on the ‘‘biased competition hypothesis’’ (Desimone and Duncan 1995). Spatial and object attention are accomplished by a multiplicative gain control that emerges dynamically through an inter-cortical mutual biased coupling. By damaging the model in different ways, authors report a variety of dysfunctions associated with visual neglect that can be simulated and explained as disruption of specific subsystems. In particular, authors were able to explain the asymmetrical effect of spatial cueing on neglect, and the phenomenon of extinction in the framework of visual search. Pouget and Sejnowski (2001) presented a representational model that can account for several behaviours shown by patients with hemi-neglect. In this model, contralateral neglect arises because the unilateral parietal lesions lead to a neuronal gradient in basis function maps producing an imbalance in the salience of stimuli that is modulated by the orientation of the body in space. Monaghan and Shillcock (2004) reported the results of a series of artificial neural network simulations of the line-bisection task that emphasized the hemispheric asymmetries in neglect cause and in its effects. They claimed that a model with neuro-anatomically realistic principles of connectivity in the nervous system could produce emergent behaviours that capture a wide range of quantitative and qualitative data observed in neglect patients.
Recent research suggests that spatial cognition models should be embodied (Coello and Delevoye-Turrell 2007; Trafton and Harrison 2011) and, in particular, some empirical data in cognitive neurosciences with USN patients (Richard et al. 2004; Saj et al. 2006) support this view showing that the general spatial processing is influenced by a distorted representation of the body, which is shifted in the direction of the lesion. Meanwhile, many projects in robotics and artificial intelligence have highlighted the value of a direct sensory-action approach where intelligence requires a body (Chaminade and Cheng 2009; De La Cruz et al. 2014; Di Nuovo et al. 2013; Fischer and Coello 2016; Levesque and Lakemeyer 2008), as opposed to classical intelligence which used the sensory-thought-action framework and involved a strong dissociation between the body and mind. But, so far, at the best of our knowledge, no other cognitive robotics model has been designed and applied to study USN.
In this paper, we present a novel artificial neural network model to control the spatial attention of the iCub robotic platform from proprioceptive information including not only visual information but also motor inputs. The architecture is designed to model the RH specialization for elaboration of the visuo-spatial information, which emerges naturally because the network initialization incorporate some mechanisms inspired by the plasticity of the human brain (Gould et al. 1999).
The model is studied and validated by replicating an experiment that was carried out with human patients affected by USN (Bisiach et al. 1985), which addresses the question of bodily reference system of space representation. The model links are damaged to simulate different USN conditions and tested in a manipulation task that requires the cognitive robot to perform a spatial exploration. The experiments aim to confirm the validity of the hemisphere specialization and to examine the relation of unilateral neglect to the sagittal mid-plane of the trunk and the line of sight. Finally, rehabilitation sessions are simulated to see the recovery capability of the network.
Details of the model and of the experimental setup are in Sect. “Materials and methods”. Section “Experimental results and discussion” reports and discusses the numerical results of the experiment with the iCub robot. Finally, Sect. “Conclusion” gives our conclusion.
Materials and methods
The iCub robotic platform and the neural network architecture
The hidden layers are divided into two regions to mimic the separation of the cerebral hemispheres. The object positions were calculated from pictures taken by the eye cameras during the training phase. These positions were represented as a 2D pixel matrix, and they are the input of our artificial neural architecture (target inputs). The other (motor) input is the neck joint angle. Input coordinates were different for the RH and LH as they were retrieved using, respectively, the right and left eye cameras pictures, this way the coordinates were relative to the camera position. To simulate the antagonist action of the real human neck muscle, we coded the right input as the opposite of the left values: if the neck was turned 40 degrees to the right, the right motor input was −40 (means that the right muscle was flexed), at the same time, the left motor input was 40 (means that the left muscle was extended); meanwhile, if the neck was turned 40° to the left, the right motor input was 40 (means that the right muscle was extended), at the same time, the left motor input was −40 (means that the left muscle was flexed).
The role of the final layer (Activator) is to simulate the final processing of the attentional biases and to produce the final output that will activate the action associated with one target area. The activator has a linear transfer function that combines the activation from LH and RH and generates the final classification likelihood of the sixteen possible target positions. In this paper, we refer to the final output as the likelihood, which can be defined as how likely it is to perform the action to explore a specific target area on the table. Note that the final output activation can be greater than 1 or lower than 0 as it is the combined result of the sum of the LH and RH activations.
the reinforcement of the intra-hemispheric connections;
the formation of new pathways.
the stronger links are modelled via the initialization of the LH connection weights in a smaller range, i.e. between −0.1 and 0.1, while the RH connection weights are greater (e.g. in the standard range [−1, 1]);
the new pathways are modelled allocating four additional neural units to the RH layers. This way, in our experiments the relevant specialization emerges naturally after the backpropagation training.
The experimental setup and procedure
In a preliminary phase, the robot was trained to accomplish the experimental task using a pre-programmed routine. To train the network we applied the gradient descent with momentum backpropagation algorithm (Rumelhart et al. 1986), which is the most widely used method for training feedforward neural networks. For each iteration of the backpropagation algorithm, called epoch, the batch training procedure is applied and all the examples in the training set are inputted to the network before the weights are updated.
The goal of the training was to associate the action routine with the spatial attentional focus that identifies a specific place in the table. The action primitives needed to perform the task were previously learned by the robot. The model was trained using all the possible target positions on the table. A total of sixteen positions were identified, equally distributed on the left and on the right side of the robot in order to have a balanced training scenario that covers the entire attentional field.
In the lesioning experiments, we simulated damages in different parts of the artificial hemisphere by cutting neural links (i.e. assigning 0 to connection weights), obtaining also an intra-hemispheric disconnection between anterior and posterior layers. A similar approach was also found to yield neglect-related behaviour in previous simulation studies (e.g. Di Ferdinando et al. 2007; Mozer 2002).
Finally, as a further experiment, we re-applied the backpropagation algorithm to simulate a rehabilitation therapy and the recovery after the damage as additional validation of the model. Every session comprised 100 applications (epochs) of the backpropagation algorithm, and we repeated the experiment and recorded the omissions. In this scenario, the results are analysed in terms of the number of sessions needed to recover and the performance of the two hemispheres is compared.
In this case, the supervised backpropagation can be seen as resembling a rehabilitation procedure in which the robot is supervised by a therapist in the exploration of the space by means of training examples.
Experimental results and discussion
In our experiments, we consider a task execution successful when the final layer (Activator) activates the output neuronal unit associated with the target area and, consequently, the primitive motor action to remove the target object from the table. A neuronal unit of the Activator layer is considered active if its output value is >0.5. Otherwise, the trial is recorded as an omission. In Tables, we highlight successful attempts in bold values, while omissions are in italicized values.
Each experiment was replicated five times with random weight initialization, and we report the median result in the following tables and text. We considered two test cases for damaging the model: (1) there is no specialization, i.e. LH and RH activate the focus only when the target is in the contralateral area of the attention focus; (2) the RH is specialized and it is able to activate the focus in any area, while the left one can only activate the focus on the right. In both cases, after the initial training phase, the robot learns to execute the task perfectly. Indeed, in a fully “healthy” status, the average likelihood associated with the correct target position is 0.9998 or 0.9999 in both test cases.
Test case 1: No specialization (control experiment)
The LH is damaged: bold values indicate the successful removal of the object in the corresponding area, while italicized values indicate that the area was omitted (i.e. the object was not removed)
The RH is damaged: bold values indicate the successful removal of the object in the corresponding area, while italicized values indicate that the area was omitted (i.e. the object was not removed)
Test case 2: Right hemisphere specialization
Experimental results when the “unspecialized” LH is damaged (control experiment): bold values indicate the successful removal of the object in the corresponding area, while italicized values indicate that the area was omitted (i.e. the object was not removed)
Experimental results when the “specialized” RH is damaged: bold values indicate the successful removal of the object in the corresponding area, while italicized values indicate that the area was omitted (i.e. the object was not removed)
From Table 3, we see that only the right side of the spatial attention focus is slightly affected; indeed, problems can be considered minor as only three omissions are registered in condition B, which is the most difficult because all the targets are in the contralateral side of the damage, and two in C. The likelihood is quite high in all cases and, often, it is above 0.4 and near to 0.5 that is the threshold for a successful activation. This confirms that the contribution given by the “unspecialized” LH is weaker than the “specialized” RH.
From Table 4 we see that our experimental results are similar to the findings reported in the work that our experiment is replicating. Indeed, in (Bisiach et al. 1985), authors report more omission (i.e. missed targets) in the contralesional side of the brain lesion, i.e. on the left as the RH is damaged. In particular, we see that the sagittal mid-plane and line of sight contribute significantly to the omissions: when the robot turns its head it is able to remove almost all objects in condition B.
The comparison between results in Tables 3 and 4 clearly suggest that neglect is less severe when LH is damaged, and this is in line with the findings reported in the literature (Mapstone et al. 2003; Monaghan and Shillcock 2004). This difference can be clearly seen both in terms of successful removal of objects, 84.38 and 43.75 %, respectively, when LH and RH are damaged, and of average likelihood, which is 0.737 and 0.345. These numbers confirm that the architecture design led to a specialization of the RH for processing the visual-spatial information from the robot sensors, as its influence to the final result is much stronger than the LH. Indeed, in our experiments, the artificial RH contribution can be estimated as more than 2/3 versus the weaker 1/3 of the artificial LH.
By comparing the two plots, we see that the recovery in terms of weights strength is similar between the two hemispheres, and it tends to stabilize around 30 % of the original weights strength. Despite the weaker connections in both cases the robot fully recovers, however faster when the damage is on the LH than RH. In fact, in the case of LH damage, there are no signs of USN after 22 re-training sessions, while in the case of RH damage a full recovery is achieved after 49 sessions. The faster recovery behaviour in case of left damage is frequently reported in the literature (e.g. De Renzi 1982), and it was also observed by Monaghan and Shillcock (2004) who suggest it is evidence of the RH specialization for the elaboration of visual-spatial information.
This article presented an embodied cognitive robotics approach to the computational modelling of the cognitive dysfunction known as USN. The aim of the study was to introduce and validate a novel model architecture that incorporates the lateral specialization for processing the visual-spatial information. The design of the model hypothesizes plasticity mechanisms that allow the emergence of spatial specialization of the right hemisphere in the experimental task. Finally, we report results of an experimental with the real iCub robot platform that shows behaviours similar to those reported in previous studies with human patients. The present study also highlights some advantages of using an artificial brain embodied in a robotic platform to simulate cognitive dysfunctions.
These results support the use of the cognitive robotics approach to supplement the classical studies to focus on specific parts of the brain and to allow hypothesis and assumptions that are difficult to test in experiments with humans and animals. As an example, we were able to test neglect with LH damage, which is less observed in patients and, moreover, it may imply other problems (e.g. memory, speech, writing, and cognitive processing) that can severely limit patient capabilities to effectively interact (Karnath et al. 2002; Springer and Deutsch 1985), these features make difficult to find subjects with the lesion in the LH available for an experiment. Another advantage is that robots are “tireless” so they can complete the experimental test right after the simulated rehabilitation training, whereas a human patient will be probably tired and this can affect its performance during the test, especially at the beginning of the therapeutic path.
Future work on the model will focus on the relation between USN and body perception, and to further investigate its use in the rehabilitation context.
The work was partially supported by the EU FP7 Project ROBOT-ERA (ICT-288899) and the UK EPSRC project BABEL.
- Chatterjee A (2003) Neglect: a disorder of spatial attention. In: D'Esposito M (ed) Neurological Foundations of Cognitive Neuroscience. The MIT Press, Cambridge, MA, pp 1–26Google Scholar
- De La Cruz VM, Di Nuovo A, Di Nuovo S, Cangelosi A (2014) Making fingers and words count in a cognitive robot. Front Behav Neurosci 8(February):13Google Scholar
- De Renzi E (1982) Disorders of space exploration and cognition. Wiley, New YorkGoogle Scholar
- Fischer MH, Coello Y (eds) (2016) Foundations of embodied cognition. Taylor & Francis, LondonGoogle Scholar
- Heilman KM, Watson RT, Valenstein E (1994) Localization of lesions in neglect and related disorders. In: Heilman KM, Valenstein E (eds) Localization and neuroimaging in neuropsychology. Oxford University Press, New York, pp 495–524Google Scholar
- Langton CG (1995) Artificial life: an overview. Artificial life. MIT PressGoogle Scholar
- Mozer MC (1991) The perception of multiple objects: a connectionist approach. The MIT Press, CambridgeGoogle Scholar
- Parasuraman R, Yantis S (1998) The attentive brain. Mit Press Cambridge, CambridgeGoogle Scholar
- Robertson IH, Halligan PW (1999) Spatial neglect: a clinical handbook for diagnosis and treatment. Psychology Press, BrightonGoogle Scholar
- Springer SP, Deutsch G (1985) Left brain, right brain. WH Freeman/Times Books/Henry Holt & Co, New YorkGoogle Scholar
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.