The neural correlates and nature of self-consciousness is an advanced topic in Cognitive Neuroscience. Only a few animal species have been testified to be with this cognitive ability. From artificial intelligence and robotics point of view, few efforts are deeply rooted in the neural correlates and brain mechanisms of biological self-consciousness. Despite the fact that the scientific understanding of biological self-consciousness is still in preliminary stage, we make our efforts to integrate and adopt known biological findings of self-consciousness to build a brain-inspired model for robot self-consciousness. In this paper, we propose a brain-inspired robot bodily self model based on extensions to primate mirror neuron system and apply it to humanoid robot for self recognition. In this model, the robot firstly learns the correlations between self-generated actions and visual feedbacks in motion by learning with spike timing dependent plasticity (STDP), and then learns the appearance of body part with the expectation that the visual feedback is consistent with its motion. Based on this model, the robot uses multisensory integration to learn its own body in real world and in mirror. Then it can distinguish itself from others. In a mirror test setting with three robots with the same appearance, with the proposed brain-inspired robot bodily self model, each of them can recognize itself in the mirror after these robots make random movements at the same time. The theoretic modeling and experimental validations indicate that the brain-inspired robot bodily self model is biologically inspired, and computationally feasible as a foundation for robot self recognition.
This is a preview of subscription content, log in to check access.
Buy single article
Instant access to the full article PDF.
Price includes VAT for USA
Subscribe to journal
Immediate online access to all issues from 2019. Subscription will auto renew annually.
This is the net price. Taxes to be calculated in checkout.
Robot Self-Consciousness Project: http://bii.ia.ac.cn/robot-self
Molnarszakacs I, Uddin LQ. Self-processing and the default mode network: Interactions with the mirror neuron system. Front Hum Neurosci 2013;7(571):571.
Uddin LQ, Kaplan JT, Molnar-Szakacs I, Zaidel E, Iacoboni M. Self-face recognition activates a frontoparietal mirror network in the right hemisphere: an event-related fmri study. Neuroimage 2005;25(3):926–35.
van Veluw SJ, Chance SA. Differentiating between self and others: an ale meta-analysis of fmri studies of self-recognition and theory of mind. Brain Imaging Behav 2014;8(1):24–38.
Sugiura M, Miyauchi CM, Kotozaki Y, Akimoto Y, Nozawa T, Yomogida Y, Hanawa S, Yamamoto Y, Sakuma A, Nakagawa S, et al. Neural mechanism for mirrored self-face recognition. Cereb Cortex 2014;25(9):2806–14.
Platek S, Wathne K, Tierney NG, Thomson JW. Neural correlates of self-face recognition: an effect-location meta-analysis. Brain Res 2008;1232:173–84.
Hu C, Di X, Eickhoff SB, Zhang M, Peng K, Guo H, Sui J. Distinct and common aspects of physical and psychological self-representation in the brain: a meta-analysis of self-bias in facial and self-referential judgements. Neurosci Biobehav Rev 2016;61:197–207.
Devue C, Collette F, Balteau E, Degueldre C, Luxen A, Maquet P, Brédart S. Here i am: the cortical correlates of visual self-recognition. Brain Res 2007;1143:169–82.
Kruse B, Bogler C, Haynes JD, Schütz-Bosbach S. Am i seeing myself, my friend or a stranger? The role of personal familiarity in visual distinction of body identities in the human brain. Cortex 2016;83:86–100.
Uddin LQ, Molnar-Szakacs I, Zaidel E, Iacoboni M. rtms to the right inferior parietal lobule disrupts self–other discrimination. Soc Cogn Affect Neurosci 2006;1(1):65.
Farrer C, Franck N, Georgieff N, Frith CD, Decety J, Jeannerod M. Modulating the experience of agency: a positron emission tomography study. Neuroimage 2003;18(2):324–33.
Macuga KL, Frey SH. Selective responses in right inferior frontal and supramarginal gyri differentiate between observed movements of oneself vs. another. Neuropsychologia 2011;49(5):1202–7.
Gold K, Scassellati B. Using probabilistic reasoning over time to self-recognize. Robot Auton Syst 2009;57(4):384–92.
Takeno J, Inaba K, Suzuki T. Experiments and examination of mirror image cognition using a small robot. Proceedings of the 2005 IEEE International Symposium on Computational Intelligence in Robotics and Automation; 2005. p. 493–498.
Broun A, Beck C, Pipe T, Mirmehdi M, Melhuish C. Bootstrapping a robots kinematic model. Robot Auton Syst 2014;62(3):330–9.
Ciliberto C, Smeraldi F, Natale L, Metta G. Online multiple instance learning applied to hand detection in a humanoid robot. Ieee/rsj International Conference on Intelligent Robots and Systems; 2011. p. 1526–1532.
Bongard J, Zykov V, Lipson H. Resilient machines through continuous self-modeling. Science 2006;314(5802):1118.
Anderson J, Gallup GG Jr. Mirror self-recognition: a review and critique of attempts to promote and engineer self-recognition in primates. Primates 2015;56(4):317.
Gallup G, Anderson J, Platek S. Self-recognition. The Oxford handbook of the self. Oxford: Oxford University Press; 2011. p. 80–110.
Zeng Y, Zhao Y, Bai J. Toward robot self-consciousness (i): Brain-inspired robot mirror neuron system model and its application in mirror self-recognition. Proceedings of the 8th International Conference on Brain Inspired Cognitive Systems (BICS 2016); 2016. p. 11–21.
Iacoboni M, Dapretto M. The mirror neuron system and the consequences of its dysfunction. Nat Rev Neurosci 2006;7(12):942–951.
Zhao W, Luo L, Li Q, Kendrick KM. What can psychiatric disorders tell us about neural processing of the self? Front Hum Neurosci 2013;7:485.
Thakkar KN, Peterman JS, Park S. Altered brain activation during action imitation and observation in schizophrenia: a translational approach to investigating social dysfunction in schizophrenia. Am J Psychiatr 2014;171(5):539–48.
Haist F, Anzures G. Functional development of the brain’s face-processing system. Wiley Interdiscip Rev Cogn Sci 2017;8(1-2):1–11.
Duchaine B, Yovel G. A revised neural framework for face processing. Ann Rev Vis Sci 2015;1:393–416.
Yovel G, OToole AJ. Recognizing people in motion. Trends Cogn Sci 2016;20(5):383–95.
Peelen MV, Downing PE. The neural basis of visual body perception. Nat Rev Neurosci 2007;8(8):636–48.
Taylor JC, Wiggett AJ, Downing PE. Functional mri analysis of body and body part representations in the extrastriate and fusiform body areas. J Neurophysiol 2007;98(3):1626–33.
Gazzaniga MS, Ivry RB, Mangun GR, Steven MS. Cognitive neuroscience: the biology of the mind. New York: W. W. Norton & Company Inc.; 2013.
Perrone JA, Thiele A. Speed skills: measuring the visual speed analyzing properties of primate mt neurons. Nat Neurosci 2001;4(5):526–32.
Grossman ED, Blake R. Brain areas active during visual perception of biological motion. Neuron 2002;35(6):1167–75.
Hamzei F, Vry MS, Saur D, Glauche V, Hoeren M, Mader I, Weiller C, Rijntjes M. The dual-loop model and the human mirror neuron system: an exploratory combined fmri and dti study of the inferior frontal gyrus. Cereb Cortex 2016;26(5):2215–24.
Georgopoulos AP, Schwartz AB, Kettner RE, et al. Neuronal population coding of movement direction. Science 1986;233(4771):1416–9.
Craig AD. How do you feelłnow? The anterior insula and human awareness. Nat Rev Neurosci 2009;10(1):59–70.
Sasaki AT, Kochiyama T, Sugiura M, Tanabe HC, Sadato N. Neural networks for action representation: a functional magnetic-resonance imaging and dynamic causal modeling study. Front Hum Neurosci 2012;6(1):236.
Mehta UM, Thirthalli J, Aneelraj D, Jadhav P, Gangadhar BN, Keshavan MS. Mirror neuron dysfunction in schizophrenia and its functional implications: a systematic review. Schizophr Res 2014;160(1):9–19.
Yun JY, Hur JW, Jung WH, Jang JH, Youn T, Kang DH, Park S, Kwon JS. Dysfunctional role of parietal lobe during self-face recognition in schizophrenia. Schizophr Res 2014;152(1):81–8.
Beyeler M, Richert M, Dutt ND, Krichmar JL. Efficient spiking neural network model of pattern motion selectivity in visual cortex. Neuroinformatics 2014;12(3):435–54.
Escobar MJ, Wohrer A, Kornprobst P, Viéville T. Biological motion recognition using a mt-like model. Proceedings of the 3rd IEEE Latin American Robotics Symposium (LARS’06); 2006. p. 47–52.
Bi GQ, Poo MM. Synaptic modification by correlated activity: Hebb’s postulate revisited. Ann Rev Neurosci 2001;24(1):139–66.
Song S, Miller KD, Abbott LF. Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nat Neurosci 2000;3(9):919–26.
Izhikevich EM. Simple model of spiking neurons. IEEE Trans Neural Netw 2003;14(6):1569–72.
Girshick R. Fast r-cnn. Proceedings of the 2015 IEEE International Conference on Computer Vision; 2015. p. 1440– 1448.
Rochat P. Five levels of self-awareness as they unfold early in life. Conscious Cogn 2003;12(4):717–31.
Henriksson L, Mur M, Kriegeskorte N. Faciotopyła face-feature map with face-like topology in the human occipital face area. Cortex 2015;72:156–67.
Liu H, Yu Y, Sun F, Gu J. Visualctactile fusion for object recognition. IEEE Trans Autom Sci Eng 2017;14(2):996–1008.
Liu H, Guo D, Sun F. Object recognition using tactile measurements: Kernel sparse coding methods. IEEE Trans Instrum Meas 2016;65(3):656–65.
Compliance with Ethical Standards
This study was funded by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB02060007), and Beijing Municipal Commission of Science and Technology (Z161100000216124).
This article does not contain any studies with human participants performed by any of the authors.
Conflict of interests
All authors declare that they have no conflict of interest.
Yi Zeng and Yuxuan Zhao have equal contribution to this work and should be regarded as co-first authors
Take one movement cycle as an example, as shown in Table 1. N denotes that the related brain regions are not functionally activated. If the value is 0, it means the value is too small and can be omitted.
During the learning phase, the motion sequences, the detected moving parts, and the visual inputs are used. The actual motion sequences and the detected motion sequences from vision are used for motor-visual associative learning, while the detected moving parts and the acquired visual inputs are used for appearance learning. The inputs for Fast R-CNN are images for left arm, and right arm of the robot itself, head of the self, and head of others (1000 pieces for each class, with different backgrounds and random movements). The training was conducted on GPU K40 for approximately 16 h.
In the multi-robots mirror self-recognition test, three robots made random movements facing a mirror. Since the three robots are making random movements at the same time, the robots will acquire all the angles for associative learning. As the times of movements grow, the robot will gradually identify which angle from vision are from itself. As an illustrative example, robot A is assigned to have repetitive movement with the angle value 88°, while the other two robots make random movements for 100 times respectively. The associative weights for motor angles (88°) and visual angles (0°–360°) are shown in Fig. 10. Based on Eq. 12, the angle with the maximum weight is 88°. At this time, when robot A generates a motion with 88°, it expect to detect 88° from vision.
There are errors for angle detection which are hard to be completely avoided. Hence, the detected visual angles may not always be the same when the motor angles are the same. We can identify from Fig. 10 that the detected visual angles are distributed in a small range. Hence, 𝜃 threshold are used to represent this range in Eq. 12 for self-recognition. Figure 11 presents the standard deviation between angles from vision and the expected angles, while the red line (with value 3.69) denotes the average value for the standard deviation from 1∘ to 360∘. Hence, as an approximation, 𝜃 threshold is set to 4° in Eq. 12. According to the time consumption of information acquisition for robots, t threshold is set to 2 s. Every movement cycle contains 20 motion points. Hence, in Eq. 12, n = 20-1.
Figure 12 provides the belief values of different positions for each robot in nine independent tests. Each robot can pass the mirror test in all the nine tests.
About this article
Cite this article
Zeng, Y., Zhao, Y., Bai, J. et al. Toward Robot Self-Consciousness (II): Brain-Inspired Robot Bodily Self Model for Self-Recognition. Cogn Comput 10, 307–320 (2018). https://doi.org/10.1007/s12559-017-9505-1
- Robot self-consciousness
- Robot bodily self model
- STDP learning