Robot in the Mirror: Toward an Embodied Computational Model of Mirror Self-Recognition


Self-recognition or self-awareness is a capacity attributed typically only to humans and few other species. The definitions of these concepts vary and little is known about the mechanisms behind them. However, there is a Turing test-like benchmark: the mirror self-recognition, which consists in covertly putting a mark on the face of the tested subject, placing her in front of a mirror, and observing the reactions. In this work, first, we provide a mechanistic decomposition, or process model, of what components are required to pass this test. Based on these, we provide suggestions for empirical research. In particular, in our view, the way the infants or animals reach for the mark should be studied in detail. Second, we develop a model to enable the humanoid robot Nao to pass the test. The core of our technical contribution is learning the appearance representation and visual novelty detection by means of learning the generative model of the face with deep auto-encoders and exploiting the prediction error. The mark is identified as a salient region on the face and reaching action is triggered, relying on a previously learned mapping to arm joint angles. The architecture is tested on two robots with completely different face.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11


  1. 1.

    Mitchell [46] discusses the “chicken-and-egg problem” of acquiring this self-image—prior recognition in the mirror may be necessary to learn it—and concludes that there are “three possibilities: (1) a visually based, incomplete self-image of the part of the organism it can see, (2) a non-visual self-image, (3) or a mixture of these images.”

  2. 2.

    Elbow pitch and wrist rotation were fixed.


  1. 1.

    Abrossimoff J, Pitti A, Gaussier P (2018) Visual learning for reaching and body-schema with gain-field networks. In: 2018 Joint IEEE 8th international conference on development and learning and epigenetic robotics (ICDL-EpiRob). IEEE, pp 197–203

  2. 2.

    Alzueta E, Melcón M, Jensen O, Capilla A (2019) The ‘Narcissus Effect’: top-down alpha-beta band modulation of face-related brain areas during self-face processing. NeuroImage 213:2020

    Google Scholar 

  3. 3.

    Alzueta E, Melcón M, Poch C, Capilla A (2019) Is your own face more than a highly familiar face? Biol Psychol 142:100–107

    Google Scholar 

  4. 4.

    Amsterdam B (1972) Mirror self-image reactions before age two. Dev Psychobiol 5(4):297–305

    Google Scholar 

  5. 5.

    Anderson JR (1984) The development of self-recognition: a review. Dev Psychobiol 17(1):35–49

    Google Scholar 

  6. 6.

    Anderson JR, Gallup GG (2015) Mirror self-recognition: a review and critique of attempts to promote and engineer self-recognition in primates. Primates 56(4):317–326

    Google Scholar 

  7. 7.

    Apps MAJ, Tsakiris M (2014) The free-energy self: a predictive coding account of self-recognition. Neurosci Biobehav Rev 41:85–97

    Google Scholar 

  8. 8.

    Asada M, Hosoda K, Kuniyoshi Y, Ishiguro H, Inui T, Yoshikawa Y, Ogino M, Yoshida C (2009) Cognitive developmental robotics: a survey. IEEE Trans Autonom Mental Dev 1(1):12–34

    Google Scholar 

  9. 9.

    Ballard Dana H (1987) Modular learning in neural networks. In AAAI, pp 279–284

  10. 10.

    Bard KA, Todd BK, Bernier C, Love J, Leavens DA (2006) Self-awareness in human and chimpanzee infants: what is measured and what is meant by the mark and mirror test? Infancy 9(2):191–219

    Google Scholar 

  11. 11.

    Bigelow AE (1981) The correspondence between self-and image movement as a cue to self-recognition for young children. J Genet Psychol 139(1):11–26

    MathSciNet  Google Scholar 

  12. 12.

    Bringsjord S, Licato J, Govindarajulu NS, Ghosh R, Sen A (2015) Real robots that pass human tests of self-consciousness. In: 2015 24th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 498–504

  13. 13.

    Bruce V, Young A (1986) Understanding face recognition. Br J Psychol 77(3):305–327

    Google Scholar 

  14. 14.

    Cangelosi A, Schlesinger M (2015) Developmental robotics: from babies to robots. MIT Press, Cambridge

    Google Scholar 

  15. 15.

    Chang L, Fang Q, Zhang S, Poo M, Gong N (2015) Mirror-induced self-directed behaviors in rhesus monkeys after visual-somatosensory training. Curr Biol 25(2):212–217

    Google Scholar 

  16. 16.

    Chinn LK (2019) Development of Self Knowledge: Tactile Localization to Self-Recognition. PhD thesis, Tulane University School of Science and Engineering

  17. 17.

    Corbetta M, Shulman GL (2002) Control of goal-directed and stimulus-driven attention in the brain. Nat Rev Neurosci 3(3):201–215

    Google Scholar 

  18. 18.

    Corbetta D, Thurman SL, Wiener RF, Guan Yu, Williams JL (2014) Mapping the feel of the arm with the sight of the object: on the embodied origins of infant reaching. Front Psychol 5:576

    Google Scholar 

  19. 19.

    de Waal Frans BM (2019) Fish, mirrors, and a gradualist perspective on self-awareness. PLoS Biol 17(2)

  20. 20.

    De Waal F (2016) Are we smart enough to know how smart animals are?. WW Norton & Company, New York

    Google Scholar 

  21. 21.

    Diez-Valencia G, Ohashi T, Lanillos P, Cheng G (2019) Sensorimotor learning for artificial body perception. arXiv preprint arXiv:1901.09792

  22. 22.

    Eimer M (2012) The face-sensitive N170 component of the event-related brain potential. Oxford Handbook of Face Perception, pp 329–344

  23. 23.

    Fitzpatrick PM, Metta G (2002) Toward manipulation-driven vision. In Proc. IEEE/RSJ Int. Conf. on intelligent robots and systems

  24. 24.

    Friston K (2010) The free-energy principle: a unified brain theory? Nat Rev Neurosci 11(2):127

    Google Scholar 

  25. 25.

    Friston KJ, Daunizeau J, Kilner J, Kiebel SJ (2010) Action and behavior: a free-energy formulation. Biol Cybern 102(3):227–260

    Google Scholar 

  26. 26.

    Fuke S, Ogino M, Asada M (2007) Body image constructed from motor and tactle images with visual informaiton. Int J Hum Robot 4:347–364

    Google Scholar 

  27. 27.

    Gallagher S (2000) Philosophical conceptions of the self: implications for cognitive science. Trends Cogn Sci 4(1):14–21

    Google Scholar 

  28. 28.

    Gallup Jr GG (1982) Self-awareness and the emergence of mind in primates. Am J Primatol 2(3):237–248

    Google Scholar 

  29. 29.

    Gallup GG (1970) Chimpanzees: self-recognition. Science 167(3914):86–87

    Google Scholar 

  30. 30.

    Gold K, Scassellati B (2009) Using probabilistic reasoning over time to self-recognize. Robot Autonom Syst 57(4):384–392

    Google Scholar 

  31. 31.

    Guillaume P (1971) Imitation in children. trans. ep halperin

  32. 32.

    Hafner VV, Loviken P, Villalpando AP, Schillaci G (2020) Prerequisites for an artificial self. Front Neurorobot:14

  33. 33.

    Hart JW (2014) Robot self-modeling. Yale University, New Haven

    Google Scholar 

  34. 34.

    Heed T, Buchholz VN, Engel AK, Röder B (2015) Tactile remapping: from coordinate transformation to integration in sensorimotor processing. Trends Cogn Sci 19(5):251–258

    Google Scholar 

  35. 35.

    Hoffmann M, Pfeifer R (2018) Robots as powerful allies for the study of embodied cognition from the bottom up. In: Newen A, de Bruin L, Gallagher S (eds) The Oxford Handbook 4e Cognition, chapter 45. Oxford University Press, Oxford, pp 841–862

    Google Scholar 

  36. 36.

    Hoffmann M, Marques HG, Arieta AH, Sumioka H, Lungarella M, Pfeifer R (2010) Body schema in robotics: a review. Autonom Mental Dev IEEE Tran 2(4):304–324

    Google Scholar 

  37. 37.

    Ida Gobbini M, Haxby JV (2007) Neural systems for recognition of familiar faces. Neuropsychologia 45(1):32–41

    Google Scholar 

  38. 38.

    Kingma DP, Adam JB (2014) A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  39. 39.

    Kuniyoshi Y (2019) Fusing autonomy and sociality via embodied emergence and development of behaviour and cognition from fetal period. Philos Trans R Soc B 374(1771):20180031

    Google Scholar 

  40. 40.

    Laflaquière A, Hafner VV (2019) Self-supervised body image acquisition using a deep neural network for sensorimotor prediction. In: 2019 Joint IEEE 9th international conference on development and learning and epigenetic robotics (ICDL-EpiRob). IEEE, pp 117–122

  41. 41.

    Lanillos P, Dean-Leon E, Cheng G (2016) Yielding self-perception in robots through sensorimotor contingencies. IEEE Trans Cogn Dev Syst 9(2):100–112

    Google Scholar 

  42. 42.

    Lanillos P, Dean-Leon E, Cheng G (2017) Enactive self: a study of engineering perspectives to obtain the sensorimotor self through enaction. In: Joint IEEE Int. Conf. on, in developmental learning and epigenetic robotics

  43. 43.

    Lanillos P, Pages J, Cheng G (2020) Robot self/other distinction: active inference meets neural networks learning in a mirror. In: European conference on artificial intelligence (ECAI 2020)

  44. 44.

    Latinus M, Taylor MJ (2006) Face processing stages: impact of difficulty and the separation of effects. Brain Res 1123(1):179–187

    Google Scholar 

  45. 45.

    Ma K, Lippelt DP, Hommel B (2017) Creating virtual-hand and virtual-face illusions to investigate self-representation. JoVE 121:e54784

    Google Scholar 

  46. 46.

    Mitchell RW (1993) Mental models of mirror-self-recognition: two theories. New Ideas Psychol 11(3):295–325

    Google Scholar 

  47. 47.

    Natale L, Orabona F, Metta G, Sandini G (2007) Sensorimotor coordination in a “baby” robot: learning about objects through grasping. Prog Brain Res 164:403–424

    Google Scholar 

  48. 48.

    Neisser U (1988) Five kinds of self-knowledge. Philos Psychol 1(1):35–59

    Google Scholar 

  49. 49.

    Oliver G, Lanillos P, Cheng G (2021) Active inference body perception and action for humanoid robots. IEEE Trans Cogn Dev Syst.

    Article  Google Scholar 

  50. 50.

    Pablo L, Emmanuel DL, Gordon C (2016) Multisensory object discovery via self-detection and artificial attention. In: Joint IEEE Int. Conf. on, In developmental learning and epigenetic robotics

  51. 51.

    Pfeifer R, Bongard JC (2007) How the body shapes the way we think: a new view of intelligence. MIT Press, Cambridge

    Google Scholar 

  52. 52.

    Piaget J (1954) The construction of reality in the child. Basic Books, New York

    Google Scholar 

  53. 53.

    Pitti A, Mori H, Kouzuma S, Kuniyoshi Y (2009) Contingency perception and agency measure in visuo-motor spiking neural networks. IEEE Trans Autonom Mental Dev 1(1):86–97

    Google Scholar 

  54. 54.

    Prescott TJ, Camilleri D (2019) The synthetic psychology of the self. Cognitive architectures. Springer, Berlin, pp 85–104

    Google Scholar 

  55. 55.

    Rao RPN, Ballard DH (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2(1):79–87

    Google Scholar 

  56. 56.

    Rood T, van Gerven M, Lanillos P (2020) A deep active inference model of the rubber-hand illusion. International Workshop on Active Inference. Springer, Cham, pp 84–91

    Google Scholar 

  57. 57.

    Sancaktar C, van Gerven M, Lanillos P (2020) End-to-end pixel-based deep active inference for body perception and action. In: Joint IEEE 10th international conference on development and learning and epigenetic robotics (ICDL-EpiRob)

  58. 58.

    Schweinberger SR, Neumann MF (2016) Repetition effects in human ERPs to faces. Cortex 80:141–153

    Google Scholar 

  59. 59.

    Steels L, Spranger M (2008) The robot in the mirror. Connect Sci 20(4):337–358

    Google Scholar 

  60. 60.

    Sui J, Xiaosi G (2017) Self as object: emerging trends in self research. Trends Neurosci 40(11):643–653

    Google Scholar 

  61. 61.

    Tani J (1998) An interpretation of the ‘self’ from the dynamical systems perspective: a constructivist approach. J Conscious Stud 5(5–6):516–542

    Google Scholar 

  62. 62.

    Tsakiris M, Longo MR, Haggard P (2010) Having a body versus moving your body: neural signatures of agency and body-ownership. Neuropsychologia 48(9):2740–2749

    Google Scholar 

  63. 63.

    Yoshikawa Y, Tsuji Y, Hosoda K, Asada M (2004) Is it my body? body extraction from uninterpreted sensory data based on the invariance of multiple sensory attributes. In: 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS)(IEEE Cat. No. 04CH37566), vol 3. IEEE, pp 2325–2330

  64. 64.

    Zaadnoordijk L, Besold TR, Hunnius S (2019) A match does not make a sense: on the sufficiency of the comparator model for explaining the sense of agency. Neurosci Conscious 1:niz006

    Google Scholar 

  65. 65.

    Lanillos P, Franklin S, Franklin DW (2020) The predictive brain in action: Involuntary actions reduce body prediction errors. bioRxiv. Cold Spring Harbor Laboratory.

Download references

Author information



Corresponding author

Correspondence to Pablo Lanillos.

Additional information

M. H. and V. O. were supported by the Czech Science Foundation (GA ČR), Project nr. 17-15697Y. P. L. was partially supported by the H2020 project Selfception (nr. 741941).

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 26051 KB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hoffmann, M., Wang, S., Outrata, V. et al. Robot in the Mirror: Toward an Embodied Computational Model of Mirror Self-Recognition. Künstl Intell 35, 37–51 (2021).

Download citation


  • Self-recognition
  • Robot
  • Mirror test
  • Novelty detection
  • Predictive brain
  • Generative models