Learning a Compositional Hierarchy of Disparity Descriptors for 3D Orientation Estimation in an Active Fixation Setting

  • Katerina Kalou
  • Agostino Gibaldi
  • Andrea Canessa
  • Silvio P. Sabatini
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10614)


Interaction with everyday objects requires by the active visual system a fast and invariant reconstruction of their local shape layout, through a series of fast binocular fixation movements that change the gaze direction on the 3-dimensional surface of the object. Active binocular viewing results in complex disparity fields that, although informative about the orientation in depth (e.g., the slant and tilt), highly depend on the relative position of the eyes. Assuming to learn the statistical relationships between the differential properties of the disparity vector fields and the gaze directions, we expect to obtain more convenient, gaze-invariant visual descriptors. In this work, local approximations of disparity vector field differentials are combined in a hierarchical neural network that is trained to represent the slant and tilt from the disparity vector fields. Each gaze-related cell’s activation in the intermediate representation is recurrently merged with the other cells’ activations to gain the desired gaze-invariant selectivity. Although the representation has been tested on a limited set of combinations of slant and tilt, the resulting high classification rate validates the generalization capability of the approach.


Active vision Binocular disparity Gaze direction Biologically-inspired neural networks 


  1. 1.
    Ban, H., Welchman, A.E.: fMRI analysis-by-synthesis reveals a dorsal hierarchy that extracts surface slant. J. Neurosci. 35(27), 9823–9835 (2015)CrossRefGoogle Scholar
  2. 2.
    Canessa, A., Gibaldi, A., Chessa, M., Fato, M., Solari, F., Sabatini, S.P.: A dataset of stereoscopic images and ground-truth disparity mimicking human fixations in peripersonal space. Sci. Data 4 (2017)Google Scholar
  3. 3.
    Dhond, U.R., Aggarwal, J.K.: Structure from stereo-a review. IEEE Trans. Syst. Man Cybern. 19(6), 1489–1510 (1989)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Gibaldi, A., Canessa, A., Sabatini, S.P.: The active side of stereopsis: fixation strategy and adaptation to natural environments. Sci. Rep. 7, 44800 (2017)CrossRefGoogle Scholar
  5. 5.
    Hansard, M., Horaud, R.: Cyclopean geometry of binocular vision. JOSA A 25(9), 2357–2369 (2008)CrossRefzbMATHGoogle Scholar
  6. 6.
    Hinkle, D.A., Connor, C.E.: Three-dimensional orientation tuning in macaque area V4. Nat. Neurosci. 5(7), 665–670 (2002)CrossRefGoogle Scholar
  7. 7.
    Koenderink, J.J., van Doorn, A.J.: The internal representation of solid shape with respect to vision. Biol. Cybern. 32(4), 211–216 (1979)CrossRefzbMATHGoogle Scholar
  8. 8.
    Koenderink, J.J., van Doorn, A.J.: Facts on optic flow. Biol. Cybern. 56(4), 247–254 (1987)CrossRefzbMATHGoogle Scholar
  9. 9.
    Liu, L., van Hulle, M.M.: Modeling the surround of MT cells and their selectivity for surface orientation in depth specified by motion. Neural Comput. 10(2), 295–312 (1998)CrossRefGoogle Scholar
  10. 10.
    LeCun, Y., Huang, F.J., Bottou, L.: Learning methods for generic object recognition with invariance to pose and lighting. In: 2004 Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2004, vol. 2, pp. II–104. IEEE (2004)Google Scholar
  11. 11.
    Medsker, L.R., Jain, L.C.: Recurrent neural networks. Des. Appl. 5 (2001)Google Scholar
  12. 12.
    Nguyenkim, J.D., DeAngelis, G.C.: Disparity-based coding of three-dimensional surface orientation by macaque middle temporal neurons. J. Neurosci. 23(18), 7117–7128 (2003)Google Scholar
  13. 13.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning internal representation by back propagation. Parallel Distrib. Process.: Explor. Microstruct. Cogn. 1 (1986)Google Scholar
  14. 14.
    Salinas, E., Abbott, L.F.: A model of multiplicative neural responses in parietal cortex. Proc. Nat. Acad. Sci. 93(21), 11956–11961 (1996)CrossRefGoogle Scholar
  15. 15.
    Tsao, D.Y., Vanduffel, W., Sasaki, Y., Fize, D., Knutsen, T.A., Mandeville, J.B., Wald, L.L., Dale, A.M., Rosen, B.R., Van Essen, D.C., Livingstone, M.S.: Stereopsis activates V3A and caudal intraparietal areas in macaques and humans. Neuron 39(3), 555–568 (2003)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Katerina Kalou
    • 1
  • Agostino Gibaldi
    • 1
    • 2
  • Andrea Canessa
    • 1
  • Silvio P. Sabatini
    • 1
  1. 1.Department of Informatics, Bioengineering, Robotics and System EngineeringUniversity of GenoaGenoaItaly
  2. 2.School of OptometryUniversity of California, BerkeleyBerkeleyUSA

Personalised recommendations