Complex-Valued Representation for RGB-D Object Recognition

  • Rim Trabelsi
  • Issam Jabri
  • Farid Melgani
  • Fethi Smach
  • Nicola Conci
  • Ammar Bouallegue
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10749)


Object recognition methods usually tend to focus on single cues coming from traditional vision based systems but ignore to incorporate multi-modal data. With the advent of depth RGB-D sensors which provide synchronized multi-modal data with good quality, new opportunities have been emerged. In this paper, we make use of RGB and depth images to propose a new object recognition approach. Using a pixel-wise scheme, we propose a novel method to describe RGB-D images with a complex-valued representation. By means of neural network, we introduce a new CVNN (Complex-Valued Neural Network) with RBF neurons. Different from many RGB-D features, the proposed approach is able to jointly use RGB and depth data within a unified end-to-end learning framework. Category and instance object recognition tasks are evaluated through experiments carried out on a large scale RGB-D object dataset. Results show that our method can efficiently recognize objects in RGB-D images and outperforms state-of-the-art approaches.


RGB-D representation Object recognition Complex-valued neural networks Multi-modal data 



This work was supported by the European Union funding through ALYSSA program (ERASMUS-MUNDUS action 2 lot 6) and by the research grant from Singapore Agency for Science, Technology and Research (A*STAR) through the ARAP program.


  1. 1.
    Andreopoulos, A., Tsotsos, J.K.: 50 years of object recognition: directions forward. Comput. Vis. Image Underst. 117(8), 827–891 (2013)CrossRefGoogle Scholar
  2. 2.
    Bucak, S.S., Jin, R., Jain, A.K.: Multiple kernel learning for visual object recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1354–1369 (2014)CrossRefGoogle Scholar
  3. 3.
    Lai, K., Bo, L., Ren, X., Fox, D.: A large-scale hierarchical multi-view RGB-D object dataset. In: 2011 IEEE International Conference on Robotics and Automation (ICRA), pp. 1817–1824. IEEE (2011)Google Scholar
  4. 4.
    Held, D., Thrun, S., Savarese, S.: Robust single-view instance recognition. In: 2016 IEEE International Conference on Robotics and Automation (ICRA), pp. 2152–2159. IEEE (2016)Google Scholar
  5. 5.
    Gupta, S., Girshick, R., Arbeláez, P., Malik, J.: Learning rich features from RGB-D images for object detection and segmentation. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 345–360. Springer, Cham (2014). Google Scholar
  6. 6.
    Li, X., Fang, M., Zhang, J.-J., Wu, J.: Learning coupled classifiers with RGB images for RGB-D object recognition. Pattern Recogn. 61, 433–446 (2017)CrossRefGoogle Scholar
  7. 7.
    Amin, M.F., Murase, K.: Single-layered complex-valued neural network for real-valued classification problems. Neurocomputing 72(4), 945–955 (2009)CrossRefGoogle Scholar
  8. 8.
    Savitha, R., Suresh, S., Sundararajan, N., Kim, H.J.: A fully complex-valued radial basis function classifier for real-valued classification problems. Neurocomputing 78(1), 104–110 (2012)CrossRefGoogle Scholar
  9. 9.
    Bo, L., Ren, X., Fox, D.: Depth kernel descriptors for object recognition. In: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 821–826. IEEE (2011)Google Scholar
  10. 10.
    Lai, K., Bo, L., Ren, X., Fox, D.: A scalable tree-based approach for joint object and pose recognition. In: AAAI, vol. 1, p. 2 (2011)Google Scholar
  11. 11.
    Bo, L., Ren, X., Fox, D.: Unsupervised feature learning for RGB-D based object recognition. In: Desai, J., Dudek, G., Khatib, O., Kumar, V. (eds.) Experimental Robotics. Springer Tracts in Advanced Robotics, vol. 88, pp. 387–402. Springer, Heidelberg (2013). CrossRefGoogle Scholar
  12. 12.
    Bo, L., Ren, X., Fox, D.: Hierarchical matching pursuit for image classification: architecture and fast algorithms. In: Advances in Neural Information Processing Systems, pp. 2115–2123 (2011)Google Scholar
  13. 13.
    Eitel, A., Springenberg, J.T., Spinello, L., Riedmiller, M., Burgard, W.: Multimodal deep learning for robust RGB-D object recognition. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 681–687. IEEE (2015)Google Scholar
  14. 14.
    Hirose, A.: Complex-Valued Neural Networks. Springer Science & Business Media, Heidelberg (2006). CrossRefMATHGoogle Scholar
  15. 15.
    Hirose, A.: Dynamics of fully complex-valued neural networks. Electron. Lett. 28(16), 1492–1494 (1992)CrossRefGoogle Scholar
  16. 16.
    Hirose, A.: Continuous complex-valued back-propagation learning. Electron. Lett. 28(20), 1854–1855 (1992)CrossRefGoogle Scholar
  17. 17.
    Hirose, A.: Complex-Valued Neural Networks: Theories and Applications, vol. 5. World Scientific, Singapore (2003)CrossRefMATHGoogle Scholar
  18. 18.
    Fiori, S.: Nonlinear complex-valued extensions of Hebbian learning: an essay. Neural Comput. 17(4), 779–838 (2005)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Fiori, S.: Learning by criterion optimization on a unitary unimodular matrix group. Int. J. Neural Syst. 18(02), 87–103 (2008)CrossRefGoogle Scholar
  20. 20.
    Savitha, R., Suresh, S., Sundararajan, N.: A fully complex-valued radial basis function network and its learning algorithm. Int. J. Neural Syst. 19(04), 253–267 (2009)CrossRefGoogle Scholar
  21. 21.
    Kim, T., Adali, T.: Fully complex multi-layer perceptron network for nonlinear signal processing. J. VLSI Sig. Process. Syst. Sig. Image Video Technol. 32(1–2), 29–43 (2002)CrossRefMATHGoogle Scholar
  22. 22.
    Kanungo, T., Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002)CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Rim Trabelsi
    • 1
    • 2
    • 3
  • Issam Jabri
    • 4
  • Farid Melgani
    • 5
  • Fethi Smach
    • 6
  • Nicola Conci
    • 5
  • Ammar Bouallegue
    • 2
  1. 1.Advanced Digital Sciences CenterSingaporeSingapore
  2. 2.SysCom Laboratory, National Engineering School of TunisUniversity of Tunis El ManarTunisTunisia
  3. 3.Hatem Bettaher IResCoMath Research Unit, National Engineering School of GabesUniversity of GabesGabèsTunisia
  4. 4.College of Computer and Information SystemsAl Yamamah UniversityRiyadhKingdom of Saudi Arabia
  5. 5.Department of Information Engineering and Computer ScienceUniversity of TrentoTrentoItaly
  6. 6.Profil TechnologyMontrougeFrance

Personalised recommendations