Descriptor Learning for Omnidirectional Image Matching

  • Jonathan Masci
  • Davide Migliore
  • Michael M. Bronstein
  • Jürgen Schmidhuber
Part of the Studies in Computational Intelligence book series (SCI, volume 532)


Feature matching in omnidirectional vision systems is a challenging problem, mainly because complicated optical systems make the theoretical modelling of invariance and construction of invariant feature descriptors hard or even impossible. In this paper, we propose learning invariant descriptors using a training set of similar and dissimilar descriptor pairs.We use the similarity-preserving hashing framework, in which we are trying to map the descriptor data to the Hamming space preserving the descriptor similarity on the training set. A neural network is used to solve the underlying optimization problem. Our approach outperforms not only straightforward descriptor matching, but also state-of-the-art similarity-preserving hashing methods.


Shape Retrieval Parabolic Mirror Invariant Descriptor Omnidirectional Image Omnidirectional Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Athitsos, V., Alon, J., Sclaroff, S., Kollios, G.: Boostmap: a method for efficient approximate similarity ranking. In: Proc. CVPR (2004)Google Scholar
  2. 2.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: SURF: Speeded Up Robust Features. Computer Vision and Image Understanding 10(3), 346–359 (2008)CrossRefGoogle Scholar
  3. 3.
    Bogdanova, I., Bresson, X., Thiran, J.P., Vandergheynst, P.: Scale space analysis and active contours for omnidirectional images. Trans. Image Processing 16(7), 1888–1901 (2007), doi:10.1109/TIP.2007.899008MathSciNetCrossRefGoogle Scholar
  4. 4.
    Bonarini, A., Burgard, W., Fontana, G., Matteucci, M., Sorrenti, D.G., Tardos, J.D.: Rawseeds: Robotics advancement through web-publishing of sensorial and elaborated extensive data sets. In: Proc. IROS Workshop on Benchmarks in Robotics Research (2006),
  5. 5.
    Briggs, A., Li, Y., Scharstein, D., Wilder, M.: Robot navigation using 1d panoramic images. In: Proc. ICRA (2006)Google Scholar
  6. 6.
    Bronstein, A., Bronstein, M., Ovsjanikov, M., Guibas, L.: Shape Google: geometric words and expressions for invariant shape retrieval. ACM TOG (2010)Google Scholar
  7. 7.
    Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Video genome. Tech. Rep. arXiv:1003.5320v1 (2010)Google Scholar
  8. 8.
    Bronstein, M.M., Bronstein, A.M., Michel, F., Paragios, N.: Data fusion through cross-modality metric learning using similarity-sensitive hashing. In: Proc. CVPR (2010)Google Scholar
  9. 9.
    Ceriani, S., Fontana, G., Giusti, A., Marzorati, D., Matteucci, M., Migliore, D., Rizzi, D., Sorrenti, D.G., Taddei, P.: Rawseeds ground truth collection systems for indoor self-localization and mapping. Autonomous Robots 27(4), 353–371 (2009),, doi:10.1007/s10514-009-9156-5CrossRefGoogle Scholar
  10. 10.
    Cruz, J., Bogdanova, I., Paquier, B., Bierlaire, M., Thiran, J.P.: Scale invariant feature transform on the sphere: Theory and applications. Tech. rep. (2009),
  11. 11.
    Freund, Y., Schapire, R.: A decision-theoretic generalization of on-line learning and an application to boosting. In: European Conference on Computational Learning Theory, pp. 23–37 (1995)Google Scholar
  12. 12.
    Geyer, C., Stewenius, H.: A nine-point algorithm for estimating para-catadioptric fundamental matrices. In: Proc. CVPR (2007)Google Scholar
  13. 13.
    Gionis, A., Indik, P., Motwani, R.: Similarity Search in High Dimensions via Hashing. In: International Conference on Very Large Databases (2004)Google Scholar
  14. 14.
    Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: Proc. CVPR (2006)Google Scholar
  15. 15.
    Hansen, P.I., Corke, P., Boles, W.: Wide-angle visual feature matching for outdoor localization. Int. J. Robotics Research 29(2/3), 267–297 (2010), CrossRefGoogle Scholar
  16. 16.
    Jain, P., Kulis, B., Grauman, K.: Fast image search for learned metrics. In: CVPR (2008)Google Scholar
  17. 17.
    Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 304–317. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Jégou, H., Douze, M., Schmid, C.: Packing Bag-of-Features. In: Proc. ICCV (2009)Google Scholar
  19. 19.
    Jégou, H., Douze, M., Schmid, C.: Product quantization for nearest neighbor search. Trans. PAMI (2010)Google Scholar
  20. 20.
    Kimmel, R., Zhang, C., Bronstein, A.M., Bronstein, M.M.: Are mser features really interesting? Trans. PAMI 32(11), 2316–2320 (2011)CrossRefGoogle Scholar
  21. 21.
    Kulis, B., Darrell, T.: Learning to hash with binary reconstructive embeddings. In: Proc. NIPS, pp. 1042–1050 (2009)Google Scholar
  22. 22.
    Lowe, D.: Distinctive Image Features from Scale-Invariant Keypoints. IJCV 20(2), 91–110 (2004)CrossRefGoogle Scholar
  23. 23.
    Mauthner, T., Fraundorfer, F., Bischof, H.: Region matching for omnidirectional images using virtual camera planes. Technology (2006)Google Scholar
  24. 24.
    Mei, C., Rives, P.: Single view point omnidirectional camera calibration from planar grids. In: Proc. ICRA (2007)Google Scholar
  25. 25.
    Micusik, B., Pajdla, T.: Structure from motion with wide circular field of view cameras. Trans. PAMI 28(7), 1135–1149 (2006), doi:10.1109/TPAMI.2006.151CrossRefGoogle Scholar
  26. 26.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. IJCV 65(1/2), 43–72 (2005)CrossRefGoogle Scholar
  27. 27.
    Nayar, S.K.: Catadioptric Omnidirectional Camera. In: Proc. CVPR (1997)Google Scholar
  28. 28.
    Puig, L., Guerrero, J.J.: Scale space for central catadioptric systems. towards a generic camera feature extractor. In: Proc. ICCV (2011)Google Scholar
  29. 29.
    Raginsky, M., Lazebnik, S.: Locality-Sensitive Binary Codes from Shift-Invariant Kernels. In: Proc. NIPS (2009)Google Scholar
  30. 30.
    Salakhutdinov, R., Hinton, G.: Semantic hashing. In: SIGIR Workshop on Information Retrieval and applications of Graphical Models (2007)Google Scholar
  31. 31.
    Scaramuzza, D., Siegwart, R., Martinelli, A.: A robust descriptor for tracking vertical lines in omnidirectional images and its use in mobile robotics. Int. J. Robotics Research 28(2), 149–171 (2009), CrossRefGoogle Scholar
  32. 32.
    Schmidhuber, J., Prelinger, D.: Discovering predictable classifications. Neural Computation 5(4), 625–635 (1993)CrossRefGoogle Scholar
  33. 33.
    Shakhnarovich, G.: Learning Task-Specific Similarity. Ph.D. thesis, MIT (2005)Google Scholar
  34. 34.
    Strecha, C., Bronstein, A.M., Bronstein, M.M., Fua, P.: LDAHash: improved matching with smaller descriptors. Trans. PAMI (2011)Google Scholar
  35. 35.
    Svoboda, T., Pajdla, T.: Matching in catadioptric images with appropriate windows, and outliers removal. In: Skarbek, W. (ed.) CAIP 2001. LNCS, vol. 2124, pp. 733–740. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  36. 36.
    Taylor, G.W., Spiro, I., Bregler, C., Fergus, R.: Learning invariance through imitation. In: Proc. CVPR (2011)Google Scholar
  37. 37.
    Tola, E., Lepetit, V., Fua, P.: Daisy: an Efficient Dense Descriptor Applied to Wide Baseline Stereo. Trans. PAMI 32(5), 815–830 (2010)CrossRefGoogle Scholar
  38. 38.
    Torralba, A., Fergus, R., Freeman, W.T.: 80 million tiny images: a large dataset for non-parametric object and scene recognition. Trans. PAMI 30(11), 1958–1970 (2008)CrossRefGoogle Scholar
  39. 39.
    Vedaldi, A.: An open implementation of the SIFT detector and descriptor. Tech. Rep. 070012, UCLA CSD (2007)Google Scholar
  40. 40.
    Wang, J., Kumar, S., Chang, S.F.: Semi-supervised hashing for scalable image retrieval. In: CVPR (2010)Google Scholar
  41. 41.
    Wang, J., Kumar, S., Chang, S.F.: Sequential projection learning for hashing with compact codes. In: ICML (2010)Google Scholar
  42. 42.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Jonathan Masci
    • 1
  • Davide Migliore
    • 2
  • Michael M. Bronstein
    • 3
  • Jürgen Schmidhuber
    • 1
  1. 1.IDSIA, USI and SUPSIManno-LuganoSwitzerland
  2. 2.Evidence S.r.l.MilanItaly
  3. 3.Dept. of InformaticsUniversità della Svizzera ItalianaLuganoSwitzerland

Personalised recommendations