International Journal of Computer Vision

, Volume 100, Issue 1, pp 78–98 | Cite as

Keypoints and Local Descriptors of Scalar Functions on 2D Manifolds

Article

Abstract

This paper addresses the problem of describing surfaces using local features and descriptors. While methods for the detection of interest points in images and their description based on local image features are very well understood, their extension to discrete manifolds has not been well investigated. We provide a methodological framework for analyzing real-valued functions defined over a 2D manifold, embedded in the 3D Euclidean space, e.g., photometric information, local curvature, etc. Our work is motivated by recent advancements in multiple-camera reconstruction and image-based rendering of 3D objects: there is a growing need for describing object surfaces, matching two surfaces, or tracking them over time. Considering polygonal meshes, we propose a new methodological framework for the scale-space representations of scalar functions defined over such meshes. We propose a local feature detector (MeshDOG) and region descriptor (MeshHOG). Unlike the standard image features, the proposed surface features capture both the local geometry of the underlying manifold and the scale-space differential properties of the real-valued function itself. We provide a thorough experimental evaluation. The repeatability of the feature detector and the robustness of feature descriptor are tested, by applying a large number of deformations to the manifold or to the scalar function.

Keywords

3D shape Meshed surfaces Riemannian manifold Scale space 3D keypoint detection Local shape descriptors Shape matching 

Notes

Acknowledgements

We would like to thank Cedric Cagniart and Artiom Kovnatsky for their help with the datasets.

References

  1. de Aguiar, E., Theobalt, C., Stoll, C., & Seidel, H. P. (2007). Marker-less 3D feature tracking for mesh-based human motion capture. In Human motion—understanding, modeling, capture and animation (pp. 1–15). CrossRefGoogle Scholar
  2. Ahmed, N., Theobalt, C., Rossl, C., Thrun, S., & Seidel, H. P. (2008). Dense correspondence finding for parametrization-free animation reconstruction from video. In Proceedings of IEEE conference on computer vision and pattern recognition. Google Scholar
  3. Bariya, P., & Nishino, K. (2010). Scale-hierarchical 3d object recognition in cluttered scenes. In Proc. of IEEE computer vision and pattern recognition (pp. 1657–1664). Google Scholar
  4. Barth, T. (1993). A 3-D least-squares upwind Euler solver for unstructured meshes. In Lecture notes in physics: Vol. 414. Thirteenth international conference on numerical methods in fluid dynamics (pp. 240–244). Berlin: Springer. CrossRefGoogle Scholar
  5. Bay, H., Ess, A., Tuytelaars, T., & Gool, L. V. (2008). Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3), 346–359. CrossRefGoogle Scholar
  6. Bolles, R. C., & Cain, RA (1982). Recognizing and locating partially visible objects, the Local-Feature-Focus method. The International Journal of Robotics Research, 1(3), 57–82. CrossRefGoogle Scholar
  7. Bolles, R. C., & Horaud, R. (1986). 3DPO: A three-dimensional part orientation system. The International Journal of Robotics Research, 5(3), 3–26. CrossRefGoogle Scholar
  8. Bronstein, A. M., Bronstein, M. M., Bustos, B., Castellani, U., Crisani, M., Falcidieno, B., Guibas, L. J., Kokkinos, I., Murino, V., Ovsjanikov, M., Patané, G., Sipiran, I., Spagnuolo, M., & Sun, J. (2010). Shrec 2010: robust feature detection and description benchmark. In Proc. EUROGRAPHICS workshop on 3D object retrieval (3DOR). Google Scholar
  9. Bronstein, A. M., Bronstein, M. M., Ovsjanikov, M., & Guibas, L. J. (2011). Shape Google: geometric words and expressions for invariant shape retrieval. ACM Transactions on Graphics, 30(1), 1–20. CrossRefGoogle Scholar
  10. Bustos, B., Keim, D. A., Saupe, D., Schreck, T., & Vranic, D. V. (2005). Feature-based similarity search in 3D object databases. ACM Computing Surveys, 34(4), 345–387. CrossRefGoogle Scholar
  11. Cagniart, C., Boyer, E., & Ilic, S. (2010). Probabilistic deformable surface tracking from multiple videos. In Proceedings of European conference on computer vision. Google Scholar
  12. Castellani, U., Cristani, M., Fantoni, S., & Murino, V. (2008). Sparse points matching by combining 3D mesh saliency with statistical descriptors. Computer Graphics Forum, 27(2), 643–652. CrossRefGoogle Scholar
  13. Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 886–893). Google Scholar
  14. Dong, C. S., & Wang, G. Z. (2005). Curvatures estimation on triangular mesh. Journal of Zhejiang University SCIENCE, 6A(1), 128–136. MATHGoogle Scholar
  15. Dufournaud, Y., Schmid, C., & Horaud, R. P. (2004). Image matching with scale adjustment. Computer Vision and Image Understanding, 93(2), 175–194. CrossRefGoogle Scholar
  16. Frome, A., Huber, D., Kolluri, R., Bulow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. In Proceedings of European conference on computer vision. Google Scholar
  17. Furukawa, Y., & Ponce, J. (2008). Dense 3D motion capture from synchronized video streams. In Proceedings of IEEE conference on computer vision and pattern recognition. Google Scholar
  18. Horn, R. A., & Johnson, C. A. (1994). Matrix analysis. Cambridge: Cambridge University Press. MATHGoogle Scholar
  19. Hou, T., & Qin, H. (2010). Efficient computation of scale-space features for deformable shape correspondences. In Proceedings of European conference on computer vision. Google Scholar
  20. Hua, J., Lai, Z., Dong, M., Gu, X., & Qin, H. (2008). Geodesic distance-weighted shape vector image diffusion. IEEE Transactions on Visualization and Computer Graphics, 14(6), 1643–1650. CrossRefGoogle Scholar
  21. Johnson, A. E., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449. CrossRefGoogle Scholar
  22. Kimmel, R., & Sethian, J. (1998). Computing geodesic paths on manifolds. In Proceedings of national academy of science (pp. 8431–8435). Google Scholar
  23. Kläser, A., Marszałek, M., & Schmid, C. (2008). A spatio-temporal descriptor based on 3D-gradients. In Proceedings of the British machine vision conference. Google Scholar
  24. Körtgen, M., Park, G. J., Novotny, M., & Klein, R. (2003) 3D shape matching with 3D shape contexts. Central European Seminar on Computer Graphics. Google Scholar
  25. Kovnatsky, A., Bronstein, M. M., Bronstein, A. M., & Kimmel, R. (2011). Photometric heat kernel signatures. In Proceedings of conference on scale space and variational methods in computer vision. Google Scholar
  26. Laptev, I. (2005). On space-time interest points. International Journal of Computer Vision, 64(2–3), 107–123. CrossRefGoogle Scholar
  27. Lay, D. (1996). Linear algebra and its applications. Reading: Addison-Wesley. Google Scholar
  28. Lee, C. H., Varshney, A., & Jacobs, D. (2005) Mesh saliency. Proceedings of SIGGRAPH. Google Scholar
  29. Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110. CrossRefGoogle Scholar
  30. Luo, C., Safa, I., & Wang, Y. (2009). Approximating gradients for meshes and point clouds via diffusion metric. In Proceedings of the Eurographics symposium on geometry processing. Google Scholar
  31. Marr, D., & Hildreth, E. (1980). Theory of edge detection. Proceedings of the Royal Society of London, B207, 187–217. CrossRefGoogle Scholar
  32. Matas, J., Chum, O., Urban, M., & Pajdla, T. (2004). Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 22(10), 761–767. CrossRefGoogle Scholar
  33. Mavriplis, D. (2003). Revisiting the least-squares procedure for gradient reconstruction on unstructured meshes. In Proc. of the 16th AIAA computational fluid dynamics conference, Orlando, FL. Google Scholar
  34. Meyer, M., Desbrun, M., Schröder, P., & Barr, A. H. (2002). Discrete differential geometry operators for triangulated 2-dimensional manifolds. In Proceedings of VisMath. Google Scholar
  35. Mian, A., Bennamoun, M., & Owens, R. (2010). On the repeatability and quality of keypoints for local feature-based 3d object retrieval from cluttered scenes. International Journal of Computer Vision, 89, 348–361. CrossRefGoogle Scholar
  36. Mikolajczyk, K., & Schmid, C. (2004). Scale & affine invariant interest point detectors. International Journal of Computer Vision, 60(1), 63–86. CrossRefGoogle Scholar
  37. Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615–1630. CrossRefGoogle Scholar
  38. Mukherjee, S., Wu, Q., & Zhou, D. X. (2010). Learning gradients on manifolds. Bernoulli, 16(1), 181–207. MathSciNetMATHCrossRefGoogle Scholar
  39. Novatnack, J., & Nishino, K. (2007). Scale-dependent 3D geometric features. In Proceedings of international conference on computer vision. Google Scholar
  40. Novatnack, J., & Nishino, K. (2008). Scale-dependent/invariant local 3D shape descriptors for fully automatic registration of multiple sets of range images. In Proceedings of European conference on computer vision (Vol. III, pp. 440–453). Google Scholar
  41. Rothganger, F., Lazebnik, S., Schmid, C., & Ponce, J. (2006). 3D object modeling and recognition using local affine-invariant image descriptors and multi-view spatial constraints. International Journal of Computer Vision, 66(3), 231–259. CrossRefGoogle Scholar
  42. Ruggeri, M. R., Patanè, G., Spagnuolo, M., & Saupe, D. (2010). Spectral-driven isometry-invariant matching of 3D shapes. International Journal of Computer Vision, 89(2–3), 248–265. CrossRefGoogle Scholar
  43. Schlattmann, M., Degener, P., & Klein, R. (2008). Scale space based feature point detection on surfaces. Journal of WSCG, 16(1–3). Google Scholar
  44. Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 519–526). Google Scholar
  45. Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2008). The Princeton shape benchmark. In Shape modeling international. Google Scholar
  46. Sibson, R. (1981). A brief description of natural neighbour interpolation (Vol. 21, pp. 21–36). New York: Wiley. Google Scholar
  47. Sipiran, I., & Bustos, B. (2010). A robust 3D interest points detector based on Harris operator. In 3DOR (pp. 7–14). Google Scholar
  48. Smith, E. R., Radke, R. J., & Stewart, C. V. (2011). Physical scale keypoints: Matching and registration for combined intensity/range images. International Journal of Computer Vision. Google Scholar
  49. Starck, J., & Hilton, A. (2007). Correspondence labelling for wide-time free-form surface matching. In Proceedings of international conference on computer vision. Google Scholar
  50. Sun, J., Ovsjanikov, M., & Guibas, L. (2009). A concise and provably informative multi-scale signature based on heat diffusion. In Proceedings of the symposium on geometry processing. Google Scholar
  51. Surazhsky, V., Surazhsky, T., Kirsanov, D., Gortler, S., & Hoppe, H. (2005). Fast exact and approximate geodesics on meshes. Proceedings of SIGGRAPH. Google Scholar
  52. Tangelder, J. W. H., & Veltkamp, R. C. (2004) A survey of content based 3D shape retrieval methods. In Shape modeling international (pp. 145–156). Google Scholar
  53. Varanasi, K., Zaharescu, A., Boyer, E., & Horaud, R. P. (2008). Temporal surface tracking using mesh evolution. In Proceedings of European conference on computer vision. Google Scholar
  54. Wong, S. F., & Cipolla, R. (2007). Extracting spatiotemporal interest points using global information. In Proceedings of international conference on computer vision. Google Scholar
  55. Wu, C., Clipp, B., Li, X., Frahm, J. M., & Pollefeys, M. (2008). 3D model matching with viewpoint invariant patches (vips). In Proceedings of IEEE conference on computer vision and pattern recognition. Google Scholar
  56. Xu, G. (2004). Convergent discrete Laplace-Beltrami operators over triangular surfaces. In Proceedings of geometric modeling and processing (pp. 195–204). Google Scholar
  57. Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In Proceedings of IEEE conference on computer vision and pattern recognition, Miami, USA (pp. 373–380). Google Scholar
  58. Zaharescu, A., Boyer, E., & Horaud, R. (2011). Topology-adaptive mesh deformation for surface evolution, morphing, and multiview reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 823–837. CrossRefGoogle Scholar
  59. Zhong, Y. (2009). Intrinsic shape signatures: A shape descriptor for 3D object recognition. In IEEE international conference on computer vision (3D) representation and recognition workshop. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Aimetis CorporationWaterlooCanada
  2. 2.INRIA Grenoble Rhône-AlpesMontbonnot Saint-MartinFrance

Personalised recommendations