Skip to main content
SpringerLink
Log in

Keypoints and Local Descriptors of Scalar Functions on 2D Manifolds

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

This paper addresses the problem of describing surfaces using local features and descriptors. While methods for the detection of interest points in images and their description based on local image features are very well understood, their extension to discrete manifolds has not been well investigated. We provide a methodological framework for analyzing real-valued functions defined over a 2D manifold, embedded in the 3D Euclidean space, e.g., photometric information, local curvature, etc. Our work is motivated by recent advancements in multiple-camera reconstruction and image-based rendering of 3D objects: there is a growing need for describing object surfaces, matching two surfaces, or tracking them over time. Considering polygonal meshes, we propose a new methodological framework for the scale-space representations of scalar functions defined over such meshes. We propose a local feature detector (MeshDOG) and region descriptor (MeshHOG). Unlike the standard image features, the proposed surface features capture both the local geometry of the underlying manifold and the scale-space differential properties of the real-valued function itself. We provide a thorough experimental evaluation. The repeatability of the feature detector and the robustness of feature descriptor are tested, by applying a large number of deformations to the manifold or to the scalar function.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://shape.cs.princeton.edu/benchmark/.

  2. http://mvviewer.gforge.inria.fr.

  3. http://tosca.cs.technion.ac.il/book/shrec_feat.html.

  4. http://4drepository.inrialpes.fr/.

References

  • de Aguiar, E., Theobalt, C., Stoll, C., & Seidel, H. P. (2007). Marker-less 3D feature tracking for mesh-based human motion capture. In Human motion—understanding, modeling, capture and animation (pp. 1–15).

    Chapter  Google Scholar 

  • Ahmed, N., Theobalt, C., Rossl, C., Thrun, S., & Seidel, H. P. (2008). Dense correspondence finding for parametrization-free animation reconstruction from video. In Proceedings of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Bariya, P., & Nishino, K. (2010). Scale-hierarchical 3d object recognition in cluttered scenes. In Proc. of IEEE computer vision and pattern recognition (pp. 1657–1664).

    Google Scholar 

  • Barth, T. (1993). A 3-D least-squares upwind Euler solver for unstructured meshes. In Lecture notes in physics: Vol. 414. Thirteenth international conference on numerical methods in fluid dynamics (pp. 240–244). Berlin: Springer.

    Chapter  Google Scholar 

  • Bay, H., Ess, A., Tuytelaars, T., & Gool, L. V. (2008). Speeded-up robust features (SURF). Computer Vision and Image Understanding, 110(3), 346–359.

    Article  Google Scholar 

  • Bolles, R. C., & Cain, RA (1982). Recognizing and locating partially visible objects, the Local-Feature-Focus method. The International Journal of Robotics Research, 1(3), 57–82.

    Article  Google Scholar 

  • Bolles, R. C., & Horaud, R. (1986). 3DPO: A three-dimensional part orientation system. The International Journal of Robotics Research, 5(3), 3–26.

    Article  Google Scholar 

  • Bronstein, A. M., Bronstein, M. M., Bustos, B., Castellani, U., Crisani, M., Falcidieno, B., Guibas, L. J., Kokkinos, I., Murino, V., Ovsjanikov, M., Patané, G., Sipiran, I., Spagnuolo, M., & Sun, J. (2010). Shrec 2010: robust feature detection and description benchmark. In Proc. EUROGRAPHICS workshop on 3D object retrieval (3DOR).

    Google Scholar 

  • Bronstein, A. M., Bronstein, M. M., Ovsjanikov, M., & Guibas, L. J. (2011). Shape Google: geometric words and expressions for invariant shape retrieval. ACM Transactions on Graphics, 30(1), 1–20.

    Article  Google Scholar 

  • Bustos, B., Keim, D. A., Saupe, D., Schreck, T., & Vranic, D. V. (2005). Feature-based similarity search in 3D object databases. ACM Computing Surveys, 34(4), 345–387.

    Article  Google Scholar 

  • Cagniart, C., Boyer, E., & Ilic, S. (2010). Probabilistic deformable surface tracking from multiple videos. In Proceedings of European conference on computer vision.

    Google Scholar 

  • Castellani, U., Cristani, M., Fantoni, S., & Murino, V. (2008). Sparse points matching by combining 3D mesh saliency with statistical descriptors. Computer Graphics Forum, 27(2), 643–652.

    Article  Google Scholar 

  • Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 886–893).

    Google Scholar 

  • Dong, C. S., & Wang, G. Z. (2005). Curvatures estimation on triangular mesh. Journal of Zhejiang University SCIENCE, 6A(1), 128–136.

    MATH  Google Scholar 

  • Dufournaud, Y., Schmid, C., & Horaud, R. P. (2004). Image matching with scale adjustment. Computer Vision and Image Understanding, 93(2), 175–194.

    Article  Google Scholar 

  • Frome, A., Huber, D., Kolluri, R., Bulow, T., & Malik, J. (2004). Recognizing objects in range data using regional point descriptors. In Proceedings of European conference on computer vision.

    Google Scholar 

  • Furukawa, Y., & Ponce, J. (2008). Dense 3D motion capture from synchronized video streams. In Proceedings of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Horn, R. A., & Johnson, C. A. (1994). Matrix analysis. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Hou, T., & Qin, H. (2010). Efficient computation of scale-space features for deformable shape correspondences. In Proceedings of European conference on computer vision.

    Google Scholar 

  • Hua, J., Lai, Z., Dong, M., Gu, X., & Qin, H. (2008). Geodesic distance-weighted shape vector image diffusion. IEEE Transactions on Visualization and Computer Graphics, 14(6), 1643–1650.

    Article  Google Scholar 

  • Johnson, A. E., & Hebert, M. (1999). Using spin images for efficient object recognition in cluttered 3D scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(5), 433–449.

    Article  Google Scholar 

  • Kimmel, R., & Sethian, J. (1998). Computing geodesic paths on manifolds. In Proceedings of national academy of science (pp. 8431–8435).

    Google Scholar 

  • Kläser, A., Marszałek, M., & Schmid, C. (2008). A spatio-temporal descriptor based on 3D-gradients. In Proceedings of the British machine vision conference.

    Google Scholar 

  • Körtgen, M., Park, G. J., Novotny, M., & Klein, R. (2003) 3D shape matching with 3D shape contexts. Central European Seminar on Computer Graphics.

  • Kovnatsky, A., Bronstein, M. M., Bronstein, A. M., & Kimmel, R. (2011). Photometric heat kernel signatures. In Proceedings of conference on scale space and variational methods in computer vision.

    Google Scholar 

  • Laptev, I. (2005). On space-time interest points. International Journal of Computer Vision, 64(2–3), 107–123.

    Article  Google Scholar 

  • Lay, D. (1996). Linear algebra and its applications. Reading: Addison-Wesley.

    Google Scholar 

  • Lee, C. H., Varshney, A., & Jacobs, D. (2005) Mesh saliency. Proceedings of SIGGRAPH.

  • Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2), 91–110.

    Article  Google Scholar 

  • Luo, C., Safa, I., & Wang, Y. (2009). Approximating gradients for meshes and point clouds via diffusion metric. In Proceedings of the Eurographics symposium on geometry processing.

    Google Scholar 

  • Marr, D., & Hildreth, E. (1980). Theory of edge detection. Proceedings of the Royal Society of London, B207, 187–217.

    Article  Google Scholar 

  • Matas, J., Chum, O., Urban, M., & Pajdla, T. (2004). Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 22(10), 761–767.

    Article  Google Scholar 

  • Mavriplis, D. (2003). Revisiting the least-squares procedure for gradient reconstruction on unstructured meshes. In Proc. of the 16th AIAA computational fluid dynamics conference, Orlando, FL.

    Google Scholar 

  • Meyer, M., Desbrun, M., Schröder, P., & Barr, A. H. (2002). Discrete differential geometry operators for triangulated 2-dimensional manifolds. In Proceedings of VisMath.

    Google Scholar 

  • Mian, A., Bennamoun, M., & Owens, R. (2010). On the repeatability and quality of keypoints for local feature-based 3d object retrieval from cluttered scenes. International Journal of Computer Vision, 89, 348–361.

    Article  Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2004). Scale & affine invariant interest point detectors. International Journal of Computer Vision, 60(1), 63–86.

    Article  Google Scholar 

  • Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615–1630.

    Article  Google Scholar 

  • Mukherjee, S., Wu, Q., & Zhou, D. X. (2010). Learning gradients on manifolds. Bernoulli, 16(1), 181–207.

    Article  MathSciNet  MATH  Google Scholar 

  • Novatnack, J., & Nishino, K. (2007). Scale-dependent 3D geometric features. In Proceedings of international conference on computer vision.

    Google Scholar 

  • Novatnack, J., & Nishino, K. (2008). Scale-dependent/invariant local 3D shape descriptors for fully automatic registration of multiple sets of range images. In Proceedings of European conference on computer vision (Vol. III, pp. 440–453).

    Google Scholar 

  • Rothganger, F., Lazebnik, S., Schmid, C., & Ponce, J. (2006). 3D object modeling and recognition using local affine-invariant image descriptors and multi-view spatial constraints. International Journal of Computer Vision, 66(3), 231–259.

    Article  Google Scholar 

  • Ruggeri, M. R., Patanè, G., Spagnuolo, M., & Saupe, D. (2010). Spectral-driven isometry-invariant matching of 3D shapes. International Journal of Computer Vision, 89(2–3), 248–265.

    Article  Google Scholar 

  • Schlattmann, M., Degener, P., & Klein, R. (2008). Scale space based feature point detection on surfaces. Journal of WSCG, 16(1–3).

  • Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., & Szeliski, R. (2006). A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 1, pp. 519–526).

    Google Scholar 

  • Shilane, P., Min, P., Kazhdan, M., & Funkhouser, T. (2008). The Princeton shape benchmark. In Shape modeling international.

    Google Scholar 

  • Sibson, R. (1981). A brief description of natural neighbour interpolation (Vol. 21, pp. 21–36). New York: Wiley.

    Google Scholar 

  • Sipiran, I., & Bustos, B. (2010). A robust 3D interest points detector based on Harris operator. In 3DOR (pp. 7–14).

    Google Scholar 

  • Smith, E. R., Radke, R. J., & Stewart, C. V. (2011). Physical scale keypoints: Matching and registration for combined intensity/range images. International Journal of Computer Vision.

  • Starck, J., & Hilton, A. (2007). Correspondence labelling for wide-time free-form surface matching. In Proceedings of international conference on computer vision.

    Google Scholar 

  • Sun, J., Ovsjanikov, M., & Guibas, L. (2009). A concise and provably informative multi-scale signature based on heat diffusion. In Proceedings of the symposium on geometry processing.

    Google Scholar 

  • Surazhsky, V., Surazhsky, T., Kirsanov, D., Gortler, S., & Hoppe, H. (2005). Fast exact and approximate geodesics on meshes. Proceedings of SIGGRAPH.

  • Tangelder, J. W. H., & Veltkamp, R. C. (2004) A survey of content based 3D shape retrieval methods. In Shape modeling international (pp. 145–156).

    Google Scholar 

  • Varanasi, K., Zaharescu, A., Boyer, E., & Horaud, R. P. (2008). Temporal surface tracking using mesh evolution. In Proceedings of European conference on computer vision.

    Google Scholar 

  • Wong, S. F., & Cipolla, R. (2007). Extracting spatiotemporal interest points using global information. In Proceedings of international conference on computer vision.

    Google Scholar 

  • Wu, C., Clipp, B., Li, X., Frahm, J. M., & Pollefeys, M. (2008). 3D model matching with viewpoint invariant patches (vips). In Proceedings of IEEE conference on computer vision and pattern recognition.

    Google Scholar 

  • Xu, G. (2004). Convergent discrete Laplace-Beltrami operators over triangular surfaces. In Proceedings of geometric modeling and processing (pp. 195–204).

    Google Scholar 

  • Zaharescu, A., Boyer, E., Varanasi, K., & Horaud, R. (2009). Surface feature detection and description with applications to mesh matching. In Proceedings of IEEE conference on computer vision and pattern recognition, Miami, USA (pp. 373–380).

    Google Scholar 

  • Zaharescu, A., Boyer, E., & Horaud, R. (2011). Topology-adaptive mesh deformation for surface evolution, morphing, and multiview reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(4), 823–837.

    Article  Google Scholar 

  • Zhong, Y. (2009). Intrinsic shape signatures: A shape descriptor for 3D object recognition. In IEEE international conference on computer vision (3D) representation and recognition workshop.

    Google Scholar 

Download references

Acknowledgements

We would like to thank Cedric Cagniart and Artiom Kovnatsky for their help with the datasets.

Author information

Authors and Affiliations

  1. Aimetis Corporation, Waterloo, Ontario, Canada

    Andrei Zaharescu

  2. INRIA Grenoble Rhône-Alpes, Montbonnot Saint-Martin, France

    Edmond Boyer & Radu Horaud

Authors
  1. Andrei Zaharescu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edmond Boyer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Radu Horaud
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Andrei Zaharescu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zaharescu, A., Boyer, E. & Horaud, R. Keypoints and Local Descriptors of Scalar Functions on 2D Manifolds. Int J Comput Vis 100, 78–98 (2012). https://doi.org/10.1007/s11263-012-0528-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-012-0528-5

Keywords

Search

Navigation