Robust 3D Pose Estimation and Efficient 2D Region-Based Segmentation from a 3D Shape Prior

  • Samuel Dambreville
  • Romeil Sandhu
  • Anthony Yezzi
  • Allen Tannenbaum
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5303)


In this work, we present an approach to jointly segment a rigid object in a 2D image and estimate its 3D pose, using the knowledge of a 3D model. We naturally couple the two processes together into a unique energy functional that is minimized through a variational approach. Our methodology differs from the standard monocular 3D pose estimation algorithms since it does not rely on local image features. Instead, we use global image statistics to drive the pose estimation process. This confers a satisfying level of robustness to noise and initialization for our algorithm, and bypasses the need to establish correspondences between image and object features. Moreover, our methodology possesses the typical qualities of region-based active contour techniques with shape priors, such as robustness to occlusions or missing information, without the need to evolve an infinite dimensional curve. Another novelty of the proposed contribution is to use a unique 3D model surface of the object, instead of learning a large collection of 2D shapes to accommodate for the diverse aspects that a 3D object can take when imaged by a camera. Experimental results on both synthetic and real images are provided, which highlight the robust performance of the technique on challenging tracking and segmentation applications.


Segmentation Result Active Contour Rigid Object Shape Prior Local Image Feature 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Quan, L., Lan, Z.D.: Linear n-point camera pose determination. IEEE Transactions on Pattern Analysis and Machine Intelligence 21, 774–780 (1999)CrossRefGoogle Scholar
  2. 2.
    Dhome, M., Richetin, M., Lapreste, J.T.: Determination of the attitude of 3d objects from a single perspective view. IEEE Trans. Pattern Anal. Mach. Intell. 11, 1265–1278 (1989)CrossRefGoogle Scholar
  3. 3.
    Marchand, E., Bouthemy, P., Chaumette, F.: A 2d-3d model-based approach to real-time visual tracking. Image and Vision Computing 19, 941–955 (2001)CrossRefGoogle Scholar
  4. 4.
    Zerroug, M., Nevatia, R.: Pose estimation of multi-part curved objects. In: ISCV 1995: Proceedings of the International Symposium on Computer Vision, p. 431 (1995)Google Scholar
  5. 5.
    Rosenhahn, B., Perwass, C., Sommer, G.: Pose estimation of free-form contours. IJCV 62, 267–289 (2005)CrossRefzbMATHGoogle Scholar
  6. 6.
    Drummond, T., Cipolla, R.: Real-time tracking of multiple articulated structures in multiple views. In: Proc. 6th European Conf.on Computer Vision, ECCV, pp. 20–36 (2000)Google Scholar
  7. 7.
    Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. IJCV, 22, 61–79 (1997)Google Scholar
  8. 8.
    Kichenassamy, S., Kumar, S., Olver, P., Tannenbaum, A., Yezzi, A.: Conformal curvature flow: From phase transitions to active vision. Archives for Rational Mechanics and Analysis 134, 275–301 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Zhu, S.C., Yuille, A.L.: Region competition: Unifying snakes, region growing, and Bayes/MDL for multiband image segmentation. IEEE Trans. PAMI 18, 884–900 (1996)CrossRefGoogle Scholar
  10. 10.
    Chan, T., Vese, L.: Active contours without edges. IEEE TIP 10, 266–277 (2001)zbMATHGoogle Scholar
  11. 11.
    Paragios, N., Deriche, R.: Geodesic active regions: A new paradigm to deal with frame partition problems in computer vision. Journal of Visual Communication and Image Representation 13, 249–268 (2002)CrossRefGoogle Scholar
  12. 12.
    Dambreville, S., Yezzi, A., Niethammer, M., Tannenbaum, A.: A variational framework combining level-sets and thresholding. In: BMVC, pp. 266–280 (2007)Google Scholar
  13. 13.
    Michailovich, O., Rathi, Y., Tannenbaum, A.: Image segmentation using active contours driven by the bhattacharyya gradient flow. IEEE TIP, 2787–2801 (2007)Google Scholar
  14. 14.
    Kim, J., Fisher, J., Yezzi, A., Cetin, M., Willsky, A.: Nonparametric methods for image segmentation using information theory and curve evolution. In: Proc. ICIP, vol. 3, pp. 797–800 (2002)Google Scholar
  15. 15.
    Paragios, N., Deriche, R.: Geodesic active regions for supervised texture segmentation. In: ICCV (2), pp. 926–932 (1999)Google Scholar
  16. 16.
    Leventon, M., Grimson, E., Faugeras, O.: Statistical shape influence in geodesic active contours. In: Proc. IEEE CVPR, pp. 1316–1324 (2000)Google Scholar
  17. 17.
    Tsai, A., Yezzi, T., Wells, W., Tempany, C., Tucker, D., Fan, A., Grimson, E., Willsky, A.: A shape-based approach to the segmentation of medical imagery using level sets. IEEE Trans. on Medical Imaging 22, 137–153 (2003)CrossRefGoogle Scholar
  18. 18.
    Cremers, D., Kohlberger, T., Schnoerr, C.: Shape statistics in kernel space for variational image segmentation. Pattern Recognition 36, 1292–1943 (2003)CrossRefGoogle Scholar
  19. 19.
    Dambreville, S., Rathi, Y., Tannenbaum, A.: Shape-based approach to robust image segmentation using kernel pca. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 977–984 (2006)Google Scholar
  20. 20.
    Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Heidelberg (2003)CrossRefzbMATHGoogle Scholar
  21. 21.
    Turk, G., Levoy, M.: Zippered polygon meshes from range images. In: Siggraph, pp. 311–318 (1994)Google Scholar
  22. 22.
    Yezzi, A., Soatto, S.: Structure from motion for scenes without features. In: Proc. IEEE CVPR, vol. 1, pp. 171–178 (2003)Google Scholar
  23. 23.
    Yezzi, A., Soatto, S.: Stereoscopic segmentation. International Journal of Computer Vision (IJCV) 53, 31–43 (2003)CrossRefGoogle Scholar
  24. 24.
    Faugeras, O.D., Keriven, R.: Variational principles, surface evolution pdes, level set methods and the stereo problem. INRIA Tech. report 3021, 1–37 (1996)zbMATHGoogle Scholar
  25. 25.
    Unal, G., Yezzi, A., Soatto, S., Slabaugh, G.: A variational approach to problems in calibration of multiple cameras. Trans. Pattern Analysis and Machine Intelligence 29, 1322–1338 (2007)CrossRefGoogle Scholar
  26. 26.
    Rosenhahn, B., Brox, T., Weickert, J.: Three-dimensional shape knowledge for joint image segmentation and pose tracking. IJCV 73, 243–262 (2007)CrossRefGoogle Scholar
  27. 27.
    Schmaltz, C., Rosenhahn, B., Brox, T., Cremers, D., Weickert, J., Wietzke, L., Sommer, G.: Region-based pose tracking. In: Pattern Recognition and Image Analysis, pp. 56–63 (2007)Google Scholar
  28. 28.
    DoCarmo, M.P.: Diferential Geometry of Curves and Surfaces. Prentice Hall, Englewood Cliffs (1976)Google Scholar
  29. 29.
    Lepetit, V., Fua, P.: Monocular model-based 3d tracking of rigid objects: A survey. Fundations and Trends in Computer Graphics and Vision 1, 1–89 (2005)CrossRefGoogle Scholar
  30. 30.
    Forsyth, D., Ponce, J.: Computer Vision. Prentice Hall, Englewood Cliffs (2003)Google Scholar
  31. 31.
    Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  32. 32.
    Ma, Y., Soatto, S., Kosecka, J., Sastry, S.: An invitation to 3D vision. Springer, HeidelbergGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Samuel Dambreville
    • 1
  • Romeil Sandhu
    • 1
  • Anthony Yezzi
    • 1
  • Allen Tannenbaum
    • 1
  1. 1.Georgia Institute of TechnologyUSA

Personalised recommendations