High Accuracy Optical Flow Serves 3-D Pose Tracking: Exploiting Contour and Flow Based Constraints

  • Thomas Brox
  • Bodo Rosenhahn
  • Daniel Cremers
  • Hans-Peter Seidel
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3952)


Tracking the 3-D pose of an object needs correspondences between 2-D features in the image and their 3-D counterparts in the object model. A large variety of such features has been suggested in the literature. All of them have drawbacks in one situation or the other since their extraction in the image and/or the matching is prone to errors. In this paper, we propose to use two complementary types of features for pose tracking, such that one type makes up for the shortcomings of the other. Aside from the object contour, which is matched to a free-form object surface, we suggest to employ the optic flow in order to compute additional point correspondences. Optic flow estimation is a mature research field with sophisticated algorithms available. Using here a high quality method ensures a reliable matching. In our experiments we demonstrate the performance of our method and in particular the improvements due to the optic flow.


Block Match Point Correspondence Contour Extraction Constancy Assumption Pose Estimation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Agarwal, A., Triggs, B.: Tracking Articulated Motion Using a Mixture of Autoregressive Models. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3023, pp. 54–65. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  2. 2.
    Araújo, H., Carceroni, R.L., Brown, C.M.: A fully projective formulation to improve the accuracy of Lowe’s pose-estimation algorithm. Computer Vision and Image Understanding 70(2), 227–238 (1998)CrossRefGoogle Scholar
  3. 3.
    Bregler, C., Malik, J., Pullen, K.: Twist based acquisition and tracking of animal and human kinematics. International Journal of Computer Vision 56(3), 179–194 (2004)CrossRefGoogle Scholar
  4. 4.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3024, pp. 25–36. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  5. 5.
    Brox, T., Rosenhahn, B., Weickert, J.: Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Estimation. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 109–116. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  6. 6.
    Brox, T., Weickert, J.: A TV flow based local scale measure for texture discrimination. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3022, pp. 578–590. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  7. 7.
    Chan, T., Vese, L.: Active contours without edges. IEEE Transactions on Image Processing 10(2), 266–277 (2001)CrossRefMATHGoogle Scholar
  8. 8.
    David, P., DeMenthon, D., Duraiswami, R., Samet, H.: Simultaneous pose and correspondence determination using line features. In: Proc. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 424–431 (2003)Google Scholar
  9. 9.
    DeCarlo, D., Metaxas, D.: Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision 38(2), 99–127 (2000)CrossRefMATHGoogle Scholar
  10. 10.
    Fua, P., Plänkers, R., Thalmann, D.: Tracking and modeling people in video sequences. Computer Vision and Image Understanding 81(3), 285–302 (2001)CrossRefMATHGoogle Scholar
  11. 11.
    Goddard, J.: Pose and motion estimation from vision using dual quaternion-based extended Kalman filtering. Technical report, University of Tennessee, Knoxville (1997)Google Scholar
  12. 12.
    Grimson, W.E.L.: Object Recognition by Computer. MIT Press, Cambridge (1990)Google Scholar
  13. 13.
    Horn, B., Schunck, B.: Determining optical flow. Artificial Intelligence 17, 185–203 (1981)CrossRefGoogle Scholar
  14. 14.
    Koch, R.: Dynamic 3D scene analysis through synthesis feedback control. IEEE Transactions on Pattern Analysis and Machine Intelligence 15(6), 556–568 (1993)CrossRefGoogle Scholar
  15. 15.
    Lowe, D.: Solving for the parameters of object models from image descriptions. In: Proc. ARPA Image Understanding Workshop, pp. 121–127 (1980)Google Scholar
  16. 16.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Marchand, E., Bouthemy, P., Chaumette, F.: A 2D-3D model-based approach to real-time visual tracking. Image and Vision Computing 19(13), 941–955 (2001)CrossRefGoogle Scholar
  18. 18.
    Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence 27(10), 1615–1630 (2005)CrossRefGoogle Scholar
  19. 19.
    Murray, R., Li, Z., Sastry, S.: Mathematical Introduction to Robotic Manipulation. CRC Press, Boca Raton (1994)MATHGoogle Scholar
  20. 20.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations. Journal of Computational Physics 79, 12–49 (1988)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Paragios, N., Deriche, R.: Geodesic active regions: A new paradigm to deal with frame partition problems in computer vision. Journal of Visual Communication and Image Representation 13(1/2), 249–268 (2002)CrossRefGoogle Scholar
  22. 22.
    Rosenhahn, B.: Pose estimation revisited. Technical Report TR-0308, Institute of Computer Science, University of Kiel, Germany (October 2003)Google Scholar
  23. 23.
    Rosenhahn, B., Perwass, C., Sommer, G.: Pose estimation of free-form contours. International Journal of Computer Vision 62(3), 267–289 (2005)CrossRefMATHGoogle Scholar
  24. 24.
    Rosenhahn, B., Sommer, G.: Pose Estimation of Free-Form Objects. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 414–427. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  25. 25.
    Shevlin, F.: Analysis of orientation problems using Plücker lines. In: International Conference on Pattern Recognition (ICPR), Brisbane, vol. 1, pp. 685–689 (1998)Google Scholar
  26. 26.
    Vacchetti, L., Lepetit, V., Fua, P.: Combining edge and texture information for real-time accurate 3D camera tracking. In: 3rd International Symposium on Mixed and Augmented Reality, pp. 48–57 (2004)Google Scholar
  27. 27.
    Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3D tracking using online and offline information. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(10), 1391–1391 (2004)CrossRefGoogle Scholar
  28. 28.
    Zang, Z.: Iterative point matching for registration of free-form curves and surfaces. International Journal of Computer Vision 13(2), 119–152 (1999)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Thomas Brox
    • 1
  • Bodo Rosenhahn
    • 2
  • Daniel Cremers
    • 1
  • Hans-Peter Seidel
    • 2
  1. 1.CVPR Group, Department of Computer ScienceUniversity of BonnBonnGermany
  2. 2.Max Planck Center for Visual Computing and CommunicationSaarbrückenGermany

Personalised recommendations