The Visual Computer

, Volume 31, Issue 6–8, pp 979–988 | Cite as

Global optimal searching for textureless 3D object tracking

  • Guofeng Wang
  • Bin Wang
  • Fan Zhong
  • Xueying QinEmail author
  • Baoquan Chen
Original Article


Textureless 3D object tracking of the object’s position and orientation is a considerably challenging problem, for which a 3D model is commonly used. The 3D–2D correspondence between a known 3D object model and 2D scene edges in an image is standardly used to locate the 3D object, one of the most important problems in model-based 3D object tracking. State-of-the-art methods solve this problem by searching correspondences independently. However, this often fails in highly cluttered backgrounds, owing to the presence of numerous local minima. To overcome this problem, we propose a new method based on global optimization for searching these correspondences. With our search mechanism, a graph model based on an energy function is used to establish the relationship of the candidate correspondences. Then, the optimal correspondences can be efficiently searched with dynamic programming. Qualitative and quantitative experimental results demonstrate that the proposed method performs favorably compared to the state-of-the-art methods in highly cluttered backgrounds.


3D tracking 3D–2D correspondence  Global optimization Dynamic programming 



The authors gratefully acknowledge the anonymous reviewers for their comments to help us to improve our paper, and also thank for their enormous help in revising this paper. This work is supported by 973 program of China (No. 2015CB352500), 863 program of China (No. 2015AA016405), and NSF of China (Nos. 61173070, 61202149).


  1. 1.
    Bresenham, J.E.: Algorithm for computer control of a digital plotter. IBM Syst. J. 4(1), 25–30 (1965)CrossRefGoogle Scholar
  2. 2.
    Brown, J., Capson, D.: A framework for 3d model-based visual tracking using a gpu-accelerated particle filter. IEEE Trans. Vis. Comput. Graph. 18(1), 68–80 (2012)CrossRefGoogle Scholar
  3. 3.
    Comport, A., Marchand, E., Pressigout, M., Chaumette, F.: Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Trans. Vis. Comput. Graph. 12(4), 615–628 (2006)CrossRefGoogle Scholar
  4. 4.
    Dambreville, S., Sandhu, R., Yezzi, A., Tannenbaum, A.: Robust 3d pose estimation and efficient 2d region-based segmentation from a 3d shape prior. In: Proceedings of European Conference on Computer Vision, pp 169–182 (2008)Google Scholar
  5. 5.
    Drummond, T., Cipolla, R.: Real-time visual tracking of complex structures. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 932–946 (2002)CrossRefGoogle Scholar
  6. 6.
    Harris, C., Stennett, C.: Rapid: a video-rate object tracker. In: Proceedings of British Machine Vision Conference, pp. 73–77 (1990)Google Scholar
  7. 7.
    Hinterstoisser, S., Benhimane, S., Navab, N.: N3m: natural 3d markers for real-time object detection and pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1–7 (2007)Google Scholar
  8. 8.
    Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P., Navab, N., Fua, P., Lepetit, V.: Gradient response maps for real-time detection of texture-less objects. IEEE Trans. Pattern Anal. Mach. Intell. 34(5), 876–888 (2012)CrossRefGoogle Scholar
  9. 9.
    Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., Fitzgibbon, A.: Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. ACM Symposium on User Interface Software and Technology (2011)Google Scholar
  10. 10.
    Kato, H., Billinghurst, M.: Marker tracking and hmd calibration for a video-based augmented reality conferencing system. In: IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 85–94 (1999)Google Scholar
  11. 11.
    Kim, K., Lepetit, V., Woo, W.: Scalable real-time planar targets tracking for digilog books. Vis. Comput. 26(6–8), 1145–1154 (2010)CrossRefGoogle Scholar
  12. 12.
    Klein, G., Murray, D.: Full-3d edge tracking with a particle filter. In: Proceedings of British Machine Vision Conference, pp. 114.1-114.10 (2006)Google Scholar
  13. 13.
    Lepetit, V., Fua, P.: Monocular model-based 3d tracking of rigid objects: a survey. Found. Trends Comput. Graph. Vis. 1(1), 1–89 (2005)CrossRefGoogle Scholar
  14. 14.
    Ma, Z., Wu, E.: Real-time and robust hand tracking with a single depth camera. Vis. Comput. 30(10), 1133–1144 (2014)CrossRefGoogle Scholar
  15. 15.
    Marchand, E., Bouthemy, P., Chaumette, F.: A 2d–3d model-based approach to real-time visual tracking. Image Vis. Comput. 19(13), 941–955 (2001)CrossRefGoogle Scholar
  16. 16.
    Prisacariu, V.A., Reid, I.D.: Pwp3d: real-time segmentation and tracking of 3d objects. Int. J. Comput. Vis. 98(3), 335–354 (2012)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Rosenhahn, B., Brox, T., Weickert, J.: Three-dimensional shape knowledge for joint image segmentation and pose tracking. Int. J. Comput. Vis. 73(3), 243–262 (2007)CrossRefGoogle Scholar
  18. 18.
    Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1508–1515 (2005)Google Scholar
  19. 19.
    Schmaltz, C., Rosenhahn, B., Brox, T., Weickert, J.: Region-based pose tracking with occlusions using 3d models. Mach. Vis. Appl. 23(3), 557–577 (2012)CrossRefGoogle Scholar
  20. 20.
    Seo, B., Park, H., Park, J., Hinterstoisser, S., Llic, S.: Optimal local searching for fast and robust textureless 3d object tracking in highly cluttered backgrounds. IEEE Trans. Vis. Comput. Graph. 20(1), 99–110 (2014)CrossRefGoogle Scholar
  21. 21.
    Vacchetti, L., Lepetit, V., Fua, P.: Combining edge and texture information for real-time accurate 3d camera tracking. In: IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 48–56 (2004)Google Scholar
  22. 22.
    Vacchetti, L., Lepetit, V., Fua, P.: Stable real-time 3d tracking using online and offline information. IEEE Trans. Pattern Anal. Mach. Intell. 26(10), 1385–1391 (2004)CrossRefGoogle Scholar
  23. 23.
    Wuest, H., Vial, F., Stricker, D.: Adaptive line tracking with multiple hypotheses for augmented reality. In: IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 62–69 (2005)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Guofeng Wang
    • 1
  • Bin Wang
    • 1
  • Fan Zhong
    • 1
  • Xueying Qin
    • 1
    Email author
  • Baoquan Chen
    • 1
  1. 1.School of Computer Science and TechnologyShandong UniversityJinanChina

Personalised recommendations