Advertisement

Robust edge-based 3D object tracking with direction-based pose validation

  • 272 Accesses

  • 1 Citations

Abstract

In this paper we propose a robust edge-based approach for 3D textureless object tracking. We first introduce an edge-based pose estimation method, which minimizes the holistic distance between the projected object contour and the query image edges, without explicitly searching for 3D-2D correspondences. This method is accurate with a good initialization; however, it is sensitive to occlusion and fast motion, thus often gets lost in real environments. To improve robustness, we exploit consistency of edge direction for validating the correctness of the estimated 3D pose, and further incorporate the validation scheme for robust estimation, non-local searching and failure recovery. The robust estimation adopts point-wise validation to reduce the effect of outlier, resulting in a direction-based robust estimator. The non-local searching is based on particle filter, with the pose validation for a faithful weighting of particles, which is shown to be better than the distance-based weighting. The failure recovery is based on fast 2D detection, and estimates the recovered pose by searching for 3D-2D point correspondences, with the validation scheme to adaptively determine state transition. The effectiveness of our approach is demonstrated using comparative experiments on real image sequences with occlusions, large motions and background clutters.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. 1.

    PWP3D: http://www.robots.ox.ac.uk/~victor/code.html

  2. 2.

    GOS: https://github.com/guofengw/GOSTracker

  3. 3.

    D2CO: https://bitbucket.org/alberto_pretto/d2co.git

  4. 4.

    For fair comparison, we closed failure recovery for comparisons with PWP3D, GOS, and D2CO.

  5. 5.

    See the accompany videos.

References

  1. 1.

    Besl PJ, Mckay ND (1992) Method for registration of 3-D shapes. In: Robotics - DL tentative, pp 239–256

  2. 2.

    Calonder M, Lepetit V, Strecha C, Fua P (2010) BRIEF: Binary robust independent elementary features. In: European conference on computer vision, pp 778–792

  3. 3.

    Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 8(6):679–98

  4. 4.

    Chen Y, Medioni G (1992) Object modelling by registration of multiple range images. Image Vis Comput 10(3):145–155

  5. 5.

    Choi C, Christensen HI (2012) 3D textureless object detection and tracking: an edge-based approach. In: International conference on intelligent robotics and system, pp 3877–3884

  6. 6.

    Choi C, Christensen HI (2012) Robust 3D visual tracking using particle filtering on the special Euclidean group: A combined approach of keypoint and edge features. Int J Robot Res 33(4):498–519

  7. 7.

    Comport AI, Marchand E, Chaumette F (2003) A real-time tracker for markerless augmented reality. In: IEEE International symposium on mixed and augmented reality, pp 36–45

  8. 8.

    Felzenszwalb PF, Huttenlocher DP (2004) Distance transforms of sampled functions. Theory of Computing 8(19):415–428

  9. 9.

    Fitzgibbon AW (2003) Robust registration of 2D and 3D point sets. Image Vis Comput 21(13):1145–1153

  10. 10.

    Gao X, Hou XR, Tang J, Cheng HF (2003) Complete solution classification for the perspective-three-point problem. IEEE Trans Pattern Anal Mach Intell 25 (8):930–943

  11. 11.

    Harris C, Stennett C (1990) RAPId - a video-rate object tracker. In: British machine vision conference, pp 73–77

  12. 12.

    Hartley R, Zisserman A (2000) Multiple view geometry in computer vision. Cambridge University Press

  13. 13.

    Hinterstoisser S, Cagniart C, Ilic S, Sturm P, Navab N, Fua P, Lepetit V (2012) Gradient response maps for real-time detection of textureless objects. IEEE Trans Pattern Anal Mach Intell 34(5):876–888

  14. 14.

    Hinterstoisser S, Lepetit V, Ilic S, Fua P, Navab N (2010) Dominant orientation templates for real-time detection of texture-less objects. In: IEEE Conference on computer vision and pattern recognition, pp 2257–2264

  15. 15.

    Hinterstoisser S, Lepetit V, Ilic S, Holzer S, Bradski G, Konolige K, Navab N (2012) Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes. In: Asian conference on computer vision, pp 548–562

  16. 16.

    Imperoli M, Pretto A (2015) D2CO: Fast and robust registration of 3D textureless objects using the directional chamfer distance. In: International conference on computer vision systems, pp 316–328

  17. 17.

    Isard M, Blake A (1998) CONDENSATION - Conditional density propagation for visual tracking. In: International journal of computer vision, pp 5–28

  18. 18.

    Kato H, Billinghurst M (1999) Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: IEEE And ACM international workshop on augmented reality, pp 85–94

  19. 19.

    Klein G, Murray DW (2006) Full-3D edge tracking with a particle filter. In: British machine vision conference, pp 1119–1128

  20. 20.

    Lepetit V, Fua P (2005) Monocular model-based 3D tracking of rigid objects: a survey. Found Trends Comput Graph Vis 1(1):1–89

  21. 21.

    Lepetit V, Moreno-Noguer F (2009) Epnp: an accurate o(n) solution to the pnp problem. Int J Comput Vis 81(2):155–166

  22. 22.

    Liu MY, Tuzel O, Veeraraghavan A, Chellappa R (2010) Fast directional chamfer matching. In: IEEE Conference on computer vision and pattern recognition, pp 1696–1703

  23. 23.

    Lourakis M, Zabulis X (2013) Model-based pose estimation for rigid objects. In: International conference on computer vision systems, pp 83–92

  24. 24.

    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

  25. 25.

    Marchand E, Bouthemy P, Chaumette F (2001) A 2Dc3D model-based approach to real-time visual tracking. Image Vis Comput 19(13):941–955

  26. 26.

    Marchand E, Uchiyama H, Spindler F (2015) Pose estimation for augmented reality: a hands-on survey. IEEE Trans Vis Comput Graph 22(12):2633–2651

  27. 27.

    Park Y, Lepetit V, Woo W (2008) Multiple 3D object tracking for augmented reality. In: IEEE International symposium on mixed and augmented reality, pp 117–120

  28. 28.

    Park Y, Lepetit V, Woo W (2011) Texture-less object tracking with online training using an rgb-d camera. In: IEEE International symposium on mixed and augmented reality, pp 121–126

  29. 29.

    Prisacariu VA, Reid ID (2012) PWP3D: Real-time segmentation and tracking of 3d objects. Int J Comput Vis 98(3):335–354

  30. 30.

    Rublee E, Rabaud V, Konolige K, Bradski G (2011) ORB: An efficient alternative to SIFT or SURF. In: IEEE International conference on computer vision, pp 2564–2571

  31. 31.

    Seo BK, Park H, Park JI, Hinterstoisser S, Ilic S (2014) Optimal local searching for fast and robust textureless 3D object tracking in highly cluttered backgrounds. IEEE Trans Vis Comput Graph 20(1):99–110

  32. 32.

    Shotton J, Glocker B, Zach C, Izadi S, Criminisi A, Fitzgibbon A (2013) Scene coordinate regression forests for camera relocalization in rgb-d images. In: IEEE Conference on computer vision and pattern recognition, pp 2930–2937

  33. 33.

    Vacchetti L, Lepetit V, Fua P (2004) Combining edge and texture information for real-time accurate 3D camera tracking. In: IEEE International symposium on mixed and augmented reality, pp 48–56

  34. 34.

    Vacchetti L, Lepetit V, Fua P (2004) Stable real-time 3D tracking using online and offline information. IEEE Trans Pattern Anal Mach Intell 26(10):1385–1391

  35. 35.

    Wang B, Zhong F, Qin X (2017) Pose optimization in edge distance field for textureless 3d object tracking. In: The computer graphics international conference, pp 1–6

  36. 36.

    Wang G, Wang B, Zhong F, Qin X, Chen B (2015) Global optimal searching for textureless 3D object tracking. Vis Comput 31(6):979–988

  37. 37.

    Wuest H, Vial F, Strieker D (2005) Adaptive line tracking with multiple hypotheses for augmented reality. In: IEEE International symposium on mixed and augmented reality, pp 62–69

Download references

Acknowledgements

The authors gratefully acknowledge the anonymous reviewers for their comments to help us to improve our paper, and also thank for their enormous help in revising this paper. This work is supported by the National Key Research and Development Program of China (No. 2016YFB1001501).

Author information

Correspondence to Fan Zhong.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 71.1 MB)

(MP4 71.1 MB)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wang, B., Zhong, F. & Qin, X. Robust edge-based 3D object tracking with direction-based pose validation. Multimed Tools Appl 78, 12307–12331 (2019) doi:10.1007/s11042-018-6727-5

Download citation

Keywords

  • 3D tracking
  • Pose optimization
  • Distance field
  • Particle filter