Advertisement

SDF-Net: Real-Time Rigid Object Tracking Using a Deep Signed Distance Network

  • Prayook Jatesiktat
  • Ming Jeat Foo
  • Guan Ming Lim
  • Wei Tech Ang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10860)

Abstract

In this paper, a deep neural network is used to model the signed distance function (SDF) of a rigid object for real-time tracking using a single depth camera. By leveraging the generalization capability of the neural network, we could better represent the model of the object implicitly. With the training stage done off-line, our proposed methods are capable of real-time performance and running as fast as 1.29 ms per frame on one CPU core, which is suitable for applications with limited hardware capabilities. Furthermore, the memory footprint of our trained SDF-Net for an object is less than 10 kilobytes. A quantitative comparison using public dataset is being carried out and our approach is comparable with the state-of-the-arts. The methods are also tested on actual depth records to evaluate their performance in real-life scenarios.

Keywords

Deep neural network Signed distance Object representation Object tracking Depth image 

References

  1. 1.
    Akkaladevi, S., Ankerl, M., Heindl, C., Pichler, A.: Tracking multiple rigid symmetric and non-symmetric objects in real-time using depth data. In: IEEE International Conference on Robotics and Automation, pp. 5644–5649 (2016).  https://doi.org/10.1109/ICRA.2016.7487784
  2. 2.
    Akkaladevi, S.C., Ankerl, M., Fritz, G., Pichler, A.: Real-time tracking of rigid objects using depth data. In: Proceedings of OAGM&ARW Joint Workshop (2016).  https://doi.org/10.3217/978-3-85125-528-7-14
  3. 3.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992).  https://doi.org/10.1109/34.121791CrossRefGoogle Scholar
  4. 4.
    Canelhas, D.R., Stoyanov, T., Lilienthal, A.J.: SDF tracker: a parallel algorithm for on-line pose estimation and scene reconstruction from depth images. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3671–3676 (2013).  https://doi.org/10.1109/IROS.2013.6696880
  5. 5.
    Choi, C., Christensen, H.I.: RGB-D object tracking: a particle filter approach on GPU. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2013).  https://doi.org/10.1109/IROS.2013.6696485
  6. 6.
    Frisken, S.F.: Designing with distance fields. In: International Conference on Shape Modeling and Applications, pp. 58–59 (2005).  https://doi.org/10.1109/SMI.2005.16
  7. 7.
    Garon, M., Lalonde, J.F.: Deep 6-DOF tracking. IEEE Trans. Vis. Comput. Graph. 23(11), 2410–2418 (2017).  https://doi.org/10.1109/TVCG.2017.2734599CrossRefGoogle Scholar
  8. 8.
    Issac, J., Wthrich, M., Cifuentes, C.G., Bohg, J., Trimpe, S., Schaal, S.: Depth-based object tracking using a robust Gaussian filter. In: 2016 IEEE International Conference on Robotics and Automation (2016).  https://doi.org/10.1109/ICRA.2016.7487184
  9. 9.
    Jabarouti Moghaddam, M., Soltanian-Zadeh, H.: Automatic segmentation of brain structures using geometric moment invariants and artificial neural networks. In: Prince, J.L., Pham, D.L., Myers, K.J. (eds.) IPMI 2009. LNCS, vol. 5636, pp. 326–337. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-642-02498-6_27CrossRefGoogle Scholar
  10. 10.
    Jatesiktat, P., Foo, M.J., Lim, G.M.: SDF-Net: Supplementary materials repository. https://koonyook.github.io/SDF-Net-materials/
  11. 11.
    Kehl, W., Tombari, F., Ilic, S., Navab, N.: Real-time 3D model tracking in color and depth on a single CPU core. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 465–473 (2017).  https://doi.org/10.1109/CVPR.2017.57
  12. 12.
    Kehl, W., Navab, N., Ilic, S.: Coloured signed distance fields for full 3D object reconstruction. In: Proceedings of the British Machine Vision Conference (2014).  https://doi.org/10.5244/C.28.41
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. Computing Research Repository abs/1412.6980 (2014). http://arxiv.org/abs/1412.6980
  14. 14.
    Krull, A., Michel, F., Brachmann, E., Gumhold, S., Ihrke, S., Rother, C.: 6-DOF model based tracking via object coordinate regression. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 384–399. Springer, Cham (2015).  https://doi.org/10.1007/978-3-319-16817-3_25CrossRefGoogle Scholar
  15. 15.
    Lepetit, V., Fua, P.: Monocular model-based 3D tracking of rigid objects. Found. Trends. Comput. Graph. Vis. 1(1), 1–89 (2005).  https://doi.org/10.1561/0600000001CrossRefGoogle Scholar
  16. 16.
    Levoy, M.: The Stanford 3D scanning repository. http://graphics.stanford.edu/data/3Dscanrep/
  17. 17.
    Lu, G., Ren, L., Kolagunda, A., Wang, X., Turkbey, I.B., Choyke, P.L., Kambhamettu, C.: Representing 3D shapes based on implicit surface functions learned from RBF neural networks. J. Vis. Commun. Image Represent. 40, 852–860 (2016).  https://doi.org/10.1016/j.jvcir.2016.08.014CrossRefGoogle Scholar
  18. 18.
    Prisacariu, V., Reid, I.: PWP3D: real-time segmentation and tracking of 3D objects. Int. J. of Comput. Vis. 98, 1–20 (2012).  https://doi.org/10.1007/s11263-011-0514-3MathSciNetCrossRefGoogle Scholar
  19. 19.
    Ren, C.Y., Prisacariu, V., Murray, D., Reid, I.: STAR3D: simultaneous tracking and reconstruction of 3D objects using RGB-D data. In: IEEE International Conference on Computer Vision, pp. 1561–1568 (2013).  https://doi.org/10.1109/ICCV.2013.197
  20. 20.
    Ren, C.Y., Prisacariu, V.A., Kähler, O., Reid, I.D., Murray, D.W.: Real-time tracking of single and multiple objects from depth-colour imagery using 3D signed distance functions. Int. J. Comput. Vis. 124(1), 80–95 (2017).  https://doi.org/10.1007/s11263-016-0978-2MathSciNetCrossRefGoogle Scholar
  21. 21.
    Rouhani, M., Sappa, A.D.: Correspondence free registration through a point-to-model distance minimization. In: International Conference on Computer Vision, pp. 2150–2157 (2011).  https://doi.org/10.1109/ICCV.2011.6126491
  22. 22.
    Rouhani, M., Sappa, A.D.: The richer representation the better registration. IEEE Trans. Image Process. 22(12), 5036–5049 (2013).  https://doi.org/10.1109/TIP.2013.2281427MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001).  https://doi.org/10.1109/IM.2001.924423
  24. 24.
    Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation, pp. 1–4 (2011).  https://doi.org/10.1109/ICRA.2011.5980567
  25. 25.
    Slavcheva, M., Kehl, W., Navab, N., Ilic, S.: SDF-2-SDF: highly accurate 3D object reconstruction. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 680–696. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_41CrossRefGoogle Scholar
  26. 26.
    Somani, N., Cai, C., Perzylo, A., Rickert, M., Knoll, A.: Object recognition using constraints from primitive shape matching. In: Bebis, G., et al. (eds.) ISVC 2014. LNCS, vol. 8887, pp. 783–792. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-14249-4_75CrossRefGoogle Scholar
  27. 27.
    Tan, D.J., Ilic, S.: Multi-forest tracker: a chameleon in tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1202–1209 (2014).  https://doi.org/10.1109/CVPR.2014.157
  28. 28.
    Tan, D.J., Navab, N., Tombari, F.: Looking beyond the simple scenarios: combining learners and optimizers in 3D temporal tracking. IEEE Trans. Vis. Comput. Graph. 23(11), 2399–2409 (2017).  https://doi.org/10.1109/TVCG.2017.2734539CrossRefGoogle Scholar
  29. 29.
    Tan, D.J., Tombari, F., Ilic, S., Navab, N.: A versatile learning-based 3D temporal tracker: scalable, robust, online. In: IEEE International Conference on Computer Vision (2015).  https://doi.org/10.1109/ICCV.2015.86
  30. 30.
    Zhang, Z.: On local matching of free-form curves. In: Hogg, D., Boyle, R. (eds.) BMVC 1992, pp. 347–356. Springer, London (1992).  https://doi.org/10.1007/978-1-4471-3201-1_36CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Prayook Jatesiktat
    • 1
  • Ming Jeat Foo
    • 1
  • Guan Ming Lim
    • 1
  • Wei Tech Ang
    • 1
  1. 1.School of Mechanical and Aerospace EngineeringNanyang Technological UniversitySingaporeSingapore

Personalised recommendations