SDF-Net: Real-Time Rigid Object Tracking Using a Deep Signed Distance Network

  • Prayook JatesiktatEmail author
  • Ming Jeat Foo
  • Guan Ming Lim
  • Wei Tech Ang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10860)


In this paper, a deep neural network is used to model the signed distance function (SDF) of a rigid object for real-time tracking using a single depth camera. By leveraging the generalization capability of the neural network, we could better represent the model of the object implicitly. With the training stage done off-line, our proposed methods are capable of real-time performance and running as fast as 1.29 ms per frame on one CPU core, which is suitable for applications with limited hardware capabilities. Furthermore, the memory footprint of our trained SDF-Net for an object is less than 10 kilobytes. A quantitative comparison using public dataset is being carried out and our approach is comparable with the state-of-the-arts. The methods are also tested on actual depth records to evaluate their performance in real-life scenarios.


Deep neural network Signed distance Object representation Object tracking Depth image 


  1. 1.
    Akkaladevi, S., Ankerl, M., Heindl, C., Pichler, A.: Tracking multiple rigid symmetric and non-symmetric objects in real-time using depth data. In: IEEE International Conference on Robotics and Automation, pp. 5644–5649 (2016).
  2. 2.
    Akkaladevi, S.C., Ankerl, M., Fritz, G., Pichler, A.: Real-time tracking of rigid objects using depth data. In: Proceedings of OAGM&ARW Joint Workshop (2016).
  3. 3.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992). Scholar
  4. 4.
    Canelhas, D.R., Stoyanov, T., Lilienthal, A.J.: SDF tracker: a parallel algorithm for on-line pose estimation and scene reconstruction from depth images. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3671–3676 (2013).
  5. 5.
    Choi, C., Christensen, H.I.: RGB-D object tracking: a particle filter approach on GPU. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (2013).
  6. 6.
    Frisken, S.F.: Designing with distance fields. In: International Conference on Shape Modeling and Applications, pp. 58–59 (2005).
  7. 7.
    Garon, M., Lalonde, J.F.: Deep 6-DOF tracking. IEEE Trans. Vis. Comput. Graph. 23(11), 2410–2418 (2017). Scholar
  8. 8.
    Issac, J., Wthrich, M., Cifuentes, C.G., Bohg, J., Trimpe, S., Schaal, S.: Depth-based object tracking using a robust Gaussian filter. In: 2016 IEEE International Conference on Robotics and Automation (2016).
  9. 9.
    Jabarouti Moghaddam, M., Soltanian-Zadeh, H.: Automatic segmentation of brain structures using geometric moment invariants and artificial neural networks. In: Prince, J.L., Pham, D.L., Myers, K.J. (eds.) IPMI 2009. LNCS, vol. 5636, pp. 326–337. Springer, Heidelberg (2009). Scholar
  10. 10.
    Jatesiktat, P., Foo, M.J., Lim, G.M.: SDF-Net: Supplementary materials repository.
  11. 11.
    Kehl, W., Tombari, F., Ilic, S., Navab, N.: Real-time 3D model tracking in color and depth on a single CPU core. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 465–473 (2017).
  12. 12.
    Kehl, W., Navab, N., Ilic, S.: Coloured signed distance fields for full 3D object reconstruction. In: Proceedings of the British Machine Vision Conference (2014).
  13. 13.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. Computing Research Repository abs/1412.6980 (2014).
  14. 14.
    Krull, A., Michel, F., Brachmann, E., Gumhold, S., Ihrke, S., Rother, C.: 6-DOF model based tracking via object coordinate regression. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9006, pp. 384–399. Springer, Cham (2015). Scholar
  15. 15.
    Lepetit, V., Fua, P.: Monocular model-based 3D tracking of rigid objects. Found. Trends. Comput. Graph. Vis. 1(1), 1–89 (2005). Scholar
  16. 16.
    Levoy, M.: The Stanford 3D scanning repository.
  17. 17.
    Lu, G., Ren, L., Kolagunda, A., Wang, X., Turkbey, I.B., Choyke, P.L., Kambhamettu, C.: Representing 3D shapes based on implicit surface functions learned from RBF neural networks. J. Vis. Commun. Image Represent. 40, 852–860 (2016). Scholar
  18. 18.
    Prisacariu, V., Reid, I.: PWP3D: real-time segmentation and tracking of 3D objects. Int. J. of Comput. Vis. 98, 1–20 (2012). Scholar
  19. 19.
    Ren, C.Y., Prisacariu, V., Murray, D., Reid, I.: STAR3D: simultaneous tracking and reconstruction of 3D objects using RGB-D data. In: IEEE International Conference on Computer Vision, pp. 1561–1568 (2013).
  20. 20.
    Ren, C.Y., Prisacariu, V.A., Kähler, O., Reid, I.D., Murray, D.W.: Real-time tracking of single and multiple objects from depth-colour imagery using 3D signed distance functions. Int. J. Comput. Vis. 124(1), 80–95 (2017). Scholar
  21. 21.
    Rouhani, M., Sappa, A.D.: Correspondence free registration through a point-to-model distance minimization. In: International Conference on Computer Vision, pp. 2150–2157 (2011).
  22. 22.
    Rouhani, M., Sappa, A.D.: The richer representation the better registration. IEEE Trans. Image Process. 22(12), 5036–5049 (2013). Scholar
  23. 23.
    Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152 (2001).
  24. 24.
    Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE International Conference on Robotics and Automation, pp. 1–4 (2011).
  25. 25.
    Slavcheva, M., Kehl, W., Navab, N., Ilic, S.: SDF-2-SDF: highly accurate 3D object reconstruction. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 680–696. Springer, Cham (2016). Scholar
  26. 26.
    Somani, N., Cai, C., Perzylo, A., Rickert, M., Knoll, A.: Object recognition using constraints from primitive shape matching. In: Bebis, G., et al. (eds.) ISVC 2014. LNCS, vol. 8887, pp. 783–792. Springer, Cham (2014). Scholar
  27. 27.
    Tan, D.J., Ilic, S.: Multi-forest tracker: a chameleon in tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1202–1209 (2014).
  28. 28.
    Tan, D.J., Navab, N., Tombari, F.: Looking beyond the simple scenarios: combining learners and optimizers in 3D temporal tracking. IEEE Trans. Vis. Comput. Graph. 23(11), 2399–2409 (2017). Scholar
  29. 29.
    Tan, D.J., Tombari, F., Ilic, S., Navab, N.: A versatile learning-based 3D temporal tracker: scalable, robust, online. In: IEEE International Conference on Computer Vision (2015).
  30. 30.
    Zhang, Z.: On local matching of free-form curves. In: Hogg, D., Boyle, R. (eds.) BMVC 1992, pp. 347–356. Springer, London (1992). Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Prayook Jatesiktat
    • 1
    Email author
  • Ming Jeat Foo
    • 1
  • Guan Ming Lim
    • 1
  • Wei Tech Ang
    • 1
  1. 1.School of Mechanical and Aerospace EngineeringNanyang Technological UniversitySingaporeSingapore

Personalised recommendations