6-DOF Model Based Tracking via Object Coordinate Regression

  • Alexander KrullEmail author
  • Frank Michel
  • Eric Brachmann
  • Stefan Gumhold
  • Stephan Ihrke
  • Carsten Rother
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9006)


This work investigates the problem of 6-Degrees-Of-Freedom (6-DOF) object tracking from RGB-D images, where the object is rigid and a 3D model of the object is known. As in many previous works, we utilize a Particle Filter (PF) framework. In order to have a fast tracker, the key aspect is to design a clever proposal distribution which works reliably even with a small number of particles. To achieve this we build on a recently developed state-of-the-art system for single image 6D pose estimation of known 3D objects, using the concept of so-called 3D object coordinates. The idea is to train a random forest that regresses the 3D object coordinates from the RGB-D image. Our key technical contribution is a two-way procedure to integrate the random forest predictions in the proposal distribution generation. This has many practical advantages, in particular better generalization ability with respect to occlusions, changes in lighting and fast-moving objects. We demonstrate experimentally that we exceed state-of-the-art on a given, public dataset. To raise the bar in terms of fast-moving objects and object occlusions, we also create a new dataset, which will be made publicly available.


Object Coordinates Proposal Distribution Pose Estimation Fast-moving Objects Random Forest 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work has partially been supported by the European Social Fund and the Federal State of Saxony within project VICCI (#100098171).

We thank Daniel Schemala for development of the manual pose annotation tool, we used to generate ground truth data.

Supplementary material (20.1 mb)
Supplementary material (zip 20,568 KB)


  1. 1.
    Avidan, S.: Ensemble tracking. IEEE Trans. PAMI 29, 261–271 (2007)CrossRefGoogle Scholar
  2. 2.
    Azad, P., Munch, D., Asfour, T., Dillmann, R.: 6-DoF model-based tracking of arbitrarily shaped 3D objects. In: IEEE ICRA, pp. 5204–5209 (2011)Google Scholar
  3. 3.
    Bersch, C., Pangercic, D., Osentoski, S., Hausman, K., Marton, Z.C., Ueda, R., Okada, K., Beetz, M.: Segmentation of textured and textureless objects through interactive perception. In: RSS Workshop on Robots in Clutter: Manipulation, Perception and Navigation in Human Environments (2012)Google Scholar
  4. 4.
    Brachmann, E., Krull, A., Michel, F., Gumhold, S., Shotton, J., Rother, C.: Learning 6D object pose estimation using 3D object coordinates. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 536–551. Springer, Heidelberg (2014) CrossRefGoogle Scholar
  5. 5.
    Bray, M., Koller-Meier, E., Van Gool, L.: Smart particle filtering for 3D hand tracking. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 675–680 (2004)Google Scholar
  6. 6.
    Chiuso, A., Soatto, S.: Monte carlo filtering on lie groups. In: 39th IEEE Conference on Decision and Control, vol. 1, pp. 304–309 (2000)Google Scholar
  7. 7.
    Choi, C., Christensen, H.I.: 3D textureless object detection and tracking: an edge-based approach. In: IEEE/RSJ International Conference on IROS, pp. 3877–3884 (2012)Google Scholar
  8. 8.
    Choi, C., Christensen, H.I.: RGB-D object tracking: A particle filter approach on GPU. In: IEEE/RSJ International Conference on IROS, pp. 1084–1091 (2013)Google Scholar
  9. 9.
    Choi, C., Christensen, H.: Robust 3D visual tracking using particle filtering on the SE(3) group. In: 2011 IEEE ICRA, pp. 4384–4390 (2011)Google Scholar
  10. 10.
    Doucet, A., Godsill, S., Andrieu, C.: On sequential monte carlo sampling methods for bayesian filtering. Stat. Comput. 10, 197–208 (2000)CrossRefGoogle Scholar
  11. 11.
    Fanelli, G., Weise, T., Gall, J., Van Gool, L.: Real time head pose estimation from consumer depth cameras. In: Mester, R., Felsberg, M. (eds.) Pattern Recognition. LNCS, vol. 6835, pp. 101–110. Springer, Heidelberg (2011) CrossRefGoogle Scholar
  12. 12.
    Gordon, N., Salmond, D., Smith, A.: Novel approach to nonlinear/non-gaussian bayesian state estimation. IEEE Radar Signal Process. 2, 107–113 (1993)CrossRefGoogle Scholar
  13. 13.
    Grabner, H., Bischof, H.: Online boosting and vision. IEEE CVPR 1, 260–267 (2006)Google Scholar
  14. 14.
    Hinterstoisser, S., Lepetit, V., Ilic, S., Holzer, S., Bradski, G., Konolige, K., Navab, N.: Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part I. LNCS, vol. 7724, pp. 548–562. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  15. 15.
    Isard, M., Blake, A.: Contour tracking by stochastic propagation of conditional density. In: Buxton, B., Cipolla, R. (eds.) Computer Vision - ECCV 1996. LNCS, vol. 1064, pp. 343–356. Springer, Heidelberg (1996) Google Scholar
  16. 16.
    Klein, G., Murray, D.W.: Full-3D edge tracking with a particle filter. In: BMVC, pp. 1119–1128 (2006)Google Scholar
  17. 17.
    Kwon, J., Choi, M., Park, F.C., Chun, C.: Particle filtering on the euclidean group: framework and applications. Robotica 25, 725–737 (2007)CrossRefGoogle Scholar
  18. 18.
    McElhone, M., Stuckler, J., Behnke, S.: Joint detection and pose tracking of multi-resolution surfel models in RGB-D. In: IEEE ECMR, pp. 131–137 (2013)Google Scholar
  19. 19.
    Okuma, K., Taleghani, A., de Freitas, N., Little, J.J., Lowe, D.G.: A boosted particle filter: multitarget detection and tracking. In: Pajdla, T., Matas, J.G. (eds.) ECCV 2004. LNCS, vol. 3021, pp. 28–39. Springer, Heidelberg (2004) CrossRefGoogle Scholar
  20. 20.
    Pupilli, M., Calway, A.: Real-time camera tracking using known 3d models and a particle filter. In: IEEE ICPR, vol. 1, pp. 199–203 (2006)Google Scholar
  21. 21.
    Rios-Cabrera, R., Tuytelaars, T.: Discriminatively trained templates for 3d object detection: a real time scalable approach. In: IEEE ICCV, pp. 2048–2055 (2013)Google Scholar
  22. 22.
    Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., Fitzgibbon, A.: Scene coordinate regression forests for camera relocalization in RGB-D images. In: IEEE CVPR, pp. 2930–2937 (2013)Google Scholar
  23. 23.
    Song, S., Xiao, J.: Tracking revisited using rgbd camera: unified benchmark and baselines. In: ICCV, pp. 233–240 (2013)Google Scholar
  24. 24.
    Stckler, J., Behnke, S.: Multi-resolution surfel maps for efficient dense 3D modeling and tracking. J. Vis. Commun. Image Represent. 25, 137–147 (2014)CrossRefGoogle Scholar
  25. 25.
    Taylor, J., Shotton, J., Sharp, T., Fitzgibbon, A.W.: The vitruvian manifold: Inferring dense correspondences for one-shot human pose estimation. In: IEEE CVPR, pp. 103–110 (2012)Google Scholar
  26. 26.
    Teuliere, C., Marchand, E., Eck, L.: Using multiple hypothesis in model-based tracking. In: IEEE ICRA, pp. 4559–4565 (2010)Google Scholar
  27. 27.
    Wu, B., Nevatia, R.: Detection and tracking of multiple, partially occluded humans by bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. 75, 247–266 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Alexander Krull
    • 1
    Email author
  • Frank Michel
    • 1
  • Eric Brachmann
    • 1
  • Stefan Gumhold
    • 1
  • Stephan Ihrke
    • 1
  • Carsten Rother
    • 1
  1. 1.TU DresdenDresdenGermany

Personalised recommendations