Machine Vision and Applications

, Volume 22, Issue 2, pp 323–335 | Cite as

Tracking rigid objects using integration of model-based and model-free cues

  • Ville Kyrki
  • Danica Kragic
Original Paper


Model-based 3-D object tracking has earned significant importance in areas such as augmented reality, surveillance, visual servoing, robotic object manipulation and grasping. Key problems to robust and precise object tracking are the outliers caused by occlusion, self-occlusion, cluttered background, reflections and complex appearance properties of the object. Two of the most common solutions to the above problems have been the use of robust estimators and the integration of visual cues. The tracking system presented in this paper achieves robustness by integrating model-based and model-free cues together with robust estimators. As a model-based cue, a wireframe edge model is used. As model-free cues, automatically generated surface texture features are used. The particular contribution of this work is the integration framework where not only polyhedral objects are considered. In particular, we deal also with spherical, cylindrical and conical objects for which the complete pose cannot be estimated using only wireframe models. Using the integration with the model-free features, we show how a full pose estimate can be obtained. Experimental evaluation demonstrates robust system performance in realistic settings with highly textured objects and natural backgrounds.


Model-based tracking Model-free tracking Cue integration Iterated extended Kalman filter 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vacchetti L., Lepetit V., Fua P.: Stable real-time 3D tracking using online and offline information. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1385–1391 (2004)CrossRefGoogle Scholar
  2. 2.
    Vincze M., Ayromlou M., Ponweiser M., Zillich M.: Edge projected integration of image and model cues for robust model-based object tracking. Int. J. Robot. Res. 20(7), 533–552 (2001)CrossRefGoogle Scholar
  3. 3.
    Taylor, G., Kleeman, L.: Fusion of multimodal visual cues for model-based object tracking. In: Australiasian Conf. on Robotics and Automation, Brisbane, Australia (2003)Google Scholar
  4. 4.
    Harris C.: Tracking with rigid models. In: Blake, A., Yuille, A. (eds) Active Vision, Ch. 4, pp. 59–73. MIT Press, Cambridge (1992)Google Scholar
  5. 5.
    Comport, A., Marchand, E., Chaumette, F.: A real-time tracker for markerless augmented reality. In: IEEE Int. Symp. on Mixed and Augmented Reality pp. 36–45 (2003)Google Scholar
  6. 6.
    Vincze, M., Ayromlou, M., Kubinger, W.: An integrating framework for robust real-time 3D object tracking. In: Int. Conf. on Comp. Vis. Syst., ICVS’99, pp. 135–150 (1999)Google Scholar
  7. 7.
    Koller D., Daniilidis K., Nagel H.: Model-based object tracking in monocular image sequences of road traffic scenes. Int. J. Comput. Vis. 10(3), 257–281 (1993)CrossRefGoogle Scholar
  8. 8.
    Drummond T., Cipolla R.: Real-time visual tracking of complex structures. IEEE Trans. PAMI 24(7), 932–946 (2002)Google Scholar
  9. 9.
    Malis E., Chaumette F., Boudet S.: 2-1/2-d visual servoing. IEEE Trans. Robot. Autom. 15(2), 238–250 (1999)CrossRefGoogle Scholar
  10. 10.
    Jurie F., Dhome M.: Real time tracking of 3D objects: an efficient and robust approach. Pattern Recognit. 35, 317–328 (2002)MATHCrossRefGoogle Scholar
  11. 11.
    Dickmanns E.D., Graefe V.: Dynamic monocular machine vision. Mach. Vis. Appl. 1, 223–240 (1988)CrossRefGoogle Scholar
  12. 12.
    Lowe D.G.: Robust model-based motion tracking through the integration of search and estimation. Int. J. Comput. Vis. 8(2), 113–122 (1992)CrossRefGoogle Scholar
  13. 13.
    Wunsch, P., Hirzinger, G.: Real-time visual tracking of 3-D objects with dynamic handling of occlusion. In: IEEE Int. Conf. on Robotics and Automation, ICRA’97, Albuquerque, New Mexico, USA, pp. 2868–2873 (1997)Google Scholar
  14. 14.
    Masson, L., Jurie, F., Dhome, M.: Robust real time tracking of 3D objects. In: Int. Conf. Pattern Recognit., vol. 4, pp. 252–255 (2004)Google Scholar
  15. 15.
    Klein, G., Drummond, T.: Robust visual tracking for non-instrumented augmented reality. In: Proc. 2nd IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 113–122 (2003)Google Scholar
  16. 16.
    Rasmussen C., Hager G.: Probabilistic data association methods for tracking complex visual objects. IEEE Trans. PAMI 23(6), 560–576 (2001)Google Scholar
  17. 17.
    Triesch, J., der Malsburg, C.V.: Self-organized integration of adaptive visual cues for face tracking. In: Fourth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 102–107 (2000)Google Scholar
  18. 18.
    Kragic D., Christensen H.I.: Cue integration for visual servoing. IEEE Trans. Robot. Autom. 17(1), 18–27 (2001)CrossRefGoogle Scholar
  19. 19.
    Hayman, E., Eklundh, J.-O.: Statistical background subtraction for a mobile observer. In: Proceedings of the Ninth IEEE International Conference on Computer Vision, pp. 67–74 (2003)Google Scholar
  20. 20.
    Vacchetti, L., Lepetit, V., Fua, P.: Combining edge and texture information for real-time accurate 3D camera tracking. In: Proceedings of International Symposium on Mixed and Augmented Reality, Arlington, VA, USA (2004)Google Scholar
  21. 21.
    Rosten, E., Drummond, T.: Fusing points and lines for high performance tracking. In: IEEE International Conference on Computer Vision, vol. 2, pp. 1508–1511. Beijing, China (2005)Google Scholar
  22. 22.
    Kyrki, V., Kragic, D.: Integration of model-based and model-free cues for visual object tracking in 3D. In: IEEE International Conference on Robotics and Automation, ICRA’05, pp. 1566–1572 (2005)Google Scholar
  23. 23.
    Kyrki, V., Schmock, K.: Integation methods of model-free features for 3D tracking. In: Scandinavian Conference on Image Analysis, pp. 557–566 (2005)Google Scholar
  24. 24.
    Brox, T., Rosenhahn, B., Cremers, D., Seidel, H.P.: High accuracy optical flow serves 3-D pose tracking: exploiting contour and flow based constraints. In: European Conference on Computer Vision, ECCV 2006, Graz, Austria, pp. 98–111 (2006)Google Scholar
  25. 25.
    Pressigout M., Marchand E.: Real-time hybrid tracking using edge and texture information. Int. J. Robot. Res. 26(7), 689–713 (2007)CrossRefGoogle Scholar
  26. 26.
    Hager G., Belhumeur P.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20(10), 1025–1039 (1998)CrossRefGoogle Scholar
  27. 27.
    Lu, S., Metaxa, D., Samaras, D., Oliensis, J.: Using multiple cues for hand tracking and model refinement. In: Computer Vision and Pattern Recognition, vol. 2, pp. 443–450. Madison, Wisconsin, USA (2003)Google Scholar
  28. 28.
    Harris, C.J., Stephens, M.: A combined corner and edge detector. In: Proc. 4th Alvey Vision Conference, Manchester, UK, pp. 147–151 (1988)Google Scholar
  29. 29.
    Lefebvre T., Bruyninckx H., De Schutter J.: Kalman filters for non-linear systems: a comparison of performance. Int. J. Control 77(7), 639–653 (2004)MATHCrossRefGoogle Scholar
  30. 30.
    Isard M., Blake A.: CONDENSATION-Conditional density propagation for visual tracking. Int. J. Comput. Vis. 29(1), 5–28 (1998)CrossRefGoogle Scholar
  31. 31.
    Bar-Shalom Y., Li X.R., Kirubarajan T.: Estimation with Applications to Tracking and Navigation. Wiley-Interscience, New York (2001)CrossRefGoogle Scholar
  32. 32.
    Welch, G., Bishop, G.: SCAAT: Incremental tracking with incomplete information. In: Proc. Computer graphics and interactive techniques, pp. 333–344. Los Angeles, CA, USA (1997)Google Scholar
  33. 33.
    Heikkila, J., Silven, O.: A four-step camera calibration procedure with implicit image correction. In: IEEE Computer Vision and Pattern Recognition, pp. 1106–1112 (1997)Google Scholar
  34. 34.
    Zhang Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)CrossRefGoogle Scholar
  35. 35.
    Huber P.J.: Robust estimation of a location parameter. Ann. Math. Stat. 35, 73–101 (1964)MATHCrossRefMathSciNetGoogle Scholar
  36. 36.
    Huber P.J.: Robust Statistics. Wiley, New York (1981)MATHCrossRefGoogle Scholar
  37. 37.
    Press W.H., Teukolsky S.A., Vetterling W.T., Flannery B.P.: Numerical Recipes in C++. Cambridge University Press, Cambridge (2002)Google Scholar
  38. 38.
    Brent R.P.: Algorithms for Minimization without Derivatives. Prentice-Hall, Englewood Cliffs (1973)MATHGoogle Scholar
  39. 39.
    Tsai R., Lenz R.K.: A new technique for fully autonomous and efficient 3D robotics hand/eye calibration. IEEE Trans. Robot. Autom. 5(3), 345–358 (1989)CrossRefGoogle Scholar
  40. 40.
    Kyrki, V., Kragic, D., Christensen, H.I.: New shortest-path approaches to visual servoing. In: IEEE/RSJ Int. Conf. Intell. Robots and Systems, Sendai, Japan (2004)Google Scholar

Copyright information

© Springer-Verlag 2009

Authors and Affiliations

  1. 1.Department of Information TechnologyLappeenranta University of TechnologyLappeenrantaFinland
  2. 2.Centre for Autonomous Systems, School of Computer Science and CommunicationRoyal Institute of TechnologyStockholmSweden

Personalised recommendations