An Analysis-by-Synthesis Camera Tracking Approach Based on Free-Form Surfaces

  • Kevin Koeser
  • Bogumil Bartczak
  • Reinhard Koch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4713)


We propose a model-based camera pose estimation approach, which makes use of GPU-assisted analysis-by-synthesis methods on a very wide field of view (e.g. fish-eye) camera. After an initial registration, the synthesis part of the tracking is performed on graphics hardware, which simulates internal and external parameters of the camera, this way minimizing lens and perspective differences between a model view and a real camera image. We show how such a model is automatically created from a scene and analyze the sensitivity of the tracking to the model accuracy, in particular the case when we represent free-form surfaces by planar patches. We also examine accuracy and show on synthetic and on real data that the system does not suffer from drift accumulation. The wide field of view of the camera and the subdivision of our reference model into many textured free-form surfaces make the system robust against moving persons and other occlusions within the environment and provide a camera pose estimate in a fixed and known coordinate system.


Camera Parameter Graphic Hardware Orientation Error Bundle Adjustment Camera Tracking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. IJCV 59(3), 207–232 (2004)CrossRefGoogle Scholar
  2. 2.
    Lepetit, V., Lagger, P., Fua, P.: Randomized Trees for Real-Time Keypoint Recognition. In: CVPR 2005, San Diego, CA (June 2005)Google Scholar
  3. 3.
    Thomas, G.A., Jin, J., Niblett, T., Urquhart, C.: A Versatile Camera Position Measurement System for Virtual Reality TV Production. In: Proceedings of International Broadcasting Convention, Amsterdam, pp. 284–289 (1997)Google Scholar
  4. 4.
    Koeser, K., Haertel, V., Koch, R.: Robust Feature Representation for Efficient Camera Registration. In: Franke, K., Müller, K.-R., Nickolay, B., Schäfer, R. (eds.) Pattern Recognition. LNCS, vol. 4174, pp. 739–749. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  5. 5.
    Skrypnyk, I., Lowe, D.G.: Scene Modelling, Recognition and Tracking with Invariant Image Features. In: ISMAR 2004, Arlington, pp. 110–119 (2004)Google Scholar
  6. 6.
    Tomasi, C., Kanade, T.: Detection and Tracking of Point Features, Carnegie Mellon University, Technical Report, CMU-CS-91-132 (April 1991)Google Scholar
  7. 7.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust Wide baseline Stereo from Maximally Stable Extremal Regions. In: Proceedings of BMVC 2002 (2002)Google Scholar
  8. 8.
    Davison, A.J.: Real-Time Simultaneous Localisation and Mapping with a Single Camera. In: Proceedings International Conference Computer Vision, Nice (2003)Google Scholar
  9. 9.
    Streckel, B., Koch, R.: Lens Model Selection for Visual Tracking. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) Pattern Recognition. LNCS, vol. 3663, pp. 41–48. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  10. 10.
    Fleck, M.: Perspective Projection: The Wrong Imaging Model. TR 95-01, Computer Science, University of Iowa (1995)Google Scholar
  11. 11.
    Koch, R.: Dynamic 3D Scene Analysis through Synthesis Feedback Control. IEEE Transactions PAMI 15(6), 556–568 (1993)Google Scholar
  12. 12.
    Evers-Senne, J., Koch, R.: Image Based Rendering from Handheld Cameras using Quad Primitives. In: Evers-Senne, J., Koch, R. (eds.) VMV 2003, November 2003, Munich, Germany (2003)Google Scholar
  13. 13.
    Scaramuzza, D., Martinelli, A., Siegwart, R.: A Toolbox for Easy Calibrating Omnidirectional Cameras. In: Proceedings of IROS 2006, October 2006, Beijing, China (2006)Google Scholar
  14. 14.
    Koeser, K., Bartczak, B., Koch, R.: Drift-free Pose Estimation with Hemispherical Cameras. In: Proceedings of CVMP 2006, November 2006, London (2006)Google Scholar
  15. 15.
    Julier, S., Uhlmann, J.: A new extension of the Kalman filter to nonlinear systems. In: Int. Symp. Aerospace/Defense Sensing, Simul. and Controls, Orlando (1997)Google Scholar
  16. 16.
    Bleser, G., Wuest, H., Stricker, D.: Online Camera Pose Estimation in Partially Known and Dynamic Scenes. In: ISMAR 2006, Los Alamitos, California, pp. 56–65 (2006)Google Scholar
  17. 17.
    Neumann, J., Fermuller, C., Aloimonos, Y.: Eyes from eyes: New cameras for structure from motion. In: IEEE Workshop on Omnidirectional Vision, pp. 19–26 (2002)Google Scholar
  18. 18.
    Micusik, B., Pajdla, T.: Structure From Motion with Wide Circular Field of View Cameras. IEEE Transactions on PAMI 28(7), 1135–1149 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Kevin Koeser
    • 1
  • Bogumil Bartczak
    • 1
  • Reinhard Koch
    • 1
  1. 1.Institut für Informatik, Christian-Albrechts-Universität Kiel, D-24098 KielGermany

Personalised recommendations