Stereo Tracking of Faces for Driver Observation

  • Markus Steffens
  • Stephan Kieneke
  • Dominik Aufderheide
  • Werner Krybus
  • Christine Kohring
  • Danny Morton
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5575)


This report contributes a coherent framework for the robust tracking of facial structures. The framework comprises aspects of structure and motion problems, as there are feature extraction, spatial and temporal matching, re-calibration, tracking, and reconstruction. The scene is acquired through a calibrated stereo sensor. A cue processor extracts invariant features in both views, which are spatially matched by geometric relations. The temporal matching takes place via prediction from the tracking module and a similarity transformation of the features’ 2D locations between both views. The head is reconstructed and tracked in 3D. The re-projection of the predicted structure limits the search space of both the cue processor as well as the re-construction procedure. Due to the focused application, the instability of calibration of the stereo sensor is limited to the relative extrinsic parameters that are re-calibrated during the re-construction process. The framework is practically applied and proven. First experimental results will be discussed and further steps of development within the project are presented.


Singular Value Decomposition Stereo Match Epipolar Line Scene Graph Advance Driver Assistance System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    European Commission, Directorate General Information Society and Media: Use of Intelligent Systems in Vehicles. Special Eurobarometer 267 / Wave 65.4. 2006 Google Scholar
  2. 2.
    Büker, U.: Innere Sicherheit in allen Fahrsituationen. Hella KGaA Hueck & Co., Lippstadt (2007)Google Scholar
  3. 3.
    Mak, K.: Analyzes Advanced Driver Assistance Systems (ADAS) and Forecasts 63M Systems For 2013, UK (2007)Google Scholar
  4. 4.
    Steffens, M., Krybus, W., Kohring, C.: Ein Ansatz zur visuellen Fahrerbeobachtung, Sensorik und Algorithmik zur Beobachtung von Autofahrern unter realen Bedingungen. In: VDI-Konferenz BV 2007, Regensburg, Deutschland (2007)Google Scholar
  5. 5.
    Lay, G., Zelinsky, A.: A fast radial symmetry transform for detecting points of interest. Technical report, Australien National University, Canberra (2003)Google Scholar
  6. 6.
    Pilu, M.: Uncalibrated stereo correspondence by singular valued decomposition. Technical report, HP Laboratories Bristol (1997)Google Scholar
  7. 7.
    Scott, G., Longuet-Higgins, H.: An algorithm for associating the features of two patterns. In: Proceedings of the Royal Statistical Society of London, vol. B244, pp. 21–26 (1991)Google Scholar
  8. 8.
    Welch, G., Bishop, G.: An introduction to the kalman filter (July 2006)Google Scholar
  9. 9.
    Steffens, M.: Polar Rectification and Correspondence Analysis. Technical Report Laboratory for Image Processing Soest, South Westphalia University of Applied Sciences, Germany (2008)Google Scholar
  10. 10.
    Cheever, E.: Kalman filter (2008)Google Scholar
  11. 11.
    Torr, P.H.S.: A structure and motion toolkit in matlab. Technical report, Microsoft Research (2002)Google Scholar
  12. 12.
    Oberle, W.F.: Stereo camera re-calibration and the impact of pixel location uncertainty. Technical Report ARL-TR-2979, U.S. Army Research Laboratory (2003)Google Scholar
  13. 13.
    Pollefeys, M.: Visual 3Dmodeling from images. Technical report, University of North Carolina - Chapel Hill, USA (2002)Google Scholar
  14. 14.
    Newman, R., Matsumoto, Y., Rougeaux, S., Zelinsky, A.: Real-Time Stereo Tracking for Head Pose and Gaze Estimation. In: FG 2000, pp. 122–128 (2000)Google Scholar
  15. 15.
    Heinzmann, J., Zelinsky, A.: 3-D Facial Pose and Gaze Point Estimation using a Robust Real-Time Tracking Paradigm, Canberra, Australia (1997)Google Scholar
  16. 16.
    Seeing Machines: WIPO Patent WO/2004/003849 Google Scholar
  17. 17.
    Loy, G., Fletcher, L., Apostoloff, N., Zelinsky, A.: An Adaptive Fusion Architecture for Target Tracking, Canberra, Australia (2002)Google Scholar
  18. 18.
    Kähler, O., Denzler, J., Triesch, J.: Hierarchical Sensor Data Fusion by Probabilistic Cue Integration for Robust 3-D Object Tracking, Passau, Deutschland (2004)Google Scholar
  19. 19.
    Mills, S., Novins, K.: Motion Segmentation in Long Image Sequences, Dunedin, New Zealand (2000)Google Scholar
  20. 20.
    Mills, S., Novins, K.: Graph-Based Object Hypothesis. Dunedin, New Zealand (1998)Google Scholar
  21. 21.
    Mills, S.: Stereo-Motion Analysis of Image Sequences. Dunedin, New Zealand (1997)Google Scholar
  22. 22.
    Kropatsch, W.: Tracking with Structure in Computer Vision TWIST-CV. Project Proposal, Pattern Recognition and Image Processing Group, TU Vienna (2005)Google Scholar
  23. 23.
    Steffens, M.: Close-Range Photogrammetry. Technical Report Laboratory for Image Processing Soest, South Westphalia University of Applied Sciences, Germany (2008)Google Scholar
  24. 24.
    Steffens, M., Krybus, W.: Analysis and Implementation of Methods for Face Tracking. Technical Report Laboratory for Image Processing Soest, South Westphalia University of Applied Sciences, Germany (2007)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Markus Steffens
    • 1
    • 2
  • Stephan Kieneke
    • 1
    • 2
  • Dominik Aufderheide
    • 1
    • 2
  • Werner Krybus
    • 1
  • Christine Kohring
    • 1
  • Danny Morton
    • 2
  1. 1.South Westphalia University of Applied SciencesSoestGermany
  2. 2.University of BoltonBoltonUK

Personalised recommendations