International Journal of Computer 11263on

, Volume 10, Issue 3, pp 257–281 | Cite as

Model-based object tracking in monocular image sequences of road traffic scenes

  • D. Koller
  • K. Daniilidis
  • H. H. Nagel


Moving vehicles are detected and tracked automatically in monocular image sequences from road traffic scenes recorded by a stationary camera. In order to exploit the a priori knowledge about shape and motion of vehicles in traffic scenes, a parameterized vehicle model is used for an intraframe matching process and a recursive estimator based on a motion model is used for motion estimation. An interpretation cycle supports the intraframe matching process with a state MAP-update step. Initial model hypotheses are generated using an image segmentation component which clusters coherently moving image features into candidate representations of images of a moving vehicle. The inclusion of an illumination model allows taking shadow edges of the vehicle into account during the matching process. Only such an elaborate combination of various techniques has enabled us to track vehicles under complex illumination conditions and over long (over 400 frames) monocular image sequences. Results on various real-world road traffic scenes are presented and open problems as well as future work are outlined.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bar-Shalom, Y., and Fortmann, T.E., 1988.Tracking and Data Association. Academic Press: New York.MATHGoogle Scholar
  2. Broida, T.J., Chandrashekhar, S., and Chellappa, R., 1990. Recursive 3-d motion estimation from a monocular image sequence,IEEE Trans. Aerospace Electron. Syst. 26: 639–656.CrossRefGoogle Scholar
  3. Deriche, R., and Faugeras, O.D., 1990. Tracking line segments,Image Vis. Comput. 8: 261–270.CrossRefGoogle Scholar
  4. Evans, R., 1990. Kalman filtering of pose estimates in applications of the rapid video rate tracker,Proc. Brit. Mach. Vis. Conf., Oxford, pp. 79–84, September 24–27.Google Scholar
  5. Gelb, A., ed., 1974.Applied Optimal Estimation. MIT Press: Cambridge, MA and London.Google Scholar
  6. Gennery, D.B., 1982. Tracking known three-dimensional objects,Proc. Conf. Amer. Assoc. Artif. Intell., Pittsburgh, pp. 13–17, August 18–20.Google Scholar
  7. Gennery, D.B., 1992. Visual tracking of known three-dimensional objects,Intern. J. Comput. Vis. 7: 243–270.CrossRefGoogle Scholar
  8. Grimson, W.E.L., 1990a. The combinatorics of object recognition in cluttered environments using constrained search,Artificial Intelligence 44: 121–165.MathSciNetCrossRefMATHGoogle Scholar
  9. Grimson, W.E.L., 1990b.Object Recognition by Computer: The Role of Geometric Constraints. MIT Press: Cambridge, MA.Google Scholar
  10. Harris, C., and Stennet, C., 1990. RAPID—a video rate object tracker,Proc. Brit. Mach. Vis. Conf., Oxford, pp. 73–77, September 24–27.Google Scholar
  11. Jazwinski, A.H., 1970.Stochastic Processes and Filtering Theory. Academic Press: New York and London.MATHGoogle Scholar
  12. Koller, D, 1992. Detektion, Verfolgung and Klassifikation bewegter Objekte in monokularen Bildfolgen am Beispiel von Straßenver-kehrsszenen. Dissertation, Fakultät für Informatik der Universität Karlsruhe (TH), available as vol. DISKI 13,Dissertationen zur Künstlichen Intelligenz, infix-Verlag, Sankt Augustin, Germany.Google Scholar
  13. Koller, D., Heinze, N., and Nagel, H.-H., 1991. Algorithmic characterization of vehicle trajectories from image sequences by motion verbs,Conf. Comput. Vis. Patt. Recog., Lahaina, Maui, Hawaii, pp. 90–95, June 3–6.Google Scholar
  14. Koller, D., Daniilidis, K., Thórhallson, T., and Nagel, H.-H., 1992. Model-based object tracking in traffic scenes,Proc. 2nd Europ. Conf. Comput. Vis., S. Margherita, Ligure, Italy, May 18–23. G. Sandini (ed.),Lecture Notes in Computer Science 588, Springer-Verlag: Berlin, Heidelberg, New York.Google Scholar
  15. Kollnig, H., 1992. Berechnung von Bewegungsverben und Ermittlung einfacher Abläufe. Diplomarbeit, Institut für Algorithmen und Kognitive Systeme, Fakultät für Informatik der Universität Karlsruhe (TH), Karlsruhe.Google Scholar
  16. Korn, A.F., 1988. Towards a symbolic representation of intensity changes in images,IEEE Trans. Patt. Anal. Mach. Intell., 10: 610–625.CrossRefGoogle Scholar
  17. Lowe, D.G., 1987. Three-dimensional object recognition from single two-dimensional images,Artificial Intelligence 31: 355–395.CrossRefGoogle Scholar
  18. Lowe, D.G., 1990. Integrated treatment of matching and measurement errors for robust model-based motion tracking,Proc. 3rd Intern. Conf. Comput. Vis., Osaka, pp. 436–440, December 4–7.Google Scholar
  19. Lowe, D.G., 1991. Fitting parameterized three-dimensional models to images.IEEE Trans. Patt. Anal. Mach. Intell. 13: 441–450.CrossRefGoogle Scholar
  20. Marslin, R.F., Sullivan, G.D., and Baker, K.D., 1991. Kalman filters in constrained model-based tracking,Proc. Brit. Mach. Vis. Conf., Glasgow, UK, pp. 371–374, September 24–26, Springer-Verlag, Berlin, Heidelberg, New York.Google Scholar
  21. Maybank, S., 1990. Filter-based estimates of depth,Proc. Brit. Mach. Vis. Conf., Oxford, pp. 349–354, September 24–27.Google Scholar
  22. Mitschke, M., 1990.Dynamik der Kraftfahrzeuge: Band C—Fahrverhalten. Springer-Verlag: Berlin, Heidelberg, New York.CrossRefGoogle Scholar
  23. Murray, D.W., Castelow, D.A., and Buxton, B.F., 1989. From image sequences to recognized moving polyhedral objects,Intern. J. Comput. Vis. 3: 181–209.CrossRefGoogle Scholar
  24. Scales, L.E., 1985.Introduction to Non-Linear Optimization. Macmillan: London.Google Scholar
  25. Schick, J., and Dickmanns, E.D., 1991. Simultaneous estimation of 3D shape and motion of objects by computer 11263on,Proc. IEEE Workshop on Visual Motion, Princeton, NJ, pp. 256–261, October 7–9.Google Scholar
  26. Sung, C.-K., 1988. Extraktion von typischen und komplexen Vorgängen aus einer Bildfolge einer Verkehrsszene. In H. Bunke, O. Kübler, and P. Stucki, (eds.),DAGM-Symposium Mustererkennung 1988, pp. 90–96, Zürich, Informatik-Fachberichte180, Springer-Verlag: Berlin, Heidelberg, New York.CrossRefGoogle Scholar
  27. Thompson, D.W., and Mundy, J.L., 1987. Model-based motion analysis—motion from motion. InRobotics Research, R. Bolles and B. Roth (eds.), MIT Press: Cambridge, MA, pp. 299–309.Google Scholar
  28. Thórhallson, T., 1991. Untersuchung zur dynamischen Modellan-passung in monokularen Bildfolgen. Diplomarbeit, Fakultät für Elektrotechnik der Universität Karlsruhe (TH), durchgeführt am Institut für Algorithmen und Kognitive Systeme, Fakultät für Informatik der Universität Karlsruhe (TH), Karlsruhe.Google Scholar
  29. Tsai, R., 1987. A versatile camera calibration technique for high accuracy 3D machine 11263on metrology using off-the-shelf TV cameras and lenses,IEEE Trans. Robot. Autom. 3: 323–344.CrossRefGoogle Scholar
  30. Verghese, G., Gale, K.L., and Dyer, C.R., 1990. Real-time, parallel motion tracking of three dimensional objects from spatiotemporal images. In V. Kumar, P.S. Gopalakrishnan, and L.N. Kanal (eds.),Parallel Algorithms for Machine Intelligence and 11263on, pp. 340–359, Springer-Verlag: Berlin, Heidelberg, New York.Google Scholar
  31. Worrall, A.D., Marslin, R.F., Sullivan, G.D., and Baker, K.D., 1991. Model-based tracking,Proc. Brit. Mach. Vis. Conf., pp. 310–318, Glasgow, September 24–26, Springer-Verlag: Berlin, Heidelberg, New York.Google Scholar
  32. Wu, J.J., Rink, R.E., Caelli, T.M., and Gourishankar, V.G., 1988. Recovery of the 3-D location and motion of a rigid object through camera image (an extended Kalman filter approach),Intern. J. Comput. Vis. 3: 373–394.Google Scholar
  33. Young, G., and Chellappa, R., 1990. 3-D motion estimation using a sequence of noisy stereo images: models, estimation and uniqueness results,IEEE Trans. Patt. Anal. Mach. Intell. 12: 735–759.CrossRefGoogle Scholar
  34. Zhang, Z., and Faugeras, O.D., 1992. Three-dimensional motion computation and object segmentation in a long sequence of stereo frames,Intern. J. Comput. Vis. 7: 211–241.CrossRefGoogle Scholar

Copyright information

© Kluwer Academic Publishers 1993

Authors and Affiliations

  • D. Koller
    • 1
  • K. Daniilidis
    • 1
  • H. H. Nagel
    • 1
    • 2
  1. 1.Institut für Algorithmen und Kognitive SystemeFakultät für Informatik der Universität Karlsruhe (TH)Karlsruhe 1Germany
  2. 2.Fraunhofer-Institut far Informations- und Datenverarbeitung (IITB)Karlsruhe

Personalised recommendations