Abstract
This contribution is based on two previously published approaches one of which automatically extracts vehicle trajectories from image sequences of traffic scenes and associates these trajectories with motion verbs. The second approach exploits machine vision in order to maneuver autonomous road vehicles. The combination of these two approaches provides a link from the evaluation of video signals via an abstract representation at the level of natural language concepts to actuator devices in automatic closed loop control of road vehicles. Building on implemented representations for elementary motion verbs and for elementary road vehicle maneuvers, a grammar to represent a nontrivial subset of more complex driving activities on a highway is formulated. Driving on a highway can thereby be investigated not only at the level of control algorithms, but simultaneously at the level of natural language descriptions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Enkelmann, W. (1991). Obstacle Detection by Evaluation of Optical Flow Fields from Image Sequences.Image and Vision Computing 9: 160–168.
Heinze, N., Krüger, W. & Nagel, H.-H. (1991). Berechnung von Bewegungsverben zur Beschreibung von aus Bildfolgen gewonnenen Fahrzeugtrajektorien in Straßenverkehrsszenen.Informatik — Forschung und Entwicklung 6: 51–61 (in German).
Herzog, G., Sung, C.-K., André, E., Enkelmann, W., Nagel, H.-H., Rist, T., Wahlster, W. & Zimmermann, G. (1989). Incremental Natural Language Description of Dynamic Imagery. In Brauer, W. & Freksa, C. (eds) InProceedings of Dritter Internationaler GI-Kongreß Wissensbasierte Systeme, 153–162. Springer-Verlag: Berlin Heidelberg New York/NY, Informatik-Fachberichte 227.
Koller, D., Heinze, N. & Nagel, H.-H. (1991). Algorithmic Characterization of Vehicle Trajectories from Image Sequences by Motion Verbs. In Proceedings ofThe IEEE Conference on Computer Vision and Pattern Recognition CVPR '91, 90–95. Lahaina, Maui/Hawaii, 3–6 June 1991.
Koller, D. (1992).Detektion, Verfolgung und Klassifikation bewegter Objekte in monokularen Bildfolgen am Beispiel von Straßenverkehrsszenen. Dissertation, Fakultät für Informatik der Universität Karlsruhe (TH); appeared in the series ‘Dissertationen zur Künstlichen Intelligenz’ DISKI vol. 13. infix: Sankt Augustin/Germany (in German).
Koller, D., Daniilidis, K. & Nagel, H.-H. (1993). Model-Based Object Tracking in Monocular Image Sequences of Road Traffic Scenes.International Journal of Computer Vision 10: 257–281.
Kollnig, H. & Nagel, H.-H. (1993). Ermittlung von begrifflichen Beschreibungen von Geschehen in Straßenverkehrsszenen mit Hilfe unscharfer Mengen.Informatik — Forschung und Entwicklung 8: 186–196 (in German).
Kollnig, H., Nagel, H.-H. & Otte, M. (1994). Association of Motion Verbs with Vehicle Movements Extracted from Dense Optical Flow Fields. In Eklundh, J.-O. (ed.) In Proceedings ofThe Third European Conference on Computer Vision ECCV '94, Stockholm/Sweden, vol. II, 338–347. Springer-Verlag: Berlin Heidelberg New York/NY, Lecture Notes in Computer Science vol. 801.
Krüger, W. (1991).Begriffsgraphen zur Situationsmodellierung in der Bildfolgenauswertung. Dissertation, Fakultät für Informatik der Universität Karlsruhe (TH), Karlsruhe/Germany. A revised version appeared asSituationsmodellierung in der Bildfolgenauswertung. Springer-Verlag: Berlin Heidelberg (1992), Informatik-Fachberichte vol. 311 (in German).
Nagel, H.-H. (1988). From Image Sequences towards Conceptual Descriptions.Image and Vision Computing 6: 59–74.
Nagel, H.-H. (1989). Zur Erkennung von Situationen durch Answertung von Bildfolgen.FhG-Berichte 1(89): 25–33.
Nagel, H.-H. (1991). The Representation of Situations and Their Recognition from Image Sequences. In Proceedings ofAFCET 8 e Congrès Reconnaissance des Formes et Intelligence Artificielle, 1221–1229. Lyon-Villeurbanne/France, 25–29 November.
Nagel, H.-H. (1993).Überlegungen zur Formalisierung der sichtsystemgestützten Straßenfahrzeugführung. Internal report, Fraunhofer-Institut für Informations- und Datenverarbeitung (IITB), Karlsruhe/Germany (in German).
Nagel, H.-H. & Enkelmann, W. (1991). Generic Road Traffic Situations and Driver Support Systems. In Proceedings of theFifth PROMETHEUS Workshop, 76–85. Munich/Germany, 15–16 October 1991.
Nagel, H.-H., Enkelmann, W. & Struck, G. (1994).FhG-Co-Driver: From Map-Guided Automatic Driving by Machine Vision to a Cooperative Driving Support. Computers and Mathematics with Applications, Special Issue on “Network, Control, Communication and Computing Technologies for Intelligent Vehicle Highway Systems” (in press).
Retz-Schmidt, G. (1991). Recognizing Intentions, Interactions, and Causes of Plan Failures.User-Modelling and User-Adapted Interaction 1: 173–202.
Retz-Schmidt, G. (1992).Die Interpretation des Verhaltens mehrerer Akteure in Szenenfolgen. Springer-Verlag: Berlin Heidelberg, Informatik-Fachberichte vol. 308 (in German).
Siegle, G., Geisler, J., Laubenstein, F., Nagel, H.-H. & Struck, G. (1992). Autonomous Driving on a Road Network. In Proceedings ofIntelligent Vehicles '92 Symposium, 403–408. Detroit/MI: 29 June–1 July 1992.
Struck, G., Geisler, J., Laubenstein, F., Nagel, H.-H. & Siegle, G. (1993). Interaction between Digital Road Map Systems and Trinocular Autonomous Driving. In Proceedings ofIntelligent Vehicles '93 Symposium, 461–466. Tokyo/Japan: 14–16 July 1993.
Winograd, T. (1972).Understanding Natural Language. Academic Press, Inc.: New York, NY. For a synopsis, see: Winograd, T. (1973) A Procedural Model of Language Understanding. In Schank, R. C. & Colby, K. M. (eds.)Computer Models of Thought and Language. W.H. Freeman and Company: San Francisco/CA.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Nagel, HH. A vision of ‘vision and language’ comprises action: An example from road traffic. Artif Intell Rev 8, 189–214 (1994). https://doi.org/10.1007/BF00849074
Issue Date:
DOI: https://doi.org/10.1007/BF00849074