Knowledge based image understanding by iterative optimization

  • H. Niemann
  • V. Fischer
  • D. Paulus
  • J. Fischer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1137)


In this paper knowledge based image interpretation is formulated and solved as an optimization problem which takes into account the observed image data, the available task specific knowledge, and the requirements of an application. Knowledge is represented by a semantic network consisting of concepts (nodes) and links (edges). Concepts are further defined by attributes, relations, and a judgment function. The interface between the symbolic knowledge base and the results of image (or signal) processing and initial segmentation is specified via primitive concepts.

We present a recently developed approach to optimal interpretation that is based on the automatic conversion of the concept oriented semantic network to an attribute centered representation and the use of iterative optimization procedures, like e.g. simulated annealing or genetic algorithms. We show that this is a feasible approach which provides ‘any-time’ capability and allows parallel processing. It provides a well-defined combination of signal and symbol oriented processing by optimizing a heuristic judgment function.

The general ideas have been applied to various problems of image and speech understanding. As an example we describe the recognition of streets from TV image sequences to demonstrate the efficiency of iterative optimization.

Key words

semantic network iterative optimization knowledge based image analysis 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    T. Matsuyama and V. Hwang. SIGMA: A Knowledge-based Aerial Image Understanding System. Plenum Press, New York, 1990.Google Scholar
  2. 2.
    I. Masaki, editor. Vision-Based Vehicle Guidance. Springer, Berlin, 1992.Google Scholar
  3. 3.
    O. Faugeras. Three-Dimensional Computer Vision. Artificial Intelligence Series. The MIT Press, Cambridge, MA, 1993.Google Scholar
  4. 4.
    H. Niemann. Pattern Analysis and Understanding. Springer Series in Information Sciences 4. Springer, Berlin, 2. edition, 1990.Google Scholar
  5. 5.
    P. Krause and D. Clark. Representing Uncertain Knowledge. Intellect Books, Oxford, 1993.Google Scholar
  6. 6.
    J.F. Sowa, editor. Principles of Semantic Networks. Morgan Kaufmann, San Mateo, Calif., 1991.Google Scholar
  7. 7.
    H. Niemann, G. Sagerer, S. Schröder, and F. Kummert. ERNEST: A semantic network system for pattern understanding. IEEE Trans. on Pattern Analysis and Machine Intelligence, 9:883–905, 1990.Google Scholar
  8. 8.
    A. Kobsa. The SB-ONE knowledge representation workbench. SFB 314 (XTRA), Memo Nr. 31, Univ. des Saarlandes, FB 10, Saarbrücken, F. R. of Germany, 1989.Google Scholar
  9. 9.
    D. Paulus, A. Winzen, and H. Niemann. Knowlege based object recognition and model generation. In Proc. Europto 93, Computer Vision for Industry, pages 382–393, München, 1994. SPIE Proc. No. 1989-47.Google Scholar
  10. 10.
    A. Winzen. Automatische Erzeugung dreidimensionaler Modelle für Bildanalysesysteme. Dissertation, Technische Fakultät, Universität Erlangen-Nürnberg, Erlangen, (1994).Google Scholar
  11. 11.
    H. Niemann, H. Bunke, I. Hofmann, G. Sagerer, F. Wolf, and H. Feistel. A knowledge based system for analysis of gated blood pool studies. IEEE Trans. Pattern Analysis and Machine Intelligence, 7:246–259, 1985.Google Scholar
  12. 12.
    H. Niemann, H. Brünig, R. Salzbrunn, and S. Schröder. A knowledge-based vision system for industrial applications. Machine Vision and Applications, 3:201–229, 1990.Google Scholar
  13. 13.
    H. Burkhardt, Y. Neuvo, and J. Simon, editors. From Pixels to Features II. Parallelism in Image Processing. North-Holland, Amsterdam, 1991.Google Scholar
  14. 14.
    L. Shastri. Semantic Networks: An Evidential Formalization and its Connectionist Realization. Research Notes in Artificial Intelligence. Pitman and Morgan Kaufmann Publishers, Inc., London and San Mateo, Calif., 1988.Google Scholar
  15. 15.
    V. Fischer and H. Niemann. Parallelism in a semantic network for image understanding. In A. Bode and M. Dal Cin, editors, Parallel Computer Architectures. Theory, Hardware, Software, Applications, volume 732 of Lecture Notes in Computer Science, pages 203–218. Springer-Verlag, Berlin, 1993.Google Scholar
  16. 16.
    V. Fischer. Parallelverarbeitung in einem semantischen Netzwerk für die wissensbasierte Musteranalyse. Dissertation, Technische Fakultät, Universität Erlangen-Nürnberg, Erlangen, 1995.Google Scholar
  17. 17.
    B.W. Wah, G. Li, and C. Yu. Multiprocessing of combinatorial search problems. In [24], pages 103–145. 1990.Google Scholar
  18. 18.
    N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller. Equation of state calculations for fast computing machines. Journal of Chemical Physics, 21(6):1087–1092, 1953.Google Scholar
  19. 19.
    L. Booker, D. Goldberg, and J. Holland. Classifier systems and genetic algorithms. Artificial Intelligence, 40(1–3):235–282, 1989.Google Scholar
  20. 20.
    S. Steuer. Erstellung eines ersten Modells in ERNESTzur Identifikation der Straße und der Position des Kamerafahrzeugs im statischen Bild. Technical Report 3.2.B1 Projekt MOVIE, Bayerisches Forschungszentrum für Wissensbasierte Systeme (FORWISS) und Bayerische Motorenwerke AG (BMW AG), München, 1991.Google Scholar
  21. 21.
    G. Dueck and T. Scheuer. Threshold accepting: A general purpose optimization algorithm appearing superior to simulated annealing. Journal of Computational Physics, 90(1):161–175, 1990.Google Scholar
  22. 22.
    G. Dueck. New optimization heuristics: The great deluge algorithm and the record-to-record-travel. Journal of Computational Physics, 104(1):86–92, 1993.Google Scholar
  23. 23.
    V. Fischer, J. Fischer, and H. Niemann. An algorithm for any-time speech understanding. In German Slovenian Workshop on Image and Speech Understanding, to appear, Ljubljana, 1996.Google Scholar
  24. 24.
    V. Kumar, P. Gopalakrishnan, and L. Kumar, editors. Parallel Algorithms for Machine Intelligence and Vision. Springer-Verlag, New York, 1990.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1996

Authors and Affiliations

  • H. Niemann
    • 1
    • 2
  • V. Fischer
    • 1
    • 3
  • D. Paulus
    • 1
  • J. Fischer
    • 1
  1. 1.Lehrstuhl für Mustererkennung (Informatik 5)Universität Erlangen-NürnbergErlangenGermany
  2. 2.Forschungsgruppe WissensverarbeitungBayerisches Forschungszentrum für Wissensbasierte SystemeErlangenGermany
  3. 3.Institut für Logik und LinguistikIBM Deutschland GmbHHeidelbergGermany

Personalised recommendations