Journal of Mathematical Imaging and Vision

, Volume 7, Issue 4, pp 309–323 | Cite as

The Dynamics of Nonlinear Relaxation Labeling Processes

  • Marcello Pelillo


We present some new results which definitively explain thebehavior of the classical, heuristic nonlinear relaxation labelingalgorithm of Rosenfeld, Hummel, and Zucker in terms of theHummel-Zucker consistency theory and dynamical systems theory. Inparticular, it is shown that, when a certain symmetry condition is met,the algorithm possesses a Liapunov function which turns out to be (thenegative of) a well-known consistency measure. This follows almostimmediately from a powerful result of Baum and Eagon developed in thecontext of Markov chain theory. Moreover, it is seen that most of theessential dynamical properties of the algorithm are retained when thesymmetry restriction is relaxed. These properties are also shown tonaturally generalize to higher-order relaxation schemes. Someapplications and implications of the presented results are finallyoutlined.

relaxation labeling processes consistency growth transformations Liapunov functions stability 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Rosenfeld, R.A. Hummel, and S.W. Zucker, “Scene labeling by relaxation operations,” IEEE Trans. Syst. Man Cybern., Vol. 6, pp. 420–433, 1976.Google Scholar
  2. 2.
    J. Kittler and J. Illingworth, “Relaxation labeling algorithms—A review,” Image Vision Comput., Vol. 3, pp. 206–216, 1985.Google Scholar
  3. 3.
    S. Peleg, “A new probabilistic relaxation scheme,” IEEE Trans. Pattern Machine Intell., Vol. 2, pp. 362–369, 1980.Google Scholar
  4. 4.
    J. Kittler and E.R. Hancock, “Combining evidence in probabilistic relaxation,” Int. J. Pattern Recognition Artif. Intell., Vol. 3, pp. 29–51, 1989.Google Scholar
  5. 5.
    W.J. Christmas, J. Kittler, and M. Petrou, “Structural matching in computer vision using probabilistic relaxation,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 17, pp. 749–764, 1995.Google Scholar
  6. 6.
    R.A. Hummel and S.W. Zucker, “On the foundations of relaxation labeling processes,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 5, pp. 267–287, 1983.Google Scholar
  7. 7.
    A.J. Stoddart, M. Petrou, and J. Kittler, “On the foundations of probabilistic relaxation with product support,” submitted for publication, 1996.Google Scholar
  8. 8.
    S. Ullman, “Relaxation and constrained optimization by local processes,” Comput. Graph. Image Processing,Vol. 10, pp. 115–125, 1979.Google Scholar
  9. 9.
    O.D. Faugeras and M. Berthod, “Improving consistency and reducing ambiguity in stochastic labeling: An optimization approach,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 3, pp. 412–424, 1981.Google Scholar
  10. 10.
    J. Illingworth and J. Kittler, “Optimization algorithms in probabilistic relaxation labeling,” in Pattern Recognition Theory and Applications, P.A. Devijver and J. Kittler (Eds.), Springer-Verlag: Berlin, 1987, pp. 109–117.Google Scholar
  11. 11.
    E.R. Hancock and J. Kittler, “Discrete relaxation,” Pattern Recognition, Vol. 23, pp. 711–733, 1990.Google Scholar
  12. 12.
    S.A. Lloyd, “An optimization approach to relaxation labeling algorithms,” Image Vision Comput., Vol. 1, pp. 85–91, 1983.Google Scholar
  13. 13.
    T. Elfving and J.-O. Eklundh, “Some properties of stochastic labeling procedures,” Comput. Graph. Image Processing, Vol. 20, pp. 158–170, 1982.Google Scholar
  14. 14.
    M. Levy, “A new theoretical approach to relaxation—Application to edge detection,” in Proc. 9th Int. Conf. Pattern Recognition, Rome, Italy, 1988, pp. 208–212.Google Scholar
  15. 15.
    S. Peleg and A. Rosenfeld, “Determining compatibility coefficients for curve enhancement relaxation processes,” IEEE Trans. Syst. Man Cybern., Vol. 8, pp. 548–555, 1978.Google Scholar
  16. 16.
    R.C. Wilson and E.R. Hancock, “A Bayesian compatibility model for graph matching,” Pattern Recognition Lett., Vol. 17, pp. 263–276, 1996.Google Scholar
  17. 17.
    M. Pelillo and M. Refice, “Learning compatibility coefficients for relaxation labeling processes,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 16, pp. 933–945, 1994.Google Scholar
  18. 18.
    M. Pelillo, F. Abbattista, and A. Maffione, “An evolutionary approach to training relaxation labeling processes,” Pattern Recognition Lett., Vol. 16, pp. 1069–1078, 1995.Google Scholar
  19. 19.
    M. Pelillo and A.M. Fanelli, “Autoassociative learning in relaxation labeling networks,” in Proc. 13th Int. Conf. Pattern Recognition, Vienna, Austria, 1996.Google Scholar
  20. 20.
    R.L. Kirby, “A product rule relaxation method,” Comput. Graph. Image Processing, Vol. 13, pp. 158–189, 1980.Google Scholar
  21. 21.
    E.R. Hancock and J. Kittler, “Edge-labeling using dictionary-based relaxation,”IEEE Trans. Pattern Anal. Machine Intell., Vol. 12, pp. 165–181, 1990.Google Scholar
  22. 22.
    S.W. Zucker, E.V. Krishnamurthy, and R.L. Haar, “Relaxation processes for scene labeling: Convergence, speed, and stability,” IEEE Trans. Syst. Man Cybern., Vol. 8, pp. 41–48, 1978.Google Scholar
  23. 23.
    X. Zhuang, R.M. Haralick, and H. Joo, “A simplex-like algorithm for the relaxation labeling process,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 11, pp. 1316–1321, 1989.Google Scholar
  24. 24.
    L.E. Baum and J.A. Eagon, “An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology,” Bull. Amer. Math. Soc., Vol. 73, pp. 360–363, 1967.Google Scholar
  25. 25.
    G.R. Blakley, “Homogeneous nonnegative symmetric quadratic transformations,” Bull. Amer. Math. Soc., Vol. 70, pp. 712–715, 1964.Google Scholar
  26. 26.
    L.E. Baum and G.R. Sell, “Growth transformations for functions on manifolds,” Pacific J. Math., Vol. 27, pp. 211–227, 1968.Google Scholar
  27. 27.
    J.L. Mohammed, R.A. Hummel, and S.W. Zucker, “A gradient projection algorithm for relaxation methods,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 5, pp. 330–332, 1983.Google Scholar
  28. 28.
    L.E. Baum, T. Petrie, G. Soules, and N. Weiss, “A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains,” Ann. Math. Statist., Vol. 41, pp. 164–171, 1970.Google Scholar
  29. 29.
    S.E. Levinson, L.R. Rabiner, and M.M. Sondhi, “An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition,” Bell. Syst. Tech. J., Vol. 62, pp. 1035–1074, 1983.Google Scholar
  30. 30.
    P. Stebe, “Invariant functions of an iterative process for maximization of a polynomial,” Pacific J. Math., Vol. 43, pp. 765–783, 1972.Google Scholar
  31. 31.
    D.S. Passman, “The Jacobian of a growth transformation,” Pacific J. Math., Vol. 44, pp. 281–290, 1973.Google Scholar
  32. 32.
    P.S. Gopalakrishnan, D. Kanevsky, A. Nádas, and D. Nahamoo, “An inequality for rational functions with applications to some statistical estimation problems,” IEEE Trans. Inform. Theory, Vol. 37, pp. 107–113, 1991.Google Scholar
  33. 33.
    D.G. Luenberger, Introduction to Dynamic Systems, Wiley: New York, 1979.Google Scholar
  34. 34.
    J.P. LaSalle, The Stability and Control of Discrete Processes, Springer-Verlag: New York, 1986.Google Scholar
  35. 35.
    S.W. Zucker, A. Dobbins, and L. Iverson, “Two stages of curve detection suggest two styles of visual computation,” Neural Computat., Vol. 1, pp. 68–81, 1989.Google Scholar
  36. 36.
    G. Zoutendijk, Mathematical Programming Methods, North-Holland: Amsterdam, 1976.Google Scholar
  37. 37.
    P. Parent and S.W. Zucker, “Radial projection: An efficient update rule for relaxation labeling,” IEEE Trans. Pattern Anal. Machine Intell., Vol. 11, pp. 886–889, 1989.Google Scholar
  38. 38.
    A.M. Ostrowski, Solution of Equations and Systems of Equations, Academic Press: New York, 1966.Google Scholar
  39. 39.
    J.-O. Eklundh and A. Rosenfeld, “Some relaxation experiments using triples of pixels,” IEEE Trans. Syst. Man Cybern., Vol. 10, pp. 150–153, 1980.Google Scholar
  40. 40.
    S. Peleg, “Ambiguity reduction in handwriting with ambiguous segmentation and uncertain interpretation,” Comput. Graph. Image Processing, Vol. 10, pp. 235–245, 1979.Google Scholar
  41. 41.
    J. Kittler and E.R. Hancock, “Contextual decision rule for region analysis,” Image Vision Comput., Vol. 5, pp. 145–154, 1987.Google Scholar
  42. 42.
    A.M. Finch, R.C. Wilson, and E.R. Hancock, “Matching Delaunay triangulations by relaxation labeling,” in Computer Analysis of Images and Patterns, V. Hlavác and R. Sára (Eds.), Springer: Berlin, 1995, pp. 350–358.Google Scholar
  43. 43.
    C.L. Giles and T. Maxwell, “Learning, invariance, and generalization in high-order neural networks,” Appl. Optics, Vol. 26, pp. 4972–4978, 1987.Google Scholar
  44. 44.
    D. Psaltis and C.H. Park, “Nonlinear discriminant functions and associative memories,” in Neural Networks for Computing, J.S. Denker (Ed.), American Institute of Physics: New York, 1986, pp. 370–375.Google Scholar
  45. 45.
    R. Durbin and D.E. Rumelhart, “Product units: A computationally powerful and biologically plausible extension to backpropagation networks,” Neural Computat., Vol. 1, pp. 133–142, 1989.Google Scholar
  46. 46.
    J.J. Hopfield and D.W. Tank, “Neural computation of decisions in optimization problems,” Biol. Cybern., Vol. 52, pp. 141–152, 1985.Google Scholar
  47. 47.
    D.E. Van den Bout and T.K. Miller, “A traveling salesman objective function that works,” in Proc. IEEE Int. Conf. Neural Networks, San Diego, CA, 1988, pp. 299–303.Google Scholar
  48. 48.
    M. Pelillo, “Relaxation labeling processes for the traveling salesman problem,” in Proc. Int. J. Conf. Neural Networks, Nagoya, Japan, 1993, pp. 2429–2432.Google Scholar
  49. 49.
    M. Pelillo, “Relaxation labeling networks for the maximum clique problem,” J. Artif. Neural Networks, Special issue on “Neural Networks for Optimization,” Vol. 2, pp. 313–328, 1995.Google Scholar
  50. 50.
    G.V. Wilson and G.S. Pawley, “On the stability of the traveling salesman problem algorithm of Hopfield and Tank,” Biol. Cybern., Vol. 58, pp. 63–70, 1988.Google Scholar
  51. 51.
    T. Poggio, V. Torre, and C. Koch, “Computational vision and regularization theory,” Nature, Vol. 317, pp. 314–319, 1985.Google Scholar
  52. 52.
    N.M. Nasrabadi and C.Y. Choo, “Hopfield network for stereo vision correspondence,” IEEE Trans. Neural Networks, Vol. 3, pp. 5–13, 1992.Google Scholar
  53. 53.
    J.J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA, Vol. 79, pp. 2554–2558, 1982.Google Scholar
  54. 54.
    D. Valentin, H. Abdi, A.J. O’Toole, and G.W. Cottrell, “Connectionist models of face processing: A survey,” Pattern Recognition, Vol. 27, pp. 1209–1230, 1994.Google Scholar
  55. 55.
    H. Wechsler, “Network representations and match filters for invariant object recognition,” in Pattern Recognition Theory and Applications, P.A. Devijver and J. Kittler (Eds.), Springer-Verlag: Berlin, 1987, pp. 269–276.Google Scholar
  56. 56.
    M. Pelillo and A.M. Fanelli, “An asymmetric associative memory model based on relaxation labeling processes,” in Proc. ESANN’95—3rd Europ. Symp. Artif. Neural Networks, Brussels, Belgium, 1995, pp. 223–228.Google Scholar
  57. 57.
    G.A. Kohring, “Finite-state neural networks: A step toward the simulation of very large systems,” J. Stat. Phys., Vol. 62, pp. 563–576, 1991.Google Scholar
  58. 58.
    F. Crick and C. Asanuma, “Certain aspects of the anatomy and physiology of the cerebral cortex,” in Parallel Distributed Processing. Vol 2: Psychological and Biological Models, J.L. McClelland and D.E. Rumelhart (Eds.), MIT Press: Cambridge, MA, 1986, pp. 333–371.Google Scholar
  59. 59.
    J.A. Anderson, “Cognitive and psychological computation with neural models,” IEEE Trans. Syst. Man Cybern., Vol. 13, pp. 799–815, 1983.Google Scholar
  60. 60.
    J.A. Anderson and G.E. Hinton, “Models of information processing in the brain,” in Parallel Models of Associative Memory, G.E. Hinton and J.A. Anderson (Eds.), Erlbaum: Hillsdale, NJ, 1981, pp. 9–48.Google Scholar
  61. 61.
    D.J. Amit, Modeling Brain Function, Cambridge University Press: Cambridge, UK, 1989.Google Scholar
  62. 62.
    T. Hogg and B.A. Huberman, “Understanding biological computation: Reliable learning and recognition,” Proc. Natl. Acad. Sci. USA, Vol. 81, pp. 6871–6875, 1984.Google Scholar
  63. 63.
    M.A. Cohen and S. Grossberg, “Absolute stability of global pattern formation and parallel memory storage by competitive neural networks,” IEEE Trans. Syst. Man Cybern., Vol. 13, pp. 815–826, 1983.Google Scholar
  64. 64.
    G.E. Hinton and T.J. Sejnowski, “Learning and relearning in Boltzmann machines,” in Parallel Distributed Processing. Vol 1: Foundations, D.E. Rumelhart and J.L. McClelland (Eds.), MIT Press: Cambridge, MA, 1986, pp. 282–317.Google Scholar
  65. 65.
    C. Torras, “Relaxation and neural learning: Points of convergence and divergence,” J. Parallel Distrib. Computing, Vol. 6, pp. 217–244, 1989.Google Scholar
  66. 66.
    S.S. Yu and W.H. Tsai, “Relaxation by the Hopfield neural network,” Pattern Recognition, Vol. 25, pp. 197–209, 1992.Google Scholar

Copyright information

© Kluwer Academic Publishers 1997

Authors and Affiliations

  • Marcello Pelillo
    • 1
  1. 1.Dipartimento di Matematica Applicata e InformaticaUniversità “Ca'Foscari” di VeneziaVenezia MestreItaly

Personalised recommendations