# The Dynamics of Nonlinear Relaxation Labeling Processes

- 148 Downloads
- 52 Citations

## Abstract

We present some new results which definitively explain thebehavior of the classical, heuristic nonlinear relaxation labelingalgorithm of Rosenfeld, Hummel, and Zucker in terms of theHummel-Zucker consistency theory and dynamical systems theory. Inparticular, it is shown that, when a certain symmetry condition is met,the algorithm possesses a Liapunov function which turns out to be (thenegative of) a well-known consistency measure. This follows almostimmediately from a powerful result of Baum and Eagon developed in thecontext of Markov chain theory. Moreover, it is seen that most of theessential dynamical properties of the algorithm are retained when thesymmetry restriction is relaxed. These properties are also shown tonaturally generalize to higher-order relaxation schemes. Someapplications and implications of the presented results are finallyoutlined.

## Preview

Unable to display preview. Download preview PDF.

## References

- 1.A. Rosenfeld, R.A. Hummel, and S.W. Zucker, “Scene labeling by relaxation operations,”
*IEEE Trans. Syst. Man Cybern.*, Vol. 6, pp. 420–433, 1976.Google Scholar - 2.J. Kittler and J. Illingworth, “Relaxation labeling algorithms—A review,”
*Image Vision Comput.*, Vol. 3, pp. 206–216, 1985.Google Scholar - 3.S. Peleg, “A new probabilistic relaxation scheme,”
*IEEE Trans. Pattern Machine Intell.*, Vol. 2, pp. 362–369, 1980.Google Scholar - 4.J. Kittler and E.R. Hancock, “Combining evidence in probabilistic relaxation,”
*Int. J. Pattern Recognition Artif. Intell.*, Vol. 3, pp. 29–51, 1989.Google Scholar - 5.W.J. Christmas, J. Kittler, and M. Petrou, “Structural matching in computer vision using probabilistic relaxation,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 17, pp. 749–764, 1995.Google Scholar - 6.R.A. Hummel and S.W. Zucker, “On the foundations of relaxation labeling processes,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 5, pp. 267–287, 1983.Google Scholar - 7.A.J. Stoddart, M. Petrou, and J. Kittler, “On the foundations of probabilistic relaxation with product support,” submitted for publication, 1996.Google Scholar
- 8.S. Ullman, “Relaxation and constrained optimization by local processes,”
*Comput. Graph. Image Processing*,Vol. 10, pp. 115–125, 1979.Google Scholar - 9.O.D. Faugeras and M. Berthod, “Improving consistency and reducing ambiguity in stochastic labeling: An optimization approach,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 3, pp. 412–424, 1981.Google Scholar - 10.J. Illingworth and J. Kittler, “Optimization algorithms in probabilistic relaxation labeling,” in
*Pattern Recognition Theory and Applications*, P.A. Devijver and J. Kittler (Eds.), Springer-Verlag: Berlin, 1987, pp. 109–117.Google Scholar - 11.E.R. Hancock and J. Kittler, “Discrete relaxation,”
*Pattern Recognition*, Vol. 23, pp. 711–733, 1990.Google Scholar - 12.S.A. Lloyd, “An optimization approach to relaxation labeling algorithms,”
*Image Vision Comput.*, Vol. 1, pp. 85–91, 1983.Google Scholar - 13.T. Elfving and J.-O. Eklundh, “Some properties of stochastic labeling procedures,”
*Comput. Graph. Image Processing*, Vol. 20, pp. 158–170, 1982.Google Scholar - 14.M. Levy, “A new theoretical approach to relaxation—Application to edge detection,” in
*Proc. 9th Int. Conf. Pattern Recognition*, Rome, Italy, 1988, pp. 208–212.Google Scholar - 15.S. Peleg and A. Rosenfeld, “Determining compatibility coefficients for curve enhancement relaxation processes,”
*IEEE Trans. Syst. Man Cybern.*, Vol. 8, pp. 548–555, 1978.Google Scholar - 16.R.C. Wilson and E.R. Hancock, “A Bayesian compatibility model for graph matching,”
*Pattern Recognition Lett.*, Vol. 17, pp. 263–276, 1996.Google Scholar - 17.M. Pelillo and M. Refice, “Learning compatibility coefficients for relaxation labeling processes,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 16, pp. 933–945, 1994.Google Scholar - 18.M. Pelillo, F. Abbattista, and A. Maffione, “An evolutionary approach to training relaxation labeling processes,”
*Pattern Recognition Lett.*, Vol. 16, pp. 1069–1078, 1995.Google Scholar - 19.M. Pelillo and A.M. Fanelli, “Autoassociative learning in relaxation labeling networks,” in
*Proc. 13th Int. Conf. Pattern Recognition*, Vienna, Austria, 1996.Google Scholar - 20.R.L. Kirby, “A product rule relaxation method,”
*Comput. Graph. Image Processing*, Vol. 13, pp. 158–189, 1980.Google Scholar - 21.E.R. Hancock and J. Kittler, “Edge-labeling using dictionary-based relaxation,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 12, pp. 165–181, 1990.Google Scholar - 22.S.W. Zucker, E.V. Krishnamurthy, and R.L. Haar, “Relaxation processes for scene labeling: Convergence, speed, and stability,”
*IEEE Trans. Syst. Man Cybern.*, Vol. 8, pp. 41–48, 1978.Google Scholar - 23.X. Zhuang, R.M. Haralick, and H. Joo, “A simplex-like algorithm for the relaxation labeling process,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 11, pp. 1316–1321, 1989.Google Scholar - 24.L.E. Baum and J.A. Eagon, “An inequality with applications to statistical estimation for probabilistic functions of Markov processes and to a model for ecology,”
*Bull. Amer. Math. Soc.*, Vol. 73, pp. 360–363, 1967.Google Scholar - 25.G.R. Blakley, “Homogeneous nonnegative symmetric quadratic transformations,”
*Bull. Amer. Math. Soc.*, Vol. 70, pp. 712–715, 1964.Google Scholar - 26.L.E. Baum and G.R. Sell, “Growth transformations for functions on manifolds,”
*Pacific J. Math.*, Vol. 27, pp. 211–227, 1968.Google Scholar - 27.J.L. Mohammed, R.A. Hummel, and S.W. Zucker, “A gradient projection algorithm for relaxation methods,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 5, pp. 330–332, 1983.Google Scholar - 28.L.E. Baum, T. Petrie, G. Soules, and N. Weiss, “A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains,”
*Ann. Math. Statist.*, Vol. 41, pp. 164–171, 1970.Google Scholar - 29.S.E. Levinson, L.R. Rabiner, and M.M. Sondhi, “An introduction to the application of the theory of probabilistic functions of a Markov process to automatic speech recognition,”
*Bell. Syst. Tech. J.*, Vol. 62, pp. 1035–1074, 1983.Google Scholar - 30.P. Stebe, “Invariant functions of an iterative process for maximization of a polynomial,”
*Pacific J. Math.*, Vol. 43, pp. 765–783, 1972.Google Scholar - 31.D.S. Passman, “The Jacobian of a growth transformation,”
*Pacific J. Math.*, Vol. 44, pp. 281–290, 1973.Google Scholar - 32.P.S. Gopalakrishnan, D. Kanevsky, A. Nádas, and D. Nahamoo, “An inequality for rational functions with applications to some statistical estimation problems,”
*IEEE Trans. Inform. Theory*, Vol. 37, pp. 107–113, 1991.Google Scholar - 33.D.G. Luenberger,
*Introduction to Dynamic Systems*, Wiley: New York, 1979.Google Scholar - 34.J.P. LaSalle,
*The Stability and Control of Discrete Processes*, Springer-Verlag: New York, 1986.Google Scholar - 35.S.W. Zucker, A. Dobbins, and L. Iverson, “Two stages of curve detection suggest two styles of visual computation,”
*Neural Computat.*, Vol. 1, pp. 68–81, 1989.Google Scholar - 36.G. Zoutendijk,
*Mathematical Programming Methods*, North-Holland: Amsterdam, 1976.Google Scholar - 37.P. Parent and S.W. Zucker, “Radial projection: An efficient update rule for relaxation labeling,”
*IEEE Trans. Pattern Anal. Machine Intell.*, Vol. 11, pp. 886–889, 1989.Google Scholar - 38.A.M. Ostrowski,
*Solution of Equations and Systems of Equations*, Academic Press: New York, 1966.Google Scholar - 39.J.-O. Eklundh and A. Rosenfeld, “Some relaxation experiments using triples of pixels,”
*IEEE Trans. Syst. Man Cybern.*, Vol. 10, pp. 150–153, 1980.Google Scholar - 40.S. Peleg, “Ambiguity reduction in handwriting with ambiguous segmentation and uncertain interpretation,”
*Comput. Graph. Image Processing*, Vol. 10, pp. 235–245, 1979.Google Scholar - 41.J. Kittler and E.R. Hancock, “Contextual decision rule for region analysis,”
*Image Vision Comput.*, Vol. 5, pp. 145–154, 1987.Google Scholar - 42.A.M. Finch, R.C. Wilson, and E.R. Hancock, “Matching Delaunay triangulations by relaxation labeling,” in
*Computer Analysis of Images and Patterns*, V. Hlavác and R. Sára (Eds.), Springer: Berlin, 1995, pp. 350–358.Google Scholar - 43.C.L. Giles and T. Maxwell, “Learning, invariance, and generalization in high-order neural networks,”
*Appl. Optics*, Vol. 26, pp. 4972–4978, 1987.Google Scholar - 44.D. Psaltis and C.H. Park, “Nonlinear discriminant functions and associative memories,” in
*Neural Networks for Computing*, J.S. Denker (Ed.), American Institute of Physics: New York, 1986, pp. 370–375.Google Scholar - 45.R. Durbin and D.E. Rumelhart, “Product units: A computationally powerful and biologically plausible extension to backpropagation networks,”
*Neural Computat.*, Vol. 1, pp. 133–142, 1989.Google Scholar - 46.J.J. Hopfield and D.W. Tank, “Neural computation of decisions in optimization problems,”
*Biol. Cybern.*, Vol. 52, pp. 141–152, 1985.Google Scholar - 47.D.E. Van den Bout and T.K. Miller, “A traveling salesman objective function that works,” in
*Proc. IEEE Int. Conf. Neural Networks*, San Diego, CA, 1988, pp. 299–303.Google Scholar - 48.M. Pelillo, “Relaxation labeling processes for the traveling salesman problem,” in
*Proc. Int. J. Conf. Neural Networks*, Nagoya, Japan, 1993, pp. 2429–2432.Google Scholar - 49.M. Pelillo, “Relaxation labeling networks for the maximum clique problem,”
*J. Artif. Neural Networks*, Special issue on “Neural Networks for Optimization,” Vol. 2, pp. 313–328, 1995.Google Scholar - 50.G.V. Wilson and G.S. Pawley, “On the stability of the traveling salesman problem algorithm of Hopfield and Tank,”
*Biol. Cybern.*, Vol. 58, pp. 63–70, 1988.Google Scholar - 51.T. Poggio, V. Torre, and C. Koch, “Computational vision and regularization theory,”
*Nature*, Vol. 317, pp. 314–319, 1985.Google Scholar - 52.N.M. Nasrabadi and C.Y. Choo, “Hopfield network for stereo vision correspondence,”
*IEEE Trans. Neural Networks*, Vol. 3, pp. 5–13, 1992.Google Scholar - 53.J.J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,”
*Proc. Natl. Acad. Sci. USA*, Vol. 79, pp. 2554–2558, 1982.Google Scholar - 54.D. Valentin, H. Abdi, A.J. O’Toole, and G.W. Cottrell, “Connectionist models of face processing: A survey,”
*Pattern Recognition*, Vol. 27, pp. 1209–1230, 1994.Google Scholar - 55.H. Wechsler, “Network representations and match filters for invariant object recognition,” in
*Pattern Recognition Theory and Applications*, P.A. Devijver and J. Kittler (Eds.), Springer-Verlag: Berlin, 1987, pp. 269–276.Google Scholar - 56.M. Pelillo and A.M. Fanelli, “An asymmetric associative memory model based on relaxation labeling processes,” in
*Proc. ESANN’95—3rd Europ. Symp. Artif. Neural Networks*, Brussels, Belgium, 1995, pp. 223–228.Google Scholar - 57.G.A. Kohring, “Finite-state neural networks: A step toward the simulation of very large systems,”
*J. Stat. Phys.*, Vol. 62, pp. 563–576, 1991.Google Scholar - 58.F. Crick and C. Asanuma, “Certain aspects of the anatomy and physiology of the cerebral cortex,” in
*Parallel Distributed Processing. Vol 2: Psychological and Biological Models*, J.L. McClelland and D.E. Rumelhart (Eds.), MIT Press: Cambridge, MA, 1986, pp. 333–371.Google Scholar - 59.J.A. Anderson, “Cognitive and psychological computation with neural models,”
*IEEE Trans. Syst. Man Cybern.*, Vol. 13, pp. 799–815, 1983.Google Scholar - 60.J.A. Anderson and G.E. Hinton, “Models of information processing in the brain,” in
*Parallel Models of Associative Memory*, G.E. Hinton and J.A. Anderson (Eds.), Erlbaum: Hillsdale, NJ, 1981, pp. 9–48.Google Scholar - 61.D.J. Amit,
*Modeling Brain Function*, Cambridge University Press: Cambridge, UK, 1989.Google Scholar - 62.T. Hogg and B.A. Huberman, “Understanding biological computation: Reliable learning and recognition,”
*Proc. Natl. Acad. Sci. USA*, Vol. 81, pp. 6871–6875, 1984.Google Scholar - 63.M.A. Cohen and S. Grossberg, “Absolute stability of global pattern formation and parallel memory storage by competitive neural networks,”
*IEEE Trans. Syst. Man Cybern.*, Vol. 13, pp. 815–826, 1983.Google Scholar - 64.G.E. Hinton and T.J. Sejnowski, “Learning and relearning in Boltzmann machines,” in
*Parallel Distributed Processing. Vol 1: Foundations*, D.E. Rumelhart and J.L. McClelland (Eds.), MIT Press: Cambridge, MA, 1986, pp. 282–317.Google Scholar - 65.C. Torras, “Relaxation and neural learning: Points of convergence and divergence,”
*J. Parallel Distrib. Computing*, Vol. 6, pp. 217–244, 1989.Google Scholar - 66.S.S. Yu and W.H. Tsai, “Relaxation by the Hopfield neural network,”
*Pattern Recognition*, Vol. 25, pp. 197–209, 1992.Google Scholar