Incorporating knowledge via regularization theory: applications in vision and image processing

  • David Suter
  • Harvey A. Cohen
Vision And Robotics
Part of the Lecture Notes in Computer Science book series (LNCS, volume 406)


In computer vision one is faced with the task of discovering the nature of objects that produced the light intensity distribution received by the camera. In order to solve this problem, one usually uses knowledge of the optics, the reflectance properties of surfaces, and of the structure of the objects one is expecting to find in a given scene. In many problems in image processing one is trying to use knowledge of the imaged objects, or of the image formation process, to reconstruct a better image from a degraded image.

A proper computational formulation of these problems recognises that they involve inverse processes that are mathematically ill-posed. This allows one to derive systematically, computational schemes based upon regularization theory that ensure existence of solution and stability of inversion process.

Such approaches often suggest particular types of algorithms for efficient solution to the problems. These, in turn, often are suggestive of implementations that are highly parallel, require only local information and simple operations. These implementations have the essential features of neural network approaches to computation; we present simulations of these.

Keywords and phrases

Computer Vision Image Processing Regularization Ill-Posed Neural Network 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    B. K. P. Horn and M. J. Brooks ”The Variational Approach to Shape from Shading,” Computer Vision, Graphics, and Image Processing, Vol. 33, pp. 174–208, 1986.Google Scholar
  2. [2]
    D. Marr Vision, Freeman, San Francisco, 1982.Google Scholar
  3. [3]
    V. A. Mozorov ”Methods for Solving Incorrectly Posed Problems,” Springer-Verlag, New York, 1984.Google Scholar
  4. [4]
    D. Terzopoulos ”Regularization of Inverse Visual Problems Involving Discontinuities,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 8 No. 4, pp. 413–424, June 1986.Google Scholar
  5. [5]
    T. Poggio and C. Koch ”Ill-posed problems in early vision: from computational theory to analogue networks,” Proc. R. Soc. Lond, Vol. B 226, pp. 303–323, 1985Google Scholar
  6. [6]
    M. Bertero, P. Brianzi, E. R. Pike, and L. Rebolia ”Linear regularizing algorithms for positive solutions of linear inverse problems,” Proc. R. Soc. Lond., A, Vol. 425, pp. 257–275, 1988.Google Scholar
  7. [7]
    A. N. Tikhonov, A. V. Goncharskii, V. V. Stepanov, and I. V. Kochikov, ”Ill-posed image processing problems,” Sov. Phys. Dokl., Vol. 32 No. 6, pp. 456–458, June 1987.Google Scholar
  8. [8]
    T. J. Carmody ”Diagnostics for Multivariate Smoothing Splines,” J. of Statistical Planning and Inference, Vol. 19, pp. 171–186, 1988.Google Scholar
  9. [9]
    D. Suter and X. Deng ”Neural Net Simulation on Transputers,” Proc. IEEE Systems, Man, and Cybernetics Conf., Beijing, Aug. 1988, (to appear).Google Scholar
  10. [10]
    D. Shulman and J. Aloimonos ”Boundary Preserving Regularization: Theory Part 1,” University of Maryland, Computer Vision Lab., Tech Report CS-TR-2011, April 1988.Google Scholar
  11. [11]
    S. Y. Kung ”VLSI Array Processors,” Prentice-Hall, London, 1988.Google Scholar
  12. [12]
    A. N. Tikhonov and A. V. Goncharsky (Eds.) ”Ill-Posed Problems in the Natural Sciences,” MIT Publishers, Moscow, 1987.Google Scholar
  13. [13]
    D. Shulman and J. Y. Aloimonos ”(Non)-rigid motion interpretation: a regularized approach,” Proc. R. Soc. Lond. B, pp. 217–234, 1988.Google Scholar
  14. [14]
    S. P. Luttrell ”The use of Markov random field models to derive sampling schemes for inverse texture problems,” Inverse Problems, Vol. 3, pp. 289–300, 1987.Google Scholar
  15. [15]
    S. P. Luttrell and C. J. Oliver ”The role of prior knowledge in coherent image processing,” Phil. Trans. R. Soc. Lond. A, Vol. 324, pp. 297–298, 1988.Google Scholar
  16. [16]
    J. J. Hopfield, ”Neural Networks and Physical Systems with Emergent Collective Computational Abilities,” Proc. Natl. Acad. Sci. USA, Vol. 79, pp. 2554–2558, 1982.Google Scholar
  17. [17]
    M. Mezard, G. Parisi, and M. Virasoro, ”Spin Glass Theory and Beyond,” World Scientific, Singapore, 1987.Google Scholar
  18. [18]
    D. Chowdhury, ”Spin Glasses and Other Frustrated Systems,” World Scientific, Singapore, 1987.Google Scholar
  19. [19]
    J. L. van Hemmen and I. Morgenstern (Ed.), ”Heidelberg Colloquium on Glassy Dynamics,” Lecture Notes in Physics 275, Springer Verlag, Berlin, 1987.Google Scholar
  20. [20]
    P. J. M. van Laarhoven and E. H. L. Aarts, ”Simulated Annealing: Theory and Applications,” Reidel, Dordrecht, 1987.Google Scholar
  21. [21]
    W. Jeffrey and R. Rosner, ”Neural Network Processing as a Tool for Function Optimization,” AIP Conference Proc. 151 Neural Networks for Computing, Snowbird, Utah U.S.A., pp. 241–246, 1986.Google Scholar
  22. [22]
    W. Jeffrey and R. Rosner, ”Optimization Algorithms: Simulated Annealing and Neural Network Processing,” The Astrophysical Journal, Vol. 310, pp. 473–481, Nov. 1986.Google Scholar
  23. [23]
    S. Geman and D. Geman ”Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE PAMI, Vol. 6, pp. 721–741, 1984.Google Scholar
  24. [24]
    J. L. Marroquin ”Probabilistic Solution of Inverse Problems,” PhD thesis, MIT, Cambridge MA, Sept. 1985.Google Scholar
  25. [25]
    J. Marroquin, S. Mitter, and T. Poggio, ”Probabilistic solution of ill-posed problems in computational vision,” J. Am. Stat. Assoc., Vol. 82, pp. 76–89, 1987Google Scholar
  26. [26]
    D. Geman, ”Stochastic model for boundary detection,” Image and Vision Computing, Vol. 5 No. 2, pp. 61–66, May 1987.Google Scholar
  27. [27]
    D. W. Murray, A. Kashko and H. Buxton, ”A parallel approach to the picture restoration algorithm of Geman and Geman on an SIMD machine,” Image and Vision Computing, Vol. 4 No. 3, pp. 133–142, Aug. 1986.Google Scholar
  28. [28]
    B. F. Buxton and D. W. Murray, ”Optic flow segmentation as an ill-posed and maximum likelihood problem,” Image and Vision Computing, Vol. 3 No. 4, pp. 163–169, Nov. 1985.Google Scholar
  29. [29]
    E. Gamble and T. Poggio, ”Visual Integration and Detection of Discontinuities: The Key Role of Intensity Edges,” ”MIT A.I. Memo No. 970, Oct. 1987.Google Scholar
  30. [30]
    J. M. Hutchinson and C. Koch, ”Simple Analog and Hybrid Networks for Surface Interpolation,” AIP Conference Proc. 151 Neural Networks for Computing, Snowbird, Utah U.S.A., pp. 235–240, 1986.Google Scholar
  31. [31]
    A. Blake and A. Zisserman, ”Visual Reconstruction,” MIT Press, Cambridge Mass., 1987.Google Scholar
  32. [32]
    J. M. Hutchinson, C. Koch, J. Luo and C. Mead, ”Computing Motion using Analog and Binary Resistive Networks,” IEEE Computer, pp. 52–63, March 1988.Google Scholar
  33. [33]
    D. M. Titterington ”General structure of regularization procedures in image reconstruction,” Astron. Astrophys., Vol. 144, pp. 381–381, 1985.Google Scholar
  34. [34]
    N. M. Grzywacz and A. Yuille, ”Motion Correspondence and Analog Networks,” AIP Conference Proc. 151 Neural Networks for Computing, Snowbird, Utah U.S.A., pp. 200–205, 1986.Google Scholar
  35. [35]
    B. M. Forrest, D. Roweth, N. Stroud, D.J. Wallace and G.V. Wilson, ”Implementing Neural Network Models on Parallel Computers,” The Computer Journal, Vol. 30 No. 5, pp. 413–419, 1987.Google Scholar
  36. [36]
    E. Goles and G. Y. Vichniac, ”Lyapunov Function for Parallel Neural Networks,” AIP Conference Proc. 151 Neural Networks for Computing, Snowbird, Utah U.S.A., pp. 129–134, 1986.Google Scholar
  37. [37]
    R. Divko and K. Schulten, ”Stochastic Spin Models for Pattern Recognition,” AIP Conference Proc. 151 Neural Networks for Computing, Snowbird, Utah U.S.A., pp. 129–134, 1986.Google Scholar
  38. [38]
    J. L. Marroquin, ”Deterministic Bayesian Estimation of Markov Random Fields with Applications in Computational Vision,” Proc. 1st Int. Conf. on Computer Vision, pp. 597–601, June 8–11, 1987.Google Scholar
  39. [39]
    D. Terzopoulos, ”Multi-Level Reconstruction of Visual Surfaces,” MIT A.I. memo 671, April 1982.Google Scholar
  40. [40]
    J. Franklin ”Well-Posed Stochastic Extensions of Ill-Posed Linear Problems,” J. of Mathematical Analysis and Applications, Vol. 31, pp. 682–716, 1970.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1990

Authors and Affiliations

  • David Suter
    • 1
  • Harvey A. Cohen
    • 1
  1. 1.La Trobe UniversityBundooraAustralia

Personalised recommendations