Skip to main content
SpringerLink
Log in

Mumford-Shah Regularizer with Contextual Feedback

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

We present a simple and robust feature preserving image regularization by letting local region measures modulate the diffusivity. The purpose of this modulation is to disambiguate low level cues in early vision. We interpret the Ambrosio-Tortorelli approximation of the Mumford-Shah model as a system with modulatory feedback and utilize this interpretation to integrate high level information into the regularization process. The method does not require any prior model or learning; the high level information is extracted from local regions and fed back to the regularization step. An important characteristic of the method is that both negative and positive feedback can be simultaneously used without creating oscillations. Experiments performed with both gray and color natural images demonstrate the potential of the method under difficult noise types, non-uniform contrast, existence of multi-scale patterns and textures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Symposium on the Role of Context in Recognition, European Conference on Visual Perception, August 2005

  2. Aleman-Flores, M., Alvarez, L., Caselles, V.: Texture-oriented anisotropic filtering and geodesic active contours in breast tumor ultrasound segmentation. J. Math. Imaging Vis. 28(1), 81–97 (2007)

    Article  MathSciNet  Google Scholar 

  3. Alicandro, R., Braides, A., Shah, J.: Free-discontinuity problems via functionals involving the L 1-norm of the gradient and their approximation. Interfaces Free Bound. 1(1), 17–37 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  4. Ambrosio, L., Tortorelli, V.: On the approximation of functionals depending on jumps by elliptic functionals via Γ-convergence. Commun. Pure Appl. Math. 43(8), 999–1036 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  5. Aslan, C., Tari, S.: An axis-based representation for recognition. In: ICCV, vol. 2, pp. 1339–1346 (2005)

  6. Aujol, J.-F., Aubert, G., Chambolle, A.: Image decomposition into a bounded variation component and an oscillating component. J. Math. Imaging Vis. 22(1), 71–88 (2005)

    Article  MathSciNet  Google Scholar 

  7. Bar, L., Kiryati, N., Sochen, N.: Image deblurring in the presence of impulsive noise. Int. J. Comput. Vis. 70(3), 279–298 (2006)

    Article  Google Scholar 

  8. Bar, L., Sochen, N., Kiryati, N.: Image deblurring in the presence of salt-and-pepper noise. In: Scale-Space, pp. 107–118 (2005)

  9. Bar, M.: Visual objects in context. Nat. Rev. Neurosci. 5, 617–629 (2004)

    Article  Google Scholar 

  10. Bayerl, P., Neumann, H.: Disambiguating visual motion through contextual feedback modulation. Neural Comput. 16, 2041–2066 (2004)

    Article  MATH  Google Scholar 

  11. Black, M.J., Rangarajan, A.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Comput. Vis. 19(1), 57–91 (1996)

    Article  Google Scholar 

  12. Black, M.J., Sapiro, G.: Edges as outliers: anisotropic smoothing using local image statistics. In: Scale-Space, pp. 259–270 (1999)

  13. Black, M.J., Sapiro, G., Marimont, D.H., Heeger, D.: Robust anisotropic diffusion. IEEE Trans. Image Process. 7(3), 421–432 (1998)

    Article  Google Scholar 

  14. Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987)

    Google Scholar 

  15. Braides, A.: Approximation of Free-Discontinuity Problems. Lecture Notes in Mathematics, vol. 1694. Springer, Berlin (1998)

    MATH  Google Scholar 

  16. Brook, A., Kimmel, R., Sochen, N.A.: Variational restoration and edge detection for color images. J. Math. Imaging Vis. 18(3), 247–268 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  17. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: ECCV, vol. 4, pp. 25–36 (2004)

  18. Brox, T., Cremers, D.: On the statistical interpretation of the piecewise smooth Mumford-Shah functional. In: SSVM, pp. 203–213 (2007)

  19. Brox, T., Weickert, J.: A TV flow based local scale estimate and its application to texture discrimination. J. Vis. Commun. Image Represent. 17(5), 1053–1073 (2006)

    Article  Google Scholar 

  20. Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: CVPR, vol. 2, pp. 60–65 (2005)

  21. Burgeth, B., Weickert, J., Tari, S.: Minimally stochastic schemes for singular diffusion equations. In: Tai, X.-C., Lie, K.-A., Chan, T.F., Osher, S. (eds.) Image Processing Based on Partial Differential Equations, Mathematics and Visualization, pp. 325–339. Springer, Berlin (2006)

    Google Scholar 

  22. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)

    Google Scholar 

  23. Chan, R.H., Ho, C.-W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005)

    Article  Google Scholar 

  24. Chan, T., Esedoglu, S., Ni, K.: Histogram based segmentation using Wasserstein distances. In: SSVM, pp. 697–708 (2007)

  25. Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (2000)

    Article  Google Scholar 

  26. Chen, S., Yang, X.: A variational method with a noise detector for impulse noise removal. In: SSVM, pp. 442–450 (2007)

  27. Chen, Y., Tagare, H.D., Thiruvenkadam, S., Huang, F., Wilson, D., Gopinath, K.S., Briggs, R.W., Geiser, E.A.: Using prior shapes in geometric active contours in a variational framework. Int. J. Comput. Vis. 50(3), 315–328 (2002)

    Article  MATH  Google Scholar 

  28. Cremers, D., Tischhäuser, F., Weickert, J., Schnörr, C.: Diffusion snakes: introducing statistical shape knowledge into the Mumford-Shah functional. Int. J. Comput. Vis. 50(3), 295–313 (2002)

    Article  MATH  Google Scholar 

  29. Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation, part I: fast and exact optimization. J. Math. Imaging Vis. 26(3), 261–276 (2006)

    Article  MathSciNet  Google Scholar 

  30. Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation, part II: levelable functions, convex priors and non-convex cases. J. Math. Imaging Vis. 26(3), 277–291 (2006)

    Article  MathSciNet  Google Scholar 

  31. Debayle, J., Pinoli, J.C.: General adaptive neighborhood image processing: part I. J. Math. Imaging Vis. 25, 245–266 (2006)

    Article  MathSciNet  Google Scholar 

  32. Deng, G.: Symbol mapping and context filtering for lossless image compression. In: ICIP, vol. 1, pp. 526–529 (1998)

  33. Desolneux, A., Moisan, L., Morel, J.-M.: Meaningful alignments. Int. J. Comput. Vis. 40(1), 7–23 (2000)

    Article  MATH  Google Scholar 

  34. Desolneux, A., Moisan, L., Morel, J.-M.: From Gestalt Theory to Image Analysis: A Probabilistic Approach. Interdisciplinary Applied Mathematics, vol. 35. Springer, New York (2007)

    Google Scholar 

  35. Erdem, E., Erdem, A., Tari, S.: Edge strength functions as shape priors in image segmentation. In: EMMCVPR, pp. 490–502 (2005)

  36. Erdem, E., Sancar Yilmaz, A., Tari, S.: Mumford-Shah regularizer with spatial coherence. In: SSVM, pp. 545–555 (2007)

  37. Esedoglu, S., Shen, J.: Digital image inpainting by the Mumford-Shah-Euler image model. Eur. J. Appl. Math. 13, 353–370 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  38. Faugeras, O.: Three-Dimensional Computer Vision. MIT Press, Cambridge (1993), Chap. 4.2

    Google Scholar 

  39. Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: ICCV, vol. 1, pp. 716–723 (2003)

  40. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–639 (1984)

    Article  MATH  Google Scholar 

  41. Gilboa, G., Darbon, J., Osher, S., Chan, T.: Nonlocal convex functionals for image regularization. UCLA CAM-report 06-57 (2006)

  42. Gilboa, G., Sochen, N., Zeevi, Y.Y.: Variational denoising of partly textured images by spatially varying constraints. IEEE Trans. Image Process. 15(8), 2281–2289 (2006)

    Article  Google Scholar 

  43. Haralick, R.M., Lee, J.S.J.: Context dependent edge detection. In: CVPR, pp. 223–228 (1988)

  44. Heiler, M., Schnörr, C.: Natural image statistics for natural image segmentation. Int. J. Comput. Vis. 63(1), 5–19 (2005)

    Article  Google Scholar 

  45. Hofmann, T., Puzicha, J., Buhmann, J.M.: An optimization approach to unsupervised hierarchical texture segmentation. In: ICIP, vol. 3, pp. 213–216 (1997)

  46. Hong, B.-W., Prados, E., Soatto, S., Vese, L.: Shape representation based on integral kernels: Application to image matching and segmentation. In: CVPR, vol. 1, pp. 833–840 (2006)

  47. Kaufhold, J.P.: Energy formulations of medical image segmentations. PhD thesis, Boston University College of Engineering, Department of Biomedical Engineering, Boston, MA (2000)

  48. Kokkinos, I., Evangelopoulos, G., Maragos, P.: Texture analysis and segmentation using modulation features, generative models and weighted curve evolution. IEEE Trans. Pattern Anal. Mach. Intell. (to appear)

  49. Lai, G.C., de Figueiredo, R.J.P.: Image interpretation using contextual feedback. In: ICIP, vol. 2, p. 2623 (1995)

  50. Leventon, M.E., Grimson, E.L., Faugeras, O.D.: Statistical shape influence in geodesic active contours. In: CVPR, pp. 1316–1323 (2000)

  51. Lombardi, P., Cantoni, V., Zavidovique, B.: Context in robotic vision: Control for real-time adaptation. In: ICINCO, vol. 3, pp. 135–142 (2004)

  52. Marr, D.: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Freeman, San Francisco (1982)

    Google Scholar 

  53. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001)

  54. Morel, J.-M., Solimini, S.: Variational Methods in Image Segmentation. Birkhauser, Basel (1995)

    Google Scholar 

  55. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  56. Nielsen, M., Florack, L., Deriche, R.: Regularization, scale-space and edge detection filters. J. Math. Imaging Vis. 7(4), 291–307 (1997)

    Article  MathSciNet  Google Scholar 

  57. Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20(1–2), 99–120 (2004)

    Article  MathSciNet  Google Scholar 

  58. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)

    Article  Google Scholar 

  59. Pien, H.H., Desai, M., Shah, J.: Segmentation of MR images using curve evolution and prior information. Int. J. Pattern Recognit. Artif. Intell. 11(8), 1233–1245 (1997)

    Article  Google Scholar 

  60. Pien, H.H., Gauch, J.M.: Variational segmentation of multi-channel MRI images. In: ICIP, vol. 3, pp. 508–512 (1994)

  61. Riklin-Raviv, T.: Kiryati, Sochen, N.A.: Unlevel-sets: geometry and prior-based segmentation. In: ECCV, vol. 4, pp. 50–61 (2004)

  62. Ritter, G.X., Wilson, J.N.: Handbook of Computer Vision Algorithms in Image Algebra. CRC Press, Boca Raton (1996)

    MATH  Google Scholar 

  63. Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: CVPR, vol. 2, pp. 860–867 (2005)

  64. Rousson, M., Brox, T., Deriche, R.: Active unsupervised texture segmentation on a diffusion based feature space. In: CVPR, vol. 2, pp. 699–704 (2003)

  65. Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    Article  MATH  Google Scholar 

  66. Scharr, H., Black, M.J., Haussecker, H.W.: Image statistics and anisotropic diffusion. In: ICCV, vol. 2, pp. 840–847 (2003)

  67. Scherzer, O., Weickert, J.: Relations between regularization and diffusion filtering. J. Math. Imaging Vis. 12(1), 43–63 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  68. Schwartz, O., Hsu, A., Dayan, P.: Space and time in visual context. Nat. Rev. Neurosci. 8, 522–535 (2007)

    Article  Google Scholar 

  69. Serra, J.: Image Analysis and Mathematical Morphology. Academic Press, New York (1982)

    MATH  Google Scholar 

  70. Shah, J.: Segmentation by nonlinear diffusion. In: CVPR, pp. 202–207 (1991)

  71. Shah, J.: A common framework for curve evolution, segmentation and anisotropic diffusion. In: CVPR, pp. 136–142 (1996)

  72. Sminchisescu, C., Kanaujia, A., Li, Z., Metaxas, D.: Conditional models for contextual human motion recognition. In: ICCV, vol. 2, pp. 1808–1815 (2005)

  73. Sonka, M., Hlavac, V., Boyle, R.: Image Processing, Analysis and Machine Vision. Chapman and Hall, New York (1993)

    Google Scholar 

  74. Srivastava, A., Lee, A.B., Simoncelli, E.P., Zhu, S.-C.: On advances in statistical modeling of natural images. J. Math. Imaging Vis. 18(1), 17–33 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  75. Tang, Q., Sang, N., Zhang, T.: Contour detection based on contextual influences. Image Vis. Comput. 25, 1282–1290 (2007)

    Article  Google Scholar 

  76. Tari, S., Shah, J., Pien, H.: Extraction of shape skeletons from grayscale images. Comput. Vis. Image Underst. 66(2), 133–146 (1997)

    Article  Google Scholar 

  77. Teboul, S., Blanc-Fraud, L., Aubert, G., Barlaud, M.: Variational approach for edge preserving regularization using coupled PDE’s. IEEE Trans. Image Process. 7(3), 387–397 (1998)

    Article  Google Scholar 

  78. Torralba, A., Oliva, A.: Statistics of natural image categories. Netw. Comput. Neural Syst. 14(3), 391–412 (2003)

    Article  Google Scholar 

  79. Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., Fan, A., Grimson, E., Willsky, A.: A shape-based approach to the segmentation of medical imagery using level sets. IEEE Trans. Med. Imaging 22(2), 137–154 (2003)

    Article  Google Scholar 

  80. Vese, L.A., Osher, S.J.: Image denoising and decomposition with total variation minimization and oscillatory functions. J. Math. Imaging Vis. 20(1–2), 7–18 (2004)

    Article  MathSciNet  Google Scholar 

  81. Weickert, J.: Coherence-enhancing diffusion filtering. Int. J. Comput. Vision 31(2–3), 111–127 (1999)

    Article  Google Scholar 

  82. Wolf, A., Swift, J.B., Swinney, H., Vastano, J.A.: Determining Lyapunov exponents from a time series. Physica D 16, 285–317 (1985)

    Article  MATH  MathSciNet  Google Scholar 

  83. Wolf, L., Bileschi, S.: A critical view of context. Int. J. Comput. Vis. 69(2), 251–261 (2006)

    Article  Google Scholar 

  84. Wolf, L., Huang, X., Martin, I., Metaxas, D.: Patch-based texture edges and segmentation. In: ECCV, vol. 2, pp. 481–493 (2006)

  85. Zheng, S., Tu, Z., Yuille, A.: Detecting object boundaries using low-, mid-, and high-level information. In: CVPR, pp. 1–8 (2007)

  86. Zhu, S.C., Mumford, D.: Prior learning and Gibbs reaction-diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 19(11), 1236–1250 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Middle East Technical University, İnönü Bulvarı, 06531, Ankara, Turkey

    Erkut Erdem & Sibel Tari

Authors
  1. Erkut Erdem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sibel Tari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sibel Tari.

Additional information

High quality and color versions of the images can be downloaded from http://www.ceng.metu.edu.tr/~sibel/mind.

Preliminary conference version of this work appeared in the Proc. of the First Int. Conf. on Scale Space and Variational Methods in Computer Vision (SSVM’07), 2007 36.

Supported in part by TUBITAK research grant 105E154 and TUBITAK Phd scholarship.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Erdem, E., Tari, S. Mumford-Shah Regularizer with Contextual Feedback. J Math Imaging Vis 33, 67–84 (2009). https://doi.org/10.1007/s10851-008-0109-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-008-0109-y

Keywords

Search

Navigation