Journal of Mathematical Imaging and Vision

, Volume 33, Issue 1, pp 67–84 | Cite as

Mumford-Shah Regularizer with Contextual Feedback

Article
First Online:
Received:
Accepted:

Abstract

We present a simple and robust feature preserving image regularization by letting local region measures modulate the diffusivity. The purpose of this modulation is to disambiguate low level cues in early vision. We interpret the Ambrosio-Tortorelli approximation of the Mumford-Shah model as a system with modulatory feedback and utilize this interpretation to integrate high level information into the regularization process. The method does not require any prior model or learning; the high level information is extracted from local regions and fed back to the regularization step. An important characteristic of the method is that both negative and positive feedback can be simultaneously used without creating oscillations. Experiments performed with both gray and color natural images demonstrate the potential of the method under difficult noise types, non-uniform contrast, existence of multi-scale patterns and textures.

Keywords

Variational and PDE methods Feature preserving diffusion Structure preserving diffusion Disambiguation in low level vision 
This is a preview of subscription content, log in to check access

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Symposium on the Role of Context in Recognition, European Conference on Visual Perception, August 2005 Google Scholar
  2. 2.
    Aleman-Flores, M., Alvarez, L., Caselles, V.: Texture-oriented anisotropic filtering and geodesic active contours in breast tumor ultrasound segmentation. J. Math. Imaging Vis. 28(1), 81–97 (2007) CrossRefMathSciNetGoogle Scholar
  3. 3.
    Alicandro, R., Braides, A., Shah, J.: Free-discontinuity problems via functionals involving the L 1-norm of the gradient and their approximation. Interfaces Free Bound. 1(1), 17–37 (1999) MATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    Ambrosio, L., Tortorelli, V.: On the approximation of functionals depending on jumps by elliptic functionals via Γ-convergence. Commun. Pure Appl. Math. 43(8), 999–1036 (1990) MATHCrossRefMathSciNetGoogle Scholar
  5. 5.
    Aslan, C., Tari, S.: An axis-based representation for recognition. In: ICCV, vol. 2, pp. 1339–1346 (2005) Google Scholar
  6. 6.
    Aujol, J.-F., Aubert, G., Chambolle, A.: Image decomposition into a bounded variation component and an oscillating component. J. Math. Imaging Vis. 22(1), 71–88 (2005) CrossRefMathSciNetGoogle Scholar
  7. 7.
    Bar, L., Kiryati, N., Sochen, N.: Image deblurring in the presence of impulsive noise. Int. J. Comput. Vis. 70(3), 279–298 (2006) CrossRefGoogle Scholar
  8. 8.
    Bar, L., Sochen, N., Kiryati, N.: Image deblurring in the presence of salt-and-pepper noise. In: Scale-Space, pp. 107–118 (2005) Google Scholar
  9. 9.
    Bar, M.: Visual objects in context. Nat. Rev. Neurosci. 5, 617–629 (2004) CrossRefGoogle Scholar
  10. 10.
    Bayerl, P., Neumann, H.: Disambiguating visual motion through contextual feedback modulation. Neural Comput. 16, 2041–2066 (2004) MATHCrossRefGoogle Scholar
  11. 11.
    Black, M.J., Rangarajan, A.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Comput. Vis. 19(1), 57–91 (1996) CrossRefGoogle Scholar
  12. 12.
    Black, M.J., Sapiro, G.: Edges as outliers: anisotropic smoothing using local image statistics. In: Scale-Space, pp. 259–270 (1999) Google Scholar
  13. 13.
    Black, M.J., Sapiro, G., Marimont, D.H., Heeger, D.: Robust anisotropic diffusion. IEEE Trans. Image Process. 7(3), 421–432 (1998) CrossRefGoogle Scholar
  14. 14.
    Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987) Google Scholar
  15. 15.
    Braides, A.: Approximation of Free-Discontinuity Problems. Lecture Notes in Mathematics, vol. 1694. Springer, Berlin (1998) MATHGoogle Scholar
  16. 16.
    Brook, A., Kimmel, R., Sochen, N.A.: Variational restoration and edge detection for color images. J. Math. Imaging Vis. 18(3), 247–268 (2003) MATHCrossRefMathSciNetGoogle Scholar
  17. 17.
    Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: ECCV, vol. 4, pp. 25–36 (2004) Google Scholar
  18. 18.
    Brox, T., Cremers, D.: On the statistical interpretation of the piecewise smooth Mumford-Shah functional. In: SSVM, pp. 203–213 (2007) Google Scholar
  19. 19.
    Brox, T., Weickert, J.: A TV flow based local scale estimate and its application to texture discrimination. J. Vis. Commun. Image Represent. 17(5), 1053–1073 (2006) CrossRefGoogle Scholar
  20. 20.
    Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: CVPR, vol. 2, pp. 60–65 (2005) Google Scholar
  21. 21.
    Burgeth, B., Weickert, J., Tari, S.: Minimally stochastic schemes for singular diffusion equations. In: Tai, X.-C., Lie, K.-A., Chan, T.F., Osher, S. (eds.) Image Processing Based on Partial Differential Equations, Mathematics and Visualization, pp. 325–339. Springer, Berlin (2006) Google Scholar
  22. 22.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986) Google Scholar
  23. 23.
    Chan, R.H., Ho, C.-W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005) CrossRefGoogle Scholar
  24. 24.
    Chan, T., Esedoglu, S., Ni, K.: Histogram based segmentation using Wasserstein distances. In: SSVM, pp. 697–708 (2007) Google Scholar
  25. 25.
    Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (2000) CrossRefGoogle Scholar
  26. 26.
    Chen, S., Yang, X.: A variational method with a noise detector for impulse noise removal. In: SSVM, pp. 442–450 (2007) Google Scholar
  27. 27.
    Chen, Y., Tagare, H.D., Thiruvenkadam, S., Huang, F., Wilson, D., Gopinath, K.S., Briggs, R.W., Geiser, E.A.: Using prior shapes in geometric active contours in a variational framework. Int. J. Comput. Vis. 50(3), 315–328 (2002) MATHCrossRefGoogle Scholar
  28. 28.
    Cremers, D., Tischhäuser, F., Weickert, J., Schnörr, C.: Diffusion snakes: introducing statistical shape knowledge into the Mumford-Shah functional. Int. J. Comput. Vis. 50(3), 295–313 (2002) MATHCrossRefGoogle Scholar
  29. 29.
    Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation, part I: fast and exact optimization. J. Math. Imaging Vis. 26(3), 261–276 (2006) CrossRefMathSciNetGoogle Scholar
  30. 30.
    Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation, part II: levelable functions, convex priors and non-convex cases. J. Math. Imaging Vis. 26(3), 277–291 (2006) CrossRefMathSciNetGoogle Scholar
  31. 31.
    Debayle, J., Pinoli, J.C.: General adaptive neighborhood image processing: part I. J. Math. Imaging Vis. 25, 245–266 (2006) CrossRefMathSciNetGoogle Scholar
  32. 32.
    Deng, G.: Symbol mapping and context filtering for lossless image compression. In: ICIP, vol. 1, pp. 526–529 (1998) Google Scholar
  33. 33.
    Desolneux, A., Moisan, L., Morel, J.-M.: Meaningful alignments. Int. J. Comput. Vis. 40(1), 7–23 (2000) MATHCrossRefGoogle Scholar
  34. 34.
    Desolneux, A., Moisan, L., Morel, J.-M.: From Gestalt Theory to Image Analysis: A Probabilistic Approach. Interdisciplinary Applied Mathematics, vol. 35. Springer, New York (2007) Google Scholar
  35. 35.
    Erdem, E., Erdem, A., Tari, S.: Edge strength functions as shape priors in image segmentation. In: EMMCVPR, pp. 490–502 (2005) Google Scholar
  36. 36.
    Erdem, E., Sancar Yilmaz, A., Tari, S.: Mumford-Shah regularizer with spatial coherence. In: SSVM, pp. 545–555 (2007) Google Scholar
  37. 37.
    Esedoglu, S., Shen, J.: Digital image inpainting by the Mumford-Shah-Euler image model. Eur. J. Appl. Math. 13, 353–370 (2002) MATHCrossRefMathSciNetGoogle Scholar
  38. 38.
    Faugeras, O.: Three-Dimensional Computer Vision. MIT Press, Cambridge (1993), Chap. 4.2 Google Scholar
  39. 39.
    Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: ICCV, vol. 1, pp. 716–723 (2003) Google Scholar
  40. 40.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–639 (1984) MATHCrossRefGoogle Scholar
  41. 41.
    Gilboa, G., Darbon, J., Osher, S., Chan, T.: Nonlocal convex functionals for image regularization. UCLA CAM-report 06-57 (2006) Google Scholar
  42. 42.
    Gilboa, G., Sochen, N., Zeevi, Y.Y.: Variational denoising of partly textured images by spatially varying constraints. IEEE Trans. Image Process. 15(8), 2281–2289 (2006) CrossRefGoogle Scholar
  43. 43.
    Haralick, R.M., Lee, J.S.J.: Context dependent edge detection. In: CVPR, pp. 223–228 (1988) Google Scholar
  44. 44.
    Heiler, M., Schnörr, C.: Natural image statistics for natural image segmentation. Int. J. Comput. Vis. 63(1), 5–19 (2005) CrossRefGoogle Scholar
  45. 45.
    Hofmann, T., Puzicha, J., Buhmann, J.M.: An optimization approach to unsupervised hierarchical texture segmentation. In: ICIP, vol. 3, pp. 213–216 (1997) Google Scholar
  46. 46.
    Hong, B.-W., Prados, E., Soatto, S., Vese, L.: Shape representation based on integral kernels: Application to image matching and segmentation. In: CVPR, vol. 1, pp. 833–840 (2006) Google Scholar
  47. 47.
    Kaufhold, J.P.: Energy formulations of medical image segmentations. PhD thesis, Boston University College of Engineering, Department of Biomedical Engineering, Boston, MA (2000) Google Scholar
  48. 48.
    Kokkinos, I., Evangelopoulos, G., Maragos, P.: Texture analysis and segmentation using modulation features, generative models and weighted curve evolution. IEEE Trans. Pattern Anal. Mach. Intell. (to appear) Google Scholar
  49. 49.
    Lai, G.C., de Figueiredo, R.J.P.: Image interpretation using contextual feedback. In: ICIP, vol. 2, p. 2623 (1995) Google Scholar
  50. 50.
    Leventon, M.E., Grimson, E.L., Faugeras, O.D.: Statistical shape influence in geodesic active contours. In: CVPR, pp. 1316–1323 (2000) Google Scholar
  51. 51.
    Lombardi, P., Cantoni, V., Zavidovique, B.: Context in robotic vision: Control for real-time adaptation. In: ICINCO, vol. 3, pp. 135–142 (2004) Google Scholar
  52. 52.
    Marr, D.: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Freeman, San Francisco (1982) Google Scholar
  53. 53.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001) Google Scholar
  54. 54.
    Morel, J.-M., Solimini, S.: Variational Methods in Image Segmentation. Birkhauser, Basel (1995) Google Scholar
  55. 55.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989) MATHCrossRefMathSciNetGoogle Scholar
  56. 56.
    Nielsen, M., Florack, L., Deriche, R.: Regularization, scale-space and edge detection filters. J. Math. Imaging Vis. 7(4), 291–307 (1997) CrossRefMathSciNetGoogle Scholar
  57. 57.
    Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20(1–2), 99–120 (2004) CrossRefMathSciNetGoogle Scholar
  58. 58.
    Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990) CrossRefGoogle Scholar
  59. 59.
    Pien, H.H., Desai, M., Shah, J.: Segmentation of MR images using curve evolution and prior information. Int. J. Pattern Recognit. Artif. Intell. 11(8), 1233–1245 (1997) CrossRefGoogle Scholar
  60. 60.
    Pien, H.H., Gauch, J.M.: Variational segmentation of multi-channel MRI images. In: ICIP, vol. 3, pp. 508–512 (1994) Google Scholar
  61. 61.
    Riklin-Raviv, T.: Kiryati, Sochen, N.A.: Unlevel-sets: geometry and prior-based segmentation. In: ECCV, vol. 4, pp. 50–61 (2004) Google Scholar
  62. 62.
    Ritter, G.X., Wilson, J.N.: Handbook of Computer Vision Algorithms in Image Algebra. CRC Press, Boca Raton (1996) MATHGoogle Scholar
  63. 63.
    Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: CVPR, vol. 2, pp. 860–867 (2005) Google Scholar
  64. 64.
    Rousson, M., Brox, T., Deriche, R.: Active unsupervised texture segmentation on a diffusion based feature space. In: CVPR, vol. 2, pp. 699–704 (2003) Google Scholar
  65. 65.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992) MATHCrossRefGoogle Scholar
  66. 66.
    Scharr, H., Black, M.J., Haussecker, H.W.: Image statistics and anisotropic diffusion. In: ICCV, vol. 2, pp. 840–847 (2003) Google Scholar
  67. 67.
    Scherzer, O., Weickert, J.: Relations between regularization and diffusion filtering. J. Math. Imaging Vis. 12(1), 43–63 (2000) MATHCrossRefMathSciNetGoogle Scholar
  68. 68.
    Schwartz, O., Hsu, A., Dayan, P.: Space and time in visual context. Nat. Rev. Neurosci. 8, 522–535 (2007) CrossRefGoogle Scholar
  69. 69.
    Serra, J.: Image Analysis and Mathematical Morphology. Academic Press, New York (1982) MATHGoogle Scholar
  70. 70.
    Shah, J.: Segmentation by nonlinear diffusion. In: CVPR, pp. 202–207 (1991) Google Scholar
  71. 71.
    Shah, J.: A common framework for curve evolution, segmentation and anisotropic diffusion. In: CVPR, pp. 136–142 (1996) Google Scholar
  72. 72.
    Sminchisescu, C., Kanaujia, A., Li, Z., Metaxas, D.: Conditional models for contextual human motion recognition. In: ICCV, vol. 2, pp. 1808–1815 (2005) Google Scholar
  73. 73.
    Sonka, M., Hlavac, V., Boyle, R.: Image Processing, Analysis and Machine Vision. Chapman and Hall, New York (1993) Google Scholar
  74. 74.
    Srivastava, A., Lee, A.B., Simoncelli, E.P., Zhu, S.-C.: On advances in statistical modeling of natural images. J. Math. Imaging Vis. 18(1), 17–33 (2003) MATHCrossRefMathSciNetGoogle Scholar
  75. 75.
    Tang, Q., Sang, N., Zhang, T.: Contour detection based on contextual influences. Image Vis. Comput. 25, 1282–1290 (2007) CrossRefGoogle Scholar
  76. 76.
    Tari, S., Shah, J., Pien, H.: Extraction of shape skeletons from grayscale images. Comput. Vis. Image Underst. 66(2), 133–146 (1997) CrossRefGoogle Scholar
  77. 77.
    Teboul, S., Blanc-Fraud, L., Aubert, G., Barlaud, M.: Variational approach for edge preserving regularization using coupled PDE’s. IEEE Trans. Image Process. 7(3), 387–397 (1998) CrossRefGoogle Scholar
  78. 78.
    Torralba, A., Oliva, A.: Statistics of natural image categories. Netw. Comput. Neural Syst. 14(3), 391–412 (2003) CrossRefGoogle Scholar
  79. 79.
    Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., Fan, A., Grimson, E., Willsky, A.: A shape-based approach to the segmentation of medical imagery using level sets. IEEE Trans. Med. Imaging 22(2), 137–154 (2003) CrossRefGoogle Scholar
  80. 80.
    Vese, L.A., Osher, S.J.: Image denoising and decomposition with total variation minimization and oscillatory functions. J. Math. Imaging Vis. 20(1–2), 7–18 (2004) CrossRefMathSciNetGoogle Scholar
  81. 81.
    Weickert, J.: Coherence-enhancing diffusion filtering. Int. J. Comput. Vision 31(2–3), 111–127 (1999) CrossRefGoogle Scholar
  82. 82.
    Wolf, A., Swift, J.B., Swinney, H., Vastano, J.A.: Determining Lyapunov exponents from a time series. Physica D 16, 285–317 (1985) MATHCrossRefMathSciNetGoogle Scholar
  83. 83.
    Wolf, L., Bileschi, S.: A critical view of context. Int. J. Comput. Vis. 69(2), 251–261 (2006) CrossRefGoogle Scholar
  84. 84.
    Wolf, L., Huang, X., Martin, I., Metaxas, D.: Patch-based texture edges and segmentation. In: ECCV, vol. 2, pp. 481–493 (2006) Google Scholar
  85. 85.
    Zheng, S., Tu, Z., Yuille, A.: Detecting object boundaries using low-, mid-, and high-level information. In: CVPR, pp. 1–8 (2007) Google Scholar
  86. 86.
    Zhu, S.C., Mumford, D.: Prior learning and Gibbs reaction-diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 19(11), 1236–1250 (1997) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.Department of Computer EngineeringMiddle East Technical UniversityAnkaraTurkey

Personalised recommendations