Mumford-Shah Regularizer with Contextual Feedback
- 142 Downloads
- 11 Citations
Abstract
We present a simple and robust feature preserving image regularization by letting local region measures modulate the diffusivity. The purpose of this modulation is to disambiguate low level cues in early vision. We interpret the Ambrosio-Tortorelli approximation of the Mumford-Shah model as a system with modulatory feedback and utilize this interpretation to integrate high level information into the regularization process. The method does not require any prior model or learning; the high level information is extracted from local regions and fed back to the regularization step. An important characteristic of the method is that both negative and positive feedback can be simultaneously used without creating oscillations. Experiments performed with both gray and color natural images demonstrate the potential of the method under difficult noise types, non-uniform contrast, existence of multi-scale patterns and textures.
Keywords
Variational and PDE methods Feature preserving diffusion Structure preserving diffusion Disambiguation in low level visionPreview
Unable to display preview. Download preview PDF.
References
- 1.Symposium on the Role of Context in Recognition, European Conference on Visual Perception, August 2005 Google Scholar
- 2.Aleman-Flores, M., Alvarez, L., Caselles, V.: Texture-oriented anisotropic filtering and geodesic active contours in breast tumor ultrasound segmentation. J. Math. Imaging Vis. 28(1), 81–97 (2007) CrossRefMathSciNetGoogle Scholar
- 3.Alicandro, R., Braides, A., Shah, J.: Free-discontinuity problems via functionals involving the L 1-norm of the gradient and their approximation. Interfaces Free Bound. 1(1), 17–37 (1999) MATHMathSciNetCrossRefGoogle Scholar
- 4.Ambrosio, L., Tortorelli, V.: On the approximation of functionals depending on jumps by elliptic functionals via Γ-convergence. Commun. Pure Appl. Math. 43(8), 999–1036 (1990) MATHCrossRefMathSciNetGoogle Scholar
- 5.Aslan, C., Tari, S.: An axis-based representation for recognition. In: ICCV, vol. 2, pp. 1339–1346 (2005) Google Scholar
- 6.Aujol, J.-F., Aubert, G., Chambolle, A.: Image decomposition into a bounded variation component and an oscillating component. J. Math. Imaging Vis. 22(1), 71–88 (2005) CrossRefMathSciNetGoogle Scholar
- 7.Bar, L., Kiryati, N., Sochen, N.: Image deblurring in the presence of impulsive noise. Int. J. Comput. Vis. 70(3), 279–298 (2006) CrossRefGoogle Scholar
- 8.Bar, L., Sochen, N., Kiryati, N.: Image deblurring in the presence of salt-and-pepper noise. In: Scale-Space, pp. 107–118 (2005) Google Scholar
- 9.Bar, M.: Visual objects in context. Nat. Rev. Neurosci. 5, 617–629 (2004) CrossRefGoogle Scholar
- 10.Bayerl, P., Neumann, H.: Disambiguating visual motion through contextual feedback modulation. Neural Comput. 16, 2041–2066 (2004) MATHCrossRefGoogle Scholar
- 11.Black, M.J., Rangarajan, A.: On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int. J. Comput. Vis. 19(1), 57–91 (1996) CrossRefGoogle Scholar
- 12.Black, M.J., Sapiro, G.: Edges as outliers: anisotropic smoothing using local image statistics. In: Scale-Space, pp. 259–270 (1999) Google Scholar
- 13.Black, M.J., Sapiro, G., Marimont, D.H., Heeger, D.: Robust anisotropic diffusion. IEEE Trans. Image Process. 7(3), 421–432 (1998) CrossRefGoogle Scholar
- 14.Blake, A., Zisserman, A.: Visual Reconstruction. MIT Press, Cambridge (1987) Google Scholar
- 15.Braides, A.: Approximation of Free-Discontinuity Problems. Lecture Notes in Mathematics, vol. 1694. Springer, Berlin (1998) MATHGoogle Scholar
- 16.Brook, A., Kimmel, R., Sochen, N.A.: Variational restoration and edge detection for color images. J. Math. Imaging Vis. 18(3), 247–268 (2003) MATHCrossRefMathSciNetGoogle Scholar
- 17.Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: ECCV, vol. 4, pp. 25–36 (2004) Google Scholar
- 18.Brox, T., Cremers, D.: On the statistical interpretation of the piecewise smooth Mumford-Shah functional. In: SSVM, pp. 203–213 (2007) Google Scholar
- 19.Brox, T., Weickert, J.: A TV flow based local scale estimate and its application to texture discrimination. J. Vis. Commun. Image Represent. 17(5), 1053–1073 (2006) CrossRefGoogle Scholar
- 20.Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: CVPR, vol. 2, pp. 60–65 (2005) Google Scholar
- 21.Burgeth, B., Weickert, J., Tari, S.: Minimally stochastic schemes for singular diffusion equations. In: Tai, X.-C., Lie, K.-A., Chan, T.F., Osher, S. (eds.) Image Processing Based on Partial Differential Equations, Mathematics and Visualization, pp. 325–339. Springer, Berlin (2006) Google Scholar
- 22.Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986) Google Scholar
- 23.Chan, R.H., Ho, C.-W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005) CrossRefGoogle Scholar
- 24.Chan, T., Esedoglu, S., Ni, K.: Histogram based segmentation using Wasserstein distances. In: SSVM, pp. 697–708 (2007) Google Scholar
- 25.Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (2000) CrossRefGoogle Scholar
- 26.Chen, S., Yang, X.: A variational method with a noise detector for impulse noise removal. In: SSVM, pp. 442–450 (2007) Google Scholar
- 27.Chen, Y., Tagare, H.D., Thiruvenkadam, S., Huang, F., Wilson, D., Gopinath, K.S., Briggs, R.W., Geiser, E.A.: Using prior shapes in geometric active contours in a variational framework. Int. J. Comput. Vis. 50(3), 315–328 (2002) MATHCrossRefGoogle Scholar
- 28.Cremers, D., Tischhäuser, F., Weickert, J., Schnörr, C.: Diffusion snakes: introducing statistical shape knowledge into the Mumford-Shah functional. Int. J. Comput. Vis. 50(3), 295–313 (2002) MATHCrossRefGoogle Scholar
- 29.Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation, part I: fast and exact optimization. J. Math. Imaging Vis. 26(3), 261–276 (2006) CrossRefMathSciNetGoogle Scholar
- 30.Darbon, J., Sigelle, M.: Image restoration with discrete constrained total variation, part II: levelable functions, convex priors and non-convex cases. J. Math. Imaging Vis. 26(3), 277–291 (2006) CrossRefMathSciNetGoogle Scholar
- 31.Debayle, J., Pinoli, J.C.: General adaptive neighborhood image processing: part I. J. Math. Imaging Vis. 25, 245–266 (2006) CrossRefMathSciNetGoogle Scholar
- 32.Deng, G.: Symbol mapping and context filtering for lossless image compression. In: ICIP, vol. 1, pp. 526–529 (1998) Google Scholar
- 33.Desolneux, A., Moisan, L., Morel, J.-M.: Meaningful alignments. Int. J. Comput. Vis. 40(1), 7–23 (2000) MATHCrossRefGoogle Scholar
- 34.Desolneux, A., Moisan, L., Morel, J.-M.: From Gestalt Theory to Image Analysis: A Probabilistic Approach. Interdisciplinary Applied Mathematics, vol. 35. Springer, New York (2007) Google Scholar
- 35.Erdem, E., Erdem, A., Tari, S.: Edge strength functions as shape priors in image segmentation. In: EMMCVPR, pp. 490–502 (2005) Google Scholar
- 36.Erdem, E., Sancar Yilmaz, A., Tari, S.: Mumford-Shah regularizer with spatial coherence. In: SSVM, pp. 545–555 (2007) Google Scholar
- 37.Esedoglu, S., Shen, J.: Digital image inpainting by the Mumford-Shah-Euler image model. Eur. J. Appl. Math. 13, 353–370 (2002) MATHCrossRefMathSciNetGoogle Scholar
- 38.Faugeras, O.: Three-Dimensional Computer Vision. MIT Press, Cambridge (1993), Chap. 4.2 Google Scholar
- 39.Galun, M., Sharon, E., Basri, R., Brandt, A.: Texture segmentation by multiscale aggregation of filter responses and shape elements. In: ICCV, vol. 1, pp. 716–723 (2003) Google Scholar
- 40.Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–639 (1984) MATHCrossRefGoogle Scholar
- 41.Gilboa, G., Darbon, J., Osher, S., Chan, T.: Nonlocal convex functionals for image regularization. UCLA CAM-report 06-57 (2006) Google Scholar
- 42.Gilboa, G., Sochen, N., Zeevi, Y.Y.: Variational denoising of partly textured images by spatially varying constraints. IEEE Trans. Image Process. 15(8), 2281–2289 (2006) CrossRefGoogle Scholar
- 43.Haralick, R.M., Lee, J.S.J.: Context dependent edge detection. In: CVPR, pp. 223–228 (1988) Google Scholar
- 44.Heiler, M., Schnörr, C.: Natural image statistics for natural image segmentation. Int. J. Comput. Vis. 63(1), 5–19 (2005) CrossRefGoogle Scholar
- 45.Hofmann, T., Puzicha, J., Buhmann, J.M.: An optimization approach to unsupervised hierarchical texture segmentation. In: ICIP, vol. 3, pp. 213–216 (1997) Google Scholar
- 46.Hong, B.-W., Prados, E., Soatto, S., Vese, L.: Shape representation based on integral kernels: Application to image matching and segmentation. In: CVPR, vol. 1, pp. 833–840 (2006) Google Scholar
- 47.Kaufhold, J.P.: Energy formulations of medical image segmentations. PhD thesis, Boston University College of Engineering, Department of Biomedical Engineering, Boston, MA (2000) Google Scholar
- 48.Kokkinos, I., Evangelopoulos, G., Maragos, P.: Texture analysis and segmentation using modulation features, generative models and weighted curve evolution. IEEE Trans. Pattern Anal. Mach. Intell. (to appear) Google Scholar
- 49.Lai, G.C., de Figueiredo, R.J.P.: Image interpretation using contextual feedback. In: ICIP, vol. 2, p. 2623 (1995) Google Scholar
- 50.Leventon, M.E., Grimson, E.L., Faugeras, O.D.: Statistical shape influence in geodesic active contours. In: CVPR, pp. 1316–1323 (2000) Google Scholar
- 51.Lombardi, P., Cantoni, V., Zavidovique, B.: Context in robotic vision: Control for real-time adaptation. In: ICINCO, vol. 3, pp. 135–142 (2004) Google Scholar
- 52.Marr, D.: Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. Freeman, San Francisco (1982) Google Scholar
- 53.Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV, vol. 2, pp. 416–423 (2001) Google Scholar
- 54.Morel, J.-M., Solimini, S.: Variational Methods in Image Segmentation. Birkhauser, Basel (1995) Google Scholar
- 55.Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989) MATHCrossRefMathSciNetGoogle Scholar
- 56.Nielsen, M., Florack, L., Deriche, R.: Regularization, scale-space and edge detection filters. J. Math. Imaging Vis. 7(4), 291–307 (1997) CrossRefMathSciNetGoogle Scholar
- 57.Nikolova, M.: A variational approach to remove outliers and impulse noise. J. Math. Imaging Vis. 20(1–2), 99–120 (2004) CrossRefMathSciNetGoogle Scholar
- 58.Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990) CrossRefGoogle Scholar
- 59.Pien, H.H., Desai, M., Shah, J.: Segmentation of MR images using curve evolution and prior information. Int. J. Pattern Recognit. Artif. Intell. 11(8), 1233–1245 (1997) CrossRefGoogle Scholar
- 60.Pien, H.H., Gauch, J.M.: Variational segmentation of multi-channel MRI images. In: ICIP, vol. 3, pp. 508–512 (1994) Google Scholar
- 61.Riklin-Raviv, T.: Kiryati, Sochen, N.A.: Unlevel-sets: geometry and prior-based segmentation. In: ECCV, vol. 4, pp. 50–61 (2004) Google Scholar
- 62.Ritter, G.X., Wilson, J.N.: Handbook of Computer Vision Algorithms in Image Algebra. CRC Press, Boca Raton (1996) MATHGoogle Scholar
- 63.Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In: CVPR, vol. 2, pp. 860–867 (2005) Google Scholar
- 64.Rousson, M., Brox, T., Deriche, R.: Active unsupervised texture segmentation on a diffusion based feature space. In: CVPR, vol. 2, pp. 699–704 (2003) Google Scholar
- 65.Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992) MATHCrossRefGoogle Scholar
- 66.Scharr, H., Black, M.J., Haussecker, H.W.: Image statistics and anisotropic diffusion. In: ICCV, vol. 2, pp. 840–847 (2003) Google Scholar
- 67.Scherzer, O., Weickert, J.: Relations between regularization and diffusion filtering. J. Math. Imaging Vis. 12(1), 43–63 (2000) MATHCrossRefMathSciNetGoogle Scholar
- 68.Schwartz, O., Hsu, A., Dayan, P.: Space and time in visual context. Nat. Rev. Neurosci. 8, 522–535 (2007) CrossRefGoogle Scholar
- 69.Serra, J.: Image Analysis and Mathematical Morphology. Academic Press, New York (1982) MATHGoogle Scholar
- 70.Shah, J.: Segmentation by nonlinear diffusion. In: CVPR, pp. 202–207 (1991) Google Scholar
- 71.Shah, J.: A common framework for curve evolution, segmentation and anisotropic diffusion. In: CVPR, pp. 136–142 (1996) Google Scholar
- 72.Sminchisescu, C., Kanaujia, A., Li, Z., Metaxas, D.: Conditional models for contextual human motion recognition. In: ICCV, vol. 2, pp. 1808–1815 (2005) Google Scholar
- 73.Sonka, M., Hlavac, V., Boyle, R.: Image Processing, Analysis and Machine Vision. Chapman and Hall, New York (1993) Google Scholar
- 74.Srivastava, A., Lee, A.B., Simoncelli, E.P., Zhu, S.-C.: On advances in statistical modeling of natural images. J. Math. Imaging Vis. 18(1), 17–33 (2003) MATHCrossRefMathSciNetGoogle Scholar
- 75.Tang, Q., Sang, N., Zhang, T.: Contour detection based on contextual influences. Image Vis. Comput. 25, 1282–1290 (2007) CrossRefGoogle Scholar
- 76.Tari, S., Shah, J., Pien, H.: Extraction of shape skeletons from grayscale images. Comput. Vis. Image Underst. 66(2), 133–146 (1997) CrossRefGoogle Scholar
- 77.Teboul, S., Blanc-Fraud, L., Aubert, G., Barlaud, M.: Variational approach for edge preserving regularization using coupled PDE’s. IEEE Trans. Image Process. 7(3), 387–397 (1998) CrossRefGoogle Scholar
- 78.Torralba, A., Oliva, A.: Statistics of natural image categories. Netw. Comput. Neural Syst. 14(3), 391–412 (2003) CrossRefGoogle Scholar
- 79.Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., Fan, A., Grimson, E., Willsky, A.: A shape-based approach to the segmentation of medical imagery using level sets. IEEE Trans. Med. Imaging 22(2), 137–154 (2003) CrossRefGoogle Scholar
- 80.Vese, L.A., Osher, S.J.: Image denoising and decomposition with total variation minimization and oscillatory functions. J. Math. Imaging Vis. 20(1–2), 7–18 (2004) CrossRefMathSciNetGoogle Scholar
- 81.Weickert, J.: Coherence-enhancing diffusion filtering. Int. J. Comput. Vision 31(2–3), 111–127 (1999) CrossRefGoogle Scholar
- 82.Wolf, A., Swift, J.B., Swinney, H., Vastano, J.A.: Determining Lyapunov exponents from a time series. Physica D 16, 285–317 (1985) MATHCrossRefMathSciNetGoogle Scholar
- 83.Wolf, L., Bileschi, S.: A critical view of context. Int. J. Comput. Vis. 69(2), 251–261 (2006) CrossRefGoogle Scholar
- 84.Wolf, L., Huang, X., Martin, I., Metaxas, D.: Patch-based texture edges and segmentation. In: ECCV, vol. 2, pp. 481–493 (2006) Google Scholar
- 85.Zheng, S., Tu, Z., Yuille, A.: Detecting object boundaries using low-, mid-, and high-level information. In: CVPR, pp. 1–8 (2007) Google Scholar
- 86.Zhu, S.C., Mumford, D.: Prior learning and Gibbs reaction-diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 19(11), 1236–1250 (1997) CrossRefGoogle Scholar