Journal of Mathematical Imaging and Vision

, Volume 47, Issue 3, pp 179–209 | Cite as

A Variational Framework for Region-Based Segmentation Incorporating Physical Noise Models

  • Alex SawatzkyEmail author
  • Daniel Tenbrinck
  • Xiaoyi Jiang
  • Martin Burger


Image segmentation is one of the fundamental problems in computer vision and image processing. In the recent years mathematical models based on partial differential equations and variational methods have led to superior results in many applications, e.g., medical imaging. A majority of works on image segmentation implicitly assume the given image to be biased by additive Gaussian noise, for instance the popular Mumford-Shah model. Since this assumption is not suitable for a variety of problems, we propose a region-based variational segmentation framework to segment also images with non-Gaussian noise models. Motivated by applications in biomedical imaging, we discuss the cases of Poisson and multiplicative speckle noise intensively. Analytical results such as the existence of a solution are verified and we investigate the use of different regularization functionals to provide a-priori information regarding the expected solution. The performance of the proposed framework is illustrated by experimental results on synthetic and real data.


Image segmentation Variational methods Maximum a-posteriori probability estimation Non-Gaussian noise models Multiplicative speckle noise Poisson noise Medical ultrasound imaging Positron emission tomography 



This study has been supported by the German Research Foundation DFG, SFB 656 MoBil (project B2, B3, C3), as well as DFG project BU 2327/1. The authors thank Jörg Stypmann (University Hospital of Münster) for providing medical ultrasound data. Furthermore, we thank Florian Büther and Klaus Schäfers (EIMI, WWU Münster) for providing PET data. The authors thank Tanja Teuber (TU Kaiserslautern) for useful hints on the speckle noise model.


  1. 1.
    Acar, R., Vogel, C.R.: Analysis of bounded variation penalty methods for ill-posed problems. Inverse Probl. 10, 1217–1229 (1994) MathSciNetzbMATHCrossRefGoogle Scholar
  2. 2.
    Adams, R.A.: Sobolev Spaces. Pure and Applied Mathematics, vol. 65. Academic Press, New York (1975) zbMATHGoogle Scholar
  3. 3.
    Adams, R.A., Fournier, J.J.F.: Sobolev Spaces. Pure and Applied Mathematics, vol. 140. Elsevier, Amsterdam (2003) zbMATHGoogle Scholar
  4. 4.
    Ambrosio, L., Tortorelli, V.M.: Approximation of functionals depending on jumps by elliptic functionals via Γ-convergence. Commun. Pure Appl. Math. 43, 999–1036 (1990) MathSciNetzbMATHCrossRefGoogle Scholar
  5. 5.
    Ambrosio, L., Fusco, N., Pallara, D.: Functions of Bounded Variation and Free Discontinuity Problems. Oxford Mathematical Monographs. Oxford University Press, Oxford (2000) zbMATHGoogle Scholar
  6. 6.
    Aubert, G., Aujol, J.-F.: A variational approach to removing multiplicative noise. SIAM J. Appl. Math. 68, 925–946 (2008) MathSciNetzbMATHCrossRefGoogle Scholar
  7. 7.
    Aubert, G., Kornprobst, P.: Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations, 2nd edn. Applied Mathematical Sciences, vol. 147. Springer, Berlin (2006) Google Scholar
  8. 8.
    Aujol, J.-F.: Some first-order algorithms for total variation based image restoration. J. Math. Imaging Vis. 34, 307–327 (2009) MathSciNetCrossRefGoogle Scholar
  9. 9.
    Barbu, V., Precupanu, T.: Convexity and Optimization in Banach Spaces. Sijthoff & Noordhoff, Rockville (1978) Google Scholar
  10. 10.
    Benning, M., Kösters, T., Wübbeling, F., Schäfers, K., Burger, M.: A nonlinear variational method for improved quantification of myocardial blood flow using dynamic \(\mathrm{H}_{2} ^{15}\)O PET. In: Nuclear Science Symposium Conference Record, pp. 4472–4477 (2008) Google Scholar
  11. 11.
    Bertero, M., Lanteri, H., Zanni, L.: Iterative image reconstruction: a point of view. In: Censor, Y., Jiang, M., Louis, A. (eds.) Mathematical Methods in Biomedical Imaging and Intensity-Modulated Radiation Therapy (IMRT). Publications of the Scuola Normale, CRM Series, vol. 7, pp. 37–63 (2008) Google Scholar
  12. 12.
    Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J.-P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. 28, 151–167 (2007) MathSciNetCrossRefGoogle Scholar
  13. 13.
    Brox, T., Weickert, J.: Level set segmentation with multiple regions. IEEE Trans. Image Process. 15, 3213–3218 (2006) CrossRefGoogle Scholar
  14. 14.
    Burger, M., Franek, M., Schönlieb, C.-B.: Regularized regression and density estimation based on optimal transport. Appl. Math. Res. Express 2012 (2012), 45 pp. Google Scholar
  15. 15.
    Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis. 22, 61–79 (1997) zbMATHCrossRefGoogle Scholar
  16. 16.
    Caselles, V., Chambolle, A., Novaga, M.: The discontinuity set of solutions of the TV denoising problem and some extensions. Multiscale Model. Simul. 6, 879–894 (2007) MathSciNetzbMATHCrossRefGoogle Scholar
  17. 17.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004) MathSciNetCrossRefGoogle Scholar
  18. 18.
    Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40, 120–145 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  19. 19.
    Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet and Stochastic Methods. SIAM, Philadelphia (2005) CrossRefGoogle Scholar
  20. 20.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10, 266–277 (2001) zbMATHCrossRefGoogle Scholar
  21. 21.
    Chan, T.F., Golub, G.H., Mulet, P.: A nonlinear primal-dual method for total variation-based image restoration. SIAM J. Sci. Comput. 20, 1964–1977 (1999) MathSciNetzbMATHCrossRefGoogle Scholar
  22. 22.
    Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66, 1632–1648 (2006) MathSciNetzbMATHCrossRefGoogle Scholar
  23. 23.
    Chesnaud, C., Réfrégier, P., Boulet, V.: Statistical region snake-based segmentation adapted to different physical noise models. IEEE Trans. Pattern Anal. Mach. Intell. 21, 1145–1157 (1999) CrossRefGoogle Scholar
  24. 24.
    Chung, G., Vese, L.A.: Energy minimization based segmentation and denoising using a multilayer level set approach. In: Proceedings of the 5th International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition. LNCS, vol. 3757, pp. 439–455. Springer, Berlin (2005) CrossRefGoogle Scholar
  25. 25.
    Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72, 195–215 (2007) CrossRefGoogle Scholar
  26. 26.
    Cremers, D., Pock, T., Kolev, K., Chambolle, A.: Convex relaxation techniques for segmentation, stereo, and multiview reconstruction. In: Markov Random Fields for Vision and Image Processing. MIT Press, New York (2011) Google Scholar
  27. 27.
    Dey, N., Blanc-Féraud, L., Zimmer, C., Roux, P., Kam, Z., Olivio-Marin, J.-C., Zerubia, J.: 3D microscopy deconvolution using Richardson-Lucy algorithm with total variation regularization. Tech. Rep. 5272, Institut National de Recherche en Informatique et en Automatique (2004) Google Scholar
  28. 28.
    Ekeland, I., Temam, R.: Convex Analysis and Variational Problems. Studies in Mathematics and Its Applications, vol. 1. North-Holland, Amsterdam (1976) zbMATHGoogle Scholar
  29. 29.
    Elman, H.C., Golub, G.H.: Inexact and preconditioned Uzawa algorithms for saddle point problems. SIAM J. Numer. Anal. 31, 1645–1661 (1994) MathSciNetzbMATHCrossRefGoogle Scholar
  30. 30.
    Fortin, M., Glowinski, R.: Augmented Lagrangian Methods: Applications to the Numerical Solution of Boundary-Value Problems. Studies in Mathematics and Its Applications, vol. 15. Elsevier, Amsterdam (1983) zbMATHGoogle Scholar
  31. 31.
    Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. J. Appl. Stat. 20, 25–62 (1993) CrossRefGoogle Scholar
  32. 32.
    Geman, S., McClure, D.E.: Bayesian image analysis: an application to single photon emission tomography. In: Statistical Computation Section, American Statistical Association, pp. 12–18 (1985) Google Scholar
  33. 33.
    Ghanem, A., et al.: Triggered replenishment imaging reduces variability of quantitative myocardial contrast echocardiography and allows assessment of myocardial blood flow reserve. Echocardiography 24, 149–158 (2007) CrossRefGoogle Scholar
  34. 34.
    Gianazza, U., Savaré, G., Toscani, G.: The Wasserstein gradient flow of the Fisher information and the quantum drift-diffusion equation. Arch. Ration. Mech. Anal. 194, 133–220 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  35. 35.
    Giusti, E.: Minimal Surfaces and Functions of Bounded Variation. Monographs in Mathematics, vol. 80. Birkhäuser, Basel (1984) zbMATHCrossRefGoogle Scholar
  36. 36.
    Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Studies in Applied Mathematics, vol. 9. SIAM, Philadelphia (1989) zbMATHCrossRefGoogle Scholar
  37. 37.
    Goldluecke, B., Cremers, D.: Convex relaxation for multilabel problems with product label spaces. In: Proceedings of the 11th European Conference on Computer Vision. LNCS, vol. 6315, pp. 225–238. Springer, Berlin (2010) Google Scholar
  38. 38.
    Goldstein, T., Osher, S.: The split Bregman method for L 1-regularized problems. SIAM J. Imaging Sci. 2, 323–343 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  39. 39.
    Goldstein, T., Bresson, X., Osher, S.: Geometric applications of the split Bregman method: segmentation and surface reconstruction. J. Sci. Comput. 45, 272–293 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  40. 40.
    Helin, T., Lassas, M.: Hierarchical models in statistical inverse problems and the Mumford-Shah functional. Inverse Probl. 27, 015008 (2011). 32 pp. MathSciNetCrossRefGoogle Scholar
  41. 41.
    Hell, S.W.: Toward fluorescence nanoscopy. Nat. Biotechnol. 21, 1347–1355 (2003) CrossRefGoogle Scholar
  42. 42.
    Ito, K., Kunisch, K.: Lagrange Multiplier Approach to Variational Problems and Applications. Advances in Design and Control, vol. 15. SIAM, Philadelphia (2008) zbMATHCrossRefGoogle Scholar
  43. 43.
    Jin, Z., Yang, X.: A variational model to remove the multiplicative noise in ultrasound images. J. Math. Imaging Vis. 39, 62–74 (2011) MathSciNetzbMATHCrossRefGoogle Scholar
  44. 44.
    Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int. J. Comput. Vis. 1, 321–331 (1988) CrossRefGoogle Scholar
  45. 45.
    Krissian, K., Kikinis, R., Westin, C.-F., Vosburgh, K.: Speckle-constrained filtering of ultrasound images. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 547–552 (2005) Google Scholar
  46. 46.
    Lantéri, H., Theys, C.: Restoration of astrophysical images—the case of Poisson data with additive Gaussian noise. EURASIP J. Appl. Signal Process. 15, 2500–2513 (2005) Google Scholar
  47. 47.
    Le, T., Chartrand, R., Asaki, T.J.: A variational approach to reconstructing images corrupted by Poisson noise. J. Math. Imaging Vis. 27, 257–263 (2007) MathSciNetCrossRefGoogle Scholar
  48. 48.
    Lellmann, J., Becker, F., Schnörr, C.: Convex optimization for multi-class image labeling with a novel family of total variation based regularizers. In: Proceedings of the IEEE 12th International Conference on Computer Vision, pp. 646–653 (2009) Google Scholar
  49. 49.
    Lellmann, J., Lenzen, F., Schnörr, C.: Optimality bounds for a variational relaxation of the image partitioning problem. In: Proceedings of the 8th International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition. LNCS, vol. 6819, pp. 132–146. Springer, Berlin (2011) Google Scholar
  50. 50.
    Li, C., Xu, C., Gui, C., Fox, M.D.: Level set evolution without re-initialization: a new variational formulation. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 430–436 (2005) Google Scholar
  51. 51.
    Llacer, J., Núñez, J.: Iterative maximum likelihood and Bayesian algorithms for image reconstruction in astronomy. In: White, R.L., Allen, R.J. (eds.) The Restoration of Hubble Space Telescope Images, pp. 62–69. The Space Telescope Science Institute, Baltimore (1990) Google Scholar
  52. 52.
    Loupas, T., McDicken, W.N., Allan, P.L.: An adaptive weighted median filter for speckle suppression in medical ultrasonic images. IEEE Trans. Circuits Syst. 36, 129–135 (1989) CrossRefGoogle Scholar
  53. 53.
    Martin, P., Réfrégier, P., Goudail, F., Guérault, F.: Influence of the noise model on level set active contour segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 26, 799–803 (2004) CrossRefGoogle Scholar
  54. 54.
    Megginson, R.E.: An Introduction to Banach Space Theory. Graduate Texts in Mathematics, vol. 183. Springer, Berlin (1998) zbMATHCrossRefGoogle Scholar
  55. 55.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42, 577–685 (1989) MathSciNetzbMATHCrossRefGoogle Scholar
  56. 56.
    Nečas, J.: Les Méthodes Directes en Théorie des Équations Elliptiques. Academia/Masson et Cie, Prague/Paris (1967) zbMATHGoogle Scholar
  57. 57.
    Noble, J.A., Boukerroui, D.: Ultrasound image segmentation: a survey. IEEE Trans. Med. Imaging 25, 987–1010 (2006) CrossRefGoogle Scholar
  58. 58.
    Obereder, A., Scherzer, O., Kovac, A.: Bivariate density estimation using BV regularisation. Comput. Stat. Data Anal. 51, 5622–5634 (2007) MathSciNetzbMATHCrossRefGoogle Scholar
  59. 59.
    Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Applied Mathematical Sciences, vol. 153. Springer, Berlin (2003) zbMATHGoogle Scholar
  60. 60.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys. 79, 12–49 (1988) MathSciNetzbMATHCrossRefGoogle Scholar
  61. 61.
    Paragois, N., Deriche, R.: Geodesic active regions: a new paradigm to deal with frame partition problems in computer vision. J. Vis. Commun. Image Represent. 13, 249–268 (2002) CrossRefGoogle Scholar
  62. 62.
    Pirich, C., Schwaiger, M.: The clinical role of positron emission tomography in management of the cardiac patient. Port. J. Cardiol. 19(Suppl 1), 89–100 (2000) Google Scholar
  63. 63.
    Pock, T., Chambolle, A., Cremers, D., Bischof, H.: A convex relaxation approach for computing minimal partitions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 810–817 (2009) Google Scholar
  64. 64.
    Pock, T., Cremers, D., Bischof, H., Chambolle, A.: Global solutions of variational models with convex regularization. SIAM J. Imaging Sci. 3, 1122–1145 (2010) MathSciNetzbMATHCrossRefGoogle Scholar
  65. 65.
    Resmerita, E., Anderssen, R.S.: Joint additive Kullback-Leibler residual minimization and regularization for linear inverse problems. Math. Methods Appl. Sci. 30, 1527–1544 (2007) MathSciNetzbMATHCrossRefGoogle Scholar
  66. 66.
    Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992) zbMATHCrossRefGoogle Scholar
  67. 67.
    Rudin, L., Lions, P.-L., Osher, S.: Multiplicative denoising and deblurring: theory and algorithms. In: Geometric Level Set Methods in Imaging, Vision, and Graphics, pp. 103–119. Springer, Berlin (2003) CrossRefGoogle Scholar
  68. 68.
    Sarti, A., Corsi, C., Mazzini, E., Lamberti, C.: Maximum likelihood segmentation of ultrasound images with Rayleigh distribution. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 52, 947–960 (2005) CrossRefGoogle Scholar
  69. 69.
    Sawatzky, A.: (Nonlocal) Total Variation in Medical Imaging. PhD thesis, University of Münster (2011). CAM Report 11-47, UCLA Google Scholar
  70. 70.
    Sawatzky, A., Brune, C., Müller, J., Burger, M.: Total variation processing of images with Poisson statistics. In: Jiang, X., Petkov, N. (eds.) Computer Analysis of Images and Patterns. Lecture Notes in Computer Science, vol. 5702, pp. 533–540 (2009) CrossRefGoogle Scholar
  71. 71.
    Schäfers, K.P., et al.: Absolute quantification of myocardial blood flow with \(\mathrm{H}_{2} ^{15}\)O and 3-dimensional PET: an experimental validation. J. Nucl. Med. 43, 1031–1040 (2002) Google Scholar
  72. 72.
    Shepp, L.A., Vardi, Y.: Maximum likelihood reconstruction for emission tomography. IEEE Trans. Med. Imaging 1, 113–122 (1982) CrossRefGoogle Scholar
  73. 73.
    Smereka, P.: Semi-implicit level set methods for curvature and surface diffusion motion. J. Sci. Comput. 19, 439–456 (2003) MathSciNetzbMATHCrossRefGoogle Scholar
  74. 74.
    Snyder, D.L., Hammoud, A.M., White, R.L.: Image recovery from data acquired with a charge-coupled-device camera. J. Opt. Soc. Am. A, Opt. Image Sci. Vis. 10, 1014–1023 (1993) CrossRefGoogle Scholar
  75. 75.
    Soret, M., Bacharach, S.L., Buvat, I.: Partial-volume effect in PET tumor imaging. J. Nucl. Med. 48, 932–945 (2007) CrossRefGoogle Scholar
  76. 76.
    Stypmann, J., et al.: Dilated cardiomyopathy in mice deficient for the lysosomal cysteine peptidase cathepsin L. Proc. Natl. Acad. Sci. USA 99, 6234–6239 (2002) CrossRefGoogle Scholar
  77. 77.
    Tur, M., Chin, K.C., Goodman, J.W.: When is speckle noise multiplicative? Appl. Opt. 21, 1157–1159 (1982) CrossRefGoogle Scholar
  78. 78.
    Vardi, Y., Shepp, L.A., Kaufman, L.: A statistical model for positron emission tomography. J. Am. Stat. Assoc. 80, 8–20 (1985) MathSciNetzbMATHCrossRefGoogle Scholar
  79. 79.
    Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50, 271–293 (2002) zbMATHCrossRefGoogle Scholar
  80. 80.
    Villani, C.: Topics in Optimal Transportation. Graduate Studies in Mathematics, vol. 58. American Mathematical Society, New York (2003) zbMATHGoogle Scholar
  81. 81.
    Vovk, U., Pernuš, F., Likar, B.: A review of methods for correction of intensity inhomogeneity in MRI. IEEE Trans. Med. Imaging 26, 405–421 (2007) CrossRefGoogle Scholar
  82. 82.
    Wellnhofer, E., et al.: Angiographic assessment of cardiac allograft vasculopathy: results of a consensus conference of the task force for thoracic organ transplantation of the German cardiac society. Transpl. Int. 23, 1094–1104 (2010) CrossRefGoogle Scholar
  83. 83.
    Wernick, M.N., Aarsvold, J.N. (eds.): Emission Tomography: The Fundamentals of PET and SPECT. Elsevier, Amsterdam (2004) Google Scholar
  84. 84.
    Wirtz, D.: SEGMEDIX: Development and application of a medical image segmentation framework. Master’s thesis, University of Münster (2009)
  85. 85.
    Xiao, G., Brady, M., Noble, J.A., Zhang, Y.: Segmentation of ultrasound B-mode images with intensity inhomogeneity correction. IEEE Trans. Med. Imaging 21, 48–57 (2002) CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Alex Sawatzky
    • 1
    Email author
  • Daniel Tenbrinck
    • 2
  • Xiaoyi Jiang
    • 2
  • Martin Burger
    • 1
  1. 1.Institut für Numerische und Angewandte MathematikWestfälische Wilhelms-Universität MünsterMünsterGermany
  2. 2.Institut für InformatikWestfälische Wilhelms-Universität MünsterMünsterGermany

Personalised recommendations