Skip to main content
SpringerLink
Log in

A Three-Stage Approach for Segmenting Degraded Color Images: Smoothing, Lifting and Thresholding (SLaT)

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

In this paper, we propose a Smoothing, Lifting and Thresholding (SLaT) method with three stages for multiphase segmentation of color images corrupted by different degradations: noise, information loss and blur. At the first stage, a convex variant of the Mumford–Shah model is applied to each channel to obtain a smooth image. We show that the model has unique solution under different degradations. In order to properly handle the color information, the second stage is dimension lifting where we consider a new vector-valued image composed of the restored image and its transform in a secondary color space to provide additional information. This ensures that even if the first color space has highly correlated channels, we can still have enough information to give good segmentation results. In the last stage, we apply multichannel thresholding to the combined vector-valued image to find the segmentation. The number of phases is only required in the last stage, so users can modify it without the need of solving the previous stages again. Experiments demonstrate that our SLaT method gives excellent results in terms of segmentation quality and CPU time in comparison with other state-of-the-art segmentation methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/

References

  1. Bar, L., Chan, T.F., Chung, G., Jung, M., Kiryati, N., Mohieddine, R., Sochen, N., Vese, L.A.: Mumford and Shah model and its applications to image segmentation and image restoration. In: Scherzer, O. (ed.) Handbook of Mathematical Methods in Imaging, pp. 1539–1598. Springer, Berlin (2015)

    Chapter  Google Scholar 

  2. Benninghoff, H., Garcke, H.: Efficient image segmentation and restoration using parametric curve evolution with junctions and topology changes. SIAM J. Imaging Sci. 7(3), 1451–1483 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  3. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  MATH  Google Scholar 

  4. Bresson, X., Esedoglu, S., Vandergheynst, P., Thiran, J.P., Osher, S.: Fast global minimization of the active contour/snake model. J. Math. Imaging Vis. 28(2), 151–167 (2007)

    Article  MathSciNet  Google Scholar 

  5. Cai, X.: Variational image segmentation model coupled with image restoration achievements. Pattern Recognit. 48(6), 2029–2042 (2015)

    Article  Google Scholar 

  6. Cai, X., Chan, R., Zeng, T.: A two-stage image segmentation method using a convex variant of the Mumford–Shah model and thresholding. SIAM J. Imaging Sci. 6(1), 368–390 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  7. Cai, X., Steidl, G.: Multiclass segmentation by iterated ROF thresholding. In: Heyden, A., Kahl, F., Olsson, C., Oskarsson, M., Tai, X.C. (eds.) Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 237–250. Springer, Berlin (2013)

    Chapter  Google Scholar 

  8. Cardelino, J., Caselles, V., Bertalmio, M., Randall, G.: A contrario selection of optimal partitions for image segmentation. SIAM J. Imaging Sci. 6(3), 1274–1317 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chambolle, A., Cremers, D., Pock, T.: A convex approach to minimal partitions. SIAM J. Imaging Sci. 5(4), 1113–1158 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  10. Chambolle, A., Pock, T.: A first-order primal-dual algorithm for convex problems with applications to imaging. J. Math. Imaging Vis. 40(1), 120–145 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  11. Chan, R., Yang, H., Zeng, T.: A two-stage image segmentation method for blurry images with poisson or multiplicative gamma noise. SIAM J. Imaging Sci. 7(1), 98–127 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  12. Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (2000)

    Article  Google Scholar 

  14. Chan, T.F., Vese, L., et al.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)

    Article  MATH  Google Scholar 

  15. Chen, Y., Lan, G., Ouyang, Y.: Optimal primal-dual methods for a class of saddle point problems. SIAM J. Optim. 24(4), 1779–1814 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  16. Cremers, D., Rousson, M., Deriche, R.: A review of statistical approaches to level set segmentation: integrating color, texture, motion and shape. Int. J. Comput. Vis. 72(2), 195–215 (2007)

    Article  Google Scholar 

  17. Ekeland, I., Temam, R.: Convex analysis and variational problems. SIAM Classics in Applied Mathematics, Philadelphia (1976)

    MATH  Google Scholar 

  18. Evans, L.C.: Partial differential equations and Monge-Kantorovich mass transfer. Curr. Dev. Math. 1997(1), 65–126 (1997)

    Article  MATH  Google Scholar 

  19. Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)

    Article  MATH  Google Scholar 

  20. Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  21. Grady, L.: Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1768–1783 (2006)

    Article  Google Scholar 

  22. Grady, L., Alvino, C.: Reformulating and optimizing the Mumford–Shah functional on a graph—faster, lower energy solution. In: ECCV 2008, pp. 248–261. Springer, Berlin (2008)

  23. Jung, Y.M., Kang, S.H., Shen, J.: Multiphase image segmentation via Modica–Mortola phase transition. SIAM J. Appl. Math. 67(5), 1213–1232 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  24. Kanungo, T., Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002)

    Article  MATH  Google Scholar 

  25. Kay, D., Tomasi, A., et al.: Color image segmentation by the vector-valued Allen–Cahn phase-field model: a multigrid solution. IEEE Trans. Image Process. 18(10), 2330–2339 (2009)

    Article  MathSciNet  Google Scholar 

  26. Levinshtein, A., Stere, A., Kutulakos, K.N., Fleet, D.J., Dickinson, S.J., Siddiqi, K.: Turbopixels: fast superpixels using geometric flows. IEEE Trans. Pattern Anal. Mach. Intell. 31(12), 2290–2297 (2009)

    Article  Google Scholar 

  27. Li, C., Huang, R., Ding, Z., Gatenby, J.C., Metaxas, D.N., C, G.J.: A level set method for image segmentation in the presence of intensity inhomogeneity with application to MRI. IEEE Trans. Image Process. 20(7), 2007–2016 (2011)

    Article  MathSciNet  Google Scholar 

  28. Li, F., Ng, M.K., Zeng, T.Y., Shen, C.: A multiphase image segmentation method based on fuzzy region competition. SIAM J. Imaging Sci. 3(3), 277–299 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  29. Lukac, R., Plataniotis, K.N.: Color Image Processing: Methods and Applications. CRC Press, Boca Raton (2007)

    Google Scholar 

  30. Luong, Q.T.: Color in computer vision. In: Chen, C.H., Pau, L.F., Wang, P.S.P. (eds.) Handbook of Pattern Recognition & Computer Vision, pp. 311–368. World Scientific Publishing Co., Inc., River Edge, NJ, USA (1993)

    Chapter  Google Scholar 

  31. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. ICCV 2, 416–423 (2001)

    Google Scholar 

  32. Mumford, D., Shah, J.: Boundary detection by minimizing functionals. In: Ullman, S., Richards, W. (eds.) Image Understanding 1989. Ablex Publishing Corporation, New Jersey (1990)

    Google Scholar 

  33. Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)

    Article  MathSciNet  MATH  Google Scholar 

  34. Paschos, G.: Perceptually uniform color spaces for color texture analysis: an empirical evaluation. IEEE Trans. Image Process. 10(6), 932–937 (2001)

    Article  MATH  Google Scholar 

  35. Plaza, A., Benediktsson, J.A., Boardman, J.W., Brazile, J., Bruzzone, L., Camps-Valls, G., Chanussot, J., Fauvel, M., Gamba, P., Gualtieri, A., et al.: Recent advances in techniques for hyperspectral image processing. Remote Sens. Environ. 113, S110–S122 (2009)

    Article  Google Scholar 

  36. Pock, T., Chambolle, A., Cremers, D., Bischof, H.: A convex relaxation approach for computing minimal partitions. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009. CVPR 2009, pp. 810–817 (2009)

  37. Potts, R.B.: Some generalized order-disorder transformations. In: Mathematical Proceedings of the Cambridge Philosophical Society, vol. 48, pp. 106–109. Cambridge University Press, Cambridge (1952)

  38. Rotaru, C., Graf, T., Zhang, J.: Color image segmentation in HSI space for automotive applications. J. Real-Time Image Process. 3(4), 311–322 (2008)

    Article  Google Scholar 

  39. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Phys. D Nonlinear Phenom. 60(1), 259–268 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  40. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  41. Storath, M., Weinmann, A.: Fast partitioning of vector-valued images. SIAM J. Imaging Sci. 7(3), 1826–1852 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Tai, C., Zhang, X., Shen, Z.: Wavelet frame based multiphase image segmentation. SIAM J. Imaging Sci. 6(4), 2521–2546 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  43. Tai, Y.W., Jia, J., Tang, C.K.: Soft color segmentation and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1520–1537 (2007)

    Article  Google Scholar 

  44. Townsend, D.: Multimodality imaging of structure and function. Phys. Med. Biol. 53(4), R1 (2008)

    Article  MathSciNet  Google Scholar 

  45. Vandenbroucke, N., Macaire, L., Postaire, J.: Color image segmentation by pixel classification in an adapted hybrid color space. Application to soccer image analysis. Comput. Vis. Image Underst. 90(2), 190–216 (2003)

    Article  Google Scholar 

  46. Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50(3), 271–293 (2002)

    Article  MATH  Google Scholar 

  47. Wang, X., Tang, Y., Masnou, S., Chen, L.: A global/local affinity graph for image segmentation. IEEE Trans. Image Process. 24(4), 1399–1411 (2015)

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors thank G. Steidl and M. Bertalmío for constructive discussions. The work of X. Cai is partially supported by Welcome Trust, Issac Newton Trust, and KAUST Award No. KUK-I1-007-43. The work of R. Chan is partially supported by HKRGC GRF Grant No. CUHK300614, CUHK14306316, CRF Grant No. CUHK2/CRF/11G, CRF Grant C1007-15G, and AoE Grant AoE/M-05/12 The work of M. Nikolova is partially supported by HKRGC GRF Grant No. CUHK300614, and by the French Research Agency (ANR) under Grant No. ANR-14-CE27-001 (MIRIAM). The work of T. Zeng is partially supported by NSFC 11271049, 11671002, RGC 211911, 12302714 and RFGs of HKBU.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics and Theoretical Physics (DAMTP), University of Cambridge, Cambridge, CB3 0WA, UK

    Xiaohao Cai

  2. Mullard Space Science Laboratory (MSSL), University College London, Dorking, Surry, RH5 6NT, UK

    Xiaohao Cai

  3. Department of Mathematics, The Chinese University of Hong Kong, Shatin, Hong Kong

    Raymond Chan

  4. CMLA, ENS Cachan, CNRS, Université Paris-Saclay, 94235, Cachan, France

    Mila Nikolova

  5. Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

    Tieyong Zeng

Authors
  1. Xiaohao Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Raymond Chan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mila Nikolova
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tieyong Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tieyong Zeng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, X., Chan, R., Nikolova, M. et al. A Three-Stage Approach for Segmenting Degraded Color Images: Smoothing, Lifting and Thresholding (SLaT). J Sci Comput 72, 1313–1332 (2017). https://doi.org/10.1007/s10915-017-0402-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10915-017-0402-2

Keywords

Search

Navigation