Journal of Mathematical Imaging and Vision

, Volume 60, Issue 1, pp 1–17 | Cite as

Image Segmentation with Depth Information via Simplified Variational Level Set Formulation

  • Lu Tan
  • Zhenkuan Pan
  • Wanquan LiuEmail author
  • Jinming Duan
  • Weibo Wei
  • Guodong Wang


Image segmentation with depth information can be modeled as a minimization problem with Nitzberg–Mumford–Shiota functional, which can be transformed into a tractable variational level set formulation. However, such formulation leads to a series of complicated high-order nonlinear partial differential equations which are difficult to solve efficiently. In this paper, we first propose an equivalently reduced variational level set formulation without using curvatures by taking level set functions as signed distance functions. Then, an alternating direction method of multipliers (ADMM) based on this simplified variational level set formulation is designed by introducing some auxiliary variables, Lagrange multipliers via using alternating optimization strategy. With the proposed ADMM method, the minimization problem for this simplified variational level set formulation is transformed into a series of sub-problems, which can be solved easily via using the Gauss–Seidel iterations, fast Fourier transform and soft thresholding formulas. The level set functions are treated as signed distance functions during computation process via implementing a simple algebraic projection method, which avoids the traditional re-initialization process for conventional variational level set methods. Extensive experiments have been conducted on both synthetic and real images, which validate the proposed approach, and show advantages of the proposed ADMM projection over algorithms based on traditional gradient descent method in terms of computational efficiency.


Segmentation with depth Nitzberg–Mumford–Shiota (NMS)functional Variational level set formulation Alternating direction method of multipliers (ADMM) Projection method Fast Fourier Transform (FFT) Soft thresholding formulas 



The work has been partially supported by the National Natural Science Foundation of China with Grant numbers 61305045, 61170106, 61363066 and 61303079.


  1. 1.
    Nitzberg, M., Mumford, D.: The 2.1D sketch. In: Proceedings of the Third IEEE International Conference on Computer Vision, pp. 138–144 (1990)Google Scholar
  2. 2.
    Yu, C.-C., Liu, Y.-J., Wu, M.T., Li, K.-Y., Fu, X.: A global energy optimization framework for 2.1D sketch extraction from monocular images. Graph. Models 76(5), 507–521 (2014)CrossRefGoogle Scholar
  3. 3.
    Amer, M.R., Yousefi, S., Raich, R., Todorovic, S.: Monocular extraction of 2.1D sketch using constrained convex optimization. Int. J. Comput. Vis. 112(1), 23 (2015)CrossRefGoogle Scholar
  4. 4.
    Zhu, W., Chan, T.F.: A variational model for capturing illusory contours using curvature. J. Math. Imaging Vis. 27(1), 29–40 (2007)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Kang, S.-H., Zhu, W., Shen, J(.J.-H).: Illusory shapes via corner fusion. SIAM J. Imaging Sci. 7(4), 1907–1936 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Nitzberg, M., Mumford, D., Shiota, T.: Filtering, Segmentation, and Depth. Lecture Notes in Computer Sciences, vol. 662. Springer-Verlag, Berlin (1993)CrossRefzbMATHGoogle Scholar
  7. 7.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Mumford, D.: Elastica and computer vision. In: Bajaj, C.L. (ed.) Algebraic Geometry and Its Applications, pp. 491–506. Springer-Verlag, New York (1994)CrossRefGoogle Scholar
  9. 9.
    Esedoglu, S., March, R.: Segmentation with depth but without detecting junctions. J. Math. Imaging Vis. 18(1), 7–15 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Ambrosio, L.A., Tortorelli, V.M.: Approximation of functionals depending on jumps by elliptic functionals via \(\Gamma \)-convegence. Commun. Pure Appl. Math. 43(8), 999–1036 (1990)CrossRefzbMATHGoogle Scholar
  11. 11.
    Loreti, P., March, R.: Propagation of fronts in a nonlinear fourth order equation. Eur. J. Appl. Math. 2, 203–213 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Zhu, W., Chan, T.F., Esedoglu, S.: Segmentation with depth: a level set approach. SIAM J. Sci. Comput. 28(5), 1957–1973 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Zhao, H.-K., Chan, T.F., Merriman, B., Osher, S.: A variational level set approach to multiphase motion. J. Comput. Phys. 127(1), 179–195 (1996)MathSciNetCrossRefzbMATHGoogle Scholar
  14. 14.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)CrossRefzbMATHGoogle Scholar
  15. 15.
    Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50(3), 271–293 (2002)CrossRefzbMATHGoogle Scholar
  16. 16.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Smereka, P.: Semi-implicit level set methods for curvature and surface diffusion motion. J. Sci. Comput. 19(1–3), 439–456 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Zhu, W., Tai, X.-C., Chan, T.F.: Image segmentation using Euler’s elastica as the regularization. J. Sci. Comput. 57(2), 414–438 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Wu, C., Tai, X.-C.: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  20. 20.
    Goldstein, T., O’Donoghue, B., Setzer, S., Baraniuk, R.: Fast alternating direction optimization methods. SIAM J. Imaging Sci. 7(3), 1588–1623 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of denoising and segmentation models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Lie, J., Lysaker, M., Tai, X.-C.: A binary level set model and some applications to Mumford–Shah image segmentation. IEEE Trans. Image Process. 15(5), 1171–1181 (2006)CrossRefzbMATHGoogle Scholar
  23. 23.
    Chan, T.F., Kang, S.-H., Shen, J(.J.-H).: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Masnou, S.: Disocclusion: a variational approach using level lines. IEEE Trans. Image Process. 11(2), 68–76 (2002)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Zhu, W., Chan, T.F.: Image denoising using mean curvature of image surface. SIAM J. Imaging Sci. 5(1), 1–32 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Yip, A., Zhu, W.: A fast modified Newton’s method for curvature based denoising of 1D signals. Inverse Probl. Imaging 7(3), 1075–1097 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Zhu, W., Tai, X.-C., Chan, T.F.: Augmented Lagrangian method for a mean curvature based image denoising model. Inverse Probl. Imaging 7(4), 1409–1432 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Myllykoski, M., Glowinski, R., Karkkainen, T., Rossi, T.: A new augmented Lagrangian approach for L1-mean curvature image denoising. SIAM J. Imaging Sci. 8(1), 95–125 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Tai, X.-C., Hahn, J., Chung, G.J.: A fast algorithm for Euler’s elastica model using augmented Lagrangian method. SIAM J. Imaging Sci. 4(1), 313–344 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Tai, X.-C.: Fast Numerical Schemes Related to Curvature Minimization: A Brief and Elementary Review, UCLA CAM Report14-40 (May, 2014)Google Scholar
  31. 31.
    Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Berlin (2003)CrossRefzbMATHGoogle Scholar
  32. 32.
    Zhao, H.-K.: Fast sweeping method for eikonal equations. Math. Comput. 74, 603–627 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Li, C., Xu, C., Gui, C., Fox, M. D.: Level set evolution without re-initialization: a new variational formulation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 430–436, June 20 (2005)Google Scholar
  34. 34.
    Liu, C., Dong, F., Zhu, S., Kong, D., Liu, K.: New variational formulations for level set evolution without re-initialization with applications to image segmentation. J. Math. Imaging Vis. 41(3), 194–209 (2011)CrossRefzbMATHGoogle Scholar
  35. 35.
    Duan, J., Pan, Z., Yin, X., Wei, W., Wang, G. (2014) Some fast projection methods based on Chan–Vese model for image segmentation. EURASIP J. Image Video Process.  10.1186/1687-5281-2014-7
  36. 36.
    Yashtini, M.: Alternating Direction Method of Multiplier for Euler’s Elastica-Based Denoising, Scale Space and Variational Methods in Computer Vision, pp. 690–701. Springer, Berlin (2015)Google Scholar
  37. 37.
    Marquina, A., Osher, S.: Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM J. Sci. Comput. 22(2), 387–405 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (1970)CrossRefGoogle Scholar
  39. 39.
    Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. arXiv preprint arXiv:1511.06324, (2015)
  40. 40.
    Glowinski, R., Pan, T.W., Tai, X.C.: Some facts about operator splitting and alternating direction methods. UCLA CAM Report: 16-10 (2016)Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.College of Information EngineeringQingdao UniversityQingdaoChina
  2. 2.School of Computer ScienceUniversity of NottinghamNottinghamUK
  3. 3.Department of ComputingCurtin UniversityBentleyAustralia

Personalised recommendations