Journal of Mathematical Imaging and Vision

, Volume 60, Issue 1, pp 1–17 | Cite as

Image Segmentation with Depth Information via Simplified Variational Level Set Formulation

  • Lu Tan
  • Zhenkuan Pan
  • Wanquan Liu
  • Jinming Duan
  • Weibo Wei
  • Guodong Wang
Article
First Online:
Received:
Accepted:

Abstract

Image segmentation with depth information can be modeled as a minimization problem with Nitzberg–Mumford–Shiota functional, which can be transformed into a tractable variational level set formulation. However, such formulation leads to a series of complicated high-order nonlinear partial differential equations which are difficult to solve efficiently. In this paper, we first propose an equivalently reduced variational level set formulation without using curvatures by taking level set functions as signed distance functions. Then, an alternating direction method of multipliers (ADMM) based on this simplified variational level set formulation is designed by introducing some auxiliary variables, Lagrange multipliers via using alternating optimization strategy. With the proposed ADMM method, the minimization problem for this simplified variational level set formulation is transformed into a series of sub-problems, which can be solved easily via using the Gauss–Seidel iterations, fast Fourier transform and soft thresholding formulas. The level set functions are treated as signed distance functions during computation process via implementing a simple algebraic projection method, which avoids the traditional re-initialization process for conventional variational level set methods. Extensive experiments have been conducted on both synthetic and real images, which validate the proposed approach, and show advantages of the proposed ADMM projection over algorithms based on traditional gradient descent method in terms of computational efficiency.

Keywords

Segmentation with depth Nitzberg–Mumford–Shiota (NMS)functional Variational level set formulation Alternating direction method of multipliers (ADMM) Projection method Fast Fourier Transform (FFT) Soft thresholding formulas 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgements

The work has been partially supported by the National Natural Science Foundation of China with Grant numbers 61305045, 61170106, 61363066 and 61303079.

References

  1. 1.
    Nitzberg, M., Mumford, D.: The 2.1D sketch. In: Proceedings of the Third IEEE International Conference on Computer Vision, pp. 138–144 (1990)Google Scholar
  2. 2.
    Yu, C.-C., Liu, Y.-J., Wu, M.T., Li, K.-Y., Fu, X.: A global energy optimization framework for 2.1D sketch extraction from monocular images. Graph. Models 76(5), 507–521 (2014)CrossRefGoogle Scholar
  3. 3.
    Amer, M.R., Yousefi, S., Raich, R., Todorovic, S.: Monocular extraction of 2.1D sketch using constrained convex optimization. Int. J. Comput. Vis. 112(1), 23 (2015)CrossRefGoogle Scholar
  4. 4.
    Zhu, W., Chan, T.F.: A variational model for capturing illusory contours using curvature. J. Math. Imaging Vis. 27(1), 29–40 (2007)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Kang, S.-H., Zhu, W., Shen, J(.J.-H).: Illusory shapes via corner fusion. SIAM J. Imaging Sci. 7(4), 1907–1936 (2014)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Nitzberg, M., Mumford, D., Shiota, T.: Filtering, Segmentation, and Depth. Lecture Notes in Computer Sciences, vol. 662. Springer-Verlag, Berlin (1993)CrossRefMATHGoogle Scholar
  7. 7.
    Mumford, D., Shah, J.: Optimal approximations by piecewise smooth functions and associated variational problems. Commun. Pure Appl. Math. 42(5), 577–685 (1989)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Mumford, D.: Elastica and computer vision. In: Bajaj, C.L. (ed.) Algebraic Geometry and Its Applications, pp. 491–506. Springer-Verlag, New York (1994)CrossRefGoogle Scholar
  9. 9.
    Esedoglu, S., March, R.: Segmentation with depth but without detecting junctions. J. Math. Imaging Vis. 18(1), 7–15 (2003)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Ambrosio, L.A., Tortorelli, V.M.: Approximation of functionals depending on jumps by elliptic functionals via \(\Gamma \)-convegence. Commun. Pure Appl. Math. 43(8), 999–1036 (1990)CrossRefMATHGoogle Scholar
  11. 11.
    Loreti, P., March, R.: Propagation of fronts in a nonlinear fourth order equation. Eur. J. Appl. Math. 2, 203–213 (2000)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Zhu, W., Chan, T.F., Esedoglu, S.: Segmentation with depth: a level set approach. SIAM J. Sci. Comput. 28(5), 1957–1973 (2006)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Zhao, H.-K., Chan, T.F., Merriman, B., Osher, S.: A variational level set approach to multiphase motion. J. Comput. Phys. 127(1), 179–195 (1996)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Trans. Image Process. 10(2), 266–277 (2001)CrossRefMATHGoogle Scholar
  15. 15.
    Vese, L.A., Chan, T.F.: A multiphase level set framework for image segmentation using the Mumford and Shah model. Int. J. Comput. Vis. 50(3), 271–293 (2002)CrossRefMATHGoogle Scholar
  16. 16.
    Osher, S., Sethian, J.A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton–Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Smereka, P.: Semi-implicit level set methods for curvature and surface diffusion motion. J. Sci. Comput. 19(1–3), 439–456 (2003)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Zhu, W., Tai, X.-C., Chan, T.F.: Image segmentation using Euler’s elastica as the regularization. J. Sci. Comput. 57(2), 414–438 (2013)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Wu, C., Tai, X.-C.: Augmented Lagrangian method, dual methods, and split Bregman iteration for ROF, vectorial TV, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Goldstein, T., O’Donoghue, B., Setzer, S., Baraniuk, R.: Fast alternating direction optimization methods. SIAM J. Imaging Sci. 7(3), 1588–1623 (2014)MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of denoising and segmentation models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Lie, J., Lysaker, M., Tai, X.-C.: A binary level set model and some applications to Mumford–Shah image segmentation. IEEE Trans. Image Process. 15(5), 1171–1181 (2006)CrossRefMATHGoogle Scholar
  23. 23.
    Chan, T.F., Kang, S.-H., Shen, J(.J.-H).: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)MathSciNetMATHGoogle Scholar
  24. 24.
    Masnou, S.: Disocclusion: a variational approach using level lines. IEEE Trans. Image Process. 11(2), 68–76 (2002)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Zhu, W., Chan, T.F.: Image denoising using mean curvature of image surface. SIAM J. Imaging Sci. 5(1), 1–32 (2012)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Yip, A., Zhu, W.: A fast modified Newton’s method for curvature based denoising of 1D signals. Inverse Probl. Imaging 7(3), 1075–1097 (2013)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Zhu, W., Tai, X.-C., Chan, T.F.: Augmented Lagrangian method for a mean curvature based image denoising model. Inverse Probl. Imaging 7(4), 1409–1432 (2013)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Myllykoski, M., Glowinski, R., Karkkainen, T., Rossi, T.: A new augmented Lagrangian approach for L1-mean curvature image denoising. SIAM J. Imaging Sci. 8(1), 95–125 (2015)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Tai, X.-C., Hahn, J., Chung, G.J.: A fast algorithm for Euler’s elastica model using augmented Lagrangian method. SIAM J. Imaging Sci. 4(1), 313–344 (2011)MathSciNetCrossRefMATHGoogle Scholar
  30. 30.
    Tai, X.-C.: Fast Numerical Schemes Related to Curvature Minimization: A Brief and Elementary Review, UCLA CAM Report14-40 (May, 2014)Google Scholar
  31. 31.
    Osher, S., Fedkiw, R.: Level Set Methods and Dynamic Implicit Surfaces. Springer, Berlin (2003)CrossRefMATHGoogle Scholar
  32. 32.
    Zhao, H.-K.: Fast sweeping method for eikonal equations. Math. Comput. 74, 603–627 (2005)MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Li, C., Xu, C., Gui, C., Fox, M. D.: Level set evolution without re-initialization: a new variational formulation. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 430–436, June 20 (2005)Google Scholar
  34. 34.
    Liu, C., Dong, F., Zhu, S., Kong, D., Liu, K.: New variational formulations for level set evolution without re-initialization with applications to image segmentation. J. Math. Imaging Vis. 41(3), 194–209 (2011)CrossRefMATHGoogle Scholar
  35. 35.
    Duan, J., Pan, Z., Yin, X., Wei, W., Wang, G. (2014) Some fast projection methods based on Chan–Vese model for image segmentation. EURASIP J. Image Video Process.  10.1186/1687-5281-2014-7
  36. 36.
    Yashtini, M.: Alternating Direction Method of Multiplier for Euler’s Elastica-Based Denoising, Scale Space and Variational Methods in Computer Vision, pp. 690–701. Springer, Berlin (2015)Google Scholar
  37. 37.
    Marquina, A., Osher, S.: Explicit algorithms for a new time dependent model based on level set motion for nonlinear deblurring and noise removal. SIAM J. Sci. Comput. 22(2), 387–405 (2000)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Chan, T.F., Sandberg, B.Y., Vese, L.A.: Active contours without edges for vector-valued images. J. Vis. Commun. Image Represent. 11(2), 130–141 (1970)CrossRefGoogle Scholar
  39. 39.
    Wang, Y., Yin, W., Zeng, J.: Global convergence of ADMM in nonconvex nonsmooth optimization. arXiv preprint arXiv:1511.06324, (2015)
  40. 40.
    Glowinski, R., Pan, T.W., Tai, X.C.: Some facts about operator splitting and alternating direction methods. UCLA CAM Report: 16-10 (2016)Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.College of Information EngineeringQingdao UniversityQingdaoChina
  2. 2.School of Computer ScienceUniversity of NottinghamNottinghamUK
  3. 3.Department of ComputingCurtin UniversityBentleyAustralia

Personalised recommendations