Journal of Mathematical Imaging and Vision

, Volume 58, Issue 2, pp 294–320 | Cite as

Two-Dimensional Compact Variational Mode Decomposition

Spatially Compact and Spectrally Sparse Image Decomposition and Segmentation
  • Dominique Zosso
  • Konstantin Dragomiretskiy
  • Andrea L. Bertozzi
  • Paul S. Weiss


Decomposing multidimensional signals, such as images, into spatially compact, potentially overlapping modes of essentially wavelike nature makes these components accessible for further downstream analysis. This decomposition enables space–frequency analysis, demodulation, estimation of local orientation, edge and corner detection, texture analysis, denoising, inpainting, or curvature estimation. Our model decomposes the input signal into modes with narrow Fourier bandwidth; to cope with sharp region boundaries, incompatible with narrow bandwidth, we introduce binary support functions that act as masks on the narrow-band mode for image recomposition. \(L^1\) and TV terms promote sparsity and spatial compactness. Constraining the support functions to partitions of the signal domain, we effectively get an image segmentation model based on spectral homogeneity. By coupling several submodes together with a single support function, we are able to decompose an image into several crystal grains. Our efficient algorithm is based on variable splitting and alternate direction optimization; we employ Merriman–Bence–Osher-like threshold dynamics to handle efficiently the motion by mean curvature of the support function boundaries under the sparsity promoting terms. The versatility and effectiveness of our proposed model is demonstrated on a broad variety of example images from different modalities. These demonstrations include the decomposition of images into overlapping modes with smooth or sharp boundaries, segmentation of images of crystal grains, and inpainting of damaged image regions through artifact detection.


Image decomposition Image segmentation Spatio-spectral decomposition Microscopy Crystal grains Artifact detection Threshold dynamics Variational methods Sparse time–frequency analysis 

Mathematics Subject Classification



  1. 1.
    Ayvaci, A., Raptis, M., Soatto, S.: Sparse occlusion detection with optical flow. Int. J. Comput. Vis. 97(3), 322–338 (2011). doi:10.1007/s11263-011-0490-7 MathSciNetCrossRefMATHGoogle Scholar
  2. 2.
    Bertsekas, D.P.: Multiplier methods: a survey. Automatica 12(2), 133–145 (1976)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Bonnell, D.A., Basov, D.N., Bode, M., Diebold, U., Kalinin, S.V., Madhavan, V., Novotny, L., Salmeron, M., Schwarz, U.D., Weiss, P.S.: Imaging physical phenomena with local probes: from electrons to photons. Rev. Mod. Phys. 84(3), 1343–1381 (2012). doi:10.1103/RevModPhys.84.1343 CrossRefGoogle Scholar
  4. 4.
    Bülow, T., Sommer, G.: A novel approach to the 2D analytic signal. In: Computer Analysis of Images and Patterns, pp. 25–32 (1999)Google Scholar
  5. 5.
    Candes, E.J., Donoho, D.L.: Curvelets: A surprisingly effective nonadaptive representation for objects with edges. In: Curve and Surface Fitting, pp. 105–120 (1999)Google Scholar
  6. 6.
    Carson, J.: Notes on the theory of modulation. Proc. IRE 10(1), 57–64 (1922). doi:10.1109/JRPROC.1922.219793 CrossRefGoogle Scholar
  7. 7.
    Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imaging Vis. 20, 89–97 (2004)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Claridge, S.A., Schwartz, J.J., Weiss, P.S.: Electrons, photons, and force: quantitative single-molecule measurements from physics to biology. ACS Nano 5(2), 693–729 (2011). doi:10.1021/nn103298x CrossRefGoogle Scholar
  9. 9.
    Clausel, M., Oberlin, T., Perrier, V.: The monogenic synchrosqueezed wavelet transform: a tool for the decomposition/demodulation of AM-FM images (2012). arxiv:1211.5082
  10. 10.
    Claridge, S.A., Thomas, J.C., Silverman, M.A., Schwartz, J.J., Yang, Y., Wang, C., Weiss, P.S.: Differentiating amino acid residues and side chain orientations in peptides using scanning tunneling microscopy. J. Am. Chem. Soc. 135(49), 18528–18535 (2013). doi:10.1021/ja408550a CrossRefGoogle Scholar
  11. 11.
    Cohen, L.D.: Auxiliary variables and two-step iterative algorithms in computer vision problems. J. Math. Imaging Vis. 6(1), 59–83 (1996). doi:10.1007/BF00127375 MathSciNetCrossRefGoogle Scholar
  12. 12.
    Combettes, P.L., Pesquet, J.C.: Proximal splitting methods in signal processing. In: Bauschke, H.H., Burachik, R.S., Combettes, P.L., Elser, V., Luke, D.R., Wolkowicz, H. (eds.) Fixed-Point Algorithms for Inverse Problems in Science and Engineering, Springer Optimization and Its Applications, vol. 49, pp. 185–212. Springer, New York, NY (2011). doi:10.1007/978-1-4419-9569-8 Google Scholar
  13. 13.
    Daubechies, I.: Orthonormal bases of compactly supported wavelets. Commun. Pure Appl. Math. 41(7), 909–996 (1988). doi:10.1002/cpa.3160410705 MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Daubechies, I., Lu, J., Wu, H.T.: Synchrosqueezed wavelet transforms: an empirical mode decomposition-like tool. Appl. Comput. Harmonic Anal. 30(2), 243–261 (2011). doi:10.1016/j.acha.2010.08.002 MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Do, M., Vetterli, M.: Pyramidal directional filter banks and curvelets. In: Proceedings 2001 International Conference on Image Processing, vol. 2, pp. 158–161. IEEE (2001). doi:10.1109/ICIP.2001.958075
  16. 16.
    Dong, W., Li, X., Lin, X., Li, Z.: A bidimensional empirical mode decomposition method for fusion of multispectral and panchromatic remote sensing images. Remote Sens. 6(9), 8446–8467 (2014). doi:10.3390/rs6098446 CrossRefGoogle Scholar
  17. 17.
    Dragomiretskiy, K., Zosso, D.: Variational mode decomposition. IEEE Trans. Signal Process. 62(3), 531–544 (2014). doi:10.1109/TSP.2013.2288675 MathSciNetCrossRefGoogle Scholar
  18. 18.
    Dragomiretskiy, K., Zosso, D.: Two-dimensional variational mode decomposition. In: Tai, X.C., Bae, E., Chan, T.F., Lysaker, M. (eds.) EMMCVPR 2015. Lecture Notes in Computer Science, vol. 8932, pp. 197–208. Springer, Berlin (2015). doi:10.1007/978-3-319-14612-6_15 Google Scholar
  19. 19.
    Elsey, M., Wirth, B.: Fast automated detection of crystal distortion and crystal defects in polycrystal images. Multiscale Model. Simul. 12(1), 1–24. doi:10.1137/130916515
  20. 20.
    Esedoglu, S., Otto, F.: Threshold dynamics for networks with arbitrary surface tensions. Commun. Pure Appl. Math. 68(5), 808–864 (2015). doi:10.1002/cpa.21527 MathSciNetCrossRefMATHGoogle Scholar
  21. 21.
    Esedoglu, S., Tsai, Y.H.R.: Threshold dynamics for the piecewise constant Mumford–Shah functional. J. Comput. Phys. 211, 367–384 (2006). doi:10.1016/ MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Estellers, V., Zosso, D., Bresson, X., Thiran, J.P.: Harmonic active contours. IEEE Trans. Image Process. 23(1), 69–82 (2014). doi:10.1109/TIP.2013.2286326 MathSciNetCrossRefGoogle Scholar
  23. 23.
    Fauchereau, N., Pegram, G.G.S., Sinclair, S.: Empirical mode decomposition on the sphere: application to the spatial scales of surface temperature variations. Hydrol. Earth Syst. Sci. 12(3), 933–941 (2008). doi:10.5194/hess-12-933-2008 CrossRefGoogle Scholar
  24. 24.
    Flandrin, P., Gonçalvès, P., Rilling, G.: EMD equivalent filter banks, from interpretation to applications. In: Hilbert–Huang Transform and Its Applications, pp. 57–74 (2005)Google Scholar
  25. 25.
    Gabor, D.: Theory of communication. J. Inst. Electr. Eng. Part III Radio Commun. Eng. 93(26), 429–457 (1946)Google Scholar
  26. 26.
    Garcia-Cardona, C., Merkurjev, E., Bertozzi, A.L., Flenner, A., Percus, A.G.: Multiclass data segmentation using diffuse interface methods on graphs. IEEE Trans. Pattern Anal. Mach. Intell. 36(8), 1600–1613 (2014). doi:10.1109/TPAMI.2014.2300478 CrossRefMATHGoogle Scholar
  27. 27.
    Georgoulas, G., Loutas, T., Stylios, C.D., Kostopoulos, V.: Bearing fault detection based on hybrid ensemble detector and empirical mode decomposition. Mech. Syst. Signal Process. 41(1–2), 510–525 (2013). doi:10.1016/j.ymssp.2013.02.020 CrossRefGoogle Scholar
  28. 28.
    Gilles, J.: Multiscale texture separation. Multiscale Model. Simul. 10(4), 1409–1427 (2012). doi:10.1137/120881579 MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Gilles, J.: Empirical wavelet transform. IEEE Trans. Signal Process. 61(16), 3999–4010 (2013). doi:10.1109/TSP.2013.2265222 MathSciNetCrossRefGoogle Scholar
  30. 30.
    Gilles, J., Tran, G., Osher, S.: 2D empirical transforms. Wavelets, ridgelets, and curvelets revisited. SIAM J. Imaging Sci. 7(1), 157–186 (2014). doi:10.1137/130923774 MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Glowinski, R., Le Tallec, P.: Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (1989)CrossRefMATHGoogle Scholar
  32. 32.
    Goldstein, T., Osher, S.: The split Bregman method for L1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009). doi:10.1137/080725891 MathSciNetCrossRefMATHGoogle Scholar
  33. 33.
    Grafakos, L.: Classical Fourier Analysis, Graduate Texts in Mathematics, vol. 249. Springer, New York, NY (2009). doi:10.1007/978-0-387-09432-8
  34. 34.
    Guo, K., Labate, D.: Optimally sparse multidimensional representation using shearlets. SIAM J. Math. Anal. 39(1), 298–318 (2007). doi:10.1137/060649781 MathSciNetCrossRefMATHGoogle Scholar
  35. 35.
    Han, J., van der Baan, M.: Empirical mode decomposition for seismic time-frequency analysis. Geophysics 78(2), O9–O19 (2013). doi:10.1190/geo2012-0199.1 CrossRefGoogle Scholar
  36. 36.
    Han, P., Kurland, A.R., Giordano, A.N., Nanayakkara, S.U., Blake, M.M., Pochas, C.M., Weiss, P.S.: Heads and tails: simultaneous exposed and buried interface imaging of monolayers. ACS Nano 3(10), 3115–3121 (2009). doi:10.1021/nn901030x CrossRefGoogle Scholar
  37. 37.
    Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4(5), 303–320 (1969)MathSciNetCrossRefMATHGoogle Scholar
  38. 38.
    Hou, T.Y., Shi, Z.: Adaptive data analysis via sparse time-frequency representation. Adv. Adapt. Data Anal. 03(1 & 2), 1–28 (2011). doi:10.1142/S1793536911000647 MathSciNetCrossRefMATHGoogle Scholar
  39. 39.
    Hou, T.Y., Shi, Z.: Data-driven time frequency analysis. Appl. Comput. Harmonic Anal. 35(2), 284–308 (2013). doi:10.1016/j.acha.2012.10.001 MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Hou, T.Y., Shi, Z.: Sparse time-frequency decomposition for multiple signals with same frequencies (2015). arXiv:1507.02037
  41. 41.
    Hu, M., Liang, H.: Intrinsic mode entropy based on multivariate empirical mode decomposition and its application to neural data analysis. Cogn. Neurodyn. 5(3), 277–284 (2011). doi:10.1007/s11571-011-9159-8 CrossRefGoogle Scholar
  42. 42.
    Hu, H., Sunu, J., Bertozzi, A.L.: Multi-class graph Mumford–Shah model for plume detection using the MBO scheme. In: Tai, X.C., Bae, E., Chan, T.F., Lysaker, M. (eds.) Energy Minimization Methods in Computer Vision and Pattern Recognition, Lecture Notes in Computer Science, vol. 8932, pp. 209–222. Springer, Berlin (2015). doi:10.1007/978-3-319-14612-6_16
  43. 43.
    Huang, N.E., Shen, Z., Long, S.R., Wu, M.C., Shih, H.H., Zheng, Q., Yen, N.C., Tung, C.C., Liu, H.H.: The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proc. R. Soc. A Math. Phys. Eng. Sci. 454(1971), 903–995 (1998). doi:10.1098/rspa.1998.0193 MathSciNetCrossRefMATHGoogle Scholar
  44. 44.
    Labate, D., Lim, W.Q., Kutyniok, G., Weiss, G.: Sparse multidimensional representation using shearlets. In: Papadakis, M., Laine, A.F., Unser, M.A. (eds.) Optics and Photonics 2005, pp. 1–9. International Society for Optics and Photonics, Bellingham (2005). doi:10.1117/12.613494 Google Scholar
  45. 45.
    Lee, T.S.: Image representation using 2D Gabor wavelets. IEEE Trans. Pattern Anal. Mach. Intell. 18(10), 959–971 (1996). doi:10.1109/34.541406
  46. 46.
    Leo, M., Piccolo, R., Distante, C., Memmolo, P., Paturzo, M., Ferraro, P.: Multilevel bidimensional empirical mode decomposition: a new speckle reduction method in digital holography. Opt. Eng. 53(11), 112,314 (2014). doi:10.1117/1.OE.53.11.112314 CrossRefGoogle Scholar
  47. 47.
    Mallat, S.: A theory for multiresolution signal decomposition: the wavelet representation. IEEE Trans. Pattern Anal. Mach. Intell. 11(7), 674–693 (1989). doi:10.1109/34.192463 CrossRefMATHGoogle Scholar
  48. 48.
    Meignen, S., Perrier, V.: A new formulation for empirical mode decomposition based on constrained optimization. IEEE Signal Process. Lett. 14(12), 932–935 (2007). doi:10.1109/LSP.2007.904706 CrossRefGoogle Scholar
  49. 49.
    Merkurjev, E., Garcia-Cardona, C., Bertozzi, A.L., Flenner, A., Percus, A.G.: Diffuse interface methods for multiclass segmentation of high-dimensional data. Appl. Math. Lett. 33, 29–34 (2014). doi:10.1016/j.aml.2014.02.008 MathSciNetCrossRefMATHGoogle Scholar
  50. 50.
    Merriman, B., Bence, J.K., Osher, S.J.: Motion of multiple junctions: a level set approach. J. Comput. Phys. 112(2), 334–363 (1994). doi:10.1006/jcph.1994.1105 MathSciNetCrossRefMATHGoogle Scholar
  51. 51.
    Moore, A.M., Yeganeh, S., Yao, Y., Claridge, S.A., Tour, J.M., Ratner, M.A., Weiss, P.S.: Polarizabilities of adsorbed and assembled molecules: measuring the conductance through buried contacts. ACS Nano 4(12), 7630–7636 (2010). doi:10.1021/nn102371z CrossRefGoogle Scholar
  52. 52.
    Nocedal, J., Wright, S.J.: Numerical Optimization, 2nd edn. Springer, Berlin (2006)MATHGoogle Scholar
  53. 53.
    Nunes, J., Bouaoune, Y., Delechelle, E., Niang, O., Bunel, P.: Image analysis by bidimensional empirical mode decomposition. Image Vis. Comput. 21(12), 1019–1026 (2003). doi:10.1016/S0262-8856(03)00094-5 CrossRefMATHGoogle Scholar
  54. 54.
    Rilling, G., Flandrin, P.: One or two frequencies? The empirical mode decomposition answers. IEEE Trans. Signal Process. 56(1), 85–95 (2008). doi:10.1109/TSP.2007.906771 MathSciNetCrossRefGoogle Scholar
  55. 55.
    Rilling, G., Flandrin, P.: Sampling effects on the empirical mode decomposition. Adv. Adapt. Data Anal. 01(01), 43–59 (2009). doi:10.1142/S1793536909000023 MathSciNetCrossRefGoogle Scholar
  56. 56.
    Rilling, G., Flandrin, P., Gonçalvès, P.: On empirical mode decomposition and its algorithms. In: IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing (2003)Google Scholar
  57. 57.
    Rockafellar, R.T.: A dual approach to solving nonlinear programming problems by unconstrained optimization. Math. Program. 5(1), 354–373 (1973). doi:10.1007/BF01580138 MathSciNetCrossRefMATHGoogle Scholar
  58. 58.
    Ruuth, S.: Efficient algorithms for diffusion-generated motion by mean curvature. J. Comput. Phys. 625, 603–625 (1998). doi:10.1006/jcph.1998.6025 MathSciNetCrossRefMATHGoogle Scholar
  59. 59.
    Schmitt, J., Pustelnik, N., Borgnat, P., Flandrin, P., Condat, L.: 2-D Prony-Huang Transform: A New Tool for 2-D Spectral Analysis, p. 24 (2014). arxiv:1404.7680
  60. 60.
    Schmitt, J., Pustelnik, N., Borgnat, P., Flandrin, P., Condat, L.: 2D Prony–Huang transform: a new tool for 2D spectral analysis. IEEE Trans. Image Process. 23(12), 5233–5248 (2014). doi:10.1109/TIP.2014.2363000 MathSciNetCrossRefGoogle Scholar
  61. 61.
    Sharpley, R.C., Vatchev, V.: Analysis of the intrinsic mode functions. Constr. Approx. 24(1), 17–47 (2005). doi:10.1007/s00365-005-0603-z MathSciNetCrossRefMATHGoogle Scholar
  62. 62.
    Strikwerda, J.C.: Finite Difference Schemes and Partial Differential Equations, 2nd edn. SIAM, Philadelphia (2004)CrossRefMATHGoogle Scholar
  63. 63.
    Sykes, E.C.H., Mantooth, B.A., Han, P., Donhauser, Z.J., Weiss, P.S.: Substrate-mediated intermolecular interactions: a quantitative single molecule analysis. J. Am. Chem. Soc. 127(19), 7255–7260 (2005). doi:10.1021/ja0472331 CrossRefGoogle Scholar
  64. 64.
    Szuts, Z.B., Blundell, J.R., Chidichimo, M.P., Marotzke, J.: A vertical-mode decomposition to investigate low-frequency internal motion across the Atlantic at \(26^{\circ }\) N. Ocean Sci. 8(3), 345–367 (2012). doi:10.5194/os-8-345-2012 CrossRefGoogle Scholar
  65. 65.
    Tang, J., Zhao, L., Yue, H., Yu, W., Chai, T.: Vibration analysis based on empirical mode decomposition and partial least square. Proc. Eng. 16, 646–652 (2011). doi:10.1016/j.proeng.2011.08.1136 CrossRefGoogle Scholar
  66. 66.
    Tavallali, P., Hou, T.Y., Shi, Z.: Extraction of intrawave signals using the sparse time-frequency representation method. Multiscale Model. Simul. 12(4), 1458–1493 (2014). doi:10.1137/140957767 MathSciNetCrossRefMATHGoogle Scholar
  67. 67.
    Thomas, J.C., Schwartz, J.J., Hohman, J.N., Claridge, S.A., Auluck, H.S., Serino, A.C., Spokoyny, A.M., Tran, G., Kelly, K.F., Mirkin, C.A., Gilles, J., Osher, S.J., Weiss, P.S.: Defect-tolerant aligned dipoles within two-dimensional plastic lattices. ACS Nano 9(5), 4734–4742 (2015). doi:10.1021/acsnano.5b01329 CrossRefGoogle Scholar
  68. 68.
    Thomas, J.C., Goronzy, D.P., Dragomiretskiy, K., Zosso, D., Gilles, J., Osher, S.J., Bertozzi, A.L., Weiss, P.S.: Mapping buried hydrogen-bonding networks. ACS Nano 10(5), 5446–5451 (2016). doi:10.1021/acsnano.6b01717 CrossRefGoogle Scholar
  69. 69.
    Unser, M., Sage, D., Van De Ville, D.: Multiresolution monogenic signal analysis using the Riesz–Laplace wavelet transform. IEEE Trans. Image Process. 18(11), 2402–2418 (2009). doi:10.1109/TIP.2009.2027628 MathSciNetCrossRefGoogle Scholar
  70. 70.
    Unser, M., Chenouard, N., Van De Ville, D.: Steerable pyramids and tight wavelet frames in \(L_{2}(\mathbb{R}^d)\). IEEE Trans. Image Process. 20(10), 2705–2721 (2011). doi:10.1109/TIP.2011.2138147 MathSciNetCrossRefGoogle Scholar
  71. 71.
    Wu, Z., Huang, N.E.: Ensemble empirical mode decomposition: a noise-assisted data analysis method. Adv. Adapt. Data Anal. 01(01), 1–41 (2009). doi:10.1142/S1793536909000047 CrossRefGoogle Scholar
  72. 72.
    Wu, H.T., Flandrin, P., Daubechies, I.: One or two frequencies? The synchrosqueezing answers. Adv. Adapt. Data Anal. 03(01n02), 29–39 (2011). doi:10.1142/S179353691100074X MathSciNetCrossRefMATHGoogle Scholar
  73. 73.
    Yan, M.: Restoration of images corrupted by impulse noise and mixed Gaussian impulse noise using blind inpainting. SIAM J. Imaging Sci. 6(3), 1227–1245 (2013). doi:10.1137/12087178X MathSciNetCrossRefMATHGoogle Scholar
  74. 74.
    Yang, H., Ying, L.: Synchrosqueezed wave packet transform for 2D mode decomposition. SIAM J. Imaging Sci. 6(4), 1979–2009 (2013). doi:10.1137/120891113 MathSciNetCrossRefMATHGoogle Scholar
  75. 75.
    Yang, H., Lu, J., Ying, L.: Crystal image analysis using 2D synchrosqueezed transforms, p. 27 (2014). arxiv:1402.1262
  76. 76.
    Yugay, D., Goronzy, D.P., Kawakami, L.M., Claridge, S.A., Song, T.B., Yan, Z., Xie, Y.H., Gilles, J., Yang, Y., Weiss, P.S.: Copper ion binding site in \(\beta \)-amyloid peptide. Nano Lett. 16(10), 6282–6289 (2016). doi:10.1021/acs.nanolett.6b02590 CrossRefGoogle Scholar
  77. 77.
    Zhu, M., Wright, S.J., Chan, T.F.: Duality-based algorithms for total-variation-regularized image restoration. Comput. Optim. Appl. 47(3), 377–400 (2010). doi:10.1007/s10589-008-9225-2 MathSciNetCrossRefMATHGoogle Scholar
  78. 78.
    Zosso, D., An, J., Stevick, J., Takaki, N., Weiss, M., Slaughter, L.S., Cao, H.H., Weiss, P.S., Bertozzi, A.L.: Image segmentation with dynamic artifacts detection and bias correction. AIMS Journal of Inverse Problems Imaging, p. 24 (2017) (accepted)Google Scholar

Copyright information

© Springer Science+Business Media New York 2017

Authors and Affiliations

  1. 1.Department of MathematicsUniversity of California, Los Angeles (UCLA)Los AngelesUSA
  2. 2.Department of Mathematical SciencesMontana State UniversityBozemanUSA
  3. 3.California NanoSystems Institute (CNSI), Department of Chemistry and Biochemistry, Department of Materials Science and EngineeringUniversity of California, Los Angeles (UCLA)Los AngelesUSA

Personalised recommendations