Advertisement

Structural Similarity-Based Optimization Problems with \(L^1\)-Regularization: Smoothing Using Mollifiers

  • Daniel OteroEmail author
  • Davide La Torre
  • Edward R. Vrscay
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9164)

Abstract

In this paper we propose a new method of solving optimization problems involving the structural similarity image quality measure with \(L^1\)-regularization. The regularization term \(\Vert x \Vert _1\) is approximated by a sequence of smooth functions \(\Vert x \Vert _1^\varepsilon \) by means of \(C^\infty _0\) functions known as mollifiers. Because the functions \(\Vert x \Vert _1^\varepsilon \) epi-converge to \(\Vert x \Vert _1\), the sequence of minimizers of the smooth objective functions converges to a minimizer of the non-smooth problem. This approach permits the use of gradient-based methods to solve the minimization problems as opposed to methods based on subdifferentials.

Keywords

Discrete Cosine Transform Multivariate Gaussian Distribution Discrete Cosine Transform Coefficient Structural Similarity Index Measure Fidelity Term 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

We gratefully acknowledge that this research has been supported in part by the Natural Sciences and Engineering Research Council of Canada (NSERC) in the form of a Discovery Grant (ERV).

References

  1. 1.
    Beck, A.A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. Arch. 2(1), 183–202 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  2. 2.
    Brezis, H.: Analyse Fonctionelle Theorie et Applications. Masson editeur, Paris (1963) Google Scholar
  3. 3.
    Bruckstein, A., Donoho, D., Elad, M.: From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM Rev. 51(1), 34–81 (2009)zbMATHMathSciNetCrossRefGoogle Scholar
  4. 4.
    Brunet, D.: A Study of the Structural Similarity Image Quality Measure with Applications to Image Processing, Ph.D. thesis, University of Waterloo (2010)Google Scholar
  5. 5.
    Brunet, D., Vrscay, E.R., Wang, Z.: On the mathematical properties of the structural similarity index. IEEE Trans. Image Proc. 21(4), 1488–1499 (2012)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Brunet, D., Vrscay, E.R., Wang, Z.: Structural similarity-based approximation of signals and images using orthogonal bases. In: Campilho, A., Kamel, M. (eds.) ICIAR 2010. LNCS, vol. 6111, pp. 11–22. Springer, Heidelberg (2010) CrossRefGoogle Scholar
  7. 7.
    Channappayya, S.S., Bovik, A.C., Caramanis, C., Heath Jr., R.W.: Design of linear equalizers optimized for the structural similarity index. IEEE Trans. Image Process. 17(6), 857–872 (2008)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Crespi, G.P., La Torre, D., Rocca, M.: Second-order mollified derivatives and optimization. Rendiconti del Circolo Matematico di Palermo 52(2), 251–262 (2003)zbMATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    Crespi, G.P., La Torre, D., Rocca, M.: Mollified derivatives and second-order optimality conditions. J. Nonlinear Convex Anal. 4(3), 437–454 (2003)zbMATHMathSciNetGoogle Scholar
  10. 10.
    Donoho, D., Elad, M.: Optimality sparse representation in general (non-orthogonal) dictionaries via \(L^1\)-minimization. Proc. Nat. Acad. Sci. 100, 2197–2202 (2003)zbMATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    Donoho, D.: Denoising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)zbMATHMathSciNetCrossRefGoogle Scholar
  12. 12.
    Ermoliev, Y.M., Norkin, V.I., Wets, R.J.B.: The minimization of semicontinuous functions: mollifier subgradients. SIAM J. Control Optim. 33, 149–167 (1995)zbMATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    Jongen, H.T., Stein, O.: Smoothing by mollifiers. Part I: semi-infinite optimization. J. Global Optim. 41(3), 319–334 (2008)zbMATHMathSciNetCrossRefGoogle Scholar
  14. 14.
    Jongen, H.T., Stein, O.: Smoothing by mollifiers. Part II: nonlinear optimization. J. Global Optim. 41(3), 335–350 (2008)zbMATHMathSciNetCrossRefGoogle Scholar
  15. 15.
    La Torre, D., Rocca, M.: Remarks on second order generalized derivatives for differentiable functions with Lipschitzian Jacobian. Appl. Math. E - Notes 3, 130–137 (2003)zbMATHMathSciNetGoogle Scholar
  16. 16.
    Mairal, J., Bach, F., Jenatton, R., Obozinski, G.: Convex Optimization with Sparsity-Inducing Norms. Optimization for Machine Learning. MIT Press, Cambridge (2011) Google Scholar
  17. 17.
    Otero, D., Vrscay, E.R.: Unconstrained structural similarity-based optimization. In: Campilho, A., Kamel, M. (eds.) ICIAR 2014, Part I. LNCS, vol. 8814, pp. 167–176. Springer, Heidelberg (2014) Google Scholar
  18. 18.
    Otero, D., Vrscay, E.R.: Solving problems that employ structural similarity as the fidelity measure. In: Proceedings of the Internartional Conference on Image Processing, Computer Vision and Patter Recognition IPCV 2014, pp. 474–479. Springer, Heidelberg (2014)Google Scholar
  19. 19.
    Rehman, A., Rostami, M., Wang, Z., Brunet, D., Vrscay, E.R.: SSIM-inspired image restoration using sparse representation. EURASIP J. Adv. Sig. Proc. (2012)Google Scholar
  20. 20.
    Rehman, A., Gao, Y., Wang, J., Wang, Z.: Image classification based on complex wavelet structural similarity. Sig. Proc. Image Comm. 28(8), 984–992 (2013)CrossRefGoogle Scholar
  21. 21.
    Richter, T., Kim. K.J.: A MS-SSIM optimal JPEG 2000 encoder. In: Data Compression Conference, pp. 401–410, Snowbird, Utah, March 2009Google Scholar
  22. 22.
    Rockafellar, R.T., Wets, R.J.-B.: Variational Analysis. Springer Verlag, Berlin (1998)zbMATHCrossRefGoogle Scholar
  23. 23.
    Turlach, B.A.: On algorithms for solving least squares problems under an \(L^1\) penalty or an \(L^1\) constraint. In: Proceedings of the American Statistical Association, Statistical Computing Section, pp. 2572–2577 (2005)Google Scholar
  24. 24.
    Voronin, S., Yoshida, D.: Gradient Based Methods for Non-Smooth Regularization Via Convolution Smoothing http://arxiv.org/pdf/1408.6795.pdf
  25. 25.
    Wainwright, M.J., Schwartz, O., Simoncelli, E.P.: Natural image statistics and divisive normalization: modeling nonlinearity and adaptation in cortical neurons. In: Rao, R., Olshausen, B., Lewicki, M. (eds.) Probabilistic Models of the Brain: Perception and Neural Function, pp. 203–222. MIT Press, Cambridge (2002)Google Scholar
  26. 26.
    Wandell, B.A.: Foundations of Vision. Sinauer Publishers, Sunderland (1995)Google Scholar
  27. 27.
    Wang, Z., Bovik, A.C.: Mean squared error: love it or leave it? a new look at signal fidelity measures. IEEE Signal Proc. Mag. 26(1), 98–117 (2009)CrossRefGoogle Scholar
  28. 28.
    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Proc. 13(4), 600–612 (2004)CrossRefGoogle Scholar
  29. 29.
    Wang, Z., Bovik, A.C.: A universal image quality index. IEEE Signal Process. Lett. 9(3), 81–84 (2002)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Daniel Otero
    • 1
    Email author
  • Davide La Torre
    • 2
    • 3
  • Edward R. Vrscay
    • 1
  1. 1.Department of Applied Mathematics, Faculty of MathematicsUniversity of WaterlooWaterlooCanada
  2. 2.Department of Economics, Management, and Quantitative MethodsUniversity of MilanMilanItaly
  3. 3.Department of Applied Mathematics and SciencesKhalifa UniversityAbu DhabiUAE

Personalised recommendations