A Study on Convex Optimization Approaches to Image Fusion

  • Jing Yuan
  • Juan Shi
  • Xue-Cheng Tai
  • Yuri Boykov
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6667)


Image fusion is an imaging technique to visualize information from multiple images in one single image, which is widely used in remote sensing, medical imaging etc. In this work, we study two variational approaches to image fusion which are closely related to the standard TV-L 2 and TV-L 1 image approximation methods. We investigate their convex optimization models under the perspective of primal and dual and propose the associated new image decompositions. In addition, we consider the TV-L 1 based image fusion approach and study the problem of fusing two discrete-constrained images \(f_1(x) \in \mathcal{L}_1\) and \(f_2(x) \in \mathcal{L}_2\), where \(\mathcal{L}_1\) and \(\mathcal{L}_2\) are the sets of linearly-ordered discrete values. We prove that the TV-L 1 based image fusion actually gives rise to an exact convex relaxation to the corresponding nonconvex image fusion given the discrete-valued constraint \(u(x) \in \mathcal{L}_1 \cup \mathcal{L}_2\). This extends the results for the global optimization of the discrete-constrained TV-L 1 image approximation [7,30] to the case of image fusion. The proposed dual models also lead to new fast and reliable algorithms in numerics, based on modern convex optimization techniques. Experiments of medical imaging, remote sensing and multi-focusing visibly show the qualitive differences between the two studied variational models of image fusion.


Image Fusion Convex Optimization Convex Relaxation Image Decomposition Image Approximation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aiazzi, B., Alparone, L., Baronti, S., Garzelli, A.: Context-driven fusion of high spatial and spectral resolution images based on oversampled multiresolution analysis. IEEE Geoscience and Remote Sensing 40(10), 2300–2312 (2002)CrossRefGoogle Scholar
  2. 2.
    Aujol, J.-F., Gilboa, G., Chan, T.F., Osher, S.: Structure-texture image decomposition - modeling, algorithms, and parameter selection. International Journal of Computer Vision 67(1), 111–136 (2006)CrossRefzbMATHGoogle Scholar
  3. 3.
    Bae, E., Yuan, J., Tai, X.C., Boykov, Y.: A fast continuous max-flow approach to non-convex multilabeling problems. Technical Report CAM10-62, UCLA (2010)Google Scholar
  4. 4.
    Bertsekas, D.P.: Constrained optimization and Lagrange multiplier methods. Academic Press Inc., New York (1982)zbMATHGoogle Scholar
  5. 5.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE PAMI 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  6. 6.
    Chambolle, A.: An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 20(1-2), 89–97 (2004)zbMATHMathSciNetGoogle Scholar
  7. 7.
    Chan, T.F., Esedoḡlu, S.: Aspects of total variation regularized L1 function approximation. SIAM J. Appl. Math. 65(5), 1817–1837 (2005) (electronic)CrossRefzbMATHMathSciNetGoogle Scholar
  8. 8.
    Chan, T.F., Esedoglu, S., Nikolova, M.: Algorithms for finding global minimizers of image segmentation and denoising models. SIAM J. Appl. Math. 66(5), 1632–1648 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  9. 9.
    Chan, T.F., Vese, L.A.: Active contours without edges. IEEE Transactions on Image Processing 10(2), 266–277 (2001)CrossRefzbMATHGoogle Scholar
  10. 10.
    Das, A., Revathy, K.: A comparative analysis of image fusion techniques for remote sensed images. In: World Congress on Engineering, pp. 639–644 (2007)Google Scholar
  11. 11.
    Ekeland, I., Téman, R.: Convex analysis and variational problems. Society for Industrial and Applied Mathematics, Philadelphia (1999)CrossRefGoogle Scholar
  12. 12.
    Fan, K.: Minimax theorems. Proc. Nat. Acad. Sci. U.S.A. 39, 42–47 (1953)CrossRefzbMATHMathSciNetGoogle Scholar
  13. 13.
    Giusti, E.: Minimal surfaces and functions of bounded variation. Australian National University, Canberra (1977)zbMATHGoogle Scholar
  14. 14.
    Ishikawa, H.: Exact optimization for markov random fields with convex priors. IEEE PAMI 25, 1333–1336 (2003)CrossRefGoogle Scholar
  15. 15.
    Kluckner, S., Pock, T., Bischof, H.: Exploiting redundancy for aerial image fusion using convex optimization. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K. (eds.) DAGM 2010. LNCS, vol. 6376, pp. 303–312. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  16. 16.
    Li, H., Manjunath, B.S., Mitra, S.K.: Multisensor image fusion using the wavelet transform. Graphical Models and Image Processing 57(3), 235–245 (1995)CrossRefGoogle Scholar
  17. 17.
    Meyer, Y.: Oscillating patterns in image processing and nonlinear evolution equations. University Lecture Series, vol. 22. American Mathematical Society, Providence (2001); The fifteenth Dean Jacqueline B. Lewis memorial lectureszbMATHGoogle Scholar
  18. 18.
    Nez, J., Otazu, X., Fors, O., Prades, A., Pal‘a, V., Arbiol, R.: Multiresolution-based image fusion with additive wavelet decomposition. IEEE Trans. On Geoscience And Remote Sensing 37(3), 1204–1211 (1999)CrossRefGoogle Scholar
  19. 19.
    Pajares, G., de la Cruz, J.M.: A wavelet-based image fusion tutorial. Pattern Recognition 37(9), 1855–1872 (2004)CrossRefGoogle Scholar
  20. 20.
    Piella, G.: Image fusion for enhanced visualization: A variational approach. International Journal of Computer Vision 83(1), 1–11 (2009)CrossRefGoogle Scholar
  21. 21.
    Rockafellar, R.T.: Augmented Lagrangians and applications of the proximal point algorithm in convex programming. Math. of Oper. Res. 1, 97–116 (1976)CrossRefzbMATHMathSciNetGoogle Scholar
  22. 22.
    Rudin, L., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60(1-4), 259–268 (1992)CrossRefzbMATHMathSciNetGoogle Scholar
  23. 23.
    Schowengerdt, R.A.: Remote Sensing: Models and Methods for Image Processing, 3rd edn. Elsevier, Amsterdam (2007)Google Scholar
  24. 24.
    Sohn, M.-J., Lee, D.-J., Yoon, S.W., Lee, H.R., Hwang, Y.J.: The effective application of segmental image fusion in spinal radiosurgery for improved targeting of spinal tumours. Acta Neurochir 151, 231–238 (2009)CrossRefGoogle Scholar
  25. 25.
    Wang, W.-W., Shui, P.-L., Feng, X.-C.: Variational models for fusion and denoising of multifocus images. IEEE Signal Processing Letters 15, 65–68 (2008)CrossRefGoogle Scholar
  26. 26.
    Wang, Z., Ziou, D., Armenakis, C., Li, D., Li, Q.: A comparative analysis of image fusion methods. IEEE Geo. and Res. 43(6), 1391–1402Google Scholar
  27. 27.
    Yang, L., Guo, B.L., Ni, W.: Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72, 203–211 (2008)CrossRefGoogle Scholar
  28. 28.
    Yuan, J., Bae, E., Tai, X.-C., Boykov, Y.: A continuous max-flow approach to potts model. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6316, pp. 379–392. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  29. 29.
    Yuan, J., Bae, E., Tai, X.-C.: A study on continuous max-flow and min-cut approaches. In: CVPR 2010, pp. 2217–2224 (2010)Google Scholar
  30. 30.
    Yuan, J., Shi, J., Tai, X.-C.: A convex and exact approach to discrete constrained tv-l1 image approximation. Technical Report CAM-10-51, UCLA (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Jing Yuan
    • 1
  • Juan Shi
    • 2
  • Xue-Cheng Tai
    • 2
    • 3
  • Yuri Boykov
    • 1
  1. 1.Computer Science DepartmentUniversity of Western OntarionLondonCanada
  2. 2.Division of Mathematical Sciences, School of Phys. and Math. Sci.Nanyang Technological UniversitySingapore
  3. 3.Department of MathematicsUniversity of BergenNorway

Personalised recommendations