Pansharpening Image Fusion Using Cross-Channel Correlation: A Framelet-Based Approach



This paper aims at developing a variational model for pansharpening image fusion. To resolve the ill-posedness of image fusion, we propose a new regularization technique that explores the cross-channel correlation of different spectral channels in wavelet tight frame (or framelet) domain. Besides using a regular cross-channel sparsity prior (Inverse Probl Imaging 7(3):777–794, 2013), the proposed model also makes efficient use of the panchromatic image as a guidance for image feature alignment. An ADMM-based iterative scheme is derived for solving the proposed model, and its performance is tested on several datasets including natural images, aerial images, and real multispectral satellite images. Numerical results suggest that the proposed approach works well on the testing datasets and outperforms some state-of-the-art algorithms in comparison.


Pansharpening image fusion Multispectral Cross-channel correlation Wavelet frames ADMM 



L. Hou and X. Zhang would like to thank the anonymous reviewers for their helpful comments in improving the presentation of this paper. They also would like to thank the authors of [11, 12] for making their resources available online for free academic use. This work is partially supported by NSFC91330102 and NSFC (GZ1025), China Postdoc Science Foundation (# 2014M551392), 973 Program (# 2015CB856000).


  1. 1.
    Amro, I., Mateos, J., Vega, M., Molina, R., Katsaggelos, A.K.: A survey of classical methods and new trends in pansharpening of multispectral images. EURASIP J. Adv. Signal Process. 2011, 79 (2011)CrossRefGoogle Scholar
  2. 2.
    Ballester, C., Caselles, V., Igual, L., Verdera, J.: A variational model for P+XS image fusion. Int. J. Comput. Vis. 69, 43–58 (2006)CrossRefGoogle Scholar
  3. 3.
    Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–124 (2011)CrossRefMATHGoogle Scholar
  4. 4.
    Cai, J., Chan, R., Shen, Z.: A framelet-based inpainting algorithm. Appl. Comput. Harmon. Anal. 24, 131–149 (2008)MathSciNetCrossRefMATHGoogle Scholar
  5. 5.
    Cai, J., Osher, S., Shen, Z.: Split Bregman methods and frame based image restoration. Multiscale Model. Simul. 8(2), 337–369 (2009)MathSciNetCrossRefMATHGoogle Scholar
  6. 6.
    Cai, J., Ji, H., Liu, C., Shen, Z.: Framelet based blind image deblurring from a single image. IEEE Trans. Image Process. 21(2), 562–572 (2012)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Chan, R.H., Chan, T.F., Shen, L., Shen, Z.: Wavelet algorithms for high-resolution image reconstrction. SIAM J. Sci. Comput. 24(4), 1408–1432 (2003)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Daubechies, I., Han, B., Ron, A., Shen, Z.: Framelets: MRA-based constructions of wavelet frames. Appl. Comput. Harmon. Anal. 14, 1–46 (2003)MathSciNetCrossRefMATHGoogle Scholar
  9. 9.
  10. 10.
    Dong, B., Shen, Z.: MRA-based Wavelet Frames and Applications. IAS/Park City Mathematics Series, vol. 19. American Mathematical Society, Providence (2010)Google Scholar
  11. 11.
    Duran, J., Buades, A., Coll, B., Sbert, C.: Implementation of nonlocal pansharpening image fusion. Image Process. Line (IPOL) 4, 1–15 (2014a)CrossRefMATHGoogle Scholar
  12. 12.
    Duran, J., Buades, A., Coll, B., Sbert, C.: A nonlocal variational model for pansharpening image fusion. SIAM J. Imaging Sci. 7(2), 761–796 (2014b)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Goldstein, T., Osher, S.: The split Bregman iteration for \(L\)1-regularized problems. SIAM J. Imaging Sci. 2(2), 323–343 (2009)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Gong, Z., Shen, Z., Toh, K.C.: Image restoration with mixed or unknown noises. Multiscale Model. Simul. 12(2), 458–487 (2014)Google Scholar
  15. 15.
    Haydan, R., Dalke, G.W., Henkel, J., Bare, J.E.: Applications of the IHS color transform to the processing of multisensor data and image enhancement. In: Proceedings of the 1st international symposium on remote sensing environment, Cairo, Egypt, pp. 599–616 (1982)Google Scholar
  16. 16.
    Hestenes, M.R.: Multiplier and gradient methods. J. Optim. Theory Appl. 4, 303–320 (1969)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Hamilton Jr, J., Adams Jr J.: Adaptive color plan interpolation in single sensor color electronic camera. U.S. Patent 5, 629–734 (1997)Google Scholar
  18. 18.
    Kakarala, R., Baharav, Z.: Adaptive demosaicing with the principal vector method. IEEE Trans. Consum. Electron. 48(4), 932–937 (2002)CrossRefGoogle Scholar
  19. 19.
    Liang, J., Li, J., Shen, Z., Zhang, X.: Wavelet frame based color image demosaicing. Inverse Probl Imaging 7(3), 777–794 (2013)MathSciNetCrossRefMATHGoogle Scholar
  20. 20.
    Mallat, S.: A Wavelet Tour of Signal Processing, 2nd edn. Academic Press Incorporation, San Diego (1998)MATHGoogle Scholar
  21. 21.
    Möller, M., Wittman, T., Bertozzi, A.L., Burger, M.: A variational approach for sharpening high dimensional images. SIAM J. Imaging Sci. 5(1), 150–178 (2012)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Pohl, Christine, van Genderen, John: Structuring contemporary remote sensing image fusion. Int. J. Image Data Fusion 6(1), 3–21 (2015)CrossRefGoogle Scholar
  23. 23.
    Powell, M.J.D.: A method for nonlinear constraints in minimization problems. In: Fletcher, R. (ed.) Optimization, pp. 283–298. Academic Press, New York (1969)Google Scholar
  24. 24.
    Ron, A., Shen, Z.: Frames and stable bases for shift-invariant subspaces of \(L_2(R^d)\). Can. J. Math. 47, 1051–1094 (1995)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Ron, A., Shen, Z.: Affine systems in \(L_2(R^d)\): the analysis of the analysis operator. J. Funct. Anal. 148, 408–447 (1997)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
  27. 27.
    Tai, C., Zhang, X., Shen, Z.: Wavelet frame based multi-phase image segmentation. SIAM J. Imaging Sci. 6, 2521–2546 (2013)MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Thomas, C., Ranchin, T., Wald, L., Chanussot, J.: Synthesis of multispectral images to high spatial resolution: a critical review of fusion methods based on remote sensing physics. IEEE Trans. Geosci. Remote Sens. 46(5), 1301–1312 (2008)CrossRefGoogle Scholar
  29. 29.
    Tsai, C.Y., TaiSong, K.: Heterogeneity-projection hard-decision color interpolation using spectral-spatial correlation. IEEE Trans. Image Process. 16(1), 78–97 (2007)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Wu, C., Tai, X.C.: Augmented lagrangian method, dual methods, and split bregman iteration for rof, vectorial tv, and high order models. SIAM J. Imaging Sci. 3(3), 300–339 (2010)MathSciNetCrossRefMATHGoogle Scholar
  31. 31.
    Zhang, L., Wu, X.: Color demosaicking via directional linear minimum mean square-error estimation. IEEE Trans. Image Process. 14(12), 2167–2178 (2005)CrossRefGoogle Scholar
  32. 32.
    Zhang, L., Wu, X., Buades, A., Li, X.: Color demosaicking by local directional interpolation and non-local adaptive thresholding. J. Electron. Imaging 20(2), 023016 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Department of Mathematics & Institute of Natural SciencesShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations