Skip to main content
SpringerLink
Log in

An optimal weighted averaging fusion strategy for thermal and visible images using dual tree discrete wavelet transform and self tunning particle swarm optimization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image fusion plays a vital role in providing better visualization of image data. In this paper, we propose a new algorithm that optimally combines information from thermal images with a visual image of the same scene to create a single comprehensive fused image. In this work, an improved version of particle swarm optimization alogithm is proposed to optimally combine the thermal and visible images. The proposed algorithm is named self tunning particle swarm optimization (STPSO). Because of the importance of the fusion rule, a weighted averaging fusion rule is formulated that uses optimal weights resulting from STPSO for the fusion of both high frequency and low frequency coefficients obtained by applying Dual Tree Discrete Wavelet Transform (DT-DWT). The objective function in STPSO is formulated with the twin objectives of maximizing the Entropy and minimizing the Root Mean Square Error (RMSE), which differentiates our work from existing fusion techniques. The efficiency of our fusion algorithm is also evaluated by adding Gaussian white noise to the source images. The fusion results are compared with existing multi-resolution based fusion techniques, such as Laplacian Pyramid (LAP), Discrete Wavelet Transform (DWT) and Non Sub-Sampled Contourlet Transform (NSCT). The simulation results indicate that the proposed fusion framework results in better quality fused images when evaluated with subjective and objective metrics. Comparision of these results with those from PSO shows that our algorithm outperforms generic PSO.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. ALEjaily AM, El Rube IA, Mangoud MA (2008) Fusion of remote sensing images using contourlet transform. Innovations and advanced techniques in systems, computing sciences and software engineering 213–218.

  2. AnoopSuraj A, Francis M, Kavya TS, Nirmal TM (2014) Discrete wavelet transform based image fusion and de-noising in FPGA. J Electr Syst Inf Technol 1:72–81

    Google Scholar 

  3. Arivazhagan S, Ganesan L, Subash Kumar TG (2009) A modified statistical approach for image fusion using wavelet transform. SIViP 3(4):137–144

    Article  Google Scholar 

  4. Bebis G, Gyaourova A, Singh S, Pavlidis I (2006) Face recognition by fusing thermal infrared and visible imagery. Image Vision Comput J 24:727–742

    Article  Google Scholar 

  5. Bhateja V, Patel H, Krishn A, Sahu A (2015) Multimodal medical image sensor fusion framework using cascade of wavelet and contourlet transform domains. IEEE Sensors J 15(12):6783–6790

    Article  Google Scholar 

  6. Bhuvaneswari C, Aruna P (2014) A new fusion model for classification of the lung diseases using genetic algorithm. Egypt Inform J 15:69–77

    Article  Google Scholar 

  7. Bulanona DM, Burksa TF, Alchanatis V (2009) Image fusion of visible and thermal images for fruit detection. Biosyst Eng 103:12–22

    Article  Google Scholar 

  8. Chen H-Y, Leou J-J (2012) Multispectral and multi-resolution image fusion using particle swarm optimization. Multimed Tools Appl 60(3):495–518

    Article  Google Scholar 

  9. Daneshvar S, Ghassemian H (2010) MRI and PET image fusion by combining IHS and retina-inspired models. Inf Fusion 11:114–123

    Article  Google Scholar 

  10. Dey T (2013) A survey on different fusion techniques of visual and thermal images for human face recognition. Int J Electron Commun Comput Eng Vol-4

  11. Duan H, Luo Q, Ma G (2013) Hybrid particle swarm optimization and genetic algorithm for multi-UAV formation reconfiguration. IEEE Comput Intell Mag 8:16–27

    Article  Google Scholar 

  12. Ellmauthaler A, Pagliari CL, da Silva EAB (2013) Multi-scale image fusion using the undecimated wavelet transform with spectral factorization and Non-orthogonal filter banks. IEEE Trans Image Process 22(3):1005–1017

    Article  MathSciNet  Google Scholar 

  13. El-Maadi A, Gregoire V, St-Laurent L, Torresan H, Turgeon B, Prevost D, Hebert P, Laurendeau D, Ricard B, Maldague X (2007) Visible and infrared imagery for surveillance applications: software and hardware considerations. QIRT J 4:25–40

    Article  Google Scholar 

  14. Farokhi S, Shamsuddin SM, Sheikh UU, Flusser J, Khansari M, Jafari-Khouzani K (2014) Near infrared face recognition by combining Zernike moments and undecimated discrete wavelet transform. Digital Signal Process 31(2014):13–27

    Article  Google Scholar 

  15. Hongbin P, Sun D-W (2014) Using wavelet textural features of visible and near infrared hyper-spectral image to differentiate between fresh and frozen–thawed pork. Food Bioprocess Technol 7(11):3088–3099

    Article  Google Scholar 

  16. Jedrasiak K, Nawrat A, Daniec K, Koteras R, Mikulski M, Grzejszczak T (2012) A prototype device for concealed weapon detection using IR and CMOS cameras fast image fusion computer vision and graphics. Volume 7594 of the series Lecture Notes in Computer Science pp 423–432

  17. Ji X, Zhang G (2015) Image fusion method of SAR and infrared image based on Curvelet transform with adaptive weighting. Multimed Tools Appl 1–17

  18. Jian M, Dong J (2010) Capture and fusion of 3d surface texture. Multimed Tools Appl 53:237–251

    Article  Google Scholar 

  19. Jin H, Wang Y (2014) A fusion method for visible and infrared images based on contrast pyramid with teaching learning based optimization. Infrared Phys Technol 64:134–142

    Article  Google Scholar 

  20. Keller W (2004) Wavelets in geodesy and geodynamics. DeGruyter, Berlin

  21. Kingsbury N (2003) Design of Q-shift complex wavelets for image processing using frequency domain energy minimization. In: International Conference on Image Processing. 1013–1016

  22. Kludas, J (2010) Information fusion for multimedia: exploiting feature interactions for semantic feature selection and construction. Thesis

  23. Kludas J, Bruno E, Marchand-Maillet S (2008) Can feature information interaction help for information fusion in multimedia problems? Multimed Tools Appl 42:57–71

    Article  Google Scholar 

  24. Kong WW, Lei YJ, Lei Y, Zhang J (2010) Technique for image fusion based on non-subsampled contourlet transform domain improved NMF. Sci China Inf Sci 53(12):2429–2440

    Article  MathSciNet  MATH  Google Scholar 

  25. Lacewell CW, Gebril M, Buaba R, Homaifar A (2010) Optimization of image fusion using genetic algorithms and discrete wavelet transform. Aerospace and Electronics Conference (NAECON), Proceedings of the IEEE 2010 National. 116–121

  26. Li X, He M, Roux M (2010) Multifocus image fusion based on redundant wavelet transform. IET Image Process 4(5):283–293

    Article  Google Scholar 

  27. Li H, Manjunath BS, Mitra SK (1995) Multi-sensor image fusion using the wavelet transform. Graph Model Image Process 57:235–245

    Article  Google Scholar 

  28. Liu Z, Blasch E, Xue Z, Zhao J, Laganiere R, Wu W (2012) Objective assessment of multi-resolution image fusion algorithms for context enhancement in night vision: A comparative study. IEEE Trans Pattern Anal Mach Intell 34(1)

  29. Liu X, Mei W, Du H, Bei J (2015) A novel image fusion algorithm based on non sub-sampled shearlet transform and morphological component analysis. SIViP 1–8.

  30. Liu L, Wang D, Yang S (2008) Compound particle swarm optimization in dynamic environments” Evo workshops. LNCS 4974:617–626

    Google Scholar 

  31. Liu X, Zhou Y, Wang J (2014) Image fusion based on shearlet transform and regional features. Int J Electron Commun 68:471–477

    Article  Google Scholar 

  32. Luo B, Wang Y, Liu Y (2010) Sensor fusion based head pose trackingfor lightweight flight cockpit systems. Multimed Tools Appl 52:235–255

    Article  Google Scholar 

  33. Mahbubur Rahman SM, Omair Ahmad M, Swamy MNS (2010) Contrast-based fusion of noisy images using discrete wavelet transform. IET Image Process 4(5):374–384

    Article  MathSciNet  Google Scholar 

  34. Malviya P, Saxena A (2014) An improved Image Fusion Technique based on Texture Feature Optimization using Wavelet Transform and Particle of Swarm Optimization (POS). Int J Comput Appl 101(6)

  35. Mansoorizadeh M, Moghaddam Charkari N (2009) Multimodal information fusion application to human emotion recognition from face and speech. Multimed Tools Appl 49:277–297

    Article  Google Scholar 

  36. Miaindarg VC, Mane AP (2013) Decimated and Un-decimated wavelet transforms based image enhancement. Int J Ind Electr, Electron, Control Robot 3(5):26–30

    Google Scholar 

  37. Miao D, Sun Z, Huang Y (2014) Fusion of Multibiometrics based on a new robust linear programming. 22nd International Conference on Pattern Recognition

  38. Muhammad A, Bala I, Shukri Salman M, Eleyan A (2014) DWT Sub-bands fusion using ant colony optimization for edge detection. IEEE 22nd Signal Processing and Communications Applications Conference (SIU 2014). 1351–1354

  39. Nickabadi A, Ebadzadeh MM, Safabakhsh R (2011) A novel particle swarm optimization algorithm with adaptive inertia weight. Appl Soft Comput 11:3658–3670

    Article  Google Scholar 

  40. Ou F, Han Z, Liu C, Ou Z (2010) Face verification with feature fusion of Gabor based and curvelet based representations. Multimed Tools Appl 57:549–563

    Article  Google Scholar 

  41. Palsson F, Sveinsson JR, OrnUlfarsson M, Benediktsson JA (2015) Model-based fusion of multi-and hyperspectral images using PCA and wavelets. IEEE Trans Geosci Remote Sens 53(5):2652–2663

    Article  Google Scholar 

  42. Petrovic V (2007) Subjective tests for image fusion evaluation and objective metric validation. Inf Fusion 8(2):2018–216

    Google Scholar 

  43. Ratnaweera A, Halgamuge SK, Watson HC (2004) Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evol Comput 8(3):240–255

    Article  Google Scholar 

  44. Saeedi J, Faez K (2012) Infrared and visible image fusion using fuzzy logic and population-based optimization. Appl Soft Comput 12:1041–1054

    Article  Google Scholar 

  45. Shamsafar F, Seyedarabi H, Aghagolzadeh A (2013) Fusing the information in visible light and near-infrared images for iris recognition. Mach Vis Appl 25(4):881–899

    Article  Google Scholar 

  46. Sharma PK, Bhavya VS, Navyashree KM, Sunil KS, Pavithra P (2012) Artificial Bee colony and its application for image fusion. IJ Inf Technol Comput Sci 11:42–49

    Google Scholar 

  47. Shreyamsha Kumar BK (2013) Multifocus and multispectral image fusion based on pixel significance using discrete cosine harmonic wavelet transform. SIViP 7(6):1125–1143

    Article  Google Scholar 

  48. Singh R, Khare A, Singh R, Khare A (2014) Fusion of multimodal medical images using Daubechies complex wavelet transform – A multi-resolution approach. Inf Fusion 19:49–60

    Article  Google Scholar 

  49. Tan H, Huang X, Tan H, He C (2013) Pixel-level image fusion algorithm based on maximum likelihood and laplacian pyramid transformation. J Comput Inf Syst 9(1):327–334

    Google Scholar 

  50. Tao J, Li S, Yang B (2010) Multimodal image fusion algorithm using dual-tree complex wavelet transform and particle swarm optimization. Adv Intell Comput Theor Appl 93:296–303

    MATH  Google Scholar 

  51. Teo T-A, Lau C-C (2012) Pyramid-based image empirical mode decomposition for the fusion of multispectral and panchromatic images. EURASIP J Adv Sig Process 2012:4

    Article  Google Scholar 

  52. Toet A (1989) Image fusion by a ratio of low-pass pyramid. Pattern Recogn Lett 9:245–253

    Article  MATH  Google Scholar 

  53. Tong Y, Zhao M, Wei Z, Liu L (2014) Compressive sensing image-fusion algorithm in wireless sensor networks based on blended basis functions. EURASIP J Wirel Commun Netw 2014:150

    Article  Google Scholar 

  54. Uniyal N, Verma SK (2014) Image fusion using morphological pyramid consistency method. Int J Comput Appl 95(25):34–38

    Google Scholar 

  55. Wang H-H (2004) A new multiwavelet-based approach to image fusion. J Math Imaging Vision 21:177–192

    Article  MathSciNet  Google Scholar 

  56. Wang W, Chang F (2011) A multi-focus image fusion method based on laplacian pyramid. J Comput 6(12):2559–2566

    Article  Google Scholar 

  57. Wang S, Gao Z, He S, He M, Ji Q (2015) Gender recognition from visible and thermal Infrared facial images. Multimed Tools Appl 75:1–24

    Google Scholar 

  58. Wang S, He S, Wu Y, He M, Ji Q (2014) Fusion of visible and thermal images for facial expression recognition. Front Comput Sci 8(2):232–242

    Article  MathSciNet  Google Scholar 

  59. Yang Y, Tong S, Huang S, Pan L (2015) Multi focus image fusion based on NSCT and focused area detection. IEEE Sensors J 15(5):2824–2838

    Google Scholar 

  60. Yin M, Liu W, Zhao X, Yin Y, Guo Y (2014) A novel image fusion algorithm based on non sub-sampled shearlet transform. Optik 125(2014):2274–2282

    Article  Google Scholar 

  61. Zhao J, Cheung S-CS (2014) Human segmentation by geometrically fusing visible-light and thermal imageries. Multimed Tools Appl 73(1):61–89

    Article  Google Scholar 

  62. Zhou D, Gao X, Liu G, Mei C, Dong J, Liu Y (2011) Randomization in particle swarm optimization for global search ability. Expert Syst Appl 38:15356–15364

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, SSN College of Engineering, Chennai, India

    Madheswari Kanmani

  2. Department of Electronics and Communication Engineering, SSN College of Engineering, Chennai, India

    Venkateswaran Narasimhan

Authors
  1. Madheswari Kanmani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Venkateswaran Narasimhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Madheswari Kanmani.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kanmani, M., Narasimhan, V. An optimal weighted averaging fusion strategy for thermal and visible images using dual tree discrete wavelet transform and self tunning particle swarm optimization. Multimed Tools Appl 76, 20989–21010 (2017). https://doi.org/10.1007/s11042-016-4030-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4030-x

Keywords

Search

Navigation