Infrared image fusion for quality enhancement

This paper presents an approach for infrared image enhancement through fusion. Firstly, the infrared image is enhanced through histogram matching to enhance its dynamic range. A reference image with a good dynamic range, such as Cameraman, Lena, and Mandrill, is used in the histogram matching process. After that, the enhanced image is fused with the original image through curvelet fusion to inject much more details in the infrared image. The proposed approach achieves high quality of infrared image enhancement compared with different techniques.


Introduction
According to the simplest definition, image processing is the act of utilizing a digital computer to remove noise and other irregularities from digital images. The noise or irregularity may creep into the image either during its formation or during transformation [1].
Image enhancement is the process of improving the visibility of an image higher-and lower-frequency details [2]. The goal is to enhance the visual details of the image or to provide a better transform representation for use in image processing applications like analysis, detection, segmentation, and recognition. It also aids in the detection of background information, which is required to comprehend object behavior through human vision and perception.
The low contrast of images prevents viewers from readily distinguishing objects against a dark background. If the colors of the items and the background are the same, the majority of color-based image processing techniques will not work in this situation. The study of image enhancement methods divides them into two major categories: transform-domain methods and spatial-domain methods.
Image fusion is the process of gathering two or more images of the same scene, to get a single image with much details. This image should have a good content and be easier to interpret. Obtaining the images resulting from the fusion process of images captured from different instruments is an important process for use in different applications, such as infrared imaging and diagnosis of diseases. Fusion algorithms enhance the details of the resulting output images, and allow immunity to the environmental influences affecting the imaging process [3].
Image fusion can be implemented at three different levels: pixel, feature, and decision. Pixel-level fusion is a low level of fusion that is used to analyze and merge data from various sources prior to estimating and recognizing the original information. The feature level is a middle level of fusion that selects important image features such as shape, length, edges, segments, and direction. The decision level is a high level of fusion that indicates the actual target. In addition, fusion methods are divided into two categories: spatial-domain fusion and transform-domain fusion. Spatialdomain fusion methods include averaging, Brovery method, and Principal Component Analysis (PCA). Unfortunately, spatial-domain methods may cause spatial distortion in the fused images. Hence, transform-domain methods can be used to solve this problem. For fusion, Discrete Wavelet Transform (DWT) image fusion can be used as shown in Figs. 1. Moreover, other transforms such as the curvelet transform can also be used. The proposed approach for infrared image enhancement through histogram matching and image fusion is illustrated in Fig. 2.

Related work
Image fusion is the process that is used to collect multiple images to obtain a single image with more details. The aim of image fusion in infrared imaging is to obtain new images that are more suitable for visual interpretation. Image fusion aims to improve image quality by decreasing redundancy to increase the applicability of infrared images. The importance of image fusion lies in the fact that each observation image contains supplementary information. When this supplementary information is merged with that of another observation, an image with much details is obtained. Different techniques can be used for image fusion, such as DWT, Dual-Tree Complex Wavelet Transform (DT-CWT), and fuzzy processing.
Wavelet transform is a multi-resolution tool that can be used for image decomposition. It allows decomposition of the image into high-frequency and low-frequency components by different filtering operations at multiple scales [4][5][6]. The DWT is the transform that decomposes the signal into a mutually-orthogonal set of wavelet scales. One of the drawbacks of the wavelet decomposition of images is the limited ability to deal with curved shapes or lines. The curved lines need some sort of piecewise approximation that is possible with the curvelet transform as shown in Fig. 3.
Dual-Tree Complex Wavelet Transform (DT-CWT) is a fusion tool, which can be implemented in more than one methodology. One of them is decomposing the source images into coefficients of high frequency and low frequency. After that, the high-frequency coefficients are fused by the maximum-choice fusion rule and the low-frequency coefficients that include some important information are fused depending on the weighted-average fusion rule. Complex basis functions are used in this implementation to allow efficient utilization of phase information. To obtain the fused image, the Inverse Dual-Tree Complex Wavelet Transform (IDT-CWT) is used [7]. The advantages of this tool are good shift invariance, selectivity, perfect reconstruction, and simple computation [8].
Fuzzy image fusion is based on the rules with which the human makes decisions. Fuzzy machines work on the same way as the human, on the condition that the decision and how to choose this decision are replaced by fuzzy groups, and the rules are changed by fuzzy rules [9].
The fuzzy image fusion steps are summarized as follows: Aghamaleki et al. [10] proposed a technique for image fusion using DT-DWT and an optimization process. Nagaraja et al. [11] introduced a method for medical image fusion using a hybrid meta-heuristic approach. Firstly, the weighted fast discrete curvelet transform is applied to obtain the image high-frequency and low-frequency sub-bands. The high-frequency sub-bands of the two images are integrated by the optimized type II fuzzy technique. The averaging approach is used to perform the fusion of low-frequency sub-bands. Finally, the inverse transform is performed to produce the final fused image.
Desale et al. [12] presented a technique for image fusion based on PCA , DCT , and DWT. DWT-based techniques achieve better image fusion results than other ones. Sruthy et al. [13] proposed an image fusion method based on DT-CWT. This method was implemented on medical images for cancer diagnosis.

The proposed approach for image fusion and enhancement
The proposed approach consists of two stages: image enhancement and image fusion. Firstly, the infrared image is enhanced by histogram matching to a visual image with good characteristics represented in a wide histogram such as Cameraman, Lena or Mandrill. After that, the enhanced image is fused with the original image. Different types of image fusion are considered including curvelet, fuzzy and DT-CWT. Figure 2 shows the block diagram of the proposed approach.

Histogram matching
The proposed approach is based on histogram matching for image enhancement. It includes the following steps:

A visual image with good histogram characteristics is
selected. 2. The mean value of the visual image is estimated as follows: 3. The standard deviation of the visual image is estimated as follows: 4. The mean of the infrared image is estimated as follows: (1) 6. A correction factor is estimated as follows: 7. Mean correction is performed as follows: 8. The enhanced image is obtained with the following formula:

Curvelet fusion
The curvelet transform is more suitable for representation of curved objects. It is fast in implementation and more y).c r + m 2m efficient in image representation. It allows piecewise representation of curved lines [14] as shown in Fig. 3.
Steps of the curvelet transform for image processing are summarized below: Steps for image fusion using the curvelet transform: • Registration for the two input images is performed. • All steps of the curvelet transform are applied on both images to get different tiles of sub-bands. • The maximum-frequency fusion rule is applied on the tiles to be fused from both images. • To get the fused image, inverse curvelet transform steps are applied.

Simulation results
The proposed approach is applied on infrared images. It consists of two stages: image enhancement and image fusion. The image enhancement is performed by histogram a. Cameraman as a reference b. Mandrill as a reference c. Lena as a reference   intensity. Figure 4 illustrates the resulting enhanced images. Figure 5 clarifies the resulting fused images with DT-CWT. Figure 6 clarifies the resulting fused images with the fuzzy technique. Figure 7 clarifies the resulting fused images with curvelet fusion. Tables 1, 2, and 3 give the evaluation metrics obtained with all types of fusion applied in the paper. Tables 1, 2, and 3 present the resulting evaluation metrics of infrared image enhancement with all utilized fusion techniques. The results reveal that the curvelet fusion technique achieves the best performance. The quality of the resulting fused images with curvelet fusion is better than those with other techniques. From the visual perspective, the resulting images with curvelet fusion are the clearest.

Conclusion
A proposed approach for quality enhancement has been applied for infrared images. It consists of two stages: image enhancement and image fusion of enhanced and original images. The image enhancement is performed by histogram matching. We compared the results of the proposed approach with those of different fusion techniques. The performance of the proposed approach is the best with the curvelet fusion technique.
Funding Open access funding provided by The Science, Technology & Innovation Funding Authority (STDF) in cooperation with The Egyptian Knowledge Bank (EKB).
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.