Abstract
A decision map contains complete and clear information about the image to be fused, which is crucial to various image fusion issues, especially multi-focus image fusion. However, in order to get a satisfactory image fusion effect, getting a decision map is very necessary and usually difficult to finish. In this letter, we address this problem with convolutional neural network (CNN), aiming to get a state-of-the-art decision map. The main idea is that the max-pooling of CNN is replaced by a convolution layer, the residuals are propagated backwards by gradient descent, and the training parameters of the individual layers of the CNN are updated layer by layer. Based on this, we propose a new all CNN (ACNN)-based multi-focus image fusion method in spatial domain. We demonstrate that the decision map obtained from the ACNN is reliable and can lead to high-quality fusion results. Experimental results clearly validate that the proposed algorithm can obtain state-of-the-art fusion performance in terms of both qualitative and quantitative evaluations.
Similar content being viewed by others
References
Chaoben Du and Shesheng Gao, IEEE Access 5, 15750 (2017).
P. Burt and E. Adelson, IEEE Trans. Commun. 31, 532 (1983).
A. Toet, Pattern Recognit. Lett. 9, 255 (1989).
Li H, B. Manjunath and S. Mitra, Graphical Models Image Process. 57, 235 (1995).
J. Lewis, R. O. Callaghan, S. Nikolov, D. Bull and N. Canagarajah, Inf. Fusion 8, 119 (2007).
Zhang Q and Guo B, Signal Process. 89, 1334 (2009).
G. Piella, Inf. Fusion 4, 259 (2003).
Li X, Li H, Yu Z and Kong Y, Opt. Eng. 54, 073 (2015).
Liu Y, Liu S and Wang Z, Inf. Fusion 24, 147 (2015).
Liu Z, Chai Y, Yin H, Zhou J and Zhu Z, Inf. Fusion 35, 102 (2017).
W. Huang and Z. Jing, Pattern Recognit. Lett. 28, 493 (2007).
S. Li, J. Kwok and Y. Wang, Inf. Fusion 2, 169 (2001).
Li S, Kwok J and Wang Y, Pattern Recognit. Lett. 23, 985 (2002).
V. Aslantas and R. Kurban, Appl. 37, 8861 (2010).
De I and Chanda B, Inf. Fusion 14, 136 (2013).
Bai X, Zhang Y, Zhou F and Xue B, Inf. Fusion 22, 105 (2015).
Li M, Cai W and Tan Z, Pattern Recognit. Lett. 27, 1948 (2006).
Li S and Yang B, Image Vis. Comput. 26, 971 (2008).
Li S, Kang X and Hu J, IEEE Trans. 22, 2864 (2015).
Liu Y, Liu S and Wang Z, Inf. Fusion 23, 139 (2013).
Zhou Z, Li S and Wang B, Inf. Fusion 20, 60 (2014).
https://en.wikipedia.org/wiki/Deep_learning
Goodfellow Ian J, Wardefarley David, Mirza Mehdi, Courville Aaron and Bengio Yoshua, Maxout Networks, Computer Science, 1319 (2013).
Stollenga Marijn F, Masci Jonathan, Gomez Faustino and Schmidhuber Jurgen, Advances in Neural Information Processing Systems 4, 3545 (2014).
Zeiler M D and Fergus R, Stochastic Pooling for Regularization of Deep Convolutional Neural Networks, Eprint Arxiv, 2013.
Lee Chen Y, Xie S, Gallagher Patrick, Zhang Z and Tu Z, Deeply Supervised Nets, In Deep Learning and Representation Learning Workshop, NIPS (2014).
JT Springenberg, A Dosovitskiy, T Brox and M Riedmiller, Striving for Simplicity: The All Convolutional Net, Eprint Arxiv, 2014.
Simonyan K and Zisserman A, Very Deep Convolutional Networks for Large-Scale Image Recognition, Computer Science, 2014.
LeCun Y, Bottou L and Bengio Y, Proceedings of the IEEE 86, 2278 (1998).
Gulcehre C aglar, Cho KyungHyun, Pascanu Razvan and Bengio Yoshua, Learned-norm Pooling for Deep Feedforward and Recurrent Neural Networks, ECML (2014).
Jia Y, Huang C and Darrell T, Beyond Spatial Pyramids: Receptive Field Learning for Pooled Image Features, Computer Vision and Pattern Recognition, IEEE, 3370 (2012).
Behnke S, Hierarchical Neural Networks for Image Interpretation, Lecture Notes in Computer Science 2766, 1345 (2003).
Li S, Kwok J and Wang Y, Pattern Recognit. Lett. 23, 985 (2002).
Gao J and Xu L, Neural Process. Lett. 43, 805 (2016).
Liu Y, Chen X, Peng H and Wang Z, Inf. Fusion 36, 191 (2017).
Zhang Y, Bai X and Wang T, Inf. Fusion 35, 81 (2017).
Guo D, Yan J.W and Qu X, Opt. Commun. 338, 138 (2015).
Author information
Authors and Affiliations
Corresponding author
Additional information
This work has been supported by the National Natural Science Foundation of China (No.61174193).
Rights and permissions
About this article
Cite this article
Du, Cb., Gao, Ss. Multi-focus image fusion with the all convolutional neural network. Optoelectron. Lett. 14, 71–75 (2018). https://doi.org/10.1007/s11801-018-7207-x
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11801-018-7207-x