Abstract
In recent years, computational intelligence has been widely used in many fields and achieved remarkable performance. Evolutionary computing and deep learning are important branches of computational intelligence. Many methods based on evolutionary computation and deep learning have achieved good performance in remote sensing image registration. This paper introduces the application of computational intelligence in remote sensing image registration from the two directions of evolutionary computing and deep learning. In the part of remote sensing image registration based on evolutionary calculation, the principles of evolutionary algorithms and swarm intelligence algorithms are elaborated and their application in remote sensing image registration is discussed. The application of deep learning in remote sensing image registration is also discussed. At the same time, the development status and future of remote sensing image registration are summarized and their prospects are examined.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
H. Ghassemian. A review of remote sensing image fusion methods. Information Fusion, vol. 32, pp. 75–89, 2016. DOI: https://doi.org/10.1016/j.inffus.2016.03.003.
P. Z. Zhang, M. G. Gong, L. Z. Su, J. J. Liu, Z. Z. Li. Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 116, pp. 24–41, 2016. DOI: https://doi.org/10.1016/j.isprsjprs.2016.02.013.
J. H. Cui. Remote sensing image feature recognition and monitoring of ecological vegetation restoration in football field. Ekoloji, vol. 28, no. 108, pp. 1571–1575, 2019.
Y. M. Zhen, Z. Sun, J. B. Li, Y. Peng. An airborne remote sensing image mosaic algorithm based on feature points. In Proceedings of the 6th International Conference on Instrumentation & Measurement, Computer, Communication and Control, IEEE, Harbin, China, pp. 202–205, 2016. DOI: https://doi.org/10.1109/IMCCC.2016.145.
S. H. Wang, J. D. Sun, P. Phillips, G. H. Zhao, Y. D. Zhang. Polarimetric synthetic aperture radar image segmentation by convolutional neural network using graphical processing units. Journal of Real-time Image Processing, vol. 15, no. 3, pp. 631–642, 2018. DOI: https://doi.org/10.1007/s11554-017-0717-0.
Y. D. Zhang, L. N. Wu. Crop classification by forward neural network with adaptive chaotic particle swarm optimization. Sensors, vol. 11, no. 5, pp. 4721–4743, 2011. DOI: https://doi.org/10.3390/s110504721.
N. Alborzi, F. Poorahangaryan, H. Beheshti. Spectral-spatial classification of hyperspectral images using signal subspace identification and edge-preserving filter. International Journal of Automation and Computing, vol. 17, no. 2, pp. 222–232, 2020. DOI: https://doi.org/10.1007/s11633-019-1188-5.
J. Du, C. X. Jiang, Q. Guo, M. Guizani, Y. Ren. Cooperative earth observation through complex space information networks. IEEE Wireless Communications, vol. 23, no. 2, pp. 136–144, 2016. DOI: https://doi.org/10.1109/MWC.2016.7462495.
J. Y. Ma, Y. Ma, C. Li. Infrared and visible image fusion methods and applications: A survey. Information Fusion, vol. 45, pp. 153–178, 2019. DOI: https://doi.org/10.1016/j.inffus.2018.02.004.
B. Zitova, J. Flusser. Image registration methods: A survey. Image and Vision Computing, vol. 21, no. 11, pp. 977–1000, 2003. DOI: https://doi.org/10.1016/S0262-8856(03)00137-9.
G. Haskins, U. Kruger, P. K. Yan. Deep learning in medical image registration: A survey. Machine Vision and Applications, vol. 31, no. 1–2, Article number 8, 2020. DOI: https://doi.org/10.1007/s00138-020-01060-x.
J. Le Moigne. Introduction to remote sensing image registration. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Fort Worth, USA, pp. 2565–2568, 2017. DOI: https://doi.org/10.1109/IGARSS.2017.8127519.
S. R. J. Ramson, K. L. Raju, S. Vishnu, T. Anagnostopoulos. Nature inspired optimization techniques for image processing — a short review. Nature Inspired Optimization Techniques for Image Processing Applications, Hemanth J., Balas V. E., Eds., Cham, Germany: Springer, pp. 113–145, 2019. DOI: https://doi.org/10.1007/978-3-319-96002-9_5.
H. Liu, G. F. Xiao. Remote sensing image registration based on improved kaze and brief descriptor. International Journal of Automation and Computing, vol. 17, no. 4, pp. 588–598, 2020. DOI: https://doi.org/10.1007/s11633-019-1218-3.
L. Ma, Y. Liu, X. L. Zhang, Y. X. Ye, G. F. Yin, B. A. Johnson. Deep learning in remote sensing applications: A meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 152, pp. 166–177, 2019. DOI: https://doi.org/10.1016/j.isprsjprs.2019.04.015.
H. Liu, G. F. Xiao, Y. L. Tan, C. J. Ouyang. Multi-source remote sensing image registration based on contourlet transform and multiple feature fusion. International Journal of Automation and Computing, vol. 16, no. 5, pp. 575–588, 2019. DOI: https://doi.org/10.1007/s11633-018-1163-6.
Y. Wu, W. P. Ma, Q. X. Su, S. D. Liu, Y. H. Ge. Remote sensing image registration based on local structural information and global constraint. Journal of Applied Remote Sensing, vol. 13, no. 1, Article number 016518, 2019. DOI: https://doi.org/10.1117/1.JRS.13.016518.
X. P. Liu, S. L. Chen, L. Zhuo, J. Li, K. N. Huang. Multi-sensor image registration by combining local self-similarity matching and mutual information. Frontiers of Earth Science, vol. 12, no. 4, pp. 779–790, 2018. DOI: https://doi.org/10.1007/s11707-018-0717-9.
X. L. Ma, X. D. Li, Q. F. Zhang, K. Tang, Z. P. Liang, W. X. Xie, Z. X. Zhu. A survey on cooperative co-evolutionary algorithms. IEEE Transactions on Evolutionary Computation, vol. 23, no. 3, pp. 421–441, 2019. DOI: https://doi.org/10.1109/TEVC.2018.2868770.
J. W. Zhang, L. N. Xing. A survey of multiobjective evolutionary algorithms. In Proceedings of IEEE International Conference on Computational Science and Engineering and IEEE International Conference on Embedded and Ubiquitous Computing, IEEE, Guangzhou, China, pp. 93–100, 2017. DOI: https://doi.org/10.1109/CSE-EUC.2017.27.
K. De Jong. Evolutionary computation: A unified approach. In Proceedings of Genetic and Evolutionary Computation Conference Companion, ACM, Denver, USA, pp. 185–199, 2016. DOI: https://doi.org/10.1145/2908961.2926973.
K. R. Opara, J. Arabas. Differential evolution: A survey of theoretical analyses. Swarm and Evolutionary Computation, vol. 44, pp. 546–558, 2019. DOI: https://doi.org/10.1016/j.swevo.2018.06.010.
Z. N. He, G. G. Yen, J. C. Lv. Evolutionary multiobjective optimization with robustness enhancement. IEEE Transactions on Evolutionary Computation, vol. 24, no. 3, pp. 494–507, 2020. DOI: https://doi.org/10.1109/TEVC.2019.2933444.
Y. F. Zhong, A. L. Ma, Y. Soon Ong, Z. X. Zhu, L. P. Zhang. Computational intelligence in optical remote sensing image processing. Applied Soft Computing, vol. 64, pp. 75–93, 2018. DOI: https://doi.org/10.1016/j.asoc.2017.11.045.
A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis. Deep learning for computer vision: A brief review. Computational Intelligence and Neuroscience, vol. 2018, Article number 7068349, 2018. DOI: https://doi.org/10.1155/2018/7068349.
B. D. de Vos, F. F. Berendsen, M. A. Viergever, M. Staring, I. Iusgum. End-to-end unsupervised deformable image registration with a convolutional neural network. In Proceedings of International Workshop on Deep Learning in Medical Image Analysis, International Workshop on Maltimodal Learning for Clinical Decision Support, Springer, Quebec city, Canada, pp. 204–212, 2017. p. 204–212, 2017. DOI: https://doi.org/10.1007/978-3-319-67558-9_24.
Z. Q. Zhao, P. Zheng, S. T. Xu, X. D. Wu. Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 11, pp. 3212–3232, 2019. DOI: https://doi.org/10.1109/TNNLS.2018.2876865.
M. G. Gong, H. L. Yang, P. Z. Zhang. Feature learning and change feature classification based on deep learning for ternary change detection in SAR images. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 129, pp. 212–225, 2017. DOI: https://doi.org/10.1016/j.isprsjprs.2017.05.001.
Y. Liu, X. Chen, Z. F. Wang, Z. J. Wang, R. K. Ward, X. S. Wang. Deep learning for pixel-level image fusion: Recent advances and future prospects. Information Fusion, vol. 42, pp. 158–173, 2018. DOI: https://doi.org/10.1016/j.inffus.2017.10.007.
F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, K. Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 0.5MB model size. https://arxiv.org/abs/1602.07360, 2016.
K. Simonyan, A. Zisserman. Very deep convolutional networks for large-scale image recognition. [Online], Available: https://arxiv.org/abs/1409.1556, 2014.
C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 1–9, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298594.
A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th Information Processing Systems, ACM, Lake Tahoe, USA, pp. 1097–1105, 2012.
J. X. Gu, Z. H. Wang, J. Kuen, L. Y. Ma, A. Shahroudy, B. Shuai, T. Liu, X. X. Wang, G. Wang, J. F. Cai, T. Chen. Recent advances in convolutional neural networks. Pattern Recognition, vol. 77, pp. 354–377, 2018. DOI: https://doi.org/10.1016/j.patcog.2017.10.013.
P. Murugan. Feed forward and backward run in deep convolution neural network. https://arxiv.org/abs/1711.03278, 2017.
A. Giusti, D. C. Cirecsan, J. Masci, L. M. Gambardella, J. Schmidhuber. Fast image scanning with deep maxpooling convolutional neural networks. In Proceedings of IEEE International Conference on Image Processing, IEEE, Melbourne, Australia pp. 4034–4038, 2013. DOI: https://doi.org/10.1109/ICIP.2013.6738831.
W. Ma, J. Lu. An equivalence of fully connected layer and convolutional layer. [Online], Available: https://arxiv.org/abs/1712.01252, 2017.
F. M. Ye, Y. F. Su, H. Xiao, X. Q. Zhao, W. D. Min. Remote sensing image registration using convolutional neural network features. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 232–236, 2018. DOI: https://doi.org/10.1109/LGRS.2017.2781741.
R. Iqbal, F. Doctor, B. More, S. Mahmud, U. Yousuf. Big data analytics: Computational intelligence techniques and application areas. Technological Forecasting and Social Change, vol. 153, Article number 119253, 2020. DOI: https://doi.org/10.1016/j.techfore.2018.03.024.
Y. S. Chen. Performance identification in large-scale class data from advanced facets of computational intelligence and soft computing techniques. International Journal of High Performance Computing and Networking, vol. 13, no. 3, pp. 283–293, 2019. DOI: https://doi.org/10.1504/IJHPCN.2019.098569.
A. E. Eiben, M. Schoenauer. Evolutionary computing. Information Processing Letters, vol. 82, no. 1, pp. 1–6, 2002. DOI: https://doi.org/10.1016/S0020-0190(02)00204-1.
H. Al-Sahaf, M. J. Zhang, A. Al-Sahaf, M. Johnston. Keypoints detection and feature extraction: A dynamic genetic programming approach for evolving rotation-invariant texture image descriptors. IEEE Transactions on Evolutionary Computation, vol. 21, no. 6, pp. 825–844, 2017. DOI: https://doi.org/10.1109/TEVC.2017.2685639.
W. A. Albukhanajer, J. A. Briffa, Y. C. Jin. Evolutionary multiobjective image feature extraction in the presence of noise. IEEE Transactions on Cybernetics, vol. 45, no. 9, pp. 1757–1768, 2015. DOI: https://doi.org/10.1109/TCYB.2014.2360074.
J. Pramanik, S. Dalai, D. Rana. Image registration using discrete wavelet transform and particle swarm optimization. International Journal of Computer Science and Information Technologies, vol. 6, pp. 1521–1525, 2015.
Y. Tian, H. D. Ma. Image registration based on improved ant colony algorithm. Advanced Materials Research, vol. 765–767, pp. 683–686, 2013. DOI: https://doi.org/10.4028/u]www.scientific.net/AMR.765-767.683.
J. Santamaria, S. Damas, J. M. Garcia-Torres, O. Cordon. Self-adaptive evolutionary image registration using differential evolution and artificial immune systems. Pattern Recognition Letters, vol. 33, no. 16, pp. 2065–2070, 2012. DOI: https://doi.org/10.1016/j.patrec.2012.07.002.
V. T. Ingole, C. N. Deshmukh, A. Joshi, D. Shete. Medical image registration using genetic algorithm. In Proceedings of the 2nd International Conference on Emerging Trends in Engineering & Technology, IEEE, Nagpur, India, pp. 63–66, 2009. DOI: https://doi.org/10.1109/ICETET.2009.143.
X. Wang, M. Adjouadi. Automatic registration of FDGCT and FLTCT images integrating genetic algorithm, powell method and wavelet decomposition. In Proceedings of IEEE Signal Processing in Medicine and Biology Symposium, IEEE, Philadelphia, USA, 2015. DOI: https://doi.org/10.1109/SPMB.2015.7405475.
L. Schwab, M. Schmitt, R. Wanka. Multimodal medical image registration using particle swarm optimization with influence of the data’s initial orientation. In Proceedings of IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology, IEEE, Niagara Falls, Canada, 2015. DOI: https://doi.org/10.1109/CIBCB.2015.7300314.
M. Abdel-Basset, A. E. Fakhry, I. El-Henawy, T. Qiu, A. K. Sangaiah. Feature and intensity based medical image registration using particle swarm optimization. Journal of Medical Systems, vol. 41, no. 12, Article number 197, 2017. DOI: https://doi.org/10.1007/s10916-017-0846-9.
K. K. Delibasis, P. A. Asvestas, G. K. Matsopoulos. Automatic point correspondence using an artificial immune system optimization technique for medical image registration. Computerized Medical Imaging and Graphics, vol. 35, no. 1, pp. 31–41, 2011. DOI: https://doi.org/10.1016/j.compmedimag.2010.09.002.
J. Inglada, F. Adragna. Automatic multi-sensor image registration by edge matching using genetic algorithms. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, Scanning the Present and Resolving the Future, IEEE, Sydney, Australia, pp. 2313–2315, 2001. DOI: https://doi.org/10.1109/IGARSS.2001.977986.
S. K. Makrogiannis, N. G. Bourbakis, S. Borek. A stochastic optimization scheme for automatic registration of aerial images. In Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelhgence, IEEE, Boca Raton, USA, pp. 328–336, 2004. DOI: https://doi.org/10.1109/ICTAI.2004.18.
Q. Zhang, G. J. Wen, C. X. Zhang, Z. R. Lin, Z. M. Shang, H. M. Wang. Image registration with position and similarity constraints based on genetic algorithm. In Proceedings of the 10th International Conference on Natural Computation, IEEE, Xiamen, China, pp. 568–572, 2014. DOI: https://doi.org/10.1109/ICNC.2014.6975897.
Z. J. Gou, H. B. Ma. An automatic registration based on genetic algorithm for multi-source remote sensing. In Proceedings of the 2nd International Conference on Control, Automation and Robotics, IEEE, Hong Kong, China, pp. 318–323, 2016. DOI: https://doi.org/10.1109/ICCAR.2016.7486748.
Y. Y. Li, Q. J. Liu, L. H. Jing, S. Liu, F. X. Miao. A genetic-optimized multi-angle normalized cross correlation SIFT for automatic remote sensing registration. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Beijing, China, pp. 2586–2589, 2016. DOI: https://doi.org/10.1109/IGARSS.2016.7729668.
S. Yavari, M. J. V. Zoej, B. Salehi. An automatic optimum number of well-distributed ground control lines selection procedure based on genetic algorithm. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 139, pp. 46–56, 2018. DOI: https://doi.org/10.1016/j.isprsjprs.2018.03.002.
I. De Falco, A. Della Cioppa, D. Maisto, E. Tarantino. Differential evolution for the registration of remotely sensed images. Soft Computing in Industrial Applications: Recent Trends, A. Saad, K. Dahal, M. Sarfraz, R. Roy, Eds., Berlin, Heidelberg: Springer, pp. 40–49, 2007. DOI: https://doi.org/10.1007/978-3-540-70706-6_4.
I. De Falco, A. Della Cioppa, D. Maisto, E. Tarantino. Differential evolution as a viable tool for satellite image registration. Applied Soft Computing, vol. 8, no. 4, pp. 1453–1462, 2008. DOI: https://doi.org/10.1016/j.asoc.2007.10.013.
I. De Falco, D. Maisto, U. Scafuri, E. Tarantino, A. Della Cioppa. Distributed differential evolution for the registration of remotely sensed images. In Proceedings of the 15th EUROMICRO International Conference on Parallel, Distributed and Network-based Processing, IEEE, Naples, Italy, pp. 358–362, 2007. DOI: https://doi.org/10.1109/PDP.2007.36.
I. De Falco, A. Della Cioppa, D. Maisto, U. Scafuri, E. Tarantino. Distributed differential evolution for the registration of satellite and multimodal medical imagery. Evolutionary Image Analysis and Signal Processing, S. Cagnoni, Ed., Berlin, Heidelberg: Springer, pp. 153–169, 2009. DOI: https://doi.org/10.1007/978-3-642-01636-3_9.
Z. B. Hu, W. Y. Gong, Z. H. Cai. Multi-resolution remote sensing image registration using differential evolution with adaptive strategy selection. Optical Engineering, vol. 51, no. 10, Article number 101707, 2012. DOI: https://doi.org/10.1117/1.OE.51.10.101707.
W. P. Ma, X. F. Fan, Y. Wu, L. C. Jiao. An orthogonal learning differential evolution algorithm for remote sensing image registration. Mathematical Problems in Engineering, vol. 2014, Article number 305980, 2014. DOI: https://doi.org/10.1155/2014/305980.
Y. Lu, Z. W. Liao, W. F. Chen. An automatic registration framework using quantum particle swarm optimization for remote sensing images. In Proceedings of International Conference on Wavelet Analysis and Pattern Recognition, IEEE, Beijing, China, pp. 484–488, 2007. DOI: https://doi.org/10.1109/ICWAPR.2007.4420718.
Y. Zhang, Y. Guo, Y. F. Gu, W. Z. Zhong. Particle swarm optimization with powell’s direction set method for remote sensing image registration. In Proceedings of the 5th International Conference on Natural Computation, IEEE, Tianjin, China, pp. 388–392, 2009. DOI: https://doi.org/10.1109/ICNC.2009.402.
R. An, C. Y. Chen, H. L. Wang. An improved particle swarm optimization algorithm for image matching. In Proceedings of International Forum on Computer Science-Technology and Applications, IEEE, Chongqing, China, pp. 7–10, 2009. DOI: https://doi.org/10.1109/IFCSTA.2009.8.
R. Gharbia, S. A. Ahmed, A. ella Hassanien. Remote sensing image registration based on particle swarm optimization and mutual information. In Proceedings of the 2nd International Conference INDIA, Vol. 2, pp. 399–408, 2015. DOI: https://doi.org/10.1007/978-81-322-2247-7_41.
S. Yavari, M. J. Valadan Zoej, M. R. Sahebi, M. Mokhtarzade. Accuracy improvement of high resolution satellite image georeferencing using an optimized line-based rational function model. International Journal of Remote Sensing, vol. 39, no. 6, pp. 1655–1670, 2018. DOI: https://doi.org/10.1080/01431161.2017.1410294.
Y. Wu, Q. G. Miao, W. P. Ma, M. G. Gong, S. F. Wang. PSOSAC: Particle swarm optimization sample consensus algorithm for remote sensing image registration. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 242–246, 2018. DOI: https://doi.org/10.1109/LGRS.2017.2783879.
Y. Wu, W. P. Ma, Q. G. Miao, S. F. Wang. Multimodal continuous ant colony optimization for multisensor remote sensing image registration with local search. Swarm and Evolutionary Computation, vol. 47, pp. 89–95, 2019. DOI: https://doi.org/10.1016/j.swevo.2017.07.004.
M. Črepinšek, S. H. Liu, M. Mernik. Exploration and exploitation in evolutionary algorithms: A survey. ACM Computing Surveys, vol. 45, no. 3, Article number 35, 2013. DOI: https://doi.org/10.1145/2480741.2480752.
J. H. Holland. Adaption in Natural and Artificial Systems. Michigan, USA: The University of Michigan Press, 1975.
N. Saini. Review of selection methods in genetic algorithms. International Journal of Engineering and Computer Science, vol. 6, no. 12, pp. 22261–22263, 2017.
P. Sharma, H. Sharma, J. C. Bansal. Effective competency based differential evolution algorithm. Journal of Statistics and Management Systems, vol. 22, no. 7, pp. 1223–1238, 2019. DOI: https://doi.org/10.1080/09720510.2019.1609560.
V. Kachitvichyanukul. Comparison of three evolutionary algorithms: GA, PSO, and DE. Industrial Engineering and Management Systems, vol. 11, no. 3, pp. 215–223, 2012. DOI: https://doi.org/10.7232/iems.2012.11.3.215.
S. Das, S. S. Mullick, P. N. Suganthan. Recent advances in differential evolution — an updated survey. Swarm and Evolutionary Computation, vol. 27, pp. 1–30, 2016. DOI: https://doi.org/10.1016/j.swevo.2016.01.004.
P. K. Yadav, N. L. Prajapati. An overview of genetic algorithm and modeling. International Journal of Scientific and Research Publications, vol. 2, no. 9, pp. 1–4, 2012.
R. D. Al-Dabbagh, F. Neri, N. Idris, M. S. Baba. Algorithmic design issues in adaptive differential evolution schemes: Review and taxonomy. Swarm and Evolutionary Computation, vol. 43, pp. 284–311, 2018. DOI: https://doi.org/10.1016/j.swevo.2018.03.008.
R. Storn, K. Price. Differential evolution — a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997. DOI: https://doi.org/10.1023/A:1008202821328.
R. S. Parpinelli, H. S. Lopes. New inspirations in swarm intelligence: A survey. International Journal of Bio-Inspired Computation, vol. 3, no. 1, pp. 1–16, 2011. DOI: https://doi.org/10.1504/IJBIC.2011.038700.
C. Kolias, G. Kambourakis, M. Maragoudakis. Swarm intelligence in intrusion detection: A survey. Computers & Security, vol. 30, no. 8, pp. 625–642, 2011. DOI: https://doi.org/10.1016/j.cose.2011.08.009.
I. Fister Jr, X. S. Yang, I. Fister, J. Brest, D. Fister. A brief review of nature-inspired algorithms for optimization. https://arxiv.org/abs/1307.4186, 2013.
J. kennedy, R. Eberhart. Particle swarm optimization. In Proceedings of IEEE International conference on Neural Netuorks, Perth, Australia, pp. 1942–1948, 1995.
Q. H. Bai. Analysis of particle swarm optimization algorithm. Computer and Information Science, vol. 3, no. 1, pp. 180–184, 2010.
D. P. Rini, S. M. Shamsuddin, S. S. Yuhaniz. Particle swarm optimization: Technique, system and challenges. International Journal of Computer Applications, vol. 14, no. 1, pp. 19–27, 2011. DOI: https://doi.org/10.5120/1810-2331.
M. Dorigo, V. Maniezzo, A. Colorni. Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics — Part B, vol. 26, no. 1, pp. 29–41, 1996.
W. Peng, R. F. Tong, G. P. Qian, J. X. Dong. A constrained ant colony algorithm for image registration. In Proceedings of International Conference on Intelligent Computing, Computational Intelligence and Bioinformatics, Springer, Kunming, China, pp. 1–11, 2006. DOI: https://doi.org/10.1007/118161021.
F. Yang, H. L. Zhang. Multiresolut ion 3D image registration using hybrid ant colony algorithm and Powell’s method. Journal of Electronics & Information Technology, vol. 29, no. 3, pp. 622–625, 2007. (in Chinese)
H. Rezaei, M. Shakeri, S. Azadi, K. Jaferzade. Multimodality image registration utilizing ant colony algorithm. In Proceedings of the 2nd International Conference on Machine Vision, IEEE, Dubai, Emirates, pp. 49–53, 2009. DOI: https://doi.org/10.1109/ICMV.2009.21.
G. S. Raghtate, S. S. Salankar. Modified fuzzy c means with optimized ant colony algorithm for image segmentation. In Proceedings of International Conference on Computational Intelligence and Communication Networks, IEEE, Jabalpur, India, pp. 1283–1288, 2015. DOI: https://doi.org/10.1109/CICN.2015.246.
B. Khorram, M. Yazdi. A new optimized thresholding method using ant colony algorithm for MR brain image segmentation. Journal of Digital Imaging, vol. 32, no. 1, pp. 162–174, 2019. DOI: https://doi.org/10.1007/s10278-018-0111-x.
G. H. Zou. Ant colony clustering algorithm and improved Markov random fusion algorithm in image segmentation of brain images. International Journal Bioautomation, vol. 20, no. 4, pp. 505–514, 2016.
G. Rellier, X. Descombes, J. Zerubia. Local registration and deformation of a road cartographic database on a spot satellite image. Pattern Recognition, vol. 35, no. 10, pp. 2213–2221, 2002. DOI: https://doi.org/10.1016/S0031-3203(01)00180-7.
X. G. Du, J. W. Dang, Y. P. Wang, X. G. Liu, S. Li. An algorithm multi-resolution medical image registration based on firefly algorithm and Powell. In Proceedings of the 3rd International Conference on Intelligent System Design and Engineering Applications, IEEE, Hong Kong, China, pp. 274–277, 2013. DOI: https://doi.org/10.1109/ISDEA.2012.68.
J. Zhang, J. L. Hu. A novel registration method based on coevolutionary strategy. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Vancouver, Canada, pp. 2375–2380, 2016. DOI: https://doi.org/10.1109/CEC.2016.7744082.
X. S. Yang, X. S. He. Bat algorithm: Literature review and applications. International Journal of Bio-inspired Computation, vol. 5, no. 3, pp. 141–149, 2013. DOI: https://doi.org/10.1504/IJBIC.2013.055093.
Y. H. Shi. Brain storm optimization algorithm. In Proceedings of the 2nd International Conference in Swarm Intelhgence, Springer, Chongqing, China, pp. 303–309, 2011. DOI: https://doi.org/10.1007/978-3-642-21515-5_36.
D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004. DOI: https://doi.org/10.1023/B:VISI.0000029664.99615.94.
E. Shelhamer, J. Long, T. Darrell. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640–651, 2014. DOI: https://doi.org/10.1109/TPAMI.2016.2572683.
L. C. Chen, Y. K. Zhu, G. Papandreou, F. Schroff, H. Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 833–851, 2018.
B. Singh, M. Najibi, L. S. Davis. SNIPER: Efficient multi-scale training. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 9310–9320, 2018.
K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90.
J. Zabalza, J. C. Ren, J. B. Zheng, H. M. Zhao, C. M. Qing, Z. J. Yang, P. J. Du, S. Marshall. Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging. Neurocomputing, vol. 185, pp. 1–10, 2016. DOI: https://doi.org/10.1016/j.neucom.2015.11.044.
Y. Q. Chen, L. C. Jiao, Y. Y. Li, J. Zhao. Multilayer projective dictionary pair learning and sparse autoencoder for polSAR image classification. IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 12, pp. 6683–6694, 2017. DOI: https://doi.org/10.1109/TGRS.2017.2727067.
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 2672–2680, 2014.
Y. Zhan, D. Hu, Y. T. Wang, X. C. Yu. Semisupervised hyperspectral image classification based on generative adversarial networks. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 212–216, 2018. DOI: https://doi.org/10.1109/LGRS.2017.2780890.
N. Dalai, B. Triggs. Histograms of oriented gradients for human detection. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, San Diego, USA, pp. 886–893, 2005. DOI: https://doi.org/10.1109/CVPR.2005.177.
R. Alshehhi, P. R. Marpu, W. Lee Woon, M. Dalla Mura. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. IS-PRS Journal of Photogrammetry and Remote Sensing, vol. 130, pp. 139–149, 2017. DOI: https://doi.org/10.1016/j.isprsjprs.2017.05.002.
W. P. Ma, J. Zhang, Y. Wu, L. C. Jiao, H. Zhu, W. Zhao. A novel two-step registration method for remote sensing images based on deep and local features. IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 7, pp. 4834–4843, 2019. DOI: https://doi.org/10.1109/TGRS.2019.2893310.
W. P. Ma, Z. L. Wen, Y. Wu, L. C. Jiao, M. G. Gong, Y. F. Zheng, L. Liu. Remote sensing image registration with modified SIFT and enhanced feature matching. IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 1, pp. 3–7, 2017. DOI: https://doi.org/10.1109/LGRS.2016.2600858.
Y. Y. Dong, W. L. Jiao, T. F. Long, L. F. Liu, G. J. He, C. J. Gong, Y. T. Guo. Local deep descriptor for remote sensing image feature matching. Remote Sensing, vol. 11, no. 4, Article number 430, 2019. DOI: https://doi.org/10.3390/rs11040430.
A. Sedaghat, M. Mokhtarzade, H. Ebadi. Uniform robust scale-invariant feature matching for optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 11, pp. 4516–4527, 2011. DOI: https://doi.org/10.1109/TGRS.2011.2144607.
A. Sedaghat, H. Ebadi. Remote sensing image matching based on adaptive binning SIFT descriptor. IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 10, pp. 5283–5293, 2015. DOI: https://doi.org/10.1109/TGRS.2015.2420659.
J. T. Springenberg, A. Dosovitskiy, T. Brox, M. Riedmiller. Striving for simplicity: The all convolutional net. https://arxiv.org/abs/1412.6806, 2014.
X. F. Han, T. Leung, Y. Q. Jia, R. Sukthankar, A. C. Berg. MatchNet: Unifying feature and metric learning for patch-based matching. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 3279–3286, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298948.
Z. Yang, T. Dan, Y. Yang. Multi-temperal remote sensing image registration using deep convolutional features. IEEE Access, vol. 6, pp. 38544–38555, 2018.
Y. Yang, S. H. Ong, K. W. C. Foong. A robust global and local mixture distance based non-rigid point set registration. Pattern Recognition, vol. 48, no. 1, pp. 156–173, 2015. DOI: https://doi.org/10.1016/j.patcog.2014.06.017.
S. Zhang, Y. Yang, K. Yang, Y. Luo, S. H. Ong. Point set registration with global-local correspondence and transformation estimation. In Proceedings of International Conference on Computer Vision, IEEE, Venice, Italy, pp. 2669–2677, 2017. DOI: https://doi.org/10.1109/ICCV.2017.291.
S. Wang, D. Quan, X. F. Liang, M. D. Ning, Y. H. Guo, L. C. Jiao. A deep learning framework for remote sensing image registration. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 145, pp. 148–164, 2018. DOI: https://doi.org/10.1016/j.isprsjprs.2017.12.012.
D. Quan, S. Wang, X. F. Liang, R. J. Wang, S. Fang, B. Hou, L. C. Jiao. Deep generative matching network for optical and SAR image registration. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Valencia, Spain, pp. 6215–6218, 2018. DOI: https://doi.org/10.1109/IGARSS.2018.8518653.
D. G. Kim, W. J. Nam, S. W. Lee. A robust matching network for gradually estimating geometric transformation on remote sensing imagery. In IEEE International Conference on Systems, Man and Cybernetics, IEEE, Bari, Italy, pp. 3889–3894, 2019. DOI: https://doi.org/10.1109/SMC.2019.8913881.
M. Vakalopoulou, S. Christodoulidis, M. Sahasrabudhe, S. Mougiakakou, N. Paragios. Image registration of satellite imagery with deep convolutional neural networks. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Yokohama, Japan, pp. 4939–4942, 2019. DOI: https://doi.org/10.1109/IGARSS.2019.8898220.
J. H. Park, W. J. Nam, S. W. Lee. A two-stream symmetric network with bidirectional ensemble for aerial image matching. Remote Sensing, vol. 12, no. 3, Article number 465, 2020. DOI: https://doi.org/10.3390/rs12030465.
J. Bromley, I. Guyon, Y. LeCun, E. Sackinger, R. Shah. Signature verification using a “Siamese” time delay neural network. In Proceedings of the 6th International Conference on Neural Information Processing Systems, Denver, USA, pp. 737–744, 1993.
L. H. Hughes, M. Schmitt, L. C. Mou, Y. Y. Wang, X. X. Zhu. Identifying corresponding patches in SAR and optical images with a pseudo-Siamese CNN. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 5, pp. 784–788, 2018. DOI: https://doi.org/10.1109/LGRS.2018.2799232.
H. Zhang, W. P. Ni, W. D. Yan, D. L. Xiang, J. Z. Wu, X. L. Yang, H. Bian. Registration of multimodal remote sensing image based on deep fully convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 3028–3042, 2019. DOI: https://doi.org/10.1109/JSTARS.2019.2916560.
A. Mishchuk, D. Mishkin, F. Radenovic, J. Matas. Working hard to know your neighbor’s margins: Local descriptor learning loss. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Long Beach, USA, pp. 4826–4837, 2017.
C. Harris, M. Stephens. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, Alvey Vision Club, Manchester, UK, pp. 147–151, 1988.
N. Merkle, W. J. Luo, S. Auer, R. Muller, R. Urtasun. Exploiting deep matching and sar data for the geo-localization accuracy improvement of optical satellite images. Remote Sensing, vol. 9, no. 6, Article number 586, 2017. DOI: https://doi.org/10.3390/rs9060586.
W. J. Luo, A. G. Schwing, R. Urtasun. Efficient deep learning for stereo matching. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 5695–5703, 2016. DOI: https://doi.org/10.1109/CVPR.2016.614.
H. Q. He, M. Chen, T. Chen, D. J. Li. Matching of remote sensing images with complex background variations via Siamese convolutional neural network. Remote Sensing, vol. 10, no. 2, Article number 355, 2018. DOI: https://doi.org/10.3390/rs10020355.
M. A. Fischler, R. C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981. DOI: https://doi.org/10.1145/358669.358692.
H. Bay, T. Tuytelaars, L. Van Gool. SURF: Speeded up robust features. In Proceedings of the 9th European Conference on Computer Vision, Springer, Graz, Austria, pp. 404–417, 2006. DOI: https://doi.org/10.1007/11744023_32.
J. M. Morel, G. S. Yu. ASIFT: A new framework for fully affine invariant image comparison. SIAM Journal on Imaging Sciences, vol. 2, no. 2, pp. 438–469, 2009. DOI: https://doi.org/10.1137/080732730.
Aknowledgements
This work was supported by National Natural Science Foundation of China (Nos. 61702392 and 61772393) and Key Research and Development Program of Shaanxi Province (Nos. 2018ZDXM-GY-045 and 2019JQ-189).
Author information
Authors and Affiliations
Corresponding author
Additional information
Yue Wu received the B. Eng. circuits and systems Ph.D. degree in control science and engineering from Xidian University, China in 2011 and 2016, respectively. Since 2016, he has been a teacher with Xidian University. He is currently an associate professor with Xidian University. He has authored or co-authored more than 40 papers in refereed journals and proceedings.
His research interests include computer vision and computational intelligence.
Jun-Wei Liu is a master student in Key Laboratory of Big Data and Intelligent Vision, School of Computer Science and Technology, Xidian University, China.
His research interests include evolutionary computing and image registration.
Chen-Zhuo Zhu is a master student in Key Laboratory of Big Data and Intelligent Vision, School of Computer Science and Technology, Xidian University, China.
His research interests include deep learning and image registration.
Zhuang-Fei Bai is a master student in Key Laboratory of Big Data and Intelligent Vision, School of Computer Science and Technology, Xidian University, China.
His research interests include computer vision and remote sensing image understanding.
Qi-Guang Miao received the M. Eng. degree in computer science from Xidian University, China in 1996, received the Ph. D. degree in computer science from Xidian University, China in 2005. He is currently a professor with the School of Computer Science and Technology, Xidian University, China.
His research interests include intelligent image processing and multiscale geometric representations for images.
Wen-Ping Ma received the B.Sc. degree in computer science and technology and the Ph.D. degree in pattern recognition and intelligent systems from Xidian University, China in 2003 and 2008, respectively. Since 2006, she has been with the Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, Xidian University, where she is currently an associate professor. She has published more than 30 SCI papers in international academic journals, including the IEEE Transactions on Evolutionary Computation, IEEE Transactions on Image Processing, Information Sciences, Pattern Recognition, Applied Soft Computing, Knowledge-based Systems, Physica A — Statistical Mechanics and its Applications, and IEEE Geoscience and Remote Sensing Letters. She is a member of the Chinese Institute of Electronics and the China Computer Federation.
Her research interests include natural computing and intelligent image processing.
Mao-Guo Gong received the B.Sc. degree in electronic engineering and the Ph.D. degree in electronic science and technology from Xidian University, China in 2003 and 2009, respectively. Since 2006, he has been a teacher with Xidian University. In 2008 and 2010, he was promoted as an associate professor and as a full professor, respectively, both with exceptive admission. He is an executive committee member of the Chinese Association for Artificial Intelligence and a senior member of the Chinese Computer Federation. He received the prestigious National Program for the support of Top-Notch Young Professionals from the Central Organization Department of China, the Excellent Young Scientist Foundation from the National Natural Science Foundation of China, and the New Century Excellent Talent in University from the Ministry of Education of China. He is currently the vice-chair of the IEEE Computational Intelligence Society Task Force on Memetic Computing.
His research interests include computational intelligence with applications to optimization, learning, data mining, and image understanding.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.
The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Wu, Y., Liu, JW., Zhu, CZ. et al. Computational Intelligence in Remote Sensing Image Registration: A survey. Int. J. Autom. Comput. 18, 1–17 (2021). https://doi.org/10.1007/s11633-020-1248-x
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11633-020-1248-x