Computational Intelligence in Remote Sensing Image Registration: A survey

Abstract

In recent years, computational intelligence has been widely used in many fields and achieved remarkable performance. Evolutionary computing and deep learning are important branches of computational intelligence. Many methods based on evolutionary computation and deep learning have achieved good performance in remote sensing image registration. This paper introduces the application of computational intelligence in remote sensing image registration from the two directions of evolutionary computing and deep learning. In the part of remote sensing image registration based on evolutionary calculation, the principles of evolutionary algorithms and swarm intelligence algorithms are elaborated and their application in remote sensing image registration is discussed. The application of deep learning in remote sensing image registration is also discussed. At the same time, the development status and future of remote sensing image registration are summarized and their prospects are examined.

References

  1. [1]

    H. Ghassemian. A review of remote sensing image fusion methods. Information Fusion, vol. 32, pp. 75–89, 2016. DOI: https://doi.org/10.1016/j.inffus.2016.03.003.

    Google Scholar 

  2. [2]

    P. Z. Zhang, M. G. Gong, L. Z. Su, J. J. Liu, Z. Z. Li. Change detection based on deep feature representation and mapping transformation for multi-spatial-resolution remote sensing images. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 116, pp. 24–41, 2016. DOI: https://doi.org/10.1016/j.isprsjprs.2016.02.013.

    Google Scholar 

  3. [3]

    J. H. Cui. Remote sensing image feature recognition and monitoring of ecological vegetation restoration in football field. Ekoloji, vol. 28, no. 108, pp. 1571–1575, 2019.

    Google Scholar 

  4. [4]

    Y. M. Zhen, Z. Sun, J. B. Li, Y. Peng. An airborne remote sensing image mosaic algorithm based on feature points. In Proceedings of the 6th International Conference on Instrumentation & Measurement, Computer, Communication and Control, IEEE, Harbin, China, pp. 202–205, 2016. DOI: https://doi.org/10.1109/IMCCC.2016.145.

    Google Scholar 

  5. [5]

    S. H. Wang, J. D. Sun, P. Phillips, G. H. Zhao, Y. D. Zhang. Polarimetric synthetic aperture radar image segmentation by convolutional neural network using graphical processing units. Journal of Real-time Image Processing, vol. 15, no. 3, pp. 631–642, 2018. DOI: https://doi.org/10.1007/s11554-017-0717-0.

    Google Scholar 

  6. [6]

    Y. D. Zhang, L. N. Wu. Crop classification by forward neural network with adaptive chaotic particle swarm optimization. Sensors, vol. 11, no. 5, pp. 4721–4743, 2011. DOI: https://doi.org/10.3390/s110504721.

    Google Scholar 

  7. [7]

    N. Alborzi, F. Poorahangaryan, H. Beheshti. Spectral-spatial classification of hyperspectral images using signal subspace identification and edge-preserving filter. International Journal of Automation and Computing, vol. 17, no. 2, pp. 222–232, 2020. DOI: https://doi.org/10.1007/s11633-019-1188-5.

    Google Scholar 

  8. [8]

    J. Du, C. X. Jiang, Q. Guo, M. Guizani, Y. Ren. Cooperative earth observation through complex space information networks. IEEE Wireless Communications, vol. 23, no. 2, pp. 136–144, 2016. DOI: https://doi.org/10.1109/MWC.2016.7462495.

    Google Scholar 

  9. [9]

    J. Y. Ma, Y. Ma, C. Li. Infrared and visible image fusion methods and applications: A survey. Information Fusion, vol. 45, pp. 153–178, 2019. DOI: https://doi.org/10.1016/j.inffus.2018.02.004.

    Google Scholar 

  10. [10]

    B. Zitova, J. Flusser. Image registration methods: A survey. Image and Vision Computing, vol. 21, no. 11, pp. 977–1000, 2003. DOI: https://doi.org/10.1016/S0262-8856(03)00137-9.

    Google Scholar 

  11. [11]

    G. Haskins, U. Kruger, P. K. Yan. Deep learning in medical image registration: A survey. Machine Vision and Applications, vol. 31, no. 1–2, Article number 8, 2020. DOI: https://doi.org/10.1007/s00138-020-01060-x.

  12. [12]

    J. Le Moigne. Introduction to remote sensing image registration. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Fort Worth, USA, pp. 2565–2568, 2017. DOI: https://doi.org/10.1109/IGARSS.2017.8127519.

    Google Scholar 

  13. [13]

    S. R. J. Ramson, K. L. Raju, S. Vishnu, T. Anagnostopoulos. Nature inspired optimization techniques for image processing — a short review. Nature Inspired Optimization Techniques for Image Processing Applications, Hemanth J., Balas V. E., Eds., Cham, Germany: Springer, pp. 113–145, 2019. DOI: https://doi.org/10.1007/978-3-319-96002-9_5.

    Google Scholar 

  14. [14]

    H. Liu, G. F. Xiao. Remote sensing image registration based on improved kaze and brief descriptor. International Journal of Automation and Computing, vol. 17, no. 4, pp. 588–598, 2020. DOI: https://doi.org/10.1007/s11633-019-1218-3.

    MathSciNet  Google Scholar 

  15. [15]

    L. Ma, Y. Liu, X. L. Zhang, Y. X. Ye, G. F. Yin, B. A. Johnson. Deep learning in remote sensing applications: A meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 152, pp. 166–177, 2019. DOI: https://doi.org/10.1016/j.isprsjprs.2019.04.015.

    Google Scholar 

  16. [16]

    H. Liu, G. F. Xiao, Y. L. Tan, C. J. Ouyang. Multi-source remote sensing image registration based on contourlet transform and multiple feature fusion. International Journal of Automation and Computing, vol. 16, no. 5, pp. 575–588, 2019. DOI: https://doi.org/10.1007/s11633-018-1163-6.

    Google Scholar 

  17. [17]

    Y. Wu, W. P. Ma, Q. X. Su, S. D. Liu, Y. H. Ge. Remote sensing image registration based on local structural information and global constraint. Journal of Applied Remote Sensing, vol. 13, no. 1, Article number 016518, 2019. DOI: https://doi.org/10.1117/1.JRS.13.016518.

  18. [18]

    X. P. Liu, S. L. Chen, L. Zhuo, J. Li, K. N. Huang. Multi-sensor image registration by combining local self-similarity matching and mutual information. Frontiers of Earth Science, vol. 12, no. 4, pp. 779–790, 2018. DOI: https://doi.org/10.1007/s11707-018-0717-9.

    Google Scholar 

  19. [19]

    X. L. Ma, X. D. Li, Q. F. Zhang, K. Tang, Z. P. Liang, W. X. Xie, Z. X. Zhu. A survey on cooperative co-evolutionary algorithms. IEEE Transactions on Evolutionary Computation, vol. 23, no. 3, pp. 421–441, 2019. DOI: https://doi.org/10.1109/TEVC.2018.2868770.

    Google Scholar 

  20. [20]

    J. W. Zhang, L. N. Xing. A survey of multiobjective evolutionary algorithms. In Proceedings of IEEE International Conference on Computational Science and Engineering and IEEE International Conference on Embedded and Ubiquitous Computing, IEEE, Guangzhou, China, pp. 93–100, 2017. DOI: https://doi.org/10.1109/CSE-EUC.2017.27.

    Google Scholar 

  21. [21]

    K. De Jong. Evolutionary computation: A unified approach. In Proceedings of Genetic and Evolutionary Computation Conference Companion, ACM, Denver, USA, pp. 185–199, 2016. DOI: https://doi.org/10.1145/2908961.2926973.

    Google Scholar 

  22. [22]

    K. R. Opara, J. Arabas. Differential evolution: A survey of theoretical analyses. Swarm and Evolutionary Computation, vol. 44, pp. 546–558, 2019. DOI: https://doi.org/10.1016/j.swevo.2018.06.010.

    Google Scholar 

  23. [23]

    Z. N. He, G. G. Yen, J. C. Lv. Evolutionary multiobjective optimization with robustness enhancement. IEEE Transactions on Evolutionary Computation, vol. 24, no. 3, pp. 494–507, 2020. DOI: https://doi.org/10.1109/TEVC.2019.2933444.

    Google Scholar 

  24. [24]

    Y. F. Zhong, A. L. Ma, Y. Soon Ong, Z. X. Zhu, L. P. Zhang. Computational intelligence in optical remote sensing image processing. Applied Soft Computing, vol. 64, pp. 75–93, 2018. DOI: https://doi.org/10.1016/j.asoc.2017.11.045.

    Google Scholar 

  25. [25]

    A. Voulodimos, N. Doulamis, A. Doulamis, E. Protopapadakis. Deep learning for computer vision: A brief review. Computational Intelligence and Neuroscience, vol. 2018, Article number 7068349, 2018. DOI: https://doi.org/10.1155/2018/7068349.

  26. [26]

    B. D. de Vos, F. F. Berendsen, M. A. Viergever, M. Staring, I. Iusgum. End-to-end unsupervised deformable image registration with a convolutional neural network. In Proceedings of International Workshop on Deep Learning in Medical Image Analysis, International Workshop on Maltimodal Learning for Clinical Decision Support, Springer, Quebec city, Canada, pp. 204–212, 2017. p. 204–212, 2017. DOI: https://doi.org/10.1007/978-3-319-67558-9_24.

    Google Scholar 

  27. [27]

    Z. Q. Zhao, P. Zheng, S. T. Xu, X. D. Wu. Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 11, pp. 3212–3232, 2019. DOI: https://doi.org/10.1109/TNNLS.2018.2876865.

    Google Scholar 

  28. [28]

    M. G. Gong, H. L. Yang, P. Z. Zhang. Feature learning and change feature classification based on deep learning for ternary change detection in SAR images. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 129, pp. 212–225, 2017. DOI: https://doi.org/10.1016/j.isprsjprs.2017.05.001.

    Google Scholar 

  29. [29]

    Y. Liu, X. Chen, Z. F. Wang, Z. J. Wang, R. K. Ward, X. S. Wang. Deep learning for pixel-level image fusion: Recent advances and future prospects. Information Fusion, vol. 42, pp. 158–173, 2018. DOI: https://doi.org/10.1016/j.inffus.2017.10.007.

    Google Scholar 

  30. [30]

    F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, K. Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and < 0.5MB model size. https://arxiv.org/abs/1602.07360, 2016.

  31. [31]

    K. Simonyan, A. Zisserman. Very deep convolutional networks for large-scale image recognition. [Online], Available: https://arxiv.org/abs/1409.1556, 2014.

  32. [32]

    C. Szegedy, W. Liu, Y. Q. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich. Going deeper with convolutions. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 1–9, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298594.

    Google Scholar 

  33. [33]

    A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th Information Processing Systems, ACM, Lake Tahoe, USA, pp. 1097–1105, 2012.

    Google Scholar 

  34. [34]

    J. X. Gu, Z. H. Wang, J. Kuen, L. Y. Ma, A. Shahroudy, B. Shuai, T. Liu, X. X. Wang, G. Wang, J. F. Cai, T. Chen. Recent advances in convolutional neural networks. Pattern Recognition, vol. 77, pp. 354–377, 2018. DOI: https://doi.org/10.1016/j.patcog.2017.10.013.

    Google Scholar 

  35. [35]

    P. Murugan. Feed forward and backward run in deep convolution neural network. https://arxiv.org/abs/1711.03278, 2017.

  36. [36]

    A. Giusti, D. C. Cirecsan, J. Masci, L. M. Gambardella, J. Schmidhuber. Fast image scanning with deep maxpooling convolutional neural networks. In Proceedings of IEEE International Conference on Image Processing, IEEE, Melbourne, Australia pp. 4034–4038, 2013. DOI: https://doi.org/10.1109/ICIP.2013.6738831.

    Google Scholar 

  37. [37]

    W. Ma, J. Lu. An equivalence of fully connected layer and convolutional layer. [Online], Available: https://arxiv.org/abs/1712.01252, 2017.

  38. [38]

    F. M. Ye, Y. F. Su, H. Xiao, X. Q. Zhao, W. D. Min. Remote sensing image registration using convolutional neural network features. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 232–236, 2018. DOI: https://doi.org/10.1109/LGRS.2017.2781741.

    Google Scholar 

  39. [39]

    R. Iqbal, F. Doctor, B. More, S. Mahmud, U. Yousuf. Big data analytics: Computational intelligence techniques and application areas. Technological Forecasting and Social Change, vol. 153, Article number 119253, 2020. DOI: https://doi.org/10.1016/j.techfore.2018.03.024.

  40. [40]

    Y. S. Chen. Performance identification in large-scale class data from advanced facets of computational intelligence and soft computing techniques. International Journal of High Performance Computing and Networking, vol. 13, no. 3, pp. 283–293, 2019. DOI: https://doi.org/10.1504/IJHPCN.2019.098569.

    Google Scholar 

  41. [41]

    A. E. Eiben, M. Schoenauer. Evolutionary computing. Information Processing Letters, vol. 82, no. 1, pp. 1–6, 2002. DOI: https://doi.org/10.1016/S0020-0190(02)00204-1.

    MathSciNet  MATH  Google Scholar 

  42. [42]

    H. Al-Sahaf, M. J. Zhang, A. Al-Sahaf, M. Johnston. Keypoints detection and feature extraction: A dynamic genetic programming approach for evolving rotation-invariant texture image descriptors. IEEE Transactions on Evolutionary Computation, vol. 21, no. 6, pp. 825–844, 2017. DOI: https://doi.org/10.1109/TEVC.2017.2685639.

    Google Scholar 

  43. [43]

    W. A. Albukhanajer, J. A. Briffa, Y. C. Jin. Evolutionary multiobjective image feature extraction in the presence of noise. IEEE Transactions on Cybernetics, vol. 45, no. 9, pp. 1757–1768, 2015. DOI: https://doi.org/10.1109/TCYB.2014.2360074.

    Google Scholar 

  44. [44]

    J. Pramanik, S. Dalai, D. Rana. Image registration using discrete wavelet transform and particle swarm optimization. International Journal of Computer Science and Information Technologies, vol. 6, pp. 1521–1525, 2015.

    Google Scholar 

  45. [45]

    Y. Tian, H. D. Ma. Image registration based on improved ant colony algorithm. Advanced Materials Research, vol. 765–767, pp. 683–686, 2013. DOI: https://doi.org/10.4028/u]www.scientific.net/AMR.765-767.683.

    Google Scholar 

  46. [46]

    J. Santamaria, S. Damas, J. M. Garcia-Torres, O. Cordon. Self-adaptive evolutionary image registration using differential evolution and artificial immune systems. Pattern Recognition Letters, vol. 33, no. 16, pp. 2065–2070, 2012. DOI: https://doi.org/10.1016/j.patrec.2012.07.002.

    Google Scholar 

  47. [47]

    V. T. Ingole, C. N. Deshmukh, A. Joshi, D. Shete. Medical image registration using genetic algorithm. In Proceedings of the 2nd International Conference on Emerging Trends in Engineering & Technology, IEEE, Nagpur, India, pp. 63–66, 2009. DOI: https://doi.org/10.1109/ICETET.2009.143.

    Google Scholar 

  48. [48]

    X. Wang, M. Adjouadi. Automatic registration of FDGCT and FLTCT images integrating genetic algorithm, powell method and wavelet decomposition. In Proceedings of IEEE Signal Processing in Medicine and Biology Symposium, IEEE, Philadelphia, USA, 2015. DOI: https://doi.org/10.1109/SPMB.2015.7405475.

    Google Scholar 

  49. [49]

    L. Schwab, M. Schmitt, R. Wanka. Multimodal medical image registration using particle swarm optimization with influence of the data’s initial orientation. In Proceedings of IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology, IEEE, Niagara Falls, Canada, 2015. DOI: https://doi.org/10.1109/CIBCB.2015.7300314.

    Google Scholar 

  50. [50]

    M. Abdel-Basset, A. E. Fakhry, I. El-Henawy, T. Qiu, A. K. Sangaiah. Feature and intensity based medical image registration using particle swarm optimization. Journal of Medical Systems, vol. 41, no. 12, Article number 197, 2017. DOI: https://doi.org/10.1007/s10916-017-0846-9.

  51. [51]

    K. K. Delibasis, P. A. Asvestas, G. K. Matsopoulos. Automatic point correspondence using an artificial immune system optimization technique for medical image registration. Computerized Medical Imaging and Graphics, vol. 35, no. 1, pp. 31–41, 2011. DOI: https://doi.org/10.1016/j.compmedimag.2010.09.002.

    Google Scholar 

  52. [52]

    J. Inglada, F. Adragna. Automatic multi-sensor image registration by edge matching using genetic algorithms. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, Scanning the Present and Resolving the Future, IEEE, Sydney, Australia, pp. 2313–2315, 2001. DOI: https://doi.org/10.1109/IGARSS.2001.977986.

    Google Scholar 

  53. [53]

    S. K. Makrogiannis, N. G. Bourbakis, S. Borek. A stochastic optimization scheme for automatic registration of aerial images. In Proceedings of the 16th IEEE International Conference on Tools with Artificial Intelhgence, IEEE, Boca Raton, USA, pp. 328–336, 2004. DOI: https://doi.org/10.1109/ICTAI.2004.18.

    Google Scholar 

  54. [54]

    Q. Zhang, G. J. Wen, C. X. Zhang, Z. R. Lin, Z. M. Shang, H. M. Wang. Image registration with position and similarity constraints based on genetic algorithm. In Proceedings of the 10th International Conference on Natural Computation, IEEE, Xiamen, China, pp. 568–572, 2014. DOI: https://doi.org/10.1109/ICNC.2014.6975897.

    Google Scholar 

  55. [55]

    Z. J. Gou, H. B. Ma. An automatic registration based on genetic algorithm for multi-source remote sensing. In Proceedings of the 2nd International Conference on Control, Automation and Robotics, IEEE, Hong Kong, China, pp. 318–323, 2016. DOI: https://doi.org/10.1109/ICCAR.2016.7486748.

    Google Scholar 

  56. [56]

    Y. Y. Li, Q. J. Liu, L. H. Jing, S. Liu, F. X. Miao. A genetic-optimized multi-angle normalized cross correlation SIFT for automatic remote sensing registration. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Beijing, China, pp. 2586–2589, 2016. DOI: https://doi.org/10.1109/IGARSS.2016.7729668.

    Google Scholar 

  57. [57]

    S. Yavari, M. J. V. Zoej, B. Salehi. An automatic optimum number of well-distributed ground control lines selection procedure based on genetic algorithm. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 139, pp. 46–56, 2018. DOI: https://doi.org/10.1016/j.isprsjprs.2018.03.002.

    Google Scholar 

  58. [58]

    I. De Falco, A. Della Cioppa, D. Maisto, E. Tarantino. Differential evolution for the registration of remotely sensed images. Soft Computing in Industrial Applications: Recent Trends, A. Saad, K. Dahal, M. Sarfraz, R. Roy, Eds., Berlin, Heidelberg: Springer, pp. 40–49, 2007. DOI: https://doi.org/10.1007/978-3-540-70706-6_4.

    Google Scholar 

  59. [59]

    I. De Falco, A. Della Cioppa, D. Maisto, E. Tarantino. Differential evolution as a viable tool for satellite image registration. Applied Soft Computing, vol. 8, no. 4, pp. 1453–1462, 2008. DOI: https://doi.org/10.1016/j.asoc.2007.10.013.

    MATH  Google Scholar 

  60. [60]

    I. De Falco, D. Maisto, U. Scafuri, E. Tarantino, A. Della Cioppa. Distributed differential evolution for the registration of remotely sensed images. In Proceedings of the 15th EUROMICRO International Conference on Parallel, Distributed and Network-based Processing, IEEE, Naples, Italy, pp. 358–362, 2007. DOI: https://doi.org/10.1109/PDP.2007.36.

    Google Scholar 

  61. [61]

    I. De Falco, A. Della Cioppa, D. Maisto, U. Scafuri, E. Tarantino. Distributed differential evolution for the registration of satellite and multimodal medical imagery. Evolutionary Image Analysis and Signal Processing, S. Cagnoni, Ed., Berlin, Heidelberg: Springer, pp. 153–169, 2009. DOI: https://doi.org/10.1007/978-3-642-01636-3_9.

    Google Scholar 

  62. [62]

    Z. B. Hu, W. Y. Gong, Z. H. Cai. Multi-resolution remote sensing image registration using differential evolution with adaptive strategy selection. Optical Engineering, vol. 51, no. 10, Article number 101707, 2012. DOI: https://doi.org/10.1117/1.OE.51.10.101707.

  63. [63]

    W. P. Ma, X. F. Fan, Y. Wu, L. C. Jiao. An orthogonal learning differential evolution algorithm for remote sensing image registration. Mathematical Problems in Engineering, vol. 2014, Article number 305980, 2014. DOI: https://doi.org/10.1155/2014/305980.

  64. [64]

    Y. Lu, Z. W. Liao, W. F. Chen. An automatic registration framework using quantum particle swarm optimization for remote sensing images. In Proceedings of International Conference on Wavelet Analysis and Pattern Recognition, IEEE, Beijing, China, pp. 484–488, 2007. DOI: https://doi.org/10.1109/ICWAPR.2007.4420718.

    Google Scholar 

  65. [65]

    Y. Zhang, Y. Guo, Y. F. Gu, W. Z. Zhong. Particle swarm optimization with powell’s direction set method for remote sensing image registration. In Proceedings of the 5th International Conference on Natural Computation, IEEE, Tianjin, China, pp. 388–392, 2009. DOI: https://doi.org/10.1109/ICNC.2009.402.

    Google Scholar 

  66. [66]

    R. An, C. Y. Chen, H. L. Wang. An improved particle swarm optimization algorithm for image matching. In Proceedings of International Forum on Computer Science-Technology and Applications, IEEE, Chongqing, China, pp. 7–10, 2009. DOI: https://doi.org/10.1109/IFCSTA.2009.8.

    Google Scholar 

  67. [67]

    R. Gharbia, S. A. Ahmed, A. ella Hassanien. Remote sensing image registration based on particle swarm optimization and mutual information. In Proceedings of the 2nd International Conference INDIA, Vol. 2, pp. 399–408, 2015. DOI: https://doi.org/10.1007/978-81-322-2247-7_41.

    Google Scholar 

  68. [68]

    S. Yavari, M. J. Valadan Zoej, M. R. Sahebi, M. Mokhtarzade. Accuracy improvement of high resolution satellite image georeferencing using an optimized line-based rational function model. International Journal of Remote Sensing, vol. 39, no. 6, pp. 1655–1670, 2018. DOI: https://doi.org/10.1080/01431161.2017.1410294.

    Google Scholar 

  69. [69]

    Y. Wu, Q. G. Miao, W. P. Ma, M. G. Gong, S. F. Wang. PSOSAC: Particle swarm optimization sample consensus algorithm for remote sensing image registration. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 242–246, 2018. DOI: https://doi.org/10.1109/LGRS.2017.2783879.

    Google Scholar 

  70. [70]

    Y. Wu, W. P. Ma, Q. G. Miao, S. F. Wang. Multimodal continuous ant colony optimization for multisensor remote sensing image registration with local search. Swarm and Evolutionary Computation, vol. 47, pp. 89–95, 2019. DOI: https://doi.org/10.1016/j.swevo.2017.07.004.

    Google Scholar 

  71. [71]

    M. Črepinšek, S. H. Liu, M. Mernik. Exploration and exploitation in evolutionary algorithms: A survey. ACM Computing Surveys, vol. 45, no. 3, Article number 35, 2013. DOI: https://doi.org/10.1145/2480741.2480752.

  72. [72]

    J. H. Holland. Adaption in Natural and Artificial Systems. Michigan, USA: The University of Michigan Press, 1975.

    Google Scholar 

  73. [73]

    N. Saini. Review of selection methods in genetic algorithms. International Journal of Engineering and Computer Science, vol. 6, no. 12, pp. 22261–22263, 2017.

    Google Scholar 

  74. [74]

    P. Sharma, H. Sharma, J. C. Bansal. Effective competency based differential evolution algorithm. Journal of Statistics and Management Systems, vol. 22, no. 7, pp. 1223–1238, 2019. DOI: https://doi.org/10.1080/09720510.2019.1609560.

    Google Scholar 

  75. [75]

    V. Kachitvichyanukul. Comparison of three evolutionary algorithms: GA, PSO, and DE. Industrial Engineering and Management Systems, vol. 11, no. 3, pp. 215–223, 2012. DOI: https://doi.org/10.7232/iems.2012.11.3.215.

    Google Scholar 

  76. [76]

    S. Das, S. S. Mullick, P. N. Suganthan. Recent advances in differential evolution — an updated survey. Swarm and Evolutionary Computation, vol. 27, pp. 1–30, 2016. DOI: https://doi.org/10.1016/j.swevo.2016.01.004.

    Google Scholar 

  77. [77]

    P. K. Yadav, N. L. Prajapati. An overview of genetic algorithm and modeling. International Journal of Scientific and Research Publications, vol. 2, no. 9, pp. 1–4, 2012.

    Google Scholar 

  78. [78]

    R. D. Al-Dabbagh, F. Neri, N. Idris, M. S. Baba. Algorithmic design issues in adaptive differential evolution schemes: Review and taxonomy. Swarm and Evolutionary Computation, vol. 43, pp. 284–311, 2018. DOI: https://doi.org/10.1016/j.swevo.2018.03.008.

    Google Scholar 

  79. [79]

    R. Storn, K. Price. Differential evolution — a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, vol. 11, no. 4, pp. 341–359, 1997. DOI: https://doi.org/10.1023/A:1008202821328.

    MathSciNet  MATH  Google Scholar 

  80. [80]

    R. S. Parpinelli, H. S. Lopes. New inspirations in swarm intelligence: A survey. International Journal of Bio-Inspired Computation, vol. 3, no. 1, pp. 1–16, 2011. DOI: https://doi.org/10.1504/IJBIC.2011.038700.

    Google Scholar 

  81. [81]

    C. Kolias, G. Kambourakis, M. Maragoudakis. Swarm intelligence in intrusion detection: A survey. Computers & Security, vol. 30, no. 8, pp. 625–642, 2011. DOI: https://doi.org/10.1016/j.cose.2011.08.009.

    Google Scholar 

  82. [82]

    I. Fister Jr, X. S. Yang, I. Fister, J. Brest, D. Fister. A brief review of nature-inspired algorithms for optimization. https://arxiv.org/abs/1307.4186, 2013.

  83. [83]

    J. kennedy, R. Eberhart. Particle swarm optimization. In Proceedings of IEEE International conference on Neural Netuorks, Perth, Australia, pp. 1942–1948, 1995.

  84. [84]

    Q. H. Bai. Analysis of particle swarm optimization algorithm. Computer and Information Science, vol. 3, no. 1, pp. 180–184, 2010.

    Google Scholar 

  85. [85]

    D. P. Rini, S. M. Shamsuddin, S. S. Yuhaniz. Particle swarm optimization: Technique, system and challenges. International Journal of Computer Applications, vol. 14, no. 1, pp. 19–27, 2011. DOI: https://doi.org/10.5120/1810-2331.

    Google Scholar 

  86. [86]

    M. Dorigo, V. Maniezzo, A. Colorni. Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics — Part B, vol. 26, no. 1, pp. 29–41, 1996.

    Google Scholar 

  87. [87]

    W. Peng, R. F. Tong, G. P. Qian, J. X. Dong. A constrained ant colony algorithm for image registration. In Proceedings of International Conference on Intelligent Computing, Computational Intelligence and Bioinformatics, Springer, Kunming, China, pp. 1–11, 2006. DOI: https://doi.org/10.1007/118161021.

    Google Scholar 

  88. [88]

    F. Yang, H. L. Zhang. Multiresolut ion 3D image registration using hybrid ant colony algorithm and Powell’s method. Journal of Electronics & Information Technology, vol. 29, no. 3, pp. 622–625, 2007. (in Chinese)

    Google Scholar 

  89. [89]

    H. Rezaei, M. Shakeri, S. Azadi, K. Jaferzade. Multimodality image registration utilizing ant colony algorithm. In Proceedings of the 2nd International Conference on Machine Vision, IEEE, Dubai, Emirates, pp. 49–53, 2009. DOI: https://doi.org/10.1109/ICMV.2009.21.

    Google Scholar 

  90. [90]

    G. S. Raghtate, S. S. Salankar. Modified fuzzy c means with optimized ant colony algorithm for image segmentation. In Proceedings of International Conference on Computational Intelligence and Communication Networks, IEEE, Jabalpur, India, pp. 1283–1288, 2015. DOI: https://doi.org/10.1109/CICN.2015.246.

    Google Scholar 

  91. [91]

    B. Khorram, M. Yazdi. A new optimized thresholding method using ant colony algorithm for MR brain image segmentation. Journal of Digital Imaging, vol. 32, no. 1, pp. 162–174, 2019. DOI: https://doi.org/10.1007/s10278-018-0111-x.

    Google Scholar 

  92. [92]

    G. H. Zou. Ant colony clustering algorithm and improved Markov random fusion algorithm in image segmentation of brain images. International Journal Bioautomation, vol. 20, no. 4, pp. 505–514, 2016.

    Google Scholar 

  93. [93]

    G. Rellier, X. Descombes, J. Zerubia. Local registration and deformation of a road cartographic database on a spot satellite image. Pattern Recognition, vol. 35, no. 10, pp. 2213–2221, 2002. DOI: https://doi.org/10.1016/S0031-3203(01)00180-7.

    MATH  Google Scholar 

  94. [94]

    X. G. Du, J. W. Dang, Y. P. Wang, X. G. Liu, S. Li. An algorithm multi-resolution medical image registration based on firefly algorithm and Powell. In Proceedings of the 3rd International Conference on Intelligent System Design and Engineering Applications, IEEE, Hong Kong, China, pp. 274–277, 2013. DOI: https://doi.org/10.1109/ISDEA.2012.68.

    Google Scholar 

  95. [95]

    J. Zhang, J. L. Hu. A novel registration method based on coevolutionary strategy. In Proceedings of IEEE Congress on Evolutionary Computation, IEEE, Vancouver, Canada, pp. 2375–2380, 2016. DOI: https://doi.org/10.1109/CEC.2016.7744082.

    Google Scholar 

  96. [96]

    X. S. Yang, X. S. He. Bat algorithm: Literature review and applications. International Journal of Bio-inspired Computation, vol. 5, no. 3, pp. 141–149, 2013. DOI: https://doi.org/10.1504/IJBIC.2013.055093.

    Google Scholar 

  97. [97]

    Y. H. Shi. Brain storm optimization algorithm. In Proceedings of the 2nd International Conference in Swarm Intelhgence, Springer, Chongqing, China, pp. 303–309, 2011. DOI: https://doi.org/10.1007/978-3-642-21515-5_36.

    Google Scholar 

  98. [98]

    D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004. DOI: https://doi.org/10.1023/B:VISI.0000029664.99615.94.

    Google Scholar 

  99. [99]

    E. Shelhamer, J. Long, T. Darrell. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640–651, 2014. DOI: https://doi.org/10.1109/TPAMI.2016.2572683.

    Google Scholar 

  100. [100]

    L. C. Chen, Y. K. Zhu, G. Papandreou, F. Schroff, H. Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proceedings of the 15th European Conference on Computer Vision, Springer, Munich, Germany, pp. 833–851, 2018.

    Google Scholar 

  101. [101]

    B. Singh, M. Najibi, L. S. Davis. SNIPER: Efficient multi-scale training. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 9310–9320, 2018.

    Google Scholar 

  102. [102]

    K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 770–778, 2016. DOI: https://doi.org/10.1109/CVPR.2016.90.

    Google Scholar 

  103. [103]

    J. Zabalza, J. C. Ren, J. B. Zheng, H. M. Zhao, C. M. Qing, Z. J. Yang, P. J. Du, S. Marshall. Novel segmented stacked autoencoder for effective dimensionality reduction and feature extraction in hyperspectral imaging. Neurocomputing, vol. 185, pp. 1–10, 2016. DOI: https://doi.org/10.1016/j.neucom.2015.11.044.

    Google Scholar 

  104. [104]

    Y. Q. Chen, L. C. Jiao, Y. Y. Li, J. Zhao. Multilayer projective dictionary pair learning and sparse autoencoder for polSAR image classification. IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 12, pp. 6683–6694, 2017. DOI: https://doi.org/10.1109/TGRS.2017.2727067.

    Google Scholar 

  105. [105]

    I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 2672–2680, 2014.

    Google Scholar 

  106. [106]

    Y. Zhan, D. Hu, Y. T. Wang, X. C. Yu. Semisupervised hyperspectral image classification based on generative adversarial networks. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 2, pp. 212–216, 2018. DOI: https://doi.org/10.1109/LGRS.2017.2780890.

    Google Scholar 

  107. [107]

    N. Dalai, B. Triggs. Histograms of oriented gradients for human detection. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, San Diego, USA, pp. 886–893, 2005. DOI: https://doi.org/10.1109/CVPR.2005.177.

    Google Scholar 

  108. [108]

    R. Alshehhi, P. R. Marpu, W. Lee Woon, M. Dalla Mura. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks. IS-PRS Journal of Photogrammetry and Remote Sensing, vol. 130, pp. 139–149, 2017. DOI: https://doi.org/10.1016/j.isprsjprs.2017.05.002.

    Google Scholar 

  109. [109]

    W. P. Ma, J. Zhang, Y. Wu, L. C. Jiao, H. Zhu, W. Zhao. A novel two-step registration method for remote sensing images based on deep and local features. IEEE Transactions on Geoscience and Remote Sensing, vol. 57, no. 7, pp. 4834–4843, 2019. DOI: https://doi.org/10.1109/TGRS.2019.2893310.

    Google Scholar 

  110. [110]

    W. P. Ma, Z. L. Wen, Y. Wu, L. C. Jiao, M. G. Gong, Y. F. Zheng, L. Liu. Remote sensing image registration with modified SIFT and enhanced feature matching. IEEE Geoscience and Remote Sensing Letters, vol. 14, no. 1, pp. 3–7, 2017. DOI: https://doi.org/10.1109/LGRS.2016.2600858.

    Google Scholar 

  111. [111]

    Y. Y. Dong, W. L. Jiao, T. F. Long, L. F. Liu, G. J. He, C. J. Gong, Y. T. Guo. Local deep descriptor for remote sensing image feature matching. Remote Sensing, vol. 11, no. 4, Article number 430, 2019. DOI: https://doi.org/10.3390/rs11040430.

  112. [112]

    A. Sedaghat, M. Mokhtarzade, H. Ebadi. Uniform robust scale-invariant feature matching for optical remote sensing images. IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 11, pp. 4516–4527, 2011. DOI: https://doi.org/10.1109/TGRS.2011.2144607.

    Google Scholar 

  113. [113]

    A. Sedaghat, H. Ebadi. Remote sensing image matching based on adaptive binning SIFT descriptor. IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 10, pp. 5283–5293, 2015. DOI: https://doi.org/10.1109/TGRS.2015.2420659.

    Google Scholar 

  114. [114]

    J. T. Springenberg, A. Dosovitskiy, T. Brox, M. Riedmiller. Striving for simplicity: The all convolutional net. https://arxiv.org/abs/1412.6806, 2014.

  115. [115]

    X. F. Han, T. Leung, Y. Q. Jia, R. Sukthankar, A. C. Berg. MatchNet: Unifying feature and metric learning for patch-based matching. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Boston, USA, pp. 3279–3286, 2015. DOI: https://doi.org/10.1109/CVPR.2015.7298948.

    Google Scholar 

  116. [116]

    Z. Yang, T. Dan, Y. Yang. Multi-temperal remote sensing image registration using deep convolutional features. IEEE Access, vol. 6, pp. 38544–38555, 2018.

    Google Scholar 

  117. [117]

    Y. Yang, S. H. Ong, K. W. C. Foong. A robust global and local mixture distance based non-rigid point set registration. Pattern Recognition, vol. 48, no. 1, pp. 156–173, 2015. DOI: https://doi.org/10.1016/j.patcog.2014.06.017.

    Google Scholar 

  118. [118]

    S. Zhang, Y. Yang, K. Yang, Y. Luo, S. H. Ong. Point set registration with global-local correspondence and transformation estimation. In Proceedings of International Conference on Computer Vision, IEEE, Venice, Italy, pp. 2669–2677, 2017. DOI: https://doi.org/10.1109/ICCV.2017.291.

    Google Scholar 

  119. [119]

    S. Wang, D. Quan, X. F. Liang, M. D. Ning, Y. H. Guo, L. C. Jiao. A deep learning framework for remote sensing image registration. ISPRS Journal of Photogrammetry and Remote Sensing, vol. 145, pp. 148–164, 2018. DOI: https://doi.org/10.1016/j.isprsjprs.2017.12.012.

    Google Scholar 

  120. [120]

    D. Quan, S. Wang, X. F. Liang, R. J. Wang, S. Fang, B. Hou, L. C. Jiao. Deep generative matching network for optical and SAR image registration. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Valencia, Spain, pp. 6215–6218, 2018. DOI: https://doi.org/10.1109/IGARSS.2018.8518653.

    Google Scholar 

  121. [121]

    D. G. Kim, W. J. Nam, S. W. Lee. A robust matching network for gradually estimating geometric transformation on remote sensing imagery. In IEEE International Conference on Systems, Man and Cybernetics, IEEE, Bari, Italy, pp. 3889–3894, 2019. DOI: https://doi.org/10.1109/SMC.2019.8913881.

    Google Scholar 

  122. [122]

    M. Vakalopoulou, S. Christodoulidis, M. Sahasrabudhe, S. Mougiakakou, N. Paragios. Image registration of satellite imagery with deep convolutional neural networks. In Proceedings of IEEE International Geoscience and Remote Sensing Symposium, IEEE, Yokohama, Japan, pp. 4939–4942, 2019. DOI: https://doi.org/10.1109/IGARSS.2019.8898220.

    Google Scholar 

  123. [123]

    J. H. Park, W. J. Nam, S. W. Lee. A two-stream symmetric network with bidirectional ensemble for aerial image matching. Remote Sensing, vol. 12, no. 3, Article number 465, 2020. DOI: https://doi.org/10.3390/rs12030465.

  124. [124]

    J. Bromley, I. Guyon, Y. LeCun, E. Sackinger, R. Shah. Signature verification using a “Siamese” time delay neural network. In Proceedings of the 6th International Conference on Neural Information Processing Systems, Denver, USA, pp. 737–744, 1993.

  125. [125]

    L. H. Hughes, M. Schmitt, L. C. Mou, Y. Y. Wang, X. X. Zhu. Identifying corresponding patches in SAR and optical images with a pseudo-Siamese CNN. IEEE Geoscience and Remote Sensing Letters, vol. 15, no. 5, pp. 784–788, 2018. DOI: https://doi.org/10.1109/LGRS.2018.2799232.

    Google Scholar 

  126. [126]

    H. Zhang, W. P. Ni, W. D. Yan, D. L. Xiang, J. Z. Wu, X. L. Yang, H. Bian. Registration of multimodal remote sensing image based on deep fully convolutional neural network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 8, pp. 3028–3042, 2019. DOI: https://doi.org/10.1109/JSTARS.2019.2916560.

    Google Scholar 

  127. [127]

    A. Mishchuk, D. Mishkin, F. Radenovic, J. Matas. Working hard to know your neighbor’s margins: Local descriptor learning loss. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Long Beach, USA, pp. 4826–4837, 2017.

    Google Scholar 

  128. [128]

    C. Harris, M. Stephens. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, Alvey Vision Club, Manchester, UK, pp. 147–151, 1988.

    Google Scholar 

  129. [129]

    N. Merkle, W. J. Luo, S. Auer, R. Muller, R. Urtasun. Exploiting deep matching and sar data for the geo-localization accuracy improvement of optical satellite images. Remote Sensing, vol. 9, no. 6, Article number 586, 2017. DOI: https://doi.org/10.3390/rs9060586.

  130. [130]

    W. J. Luo, A. G. Schwing, R. Urtasun. Efficient deep learning for stereo matching. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 5695–5703, 2016. DOI: https://doi.org/10.1109/CVPR.2016.614.

    Google Scholar 

  131. [131]

    H. Q. He, M. Chen, T. Chen, D. J. Li. Matching of remote sensing images with complex background variations via Siamese convolutional neural network. Remote Sensing, vol. 10, no. 2, Article number 355, 2018. DOI: https://doi.org/10.3390/rs10020355.

  132. [132]

    M. A. Fischler, R. C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, vol. 24, no. 6, pp. 381–395, 1981. DOI: https://doi.org/10.1145/358669.358692.

    MathSciNet  Google Scholar 

  133. [133]

    H. Bay, T. Tuytelaars, L. Van Gool. SURF: Speeded up robust features. In Proceedings of the 9th European Conference on Computer Vision, Springer, Graz, Austria, pp. 404–417, 2006. DOI: https://doi.org/10.1007/11744023_32.

    Google Scholar 

  134. [134]

    J. M. Morel, G. S. Yu. ASIFT: A new framework for fully affine invariant image comparison. SIAM Journal on Imaging Sciences, vol. 2, no. 2, pp. 438–469, 2009. DOI: https://doi.org/10.1137/080732730.

    MathSciNet  MATH  Google Scholar 

Download references

Aknowledgements

This work was supported by National Natural Science Foundation of China (Nos. 61702392 and 61772393) and Key Research and Development Program of Shaanxi Province (Nos. 2018ZDXM-GY-045 and 2019JQ-189).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Mao-Guo Gong.

Additional information

Yue Wu received the B. Eng. circuits and systems Ph.D. degree in control science and engineering from Xidian University, China in 2011 and 2016, respectively. Since 2016, he has been a teacher with Xidian University. He is currently an associate professor with Xidian University. He has authored or co-authored more than 40 papers in refereed journals and proceedings.

His research interests include computer vision and computational intelligence.

Jun-Wei Liu is a master student in Key Laboratory of Big Data and Intelligent Vision, School of Computer Science and Technology, Xidian University, China.

His research interests include evolutionary computing and image registration.

Chen-Zhuo Zhu is a master student in Key Laboratory of Big Data and Intelligent Vision, School of Computer Science and Technology, Xidian University, China.

His research interests include deep learning and image registration.

Zhuang-Fei Bai is a master student in Key Laboratory of Big Data and Intelligent Vision, School of Computer Science and Technology, Xidian University, China.

His research interests include computer vision and remote sensing image understanding.

Qi-Guang Miao received the M. Eng. degree in computer science from Xidian University, China in 1996, received the Ph. D. degree in computer science from Xidian University, China in 2005. He is currently a professor with the School of Computer Science and Technology, Xidian University, China.

His research interests include intelligent image processing and multiscale geometric representations for images.

Wen-Ping Ma received the B.Sc. degree in computer science and technology and the Ph.D. degree in pattern recognition and intelligent systems from Xidian University, China in 2003 and 2008, respectively. Since 2006, she has been with the Key Laboratory of Intelligent Perception and Image Understanding of the Ministry of Education, Xidian University, where she is currently an associate professor. She has published more than 30 SCI papers in international academic journals, including the IEEE Transactions on Evolutionary Computation, IEEE Transactions on Image Processing, Information Sciences, Pattern Recognition, Applied Soft Computing, Knowledge-based Systems, Physica A — Statistical Mechanics and its Applications, and IEEE Geoscience and Remote Sensing Letters. She is a member of the Chinese Institute of Electronics and the China Computer Federation.

Her research interests include natural computing and intelligent image processing.

Mao-Guo Gong received the B.Sc. degree in electronic engineering and the Ph.D. degree in electronic science and technology from Xidian University, China in 2003 and 2009, respectively. Since 2006, he has been a teacher with Xidian University. In 2008 and 2010, he was promoted as an associate professor and as a full professor, respectively, both with exceptive admission. He is an executive committee member of the Chinese Association for Artificial Intelligence and a senior member of the Chinese Computer Federation. He received the prestigious National Program for the support of Top-Notch Young Professionals from the Central Organization Department of China, the Excellent Young Scientist Foundation from the National Natural Science Foundation of China, and the New Century Excellent Talent in University from the Ministry of Education of China. He is currently the vice-chair of the IEEE Computational Intelligence Society Task Force on Memetic Computing.

His research interests include computational intelligence with applications to optimization, learning, data mining, and image understanding.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.

The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Liu, JW., Zhu, CZ. et al. Computational Intelligence in Remote Sensing Image Registration: A survey. Int. J. Autom. Comput. 18, 1–17 (2021). https://doi.org/10.1007/s11633-020-1248-x

Download citation

Keywords

  • Computational intelligence
  • evolutionary computation
  • neural network
  • deep learning
  • remote sensing image registration