Abstract
We present a novel, robust target detection method to locate a target from a reference image (UAV image) according to a target image (remote sensing satellite image). Using sparse parameterization diffeomorphic matching based on multiscale kernels, the approach modeling the nonrigid transformation function between the reference image and target image is proposed to complete target detection. Furthermore, it designs an feature point matching fusing intensity and phase information to determine the corresponding keypoints, which solves the cross-view problem. Then, the displacements of the corresponding keypoint sets are classified into several subsets using the probabilistic mixture model. The sparse parameterization diffeomorphic matching is executed in the subsets, removing the influence of outliers in the corresponding keypoints. The subset with the maximum evaluation for the transformation is utilized to locate the target. Finally, multiscale kernels based on sparse parameterization are integrated into diffeomorphic matching, solving the large deformation problems between target and reference images. The proposed approach incorporates the stationary velocity field into the diffeomorphism and utilizes the Lie group idea for the stationary velocity to trade off the matching accuracy and computational time. On the University-1652 image dataset with multi-view and multisource properties, experimental results show that the proposed approach is robust to noise and large deformations.
Similar content being viewed by others
References
Zamir A R, Shah M (2010) Accurate image localization based on google maps street view. In: European conference on computer vision, pp 255–268
Amir R Z, Mubarak S (2014) Image geo-localization based on multiplenearest neighbor feature matching usinggeneralized graphs. IEEE Trans Pattern Anal Mach Intell 36(8):1546–1588
Hays J, Efros A (2008) Im2gps: estimating geographic information from a single image. In: in IEEE Conference on computer vision and pattern recognition, pp 1–8
Schindler G, Brown M (2007) City-scale location recognition. In: in IEEE Conference on computer vision and pattern recognition, pp 1–7
Zhu P, Wen L (2018) Vision meets drones: A challenge. In: in European conference on computer vision, pp 437–468
Liu L, Li H (2019) Lending orientation to neural networks for cross-view geo-localization. In: in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 5617–5626
Uss M L, Voze B, Lukin VV, Chehdi K (2016) Efficient rotation-scaling-translation parameter estimation based on the fractal image model. IEEE Trans Geosci Remote Sens 54(1):197–212
Zemene E, Tesfaye Y T, Idrees H, Prati A, Pelillo M, Shah M (2019) Large-scale image geo-localization using dominant sets. IEEE Trans Pattern Anal Mach Intell 41(1):148–161. https://doi.org/10.1109/TPAMI.2017.2787132
Chen Y, Zheng W, Lai J, Yuen PC (2017) An asymmetric distance model for cross-view feature mapping in person reidentification. IEEE Trans Circ Syst Video Technol 27(8):1661–1675
Ben X, Gong C, Zhang P, Jia X, Wu Q, Meng W (2019) Coupled patch alignment for matching cross-view gaits. IEEE Trans Image Process 28(6):3142–3157
Liu L, Li H (2019) Lending orientation to neural networks for cross-view geo-localization in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 5617–5626
Yang Y, Chen Z, Li X, Guan W, Zhong D, Xu M (2020) Robust template matching with large angle localization. Neurocomputing 398:0925–2312
Guo H, Rangarajan A, Joshi S (2006) Diffeomorphic point matching, Handbook of Mathematical Models in Computer Vision, 205–219
Lowe D G (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60:91–110
Tan H, Ngo C (2009) Localized matching using earth mover’s distance towards discovery of common patterns from small image samples. Image Vis Comput 27(10):1470–1483
Aldana-Iuit J C O (2016) In the saddle: Chasing fast and repeatable features In Proceedings of the international conference on pattern recognition, pp 675–680
DeTone D, Malisiewicz T, Rabinovich A (2018) Superpoint: Self-supervised interest point detection and description. In: in IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp 337–349
Luo Z, Zhou L, Bai X, Chen H, Zhang J, Yao Y, Li S, Fang T, Quan L (2020) Aslfeat: Learning local features of accurate shape and localization. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 6588–6597
Li Z, Mahapatra D, Tielbeek J A, Stoker J (2015) Image registration based on autocorrelation of local structure. IEEE Trans Image Process 35:63–75
Cao S Y, Shen H L, Chen S J, Li C (2020) Boosting structure consistency for multispectral and multimodal image registration. IEEE Trans Image Process 29:5147–5162
Ma J, Jiang X, Fan A, Jiang J, Yan J (2020) Image matching from handcrafted to deep features: A survey. International Journal of Computer Vision, https://doi.org/10.1007/s11263-020-01359-2
Mirza F, Michael M, Alain T, Laurent Y (2005) Computing large deformation metric mapping via geodesic flows of diffeomorphisms. Int J Comput Vis 61:139–157
Krebs J, Delingette H, Mailhe B, Ayache N, Mansi T (2019) Learning a probabilistic model for diffeomorphic registration. IEEE Trans Med Imaging 38(9):2165–2176
Zhang M, Fletcher P (2015) Finite-dimensional lie algebras for fast diffeomorphic image registration, Springer International Publishing
Ivan K, Jehoon L, Gregory S, Patricio V, Allen T (2016) A stochastic approach to diffeomorphic point set registration with landmark constraints. IEEE Trans Pattern Anal Mach Intell 38:238–251
Vincent AR (2006) Processing data in lie groups: An algebraic approach. application to non-linear registration and diffusion tensor mri. INRIA Sophia-Antipolis Projet de recherche Asclepios, 181–222
Safdari M, Moallem P, Satari M (2019) Sift detector boosted by adaptive contrast threshold to improve matching robustness of remote sensing panchromatic images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12(2):675–684. https://doi.org/10.1109/JSTARS.2019.2892360
Zhang Z (1994) Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis 13(2):119–152
Almhdie A, Christophe L, Deriche M, Roger L (2007) 3d registration using a new implementation of the icp algorithm based on a comprehensive lookup matrix: Application to medical imaging. Pattern Recogn Lett 28(12):1523–1533
Dirk H, Thrun S, Burgard W (2003) An extension of the icp algorithm for modeling nonrigid objects with mobile robots. In: in Proceedings of the eighteenth international joint conference on artificial intelligence, pp 9–15
Sotiras A, Davatzikos C, Paragios N (2013) Deformable medical image registration: A survey. IEEE Trans Med Imaging 32(7):1153–1190
Arsigny V, Commowick O, Pennec X, Ayache N (2006) A log-euclidean framework for statistics on diffeomorphisms. In: International Conference on Medical Image Computing and Computer-Assisted Intervention
Vercauteren T, Pennec X, Perchant A, Ayache N (2008) Symmetric log-domain diffeomorphic registration: A demons-based approach
Avants B, Epstein C, Grossman M, Gee J (2008) Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal 12(1):26–41
Younes, Laurent (2007) Jacobi fields in groups of diffeomorphisms and applications. Q Appl Math 65(1):113–134
Ashburner J, Friston J (2007) Diffeomorphic registration using geodesic shooting and gauss-newton optimisation. NeuroImage 55(3):113–134
Vialard F, Risser L, Rueckert D, Ashburner J, Friston J (2010) Diffeomorphic 3d image registration via geodesic shooting using an efficient adjoint calculation
Singh N, Hinkle J, Joshi S, Fletcher P T (2013) A vector momenta formulation of diffeomorphisms for improved geodesic regression and atlas construction
Ruhnau P, Christoph S (2006) Optical stokes flow estimation: An imaging-based control approach. Exp Fluids 42:61–78
Hernandez M (2018) Band-limited stokes large deformation diffeomorphic metric mapping. IEEE J Biomed Health Inform 23:362–373
Hernandez M (2018) Newton-krylov optimization in pde-constrained diffeomorphic registration parameterized in the space of band-limited vector fields. arXiv:1807.05117
Ashburner J (2007) A fast diffeomorphic image registration algorithm. Neuroimage 38:95–113
Darkner S, Pai A, Liptrot MG, Sporring J (2018) Collocation for diffeomorphic deformations in medical image registration. IEEE Trans Pattern Anal Mach Intell 40:1570–1583
Reaungamornrat S, Silva T, Uneri A, Vogt S (2016) Mind demons: Symmetric diffeomorphic deformable registration of mr and ct for image-guided spine surgery. IEEE Trans Med Imaging 35(11):2413–2424
Bruveris M, Risser L, V. F (2012) Mixture of kernels and iterated semidirect product of diffeomorphisms groups. Siam Journal on Multiscale Modeling and Simulation 10(4):1–21
Sommer S, L F, Nielsen M, Pennec X (2013) Sparse multi-scale diffeomorphic registration: the kernel bundle framework. J Math Imaging Vision 46(3):292–308
Pai A, Sommer S, Srensen L, Darkner S, Sporring J, Nielsen M (2016) Kernel bundle diffeomorphic image registration using stationary velocity fields and wendland basis functions. IEEE Trans Med Imaging 35(6):1369
Mingzhen T, Anqi Q (2018) Multiscale frame-based kernels for large deformation diffeomorphic metric mapping. IEEE TRANSACTIONS ON MEDICAL IMAGING 37(10):2344–2355
Ren Z, Sun Q (2020) Simultaneous global and local graph structure preserving for multiple kernel clustering, IEEE Transactions on Neural Networks and Learning Systems, PP 99
Ren Z, Yang S X, Sun Q, Wang T (2020) Consensus affinity graph learning for multiple kernel clustering. IEEE Transactions on Cybernetics, PP 99
Rueckert D, Frangi J (2001) Automatic construction of 3d statistical deformation models using non-rigid registration, International Conference on Medical Image Computing and Computer-Assisted Intervention, 77–84
Marsland S, Twining C (2003) Constructing data-driven optimal representations for iterative pairwise non-rigid registration
Vaillant M, Miller MI, Younes A (2004) Statistics on diffeomorphisms via tangent space representations. NeuroImage 23(2004):S161–S169
Commowick O, Stefanescu R, Fillard P (2005) Incorporating statistical measures of anatomical variability in atlas-to-subject registration for conformal brain radiotherapy
Dalca A V, Balakrishnan G, Guttag J (2019) Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces. Med Image Anal 57:226–236
Morrone MC, Owens RA (1987) Feature detection from local energy. Pattern Recogn Lett 6(5):303–313. https://doi.org/10.1016/0167-8655(87)90013-4https://doi.org/10.1016/0167-8655(87)90013-4http://www.sciencedirect.com/science/article/pii/0167865587900134http://www.sciencedirect.com/science/article/pii/0167865587900134
P.D.Kovesi (2003) Phase congruency detects corners and edges. The Australian Pattern Recognition Society Conference APRSC2007:309–318. http://www.oalib.com/references/7047456
Jian B, Vemuri B (2011) Robust point set registration using gaussian mixture models. IEEE Trans Pattern Anal Mach Intell 33(8):1633–1645
Blei D M, Kucukelbir A, Mcauliffe J D (2016) Variational inference: A review for statisticians
Zheng Z, Yunchao W, Yi Y (2020) University-1652: A multi-view multi-source benchmark for drone-based geo-localization. arXiv:2002.12186
(2018) Bing maps, https://www.microsoft.com/en-us/maps
Yan P, Tan Y, Tai Y, Wu D, Hao X (2021) Unsupervised learning framework for interest point detection and description via properties optimization. Pattern Recogn 112(1–2):107808
Rublee E, Rabaud V, Konolige K, Bradski G R (2011November) Orb: an efficient alternative to sift or surf. In: IEEE International Conference on Computer Vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011, Barcelona, Spain
Hu S, Lee G H (2020) Image-based geo-localization using satellite imagery. Int J Comput Vis 128(5):1205–1219
Liu L, Li H (2019) Lending orientation to neural networks for cross-view geo-localization. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Zhu S, Yang T, Chen C (2021) Vigor: Cross-view image geo-localization beyond one-to-one retrieval. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 3640–3649
Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: Towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137–1149
Wang C Y, Bochkovskiy A, Liao H (2021) Scaled-yolov4: Scaling cross stage partial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 13029–13038
Acknowledgements
This work is supported by the Interdisciplinary Research Foundation of HIT under Grant No. IR2021104, Research Foundation of Science and Technology on Electro-Optical Information Security Control Laboratory under Grant No. 6142107200209, Aeronautical Science Foundation of China under Grant No. 2019ZC077006. Basic Scientific Research Business Expenses of Provincial Universities in Heilongjiang Province No. 2019-KYYWF-1384, and Excellent Discipline Team project no. JDXKTD-2019008.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, X., Yuan, D., Xue, K. et al. Diffeomorphic matching with multiscale kernels based on sparse parameterization for cross-view target detection. Appl Intell 53, 9689–9707 (2023). https://doi.org/10.1007/s10489-022-03668-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-022-03668-0