Abstract
Transfer learning is a challenging task in computer vision, due to the differences of data distribution. Although convolutional neural network (CNN) could learn different levels of image abstraction, the single-view features extracted from the final layer of a pre-trained or fine-tuned CNN may result in insufficient image description over different datasets. To address this issue, we focus on deep representations with multi-view analysis for remote sensing image (RSI) retrieval and classification tasks using five recently released large-scale datasets. First, cross-dataset transfer learning is presented by fine-tuning a pre-trained CNN on one dataset and testing the fine-tuned network on another one. Second, multi-view image representations are explored in terms of different activation vectors as well as CNNs. Finally, a multi-view fusion and a random projection (RP) strategy are proposed to improve the accuracies and computational cost of both RSI tasks, respectively. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bingham E, Mannila H (2001) Random projection in dimensionality reduction: applications to image and text data[C]. In: Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 245–250
Cao G, Iosifidis A, Gabbouj M et al (2019) Deep multi-view learning to rank[J]. IEEE Trans Knowl Data Eng
Cheng G, Han J, Lu X (2017) Remote sensing image scene classification: Benchmark and state of the art[J]. Proc IEEE 105(10):1865–1883
Davis J, Goadrich M (2006) The relationship between Precision-Recall and ROC curves[C]. In: Proceedings of the 23rd international conference on Machine learning. ACM, pp 233–240
Jia Y, Shelhamer E, Donahue J et al (2014) Caffe: Convolutional architecture for fast feature embedding[C]. In: Proceedings of the 22nd ACM international conference on Multimedia. ACM, pp 675–678
Johnson WB, Lindenstrauss J (1984) Extensions of Lipschitz mappings into a Hilbert space[J]. Contemp Math 26(189-206):1
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks[C]. Advances in neural information processing systems, pp 1097–1105
Li H, Tao C, Wu Z et al (2017) Rsi-cb: A large scale remote sensing image classification benchmark via crowdsource data[J]. arXiv preprint arXiv 1705.10450
Li J, Zhang B, Lu G et al (2019) Generative multi-view and multi-feature learning for classification[J]. Information Fusion 45:215–226
Liu N, Wan L, Zhang Y et al (2018) Exploiting convolutional neural networks with deeply local description for remote sensing image classification [J]. IEEE access 6:11215–11228
Lu X, Ma C, Ni B et al (2018) Deep regression tracking with shrinkage loss[C]. Proceedings of the European Conference on Computer Vision (ECCV):353–369
Lu X, Guo Y, Liu N et al (2018) Non-convex joint bilateral guided depth upsampling[J]. Multimed Tools Appl 77(12):15521–15544
Lu X, Ni B, Ma C et al (2019) Learning transform-aware attentive network for object tracking [J]. Neurocomputing 349:133–144
Lu X, Wang W, Ma C et al (2019) See More, Know More: Unsupervised Video Object Segmentation with Co-Attention Siamese Networks[C]. Proc IEEE Conf Comput Vis Pattern Recognit:3623–3632
Martnez AM, Kak AC (2001) Pca versus lda[J]. IEEE Trans Pattern Anal Mach Intell 23(2):228–233
Nogueira K, Penatti OAB, dos Santos JA (2017) Towards better exploiting convolutional neural networks for remote sensing scene classification [J]. Pattern Recogn 61:539–556
Penatti OAB, Nogueira K, Dos Santos JA (2015) Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 44–51
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556
Su H, Maji S, Kalogerakis E et al (2015) Multi-view convolutional neural networks for 3d shape recognition[C]. In: Proceedings of the IEEE international conference on computer vision, pp 945–953
Sun S (2013) A survey of multi-view machine learning[J]. Neural Comput & Applic 23(7-8):2031–2038
Vedaldi A, Lenc K (2015) Matconvnet: Convolutional neural networks for matlab[C]. In: Proceedings of the 23rd ACM international conference on Multimedia. ACM, pp 689–692
Xia GS, Hu J, Hu F et al (2017) AID: A benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Trans Geosci Remote Sens 55(7):3965–3981
Yang Y, Newsam S (2010) Bag-of-visual-words and spatial extensions for land-use classification[C]. In: Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems. ACM, pp 270–279
Zhou W, Newsam S, Li C et al (2017) Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval[J]. Remote Sens 9(5):489
Zhou W, Newsam S, Li C et al (2018) PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval [J]. ISPRS J Photogramm Remote Sens 145:197–209
Zhou T, Zhang C, Gong C et al (2018) Multiview latent space learning with feature redundancy minimization[J]. IEEE transactions on cybernetics
Zhou T, Zhang C, Peng X et al (2019) Dual shared-specific multiview subspace clustering[J]. IEEE transactions on cybernetics
Zhou T, Liu M, Thung KH et al (2019) Latent representation learning for Alzheimer’s disease diagnosis with incomplete multi-modality neuroimaging and genetic data[J]. IEEE transactions on medical imaging 38(10):2411–2422
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, N., Wan, L., Huang, Q. et al. Multi-view Deep Representations with Cross-Dataset Transfer for Remote Sensing Image Retrieval and Classification. Multimed Tools Appl 80, 22891–22905 (2021). https://doi.org/10.1007/s11042-020-08712-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-08712-0