Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A privacy-preserving image retrieval method based on deep learning and adaptive weighted fusion

  • 126 Accesses

  • 1 Citations


With the development of big data and cloud computing, more and more data owners store the data in cloud server. Considering privacy preserving, image data need to be encrypted before uploaded to the cloud, which will lead to inefficient image retrieval of ciphertext domain. Therefore, the challenge of encrypted image retrieval is how to improve the performance. Toward this goal, this paper proposes a privacy-preserving image retrieval method based on deep learning and adaptive weighted fusion. Firstly, extracting low-level feature EHD (edge histogram descriptor), BOW (bag of words) and high-level semantic feature of images. Secondly, reducing the dimension of 1024-dim high-level semantic feature by PCA (principal component analysis), and the three features were binarized. Then these types of features are adaptively fused. Finally, constructing a prefilter table for fusion features to improve search efficiency by locality sensitive hashing (LSH) algorithm. K-nearest neighbor (KNN) algorithm and logistic encryption method were used to protect the privacy of fused features and images, respectively. The experiments show that the proposed method can not only ensure image security but also improve the retrieval accuracy of encrypted image.

This is a preview of subscription content, log in to check access.

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10


  1. 1.

    Xia, Z.H., Zhang, L.G., Liu, D.D.: Attribute-based access control scheme with efficient revocation in cloud computing. China Commun. 13(7), 92–99 (2016)

  2. 2.

    Qi, L.Y., Meng, S.M., Zhang, X.Y., Wang, R.L., Xu, X.L., Zhou, Z.L., Dou, W.H.: An exception handling approach for privacy-preserving service recommendation failure in a cloud environment. Sensors 18(7), 1–11 (2018)

  3. 3.

    Qi, L.Y., Yu, J.G., Zhou, Z.L.: An invocation cost optimization method for web services in cloud environment. Sci. Program. 2017, 4358536–4358544 (2017)

  4. 4.

    Xia, Z.H., Zhu, Y., Sun, X.M., Chen, L.: Secure semantic expansion based search over encrypted cloud data supporting similarity ranking. J. Cloud Comput. 3(1), 1–11 (2014)

  5. 5.

    Fu, Z., Wu, X., Guan, C., Sun, X.M., Ren, K.: Toward efficient multi-keyword fuzzy search over encrypted outsourced data with accuracy improvement. IEEE Trans. Inf. Forensics Secur. 11(12), 2706–2716 (2016)

  6. 6.

    Lu, W., Varna, A. L., Swaminathan, A., Wu, M.: Secure image retrieval through feature protection. In: Proceedings of IEEE international conference on acoustics, speech and signal processing, pp. 1533–1536 (2009)

  7. 7.

    Hsu, C.Y., Lu, C.S., Pei, S.C.: Image feature extraction in encrypted domain with privacy-preserving SIFT. IEEE Trans. Image Process. 21(11), 4593–4607 (2012)

  8. 8.

    Bai, Y., Zhuo, L., Cheng, B., Peng, Y. F.: Surf feature Extraction in encrypted domain. In: 2014 IEEE international conference on multimedia and expo, pp. 1–6 (2014)

  9. 9.

    Lu, W., Varna, A.L., Wu, M.: Confidentiality-preserving image search: a comparative study between homomorphic encryption and distance-preserving randomization. IEEE Access. 2(1), 125–141 (2014)

  10. 10.

    Cheng, H., Zhang, X., Yu, J., Li, F.: Markov process-based retrieval for encrypted jpeg images. Eurasip J. Inf. Security. 2016(1), 1–9 (2016)

  11. 11.

    Xia, Z.H., Zhu, Y., Sun, X.M., Qin, Z., Ren, K.: Towards privacy-preserving content-based image retrieval in cloud computing. IEEE Trans. Cloud Comput. 6(1), 276–286 (2018)

  12. 12.

    Xia, Z.H., Xiong, N., Vasilakos, V., Sun, X.M.: EPCBIR: an efficient and privacy-preserving content-based image retrieval scheme in cloud computing. Inf. Sci. 387, 195–204 (2017)

  13. 13.

    Qin, J.H., Li, H., Xiang, X.Y., Tan, Y., Pan, W.Y., Xiong, N.N.: An encrypted image retrieval method based on Harris Corner optimization and LSH in cloud computing. IEEE Access. 7(1), 24626–24633 (2018)

  14. 14.

    Qi, L.Y., Zhang, X.Y., Dou, W.C., Hu, C.H., Yang, C., Chen, Jinjun: A two-stage locality-sensitive hashing based approach for privacy-preserving mobile service recommendation in Cross-platform edge environment. Future Gener. Comput. Syst. 88, 636–643 (2018)

  15. 15.

    Xu, Y.W., Qi, L.Y., Dou, W.C., Yu, J.G.: Privacy-preserving and scalable service recommendation based on SimHash in a distributed cloud environment. Complexity 2017, 3437854–3437862 (2017)

  16. 16.

    Xiang, L.Y., Shen, X.B., Qin, J.H., Hao, W.: Discrete multi-graph hashing for large-scale visual search. Neural Process. Lett. 49(3), 1055–1069 (2019)

  17. 17.

    Li, H., Qin, J.H., Xiang, X.Y., Pan, L.L., Ma, W.T., Xiong, N.N.: An efficient image matching algorithm based on adaptive threshold and RANSAC. IEEE Access. 6(1), 66963–66971 (2018)

  18. 18.

    Liu, X.W., Wang, L., Zhang, J., Yin, J.P., Liu, H.: Global and local structure preservation for feature selection. IEEE Trans. Neural Netw. Learn. Syst. 25(6), 1083–1095 (2014)

  19. 19.

    Ma, W.T., Qin, J.H., Xiang, X.Y., Tan, Y., Luo, Y.J., Xiong, N.N.: Adaptive median filtering algorithm based on divide and conquer and its application in CAPTCHA recognition. Comput. Mater. Continua. 58(3), 665–677 (2019)

  20. 20.

    Liu, Z., Blasch, E., John, V.: Statistical comparison of image fusion algorithms: recommendations. Inf. Fusion. 36, 251–260 (2017)

  21. 21.

    Xie, B., Qin, J.H., Xiang, X.Y., Li, H., Pan, L.L.: An image retrieval algorithm based on GIST and SIFT features. Int. J. Netw. Secur. 20(4), 609–616 (2018)

  22. 22.

    Qin, J.H., Sun, X.M., Xiang, X.Y., Niu, C.M.: Principal feature selection and fusion method for image steganalysis. J. Electron. Imaging. 18(3), 1–14 (2009)

  23. 23.

    Yu, S., Niu, D., Zhao, X., Liu, M.: Color image retrieval based on the hypergraph and the fusion of two descriptors. In: Proceedings of the 10th international congress on image and signal processing, BioMedical engineering and informatics, pp. 1–6 (2017)

  24. 24.

    Zheng, L., Wang, S., Tian, L., He, F., Liu, Z., Tian, Q.: Query-adaptive late fusion for image search and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1741–1750 (2015)

  25. 25.

    Zhu, X., Jing, X.Y., Wu, F., Wang, Y., Zuo, W., Zheng, W.S.: Learning heterogeneous dictionary pair with feature projection matrix for pedestrian video retrieval via single query image. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, pp. 4341–4348 (2017)

  26. 26.

    Zhang, J.M., Wu, Y., Feng, W.J., Wang, J.: Spatially attentive visual tracking using multi-model adaptive response fusion. IEEE Access. 7(1), 83873–83887 (2019)

  27. 27.

    Pan, L.L., Qin, J.H., Chen, H., Xiang, X.Y., Li, C., Chen, R.: Image augmentation-based food recognition with convolutional neural networks. Comput. Mater. Continua. 59(1), 297–313 (2019)

  28. 28.

    Pan, W.Y., Qin, J.H., Xiang, X.Y., Wu, Y., Tan, Y., Xiang, L.Y.: A smart mobile diagnosis system for citrus diseases based on densely connected convolutional networks. IEEE Access. 7, 87534–87542 (2019)

  29. 29.

    Zhang, J.M., Lu, C.Q., Li, X.D., Kim, H.J., Wang, J.: A full convolutional network based on DenseNet for remote sensing scene classification. Math. Biosci. Eng. 16(5), 3345–3367 (2019)

  30. 30.

    Wang, J., Qin, J.H., Xiang, X.Y., Tan, Y., Pan, N.: CAPTCHA recognition based on deep convolutional neural network. Math. Biosci. Eng. 16(5), 5851–5861 (2019)

  31. 31.

    Zhang, J.M., Jin, X.K., Sun, J., Wang, J., Sangaiah, A.K.: Spatial and semantic convolutional features for robust visual object tracking. Multimedia Tools Appl. (2018). https://doi.org/10.1007/s11042-018-6562-8

  32. 32.

    Zhang, J.M., Lu, C.Q., Wang, J., Wang, L., Yue, X.G.: Concrete cracks detection based on FCN with dilated convolution. Appl. Sci. 9(13), 2686 (2019)

  33. 33.

    Zhang, J.M., Jin, X.K., Sun, J., Wang, J., Li, K.Q.: Dual model learning combined with multiple feature selection for accurate visual tracking. IEEE Access. 7(1), 43956–43969 (2019)

  34. 34.

    Datar, M., Nicole, I., Piotr, I., Vahab, S.M.: Locality-sensitive hashing scheme based on p-stable distributions. In: Proceedings of the twentieth annual symposium on Computational geometry, pp. 253–262 (2004)

  35. 35.

    Delfs, H., Knebl, H.: Introduction to cryptography: principles and applications. Inf. Secur. Cryptogr. Texts Monogr. 4(4), 206–207 (2015)

  36. 36.

    Wang, J.Z., Li, J., Wiederhold, G.: Simplicity: semantics-sensitive integrated matching for picture libraries. IEEE Trans. Pattern Anal. Mach. Intell. 23(9), 947–963 (2007)

  37. 37.

    Zhou, Z.L., Wu, J.N., Sun, X.M.: Multiple distances-based coding: toward scalable feature matching for large-scale web image search. IEEE Trans. Big Data. (2019). https://doi.org/10.1109/TBDATA.2019.2919570

Download references


This work is supported by the National Natural Science Foundation of China (no. 61772561), the Key Research & Development Plan of Hunan Province (nos. 2018NK2012, 2019SK2022), the Science Research Projects of Hunan Provincial Education Department (nos. 18A174 and 18C0262), and the Science & Technology Innovation Platform and Talent Plan of Hunan Province (2017TP1022).

Author information

Correspondence to Jiaohua Qin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Qin, J., Chen, J., Xiang, X. et al. A privacy-preserving image retrieval method based on deep learning and adaptive weighted fusion. J Real-Time Image Proc 17, 161–173 (2020). https://doi.org/10.1007/s11554-019-00909-3

Download citation


  • Image retrieval
  • Adaptive weighted fusion
  • PCA dimension reduction
  • Locality-sensitive hashing
  • DenseNet