Unsupervised Deep Features for Privacy Image Classification

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11854)


Sharing images online poses security threats to a wide range of users due to the unawareness of privacy information. Deep features have been demonstrated to be a powerful representation for images. However, deep features usually suffer from the issues of a large size and requiring a huge amount of data for fine-tuning. In contrast to normal images (e.g., scene images), privacy images are often limited because of sensitive information. In this paper, we propose a novel approach that can work on limited data and generate deep features of smaller size. For training images, we first extract the initial deep features from the pre-trained model and then employ the K-means clustering algorithm to learn the centroids of these initial deep features. We use the learned centroids from training features to extract the final features for each testing image and encode our final features with the triangle encoding. To improve the discriminability of the features, we further perform the fusion of two proposed unsupervised deep features obtained from different layers. Experimental results show that the proposed features outperform state-of-the-art deep features, in terms of both classification accuracy and testing time.


Privacy images Unsupervised deep features Image classification ResNet-50 Privacy and security 


  1. 1.
    Chollet, F.: Xception: deep learning with depthwise separable convolutions. arXiv preprint, pp. 1610–02357 (2017)Google Scholar
  2. 2.
    Chollet, F., Allaire, J.: R interface to Keras.
  3. 3.
    Coates, A., Ng, A., Lee, H.: An analysis of single-layer networks in unsupervised feature learning. In: AISTATS, pp. 215–223 (2011)Google Scholar
  4. 4.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: ImageNet: a large-scale hierarchical image database. In: CVPR (2009)Google Scholar
  5. 5.
    Fan, R.E., Chang, K.W., Hsieh, C.J., Wang, X.R., Lin, C.J.: Liblinear: a library for large linear classification. J. Mach. Learn. Res. 9, 1871–1874 (2008)zbMATHGoogle Scholar
  6. 6.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770–778 (2016)Google Scholar
  7. 7.
    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.: Densely connected convolutional networks. In: CVPR, vol. 1, p. 3 (2017)Google Scholar
  8. 8.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1097–1105 (2012)Google Scholar
  9. 9.
    Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear CNN models for fine-grained visual recognition. In: ICCV, pp. 1449–1457 (2015)Google Scholar
  10. 10.
    Lin, T.Y., RoyChowdhury, A., Maji, S.: Bilinear convolutional neural networks for fine-grained visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 40(6), 1309–1322 (2017)CrossRefGoogle Scholar
  11. 11.
    R Core Team: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria (2014)Google Scholar
  12. 12.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
  13. 13.
    Sitaula, C., Xiang, Y., Basnet, A., Aryal, S., Lu, X.: Tag-based semantic features for scene image classification. arXiv preprint arXiv:1909.09999 (2019)
  14. 14.
    Sitaula, C., Xiang, Y., Zhang, Y., Lu, X., Aryal, S.: Indoor image representation by high-level semantic features. IEEE Access 7, 84967–84979 (2019)CrossRefGoogle Scholar
  15. 15.
    Spyromitros-Xioufis, E., Papadopoulos, S., Popescu, A., Kompatsiaris, Y.: Personalized privacy-aware image classification. In: ICMR, pp. 71–78 (2016)Google Scholar
  16. 16.
    Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: AAAI, vol. 4, p. 12 (2017)Google Scholar
  17. 17.
    Szegedy, C., et al.: Going deeper with convolutions. In: CVPR, pp. 1–9 (2015)Google Scholar
  18. 18.
    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision. In: CVPR, pp. 2818–2826 (2016)Google Scholar
  19. 19.
    Tonge, A., Caragea, C.: Privacy prediction of images shared on social media sites using deep features. arXiv preprint arXiv:1510.08583 (2015)
  20. 20.
    Tonge, A., Caragea, C.: On the use of deep features for online image sharing. In: Companion of the The Web Conference, pp. 1317–1321 (2018)Google Scholar
  21. 21.
    Tonge, A., Caragea, C., Squicciarini, A.C.: Uncovering scene context for predicting privacy of online shared images. In: AAAI, pp. 8167–8168 (2018)Google Scholar
  22. 22.
    Tonge, A.K., Caragea, C.: Image privacy prediction using deep features. In: AAAI, pp. 4266–4267 (2016)Google Scholar
  23. 23.
    Tran, L., Kong, D., Jin, H., Liu, J.: Privacy-CNH: a framework to detect photo privacy with convolutional neural network using hierarchical features. In: AAAI, pp. 1317–1323 (2016)Google Scholar
  24. 24.
    Yang, J., Yang, J., Zhang, D., Lu, J.F.: Feature fusion: parallel strategy vs. serial strategy. Pattern Recognit. 36(6), 1369–1381 (2003)CrossRefGoogle Scholar
  25. 25.
    Zerr, S., Siersdorfer, S., Hare, J.: Picalert!: a system for privacy-aware image classification and retrieval. In: CIKM, pp. 2710–2712. ACM (2012)Google Scholar
  26. 26.
    Zerr, S., Siersdorfer, S., Hare, J., Demidova, E.: Privacy-aware image classification and search. In: SIGIR, pp. 35–44 (2012)Google Scholar
  27. 27.
    Zhong, H., Squicciarini, A., Miller, D., Caragea, C.: A group-based personalized model for image privacy classification and labeling. In: IJCAI, pp. 3952–3958 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Information TechnologyDeakin UniversityGeelongAustralia

Personalised recommendations