Advertisement

Image Retrieval Research Based on Significant Regions

  • Jie XuEmail author
  • Shuwei Sheng
  • Yuhao Cai
  • Yin Bian
  • Du Xu
Conference paper
Part of the Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering book series (LNICST, volume 262)

Abstract

Deep Convolution neural networks (CNN) has achieved great success in the field of image recognition. But in the image retrieval task, the global CNN features ignore local detail description for paying too much attention to semantic information of images. So the MAP of image retrieval remains to be improved. Aiming at this problem, this paper proposes a local CNN feature extraction algorithm based on image understanding, which includes three steps: significant regions extraction, significant regions description and pool coding. This method overcomes the semantic gap problem in traditional local characteristic and improves the retrieval effect of global CNN features. Then, we apply this local CNN feature in the image retrieval task, including the same category retrieval task by feature fusion strategy and the instance retrieval task by re-ranking strategy. The experimental results show that this method has achieved good performance on the Caltech 101 and Caltech 256 classification datasets, and competitive results on the Oxford 5k and Paris 6k instance retrieval datasets.

Keywords

Significant regions Image understanding CNN Image retrieval 

Notes

Acknowledgements

This work was supported by National Key Research and Development Program (Grant No. 2016YFB0800105), Sichuan Province Scientific and Technological Support Project (Grant Nos. 2016GZ0093, 2018GZ0255), the Fundamental Research Funds for the Central Universities (Grant No. ZYGX2015J009).

References

  1. 1.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: IEEE International Conference on Computer Vision (2001)Google Scholar
  2. 2.
    Sivic, J.: A text retrieval approach to object matching in videos. In: Proceedings of IEEE International Conference on Computer Vision (2003)Google Scholar
  3. 3.
    Perronnin, F., Liu, Y., Sánchez, J., Poirier, H.: Large-scale image retrieval with compressed Fisher vectors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Workshops, pp. 3384–3391 (2010)Google Scholar
  4. 4.
    Jegou, H., Perronnin, F., Douze, M., Sánchez, J., Perez, P.: Aggregating local image descriptors into compact codes. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1704–1716 (2012)CrossRefGoogle Scholar
  5. 5.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems (NIPS 2012), pp. 1097–1105 (2012)Google Scholar
  6. 6.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv:1409.1556
  7. 7.
    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Workshops, pp. 1–9 (2015)Google Scholar
  8. 8.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Proceedings of the European Conference on Computer Vision, pp. 818–833 (2014)Google Scholar
  9. 9.
    Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Workshops, pp. 1717–1724 (2014)Google Scholar
  10. 10.
    Hoang, T., Do, T.T., Tan, D.K.L., Cheung, N.M.: Selective deep convolutional features for image retrieval. In: ACM, pp. 1600–1608 (2017)Google Scholar
  11. 11.
    Xu, J., Shi, C.Z., Qi, C.Z., Wang, C.H., Xiao, B.H.: Unsupervised part-based weighting aggregation of deep convolutional features for image retrieval. In: AAAI2018 (2018)Google Scholar
  12. 12.
    Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Workshops, pp. 512–519 (2014)Google Scholar
  13. 13.
    Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: Proceedings of the European Conference on Computer Vision, pp. 584–599 (2014)Google Scholar
  14. 14.
    Lin, K., Yang, H.F., Hsiao, J.H., Chen, C.S.: Deep learning of binary hash codes for fast image retrieval. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Workshops, pp. 25–37 (2015)Google Scholar
  15. 15.
    Wang, J., Song, Y., Leung, T., Rosenberg, C., Wang, J.: Learning fine-grained image similarity with deep ranking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1386–1393 (2014)Google Scholar
  16. 16.
    Chen, Q., Huang, J., Feris, R., Brown, LM., Dong, J.: Deep domain adaptation for describing people based on fine-grained clothing attributes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5315–5324 (2015)Google Scholar
  17. 17.
    Mairal, J., Koniusz, P., Harchaoui, Z., Schmid, C.: Convolutional kernel networks. In: International Conference on Neural Information Processing Systems. MIT Press, pp. 2627–2635 (2014)Google Scholar
  18. 18.
    Paulin, M., Douze, M., Harchaoui, Z., Mairal, J., Perronin, F.: Local convolutional features with unsupervised training for image retrieval. In: IEEE International Conference on Computer Vision, pp. 91–99 (2015)Google Scholar
  19. 19.
    Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K.: Visual genome: connecting language and vision using crowdsourced dense image annotations. Int. J. Comput. Vis. 123(1), 32–73 (2017)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Kalantidis, Y., Mellina, C., Osindero, S.: Cross-dimensional weighting for aggregated deep convolutional features. In: Proceedings of the European Conference on Computer Vision. Workshops, pp. 685–701 (2016)Google Scholar
  21. 21.
    Tolias, G., Sicre, R., Jegou, H.: Particular object retrieval with integral maxpooling of CNN activations. In: Proceedings of the International Conference on Learning Representations, pp. 1–12 (2016)Google Scholar
  22. 22.
    Jegou, H., Zisserman, A.: Triangulation embedding and democratic aggregation for image search. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3310–3317 (2014)Google Scholar
  23. 23.
    Razavian, A.S., Sullivan, J., Maki, A., Carlsson, S.: Visual instance retrieval with deep convolutional networks (2014). arXiv:1412.6574

Copyright information

© ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2019

Authors and Affiliations

  • Jie Xu
    • 1
    Email author
  • Shuwei Sheng
    • 1
  • Yuhao Cai
    • 1
  • Yin Bian
    • 1
  • Du Xu
    • 1
  1. 1.School of Information and Communication EngineeringUniversity of Electronic Science and Technology of ChinaChengduChina

Personalised recommendations