Skip to main content
SpringerLink
Log in
Book cover

International Conference on Multimedia Modeling

MMM 2020: MultiMedia Modeling pp 381–392Cite as

Instance Image Retrieval with Generative Adversarial Training

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11961))

Abstract

While generative adversarial training becomes promising technology for many computer vision tasks especially in image processing domain, it has few works so far on instance level image retrieval domain. In this paper, we propose an instance level image retrieval method with generative adversarial training (ILRGAN). In this proposal, adversarial training is adopted in the retrieval procedure. Both generator and discriminator are redesigned for retrieval task: the generator tries to retrieve similar images and passes them to the discriminator. And the discriminator tries to discriminate the dissimilar images from the images retrieved and then passes the decision to the generator. Generator and discriminator play min-max game until the generator retrieves images that the discriminator can not discriminate the dissimilar images. Experiments on four widely used databases show that adversarial training really works for instance level image retrieval and the proposed ILRGAN can get promising retrieval performances.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Yelamarthi, S.K., Reddy, S.K., Mishra, A., Mittal, A.: A zero-shot framework for sketch based image retrieval. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11208, pp. 316–333. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01225-0_19

    Chapter  Google Scholar 

  2. Babenko, A., Lempitsky, V.: Aggregating local deep features for image retrieval. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1269–1277 (2015)

    Google Scholar 

  3. Babenko, A., Slesarev, A., Chigorin, A., Lempitsky, V.: Neural codes for image retrieval. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 584–599. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_38

    Chapter  Google Scholar 

  4. Bell, S., Bala, K.: Learning visual similarity for product design with convolutional neural networks. ACM Trans. Graph. 34(4), 98:1–98:10 (2015)

    Article  Google Scholar 

  5. Chen, Z., Lin, J., Chandrasekhar, V., Duan, L.Y.: Gated square-root pooling for image instance retrieval. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 1982–1986. IEEE (2018)

    Google Scholar 

  6. Cong, B., Ling, H., Xiang, P., Zheng, J., Chen, S.: Optimization of deep convolutional neural network for large scale image retrieval. Neurocomputing 303, 60–67 (2018)

    Article  Google Scholar 

  7. Dizaji, K.G., Zheng, F., Nourabadi, N.S., Yang, Y., Deng, C., Huang, H.: Unsupervised deep generative adversarial hashing network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3664–3673 (2018)

    Google Scholar 

  8. Goodfellow, I., et al.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)

    Google Scholar 

  9. Gordo, A., Almazán, J., Revaud, J., Larlus, D.: Deep image retrieval: learning global representations for image search. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 241–257. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_15

    Chapter  Google Scholar 

  10. Gordo, A., Almazán, J., Revaud, J., Larlus, D.: End-to-end learning of deep visual representations for image retrieval. Int. J. Comput. Vis. 124(2), 237–254 (2017)

    Article  MathSciNet  Google Scholar 

  11. Guo, L., Liu, J., Wang, Y., Luo, Z., Wen, W., Lu, H.: Sketch-based image retrieval using generative adversarial networks. In: Proceedings of the ACM on Multimedia Conference, pp. 1267–1268 (2017)

    Google Scholar 

  12. Hoang, T., Do, T.T., Le Tan, D.K., Cheung, N.M.: Selective deep convolutional features for image retrieval. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1600–1608. ACM (2017)

    Google Scholar 

  13. Huang, L., Bai, C., Lu, Y., Chen, S., Tian, Q.: Adversarial learning for content-based image retrieval. In: 2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 97–102 (2019)

    Google Scholar 

  14. Huang, X., Peng, Y., Yuan, M.: MHTN: modal-adversarial hybrid transfer network for cross-modal retrieval. IEEE Trans. Cybern. 1–13 (2018). https://doi.org/10.1109/TCYB.2018.2879846

    Article  Google Scholar 

  15. Kalantidis, Y., Mellina, C., Osindero, S.: Cross-dimensional weighting for aggregated deep convolutional features. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9913, pp. 685–701. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46604-0_48

    Chapter  Google Scholar 

  16. Kim, J., Yoon, S.E.: Regional attention based deep feature for image retrieval. In: British Machine Vision Conference (BMVC). BMVA (2018)

    Google Scholar 

  17. Li, H., Sun, F., Liu, L., Ling, W.: A novel traffic sign detection method via color segmentation and robust shape matching. Neurocomputing 169, 77–88 (2015)

    Article  Google Scholar 

  18. Li, H., Wang, X., Tang, J., Zhao, C.: Combining global and local matching of multiple features for precise item image retrieval. Multimedia Syst. 19(1), 37–49 (2013)

    Article  Google Scholar 

  19. Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

  20. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  21. Lv, Y., Zhou, W., Tian, Q., Li, H.: Scalable bag of selected deep features for visual instance retrieval. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 239–251. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73600-6_21

    Chapter  Google Scholar 

  22. Mei, S., Min, W., Duan, H., Jiang, S.: Instance-level object retrieval via deep region CNN. Multimedia Tools Appl. 78(10), 13247–13261 (2018)

    Article  Google Scholar 

  23. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Lost in quantization: improving particular object retrieval in large scale image databases. In: 2008 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, June 2008

    Google Scholar 

  24. Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)

    Google Scholar 

  25. Radenović, F., Iscen, A., Tolias, G., Avrithis, Y., Chum, O.: Revisiting Oxford and Paris: large-scale image retrieval benchmarking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5706–5715 (2018)

    Google Scholar 

  26. Radenović, F., Tolias, G., Chum, O.: CNN image retrieval learns from bow: unsupervised fine-tuning with hard examples. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 3–20. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_1

    Chapter  Google Scholar 

  27. Rui, Y., Huang, T.S., Chang, S.F.: Image retrieval: current techniques, promising directions, and open issues. J. Vis. Commun. Image Represent. 10(1), 39–62 (1999)

    Article  Google Scholar 

  28. Sivic, Z.: Video Google: a text retrieval approach to object matching in videos. In: Proceedings Ninth IEEE International Conference on Computer Vision, vol. 2, pp. 1470–1477. IEEE (2003)

    Google Scholar 

  29. Song, J., He, T., Gao, L., Xu, X., Hanjalic, A., Shen, H.T.: Binary generative adversarial networks for image retrieval. In: AAAI Conference on Artificial Intelligence, pp. 394–401 (2018)

    Google Scholar 

  30. Tolias, G., Sicre, R., Jégou, H.: Particular object retrieval with integral max-pooling of CNN activations. In: International Conference on Learning Representations (ICRL), San Juan, Puerto Rico, pp. 1–12 (2016)

    Google Scholar 

  31. Xu, X., He, L., Lu, H., Gao, L., Ji, Y.: Deep adversarial metric learning for cross-modal retrieval. World Wide Web 22(2), 657–672 (2019)

    Article  Google Scholar 

  32. Zheng, L., Yang, Y., Tian, Q.: SIFT meets CNN: a decade survey of instance retrieval. IEEE Trans. Pattern Anal. Mach. Intell. 40(5), 1224–1244 (2017)

    Article  Google Scholar 

Download references

Acknowledgement

This work is funded by Zhejiang Provincial Natural Science Foundation of China under Grant No. LY18F020032, Natural Science Foundation of China under Grant No. 61976192, 61502424 and U1509207. The source code of this work will be released on http://www.escience.cn/people/congbai/index.html.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China

    Hongkai Li, Cong Bai & Ling Huang

  2. College of Computer Science, Fudan University, Shanghai, China

    Yugang Jiang

  3. College of Computer Science and Engineering, Tianjin University of Technology, Tianjin, China

    Shengyong Chen

Authors
  1. Hongkai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Cong Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ling Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yugang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shengyong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Bai .

Editor information

Editors and Affiliations

  1. Korea Advanced Institute of Science and, Daejeon, Korea (Republic of)

    Yong Man Ro

  2. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  3. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Junmo Kim

  4. National Cheng Kung University, Tainan City, Taiwan

    Wei-Ta Chu

  5. Tsinghua University, Beijing, China

    Peng Cui

  6. Korea Advanced Institute of Science and Technology, Daejeon, Korea (Republic of)

    Jung-Woo Choi

  7. National Tsing Hua University, Hsinchu, Taiwan

    Min-Chun Hu

  8. Ghent University, Ghent, Belgium

    Wesley De Neve

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, H., Bai, C., Huang, L., Jiang, Y., Chen, S. (2020). Instance Image Retrieval with Generative Adversarial Training. In: Ro, Y., et al. MultiMedia Modeling. MMM 2020. Lecture Notes in Computer Science(), vol 11961. Springer, Cham. https://doi.org/10.1007/978-3-030-37731-1_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37731-1_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37730-4

  • Online ISBN: 978-3-030-37731-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation