Hard Negative Mining for Metric Learning Based Zero-Shot Classification

  • Maxime Bucher
  • Stéphane Herbin
  • Frédéric Jurie
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9915)


Zero-Shot learning has been shown to be an efficient strategy for domain adaptation. In this context, this paper builds on the recent work of Bucher et al. [1], which proposed an approach to solve Zero-Shot classification problems (ZSC) by introducing a novel metric learning based objective function. This objective function allows to learn an optimal embedding of the attributes jointly with a measure of similarity between images and attributes. This paper extends their approach by proposing several schemes to control the generation of the negative pairs, resulting in a significant improvement of the performance and giving above state-of-the-art results on three challenging ZSC datasets.


Domain adaptation Zero-shot learning Hard negative mining Bootstrapping 


  1. 1.
    Bucher, M., Herbin, S., Jurie, F.: Improving Semantic Embedding Consistency by Metric Learning for Zero-Shot Classiffication. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9909, pp. 730–746. Springer, Heidelberg (2016). doi: 10.1007/978-3-319-46454-1_44 CrossRefGoogle Scholar
  2. 2.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Attribute-based classification for zero-shot visual object categorization. IEEE Trans. Pattern Anal. Mach. Intell. 36(3), 453–465 (2014)CrossRefGoogle Scholar
  3. 3.
    Akata, Z., Reed, S., Walter, D., Lee, H., Schiele, B.: Evaluation of output embeddings for fine-grained image classification. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  4. 4.
    Romera-Paredes, B., Torr, P.H.: An embarrassingly simple approach to zero-shot learning. In: ICML, pp. 2152–2161(2015)Google Scholar
  5. 5.
    Zhang, Z., Saligrama, V.: Zero-shot learning via semantic similarity embedding. In: IEEE International Conference on Computer Vision (ICCV) (2015)Google Scholar
  6. 6.
    Zhang, Z., Saligrama, V.: Zero-shot learning via joint latent similarity embedding. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6034–6042 (2016)Google Scholar
  7. 7.
    Wang, Q., Chen, K.: Zero-shot visual recognition via bidirectional latent embedding. arXiv preprint arXiv:1607.02104 (2016)
  8. 8.
    Xian, Y., Akata, Z., Sharma, G., Nguyen, Q., Hein, M., Schiele, B.: Latent embeddings for zero-shot classification. arXiv preprint arXiv:1603.08895 (2016)
  9. 9.
    Fu, Y., Zhu, X., Li, B.: A survey on instance selection for active learning. Knowl. Inf. Syst. 35(2), 249–283 (2013)CrossRefGoogle Scholar
  10. 10.
    Shrivastava, A., Gupta, A., Girshick, R.: Training region-based object detectors with online hard example mining. arXiv preprint arXiv:1604.03540 (2016)
  11. 11.
    Li, X., Snoek, C.M., Worring, M., Koelma, D., Smeulders, A.W.: Bootstrapping visual categorization with relevant negatives. IEEE Trans. Multimed. 15(4), 933–945 (2013)CrossRefGoogle Scholar
  12. 12.
    Canévet, O., Fleuret, F.: Efficient sample mining for object detection. In: ACML (2014)Google Scholar
  13. 13.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  14. 14.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  15. 15.
    Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 dataset. Technical report, July 2011Google Scholar
  16. 16.
    Patterson, G., Xu, C., Su, H., Hays, J.: The SUN attribute database: beyond categories for deeper scene understanding. Int. J. Comput. Vis. 108(1–2), 59–81 (2014)CrossRefGoogle Scholar
  17. 17.
    Jayaraman, D., Grauman, K.: Zero-shot recognition with unreliable attributes. In: Conference on Neural Information Processing Systems (NIPS) (2014)Google Scholar
  18. 18.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: ICLR (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Maxime Bucher
    • 1
    • 2
  • Stéphane Herbin
    • 1
  • Frédéric Jurie
    • 2
  1. 1.ONERA - The French Aerospace LabPalaiseauFrance
  2. 2.Normandie Univ, UNICAEN, ENSICAEN, CNRSCaenFrance

Personalised recommendations