Advertisement

Localizing and Visualizing Relative Attributes

  • Fanyi Xiao
  • Yong Jae Lee
Chapter
Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)

Abstract

In this chapter, we present a weakly supervised approach that discovers the spatial extent of relative attributes, given only pairs of ordered images. In contrast to traditional approaches that use global appearance features or rely on keypoint detectors, our goal is to automatically discover the image regions that are relevant to the attribute, even when the attribute’s appearance changes drastically across its attribute spectrum. To accomplish this, we first develop a novel formulation that combines a detector with local smoothness to discover a set of coherent visual chains across the image collection. We then introduce an efficient way to generate additional chains anchored on the initial discovered ones. Finally, we automatically identify the visual chains that are most relevant to the attribute (those whose appearance has high correlation with attribute strength), and create an ensemble image representation to model the attribute. Through extensive experiments, we demonstrate our method’s promise relative to several baselines in modeling relative attributes.

Keywords

Visual Concept Attribute Spectrum Facial Landmark Spatial Pyramid Smoothness Term 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgements

The work presented in this chapter was supported in part by an Amazon Web Services Education Research Grant and GPUs donated by NVIDIA.

References

  1. 1.
    Bengio, Y., Louradour, J., Collobert, R., Weston, J.: Curriculum learning. In: International Conference on Machine Learning (ICML) (2009)Google Scholar
  2. 2.
    Berg, T.L., Berg, A.C., Shih, J.: Automatic attribute discovery and characterization from noisy web data. In: European Conference on Computer Vision (ECCV) (2010)Google Scholar
  3. 3.
    Bourdev, L., Maji, S., Malik, J.: Describing people: poselet-based approach to attribute classification. In: International Conference on Computer Vision (ICCV) (2011)Google Scholar
  4. 4.
    Branson, S., Wah, C., Schroff, F., Babenko, B., Welinder, P., Perona, P., Belongie, S.: Visual recognition with humans in the loop. In: European Conference on Computer Vision (ECCV) (2010)Google Scholar
  5. 5.
    Chaudhuri, S., Kalogerakis, E., Giguere, S., Funkhouser, T.: Attribit: content creation with semantic attributes. In: ACM User Interface Software and Technology Symposium (UIST) (2013)Google Scholar
  6. 6.
    Doersch, C., Singh, S., Gupta, A., Sivic, J., Efros, A.A.: What makes Paris look like Paris? ACM Trans. Graph. (SIGGRAPH) 31(4), 101:1–101:9 (2012)Google Scholar
  7. 7.
    Duan, K., Parikh, D., Crandall, D., Grauman, K.: Discovering localized attributes for fine-grained recognition. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  8. 8.
    Faktor, A., Irani, M.: “Clustering by composition”—unsupervised discovery of image categories. In: European Conference on Computer Vision (ECCV) (2012)Google Scholar
  9. 9.
    Farhadi, A., Endres, I., Hoiem, D.: Attribute-centric recognition for cross-category generalization. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2010)Google Scholar
  10. 10.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  11. 11.
    Felzenszwalb, P.F., Girshick, R.B., McAllester, D., Ramanan, D.: Object detection with discriminatively trained part based models. IEEE Trans. Pattern Anal. Mach. Intell. (PAMI) 32(9), 1627–1645 (2010)CrossRefGoogle Scholar
  12. 12.
    Ferrari, V., Zisserman, A.: Learning visual attributes. In: Conference on Neural Information Processing Systems (NIPS) (2008)Google Scholar
  13. 13.
    Grauman, K., Darrell, T.: Unsupervised learning of categories from sets of partially matching image features. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2006)Google Scholar
  14. 14.
    Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R.B., Guadarrama, S., Darrell, T.: Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)
  15. 15.
    Joachims, T.: Optimizing search engines using clickthrough data. In: Knowledge Discovery in Databases (PKDD) (2002)Google Scholar
  16. 16.
    Kiapour, M., Yamaguchi, K., Berg, A.C., Berg, T.L.: Hipster wars: discovering elements of fashion styles. In: European Conference on Computer Vision (ECCV) (2014)Google Scholar
  17. 17.
    Kovashka, A., Grauman, K.: Attribute adaptation for personalized image search. In: International Conference on Computer Vision (ICCV) (2013)Google Scholar
  18. 18.
    Kovashka, A., Parikh, D., Grauman, K.: Whittlesearch: Image search with relative attribute feedback. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar
  19. 19.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Conference on Neural Information Processing Systems (NIPS) (2012)Google Scholar
  20. 20.
    Kumar, N., Belhumeur, P., Nayar, S.: FaceTracer: A search engine for large collections of images with faces. In: European Conference on Computer Vision (ECCV) (2008)Google Scholar
  21. 21.
    Kumar, N., Berg, A., P.Belhumeur, S.Nayar: Attribute and simile classifiers for face verification. In: International Conference on Computer Vision (ICCV) (2009)Google Scholar
  22. 22.
    Kumar, P., Packer, B., Koller, D.: Self-paced learning for latent variable models. In: Conference on Neural Information Processing Systems (NIPS) (2010)Google Scholar
  23. 23.
    Laffont, P.Y., Ren, Z., Tao, X., Qian, C., Hays, J.: Transient attributes for high-level understanding and editing of outdoor scenes. ACM Trans. Graph. (TOG) 33(4), 149 (2014)CrossRefGoogle Scholar
  24. 24.
    Lampert, C.H., Nickisch, H., Harmeling, S.: Learning to detect unseen object classes by between-class attribute transfer. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  25. 25.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2006)Google Scholar
  26. 26.
    Lee, Y.J., Efros, A.A., Hebert, M.: Style-aware mid-level representation for discovering visual connections in space and time. In: International Conference on Computer Vision (ICCV) (2013)Google Scholar
  27. 27.
    Lee, Y.J., Grauman, K.: Foreground focus: Unsupervised learning from partially matching images. Int. J. Comput. Vis. (IJCV) 85(2), 143–166 (2009)CrossRefGoogle Scholar
  28. 28.
    Lee, Y.J., Grauman, K.: Learning the easy things first: self-paced visual category discovery. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  29. 29.
    Li, Q., Wu, J., Tu, Z.: Harvesting mid-level visual concepts from large-scale internet images. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  30. 30.
    Palatucci, M., Pomerleau, D., Hinton, G., Mitchell, T.: Zero-shot learning with semantic output codes. In: Conference on Neural Information Processing Systems (NIPS) (2009)Google Scholar
  31. 31.
    Parikh, D., Grauman, K.: Relative attributes. In: International Conference on Computer Vision (ICCV) (2011)Google Scholar
  32. 32.
    Payet, N., Todorovic, S.: From a set of shapes to object discovery. In: European Conference on Computer Vision (ECCV) (2010)Google Scholar
  33. 33.
    Rastegari, M., Farhadi, A., Forsyth, D.: Attribute discovery via predictable discriminative binary codes. In: European Conference on Computer Vision (ECCV) (2012)Google Scholar
  34. 34.
    Saleh, B., Farhadi, A., Elgammal, A.: Object-centric anomaly detection by attribute-based reasoning. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  35. 35.
    Sandeep, R.N., Verma, Y., Jawahar, C.V.: Relative parts: distinctive parts for learning relative attributes. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  36. 36.
    Shrivastava, A., Singh, S., Gupta, A.: Constrained semi-supervised learning using attributes and comparative attributes. In: European Conference on Computer Vision (ECCV) (2012)Google Scholar
  37. 37.
    Singh, S., Gupta, A., Efros, A.A.: Unsupervised discovery of mid-level discriminative patches. In: European Conference on Computer Vision (ECCV) (2012)Google Scholar
  38. 38.
    Sivic, J., Russell, B.C., Efros, A.A., Zisserman, A., Freeman, W.T.: Discovering object categories in image collections. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2005)Google Scholar
  39. 39.
    Supancic, J., Ramanan, D.: Self-paced learning for long-term tracking. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  40. 40.
    Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  41. 41.
    Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2010)Google Scholar
  42. 42.
    Wang, S., Joo, J., Wang, Y., Zhu, S.C.: Weakly supervised learning for attribute localization in outdoor scenes. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  43. 43.
    Xiao, F., Lee, Y.J.: Discovering the spatial extent of relative attributes. In: International Conference on Computer Vision (ICCV) (2015)Google Scholar
  44. 44.
    Yu, A., Grauman, K.: Fine-grained visual comparisons with local learning. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  45. 45.
    Zhang, N., Paluri, M., Ranzato, M., Darrell, T., Bourdev, L.: PANDA: pose aligned networks for deep attribute modeling. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2014)Google Scholar
  46. 46.
    Zhu, X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.University of California DavisDavisUSA

Personalised recommendations