Visual Attribute Extraction Using Human Pose Estimation

  • Angelo Nodari
  • Marco Vanetti
  • Ignazio Gallo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7887)


We propose a method to describe how a person is dressed, using an innovative way to extract Visual Information exploiting the Human Pose Estimation. Given the lack of algorithms in this field, we aims to pave the way giving a baseline and publishing a detailed dataset for future comparisons. In particular in this study we show how using the Human Pose Estimation, we are able to extract the essential features for the description of the Visual Attributes. Furthermore, the proposed method is able to manage the problems highlighted in literature regarding the extraction of features from images of people due to their articulated poses. For this reason we also propose a formalization of how describe people’s clothing in order to give a starting point and facilitate the analysis and the Visual Attributes extraction phase. Moreover we show how the use of Deformable Structures let us to extract Visual Attributes without the using of segmentation algorithms.


Clothing Parsing Visual Attributes Human Pose Estimation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anguelov, D., Lee, K.C., Göktürk, S.B., Sumengen, B., Inc, R.: Contextual identity recognition in personal photo albums. In: IEEE Conference on In Computer Vision and Pattern Recognition, CVPR (2007)Google Scholar
  2. 2.
    Bosch, A., Zisserman, A., Muoz, X.: Representing shape with a spatial pyramid kernel. In: Conference on Image and Video Retrieval (2007)Google Scholar
  3. 3.
    Bourdev, L., Maji, S., Malik, J.: Describing people: Poselet-based attribute classification. In: International Conference on Computer Vision, ICCV (2011)Google Scholar
  4. 4.
    Chen, H., Xu, Z., Liu, Z., Zhu, S.C.: Composite Templates for Cloth Modeling and Sketching. In: Computer Vision and Pattern Recognition (2006)Google Scholar
  5. 5.
    Cohen, J.: A coefficient of agreement for nominal scales. Educational and Psychological Measurement 20(1), 37–46 (1960)CrossRefGoogle Scholar
  6. 6.
    Datta, R., Li, J., Wang, J.Z.: Content-based image retrieval: approaches and trends of the new age. In: Multimedia Information Retrieval, pp. 253–262 (2005)Google Scholar
  7. 7.
    Farhadi, A., Endres, I., Hoiem, D., Forsyth, D.: Describing objects by their attributes. In: Proceedings of the IEEE Computer Society Conference on CVPR (2009)Google Scholar
  8. 8.
    Ferrari, V., Marin-Jimenez, M., Zisserman, A.: Pose search: Retrieving people using their pose. In: Computer Vision and Pattern Recognition, pp. 1–8 (2009)Google Scholar
  9. 9.
    Ferrari, V., Zisserman, A.: Learning visual attributes. In: Advances in Neural Information Processing Systems (December 2007)Google Scholar
  10. 10.
    Ferrari, V., Marín-jiménez, M.J., Zisserman, A.: Progressive search space reduction for human pose estimation. In: Computer Vision and Pattern Recognition (2008)Google Scholar
  11. 11.
    Fischler, M.A., Elschlager, R.A.: The representation and matching of pictorial structures. IEEE Transactions on Computers C-22, 67–92 (1973)CrossRefGoogle Scholar
  12. 12.
    Hay, G.J., Castilla, G.: Geographic Object-Based Image Analysis (GEOBIA): A new name for a new discipline. Springer (2008)Google Scholar
  13. 13.
    Naga Jyothi, B., Babu, G.R., Murali Krishna, I.V.: Object Oriented and Multi-Scale Image Analysis: Strengths, Weaknesses, Opportunities and Threats-A Review. Journal of Computer Science 4, 706–712 (2008)CrossRefGoogle Scholar
  14. 14.
    Khakimdjanova, L., Park, J.: Online visual merchandising practice of apparel e-merchants. Journal of Retailing and Consumer Services 12, 307–318 (2005)CrossRefGoogle Scholar
  15. 15.
    Landis, J.R., Koch, G.G.: The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)MathSciNetzbMATHCrossRefGoogle Scholar
  16. 16.
    Liu, Y., Zhang, D., Lu, G., Ma, W.Y.: A survey of content-based image retrieval with high-level semantics. Pattern Recognition 40, 262–282 (2007)zbMATHCrossRefGoogle Scholar
  17. 17.
    Ma, W.-Y., Zhang, H.J.: Benchmarking of image features for content-based retrieval. In: Asilomar Conference on Signals, Systems Computers, vol. 1 (1998)Google Scholar
  18. 18.
    Mei, T., Hua, X.S., Li, S.: Contextual in-image advertising. In: ACM Multimedia Conference, pp. 439–448 (2008)Google Scholar
  19. 19.
    Nodari, A., Gallo, I., Vanetti, M.: Automatic visual attributes extraction from web offers images. In: Computational Modeling of Objects Presented in Images: Foundamentals Method and Applications (2012)Google Scholar
  20. 20.
    Nodari, A., Vanetti, M., Gallo, I., Albertini, S.: Color and texture indexing using an object segmentation approach. In: EANN/AIAI. CRL Publisher (2012)Google Scholar
  21. 21.
    Ramanan, D., Sminchisescu, C.: Training deformable models for localization. In: Computer Vision and Pattern Recognition, vol. 1, pp. 206–213 (2006)Google Scholar
  22. 22.
    Sivic, J., Zitnick, C.L., Szeliski, R.: Finding people in repeated shots of the same scene. In: Proceedings of the British Machine Vision Conference (2006)Google Scholar
  23. 23.
    Vaquero, D.A., Feris, R.S., Tran, D., Brown, L., Hampapur, A., Turk, M.: Attribute-based people search in surveillance environments. In: IEEE Workshop on Applications of Computer Vision (2009)Google Scholar
  24. 24.
    Weber, M., Bauml, M.: Part-based clothing segmentation for person retrieval. In: Advanced Video and Signal Based Surveillance (2011)Google Scholar
  25. 25.
    Yamaguchi, K., Kiapour, M.H., Ortiz, L.E., Berg, T.L.: Parsing clothing in fashion photographs. In: CVPR, pp. 3570–3577. IEEE (2012)Google Scholar
  26. 26.
    Zuffi, S., Freifeld, O., Black, M.J.: From pictorial structures to deformable structures. In: Proceedings of the 2012 IEEE, CVPR 2012. IEEE Computer Society (2012)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Angelo Nodari
    • 1
  • Marco Vanetti
    • 1
  • Ignazio Gallo
    • 1
  1. 1.Dipartimento di Scienze Teoriche e ApplicateUniversità dell’InsubriaVareseItaly

Personalised recommendations