Computing Image Descriptors from Annotations Acquired from External Tools

  • Jose Carlos Rangel
  • Miguel Cazorla
  • Ismael García-Varea
  • Jesús Martínez-Gómez
  • Élisa Fromont
  • Marc Sebban
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 418)


Visual descriptors are widely used in several recognition and classification tasks in robotics. The main challenge for these tasks is to find a descriptor that could represent the image content without losing representative information of the image. Nowadays, there exists a wide range of visual descriptors computed with computer vision techniques and different pooling strategies. This paper proposes a novel way for building image descriptors using an external tool, namely: Clarifai. This is a remote web tool that allows to automatically describe an input image using semantic tags, and these tags are used to generate our descriptor. The descriptor generation procedure has been tested in the ViDRILO dataset, where it has been compared and merged with some well-known descriptors. Moreover, subset variable selection techniques have been evaluated. The experimental results show that our descriptor is competitive in classification tasks with the results obtained with other kind of descriptors.


Descriptor generation Computer vision Semantic localization Robotics 


  1. 1.
    Banerji, S., Sinha, A., Liu, C.: Novel color, shape and texture-based scene image descriptors. In: 2012 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 245–248, August 2012Google Scholar
  2. 2.
    Chang, C.C., Lin, C.J.: LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2, 271–2727 (2011)CrossRefGoogle Scholar
  3. 3.
    Clarifai: Clarifai: Amplifying Intelligence (2015).
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Int. Conf. on CVPR, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  5. 5.
    Foster, I., Zhao, Y., Raicu, I., Lu, S.: Cloud computing and grid computing 360-degree compared. In: Grid Computing Environments Workshop, GCE 2008, pp. 1–10. Ieee (2008)Google Scholar
  6. 6.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)Google Scholar
  7. 7.
    Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C., Bottou, L., Weinberger, K. (eds.) Advances in Neural Information Processing Systems, vol. 25, pp. 1097–1105. Curran Associates, Inc. (2012)Google Scholar
  8. 8.
    Lampert, C., Nickisch, H., Harmeling, S.: Attribute-based classification for zero-shot visual object categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence 36(3), 453–465 (2014)CrossRefGoogle Scholar
  9. 9.
    Li, L.J., Fei-Fei, L.: What, where and who? Classifying events by scene and object recognition. In: IEEE 11th International Conference on Computer Vision, ICCV 2007, pp. 1–8. IEEE (2007)Google Scholar
  10. 10.
    Li, L.J., Su, H., Lim, Y., Fei-Fei, L.: Objects as attributes for scene classification. In: Kutulakos, K. (ed.) Trends and Topics in Computer Vision. Lecture Notes in Computer Science, vol. 6553, pp. 57–69. Springer, Heidelberg (2012)Google Scholar
  11. 11.
    Martinez-Gomez, J., Cazorla, M., Garcia-Varea, I., Morell, V.: Vidrilo: The visual and depth robot indoor localization with objects information dataset. International Journal of Robotics Research (2015)Google Scholar
  12. 12.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42(3), 145–175 (2001)CrossRefMATHGoogle Scholar
  13. 13.
    Petty, K.F., Moylan, A.J., Kwon, J., Mewes, J.J.: Traffic state estimation with integration of traffic, weather, incident, pavement condition, and roadway operations data (February 5, 2014), uS Patent App. 14/173,611Google Scholar
  14. 14.
    Szummer, M., Picard, R.W.: Indoor-outdoor image classification. In: IEEE International Workshop on Proceedings of the Content-Based Access of Image and Video Database, pp. 42–51. IEEE (1998)Google Scholar
  15. 15.
    Wang, C., Blei, D., Li, F.F.: Simultaneous image classification and annotation. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1903–1910, June 2009Google Scholar
  16. 16.
    Wang, G., Hoiem, D., Forsyth, D.: Learning image similarity from flickr groups using fast kernel machines. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(11), 2177–2188 (2012)CrossRefGoogle Scholar
  17. 17.
    Winder, S., Brown, M.: Learning local image descriptors. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8, June 2007Google Scholar
  18. 18.
    Wohlkinger, W., Vincze, M.: Ensemble of shape functions for 3D object classification. In: 2011 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 2987–2992. IEEE (2011)Google Scholar
  19. 19.
    Wu, J., Rehg, J.M.: Centrist: A visual descriptor for scene categorization. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(8), 1489–1501 (2011)CrossRefGoogle Scholar
  20. 20.
    Zhou, X., Yu, K., Zhang, T., Huang, T.: Image classification using super-vector coding of local image descriptors. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) Computer Vision ECCV 2010. Lecture Notes in Computer Science, vol. 6315, pp. 141–154. Springer, Heidelberg (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Jose Carlos Rangel
    • 1
  • Miguel Cazorla
    • 1
  • Ismael García-Varea
    • 2
  • Jesús Martínez-Gómez
    • 2
  • Élisa Fromont
    • 3
  • Marc Sebban
    • 3
  1. 1.Computer Science Research InstituteUniversity of AlicanteAlicanteSpain
  2. 2.University of Castilla-La ManchaAlbaceteSpain
  3. 3.Jean Monnet UniversitySaint EtienneFrance

Personalised recommendations