A Study on the Use of a Binary Local Descriptor and Color Extensions of Local Descriptors for Video Concept Detection

  • Foteini Markatopoulou
  • Nikiforos Pittaras
  • Olga Papadopoulou
  • Vasileios Mezaris
  • Ioannis Patras
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8935)


In this work we deal with the problem of how different local descriptors can be extended, used and combined for improving the effectiveness of video concept detection. The main contributions of this work are: 1) We examine how effectively a binary local descriptor, namely ORB, which was originally proposed for similarity matching between local image patches, can be used in the task of video concept detection. 2) Based on a previously proposed paradigm for introducing color extensions of SIFT, we define in the same way color extensions for two other non-binary or binary local descriptors (SURF, ORB), and we experimentally show that this is a generally applicable paradigm. 3) In order to enable the efficient use and combination of these color extensions within a state-of-the-art concept detection methodology (VLAD), we study and compare two possible approaches for reducing the color descriptor’s dimensionality using PCA. We evaluate the proposed techniques on the dataset of the 2013 Semantic Indexing Task of TRECVID.


Video feature extraction concept detection concept-based video retrieval binary descriptors 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Alahi, A., Ortiz, R., Vandergheynst, P.: Freak: Fast retina keypoint. In: IEEE Int. Conf., CVPR 2012, pp. 510–517 (2012)Google Scholar
  2. 2.
    Bay, H., Ess, A., Tuytelaars, T., Gool, L.V.: Speeded-up robust features (surf). Computer Vision and Image Understing 110(3), 346–359 (2008)CrossRefGoogle Scholar
  3. 3.
    Bingham, E., Mannila, H.: Random projection in dimensionality reduction: Applications to image and text data. In: 7th ACM SIGKDD Int. Conf. on Knowledge Discovery and Data Mining, pp. 245–250. ACM, NY (2001)Google Scholar
  4. 4.
    Bosch, A., Zisserman, A., Muoz, X.: Image classification using random forests and ferns. In: IEEE Int. Conf. ICCV 2007, Rio de Janeiro, pp. 1–8 (2007)Google Scholar
  5. 5.
    Calonder, M., Lepetit, V., Ozuysal, M., Trzcinski, T., Strecha, C., Fua, P.: BRIEF: Computing a Local Binary Descriptor Very Fast. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(7), 1281–1298 (2012)CrossRefGoogle Scholar
  6. 6.
    Canclini, A., Cesana, M., Redondi, A., Tagliasacchi, M., Ascenso, J., Cilla, R.: Evaluation of low-complexity visual feature detectors and descriptors. In: 18th Int. Conf. on Digital Signal Processing (DSP), pp. 1–7 (2013)Google Scholar
  7. 7.
    Chatfield, K., Lempitsky, V., Vedaldi, A., Zisserman, A.: The devil is in the details: an evaluation of recent feature encoding methods. In: British Machine Vision Conference, pp. 76.1–76.12. British Machine Vision Association (2011)Google Scholar
  8. 8.
    Chen, D.M., Makar, M., de Araújo, A.F., Girod, B.: Interframe coding of global image signatures for mobile augmented reality. In: DCC, pp. 33–42 (2014)Google Scholar
  9. 9.
    Chu, D.M., Smeulders, A.W.M.: Color invariant SURF in discriminative object tracking. In: Kutulakos, K.N. (ed.) ECCV 2010 Workshops, Part II. LNCS, vol. 6554, pp. 62–75. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Fan, P., Men, A., Chen, M., Yang, B.: Color-SURF: A surf descriptor with local kernel color histograms. In: IEEE Int. Conf. on Network Infrastructure and Digital Content, pp. 726–730 (2009)Google Scholar
  11. 11.
    Fu, J., Jing, X., Sun, S., Lu, Y., Wang, Y.: C-surf: Colored speeded up robust features. In: Yuan, Y., Wu, X., Lu, Y. (eds.) Trustworthy Computing and Services. CCIS, vol. 320, pp. 203–210. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  12. 12.
    Grana, C., Borghesani, D., Manfredi, M., Cucchiara, R.: A fast approach for integrating ORB descriptors in the bag of words model. In: SPIE, vol. 8667, pp. 866709–866709–8 (2013)Google Scholar
  13. 13.
    Jegou, H., Douze, M., Schmid, C., Perez, P.: Aggregating local descriptors into a compact image representation. In: IEEE on Computer Vision and Pattern Recognition (CVRP 2010), San Francisco, CA, pp. 3304–3311 (2010)Google Scholar
  14. 14.
    Jegou, H., Perronnin, F., Douze, M., Sanchez, J., Perez, P., Schmid, C.: Aggregating local image descriptors into compact codes. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(9), 1704–1716 (2012)CrossRefGoogle Scholar
  15. 15.
    Leutenegger, S., Chli, M., Siegwart, R.: Brisk: Binary robust invariant scalable keypoints. In: IEEE Int. Conf. ICCV 2011, pp. 2548–2555 (2011)Google Scholar
  16. 16.
    Lowe, D.G.: Distinctive Image Features from Scale-Invariant Keypoints. Int. Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Markatopoulou, F., Moumtzidou, A., Tzelepis, C., Avgerinakis, K., Gkalelis, N., Vrochidis, S., Mezaris, V., Kompatsiaris, I.: ITI-CERTH participation to TRECVID 2013. In: TRECVID 2013 Workshop, Gaithersburg, MD, USA (2013)Google Scholar
  18. 18.
    Markatopoulou, F., Mezaris, V., Kompatsiaris, I.: A comparative study on the use of multi-label classification techniques for concept-based video indexing and annotation. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014, Part I. LNCS, vol. 8325, pp. 1–12. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  19. 19.
    Over, P., Awad, G., Michel, M., Fiscus, J., Sanders, G., Kraaij, W., Smeaton, A.F.: Trecvid 2013 – an overview of the goals, tasks, data, evaluation mechanisms and metrics. In: Proceedings of TRECVID 2013, NIST, USA (2013)Google Scholar
  20. 20.
    Perronnin, F., Sánchez, J., Mensink, T.: Improving the fisher kernel for large-scale image classification. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 143–156. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  21. 21.
    Qiu, G.: Indexing chromatic and achromatic patterns for content-based colour image retrieval. Pattern Recognition 35, 1675–1686 (2002)CrossRefzbMATHGoogle Scholar
  22. 22.
    Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: An efficient alternative to SIFT or SURF. In: IEEE Int. Conf. on Computer Vision, pp. 2564–2571 (2011)Google Scholar
  23. 23.
    Safadi, B., Quénot, G.: Re-ranking by local re-scoring for video indexing and retrieval. In: 20th ACM Int. Conf. on Information and Knowledge Management, UK, pp. 2081–2084. ACM, NY (2011)Google Scholar
  24. 24.
    Van de Sande, K.E.A., Gevers, T., Snoek, C.G.M.: Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 32(9), 1582–1596 (2010)CrossRefGoogle Scholar
  25. 25.
    Van de Sande, K.E.A., Snoek, C.G.M., Smeulders, A.W.M.: Fisher and vlad with flair. In: IEEE Conference on Computer Vision and Pattern Recognition (2014)Google Scholar
  26. 26.
    Sidiropoulos, P., Mezaris, V., Kompatsiaris, I.: Video tomographs and a base detector selection strategy for improving large-scale video concept detection. IEEE Transactions on Circuits and Systems for Video Technology 24(7), 1251–1264 (2014)CrossRefGoogle Scholar
  27. 27.
    Snoek, C.G.M., Worring, M.: Concept-Based Video Retrieval. Foundations and Trends in Information Retrieval 2(4), 215–322 (2009)CrossRefGoogle Scholar
  28. 28.
    Witten, I., Frank, E.: Data Mining Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)Google Scholar
  29. 29.
    Yilmaz, E., Kanoulas, E., Aslam, J.A.: A simple and efficient sampling method for estimating ap and ndcg. In: 31st ACM SIGIR Int. Conf. on Research and Development in Information Retrieval, pp. 603–610. ACM, USA (2008)Google Scholar
  30. 30.
    Zhou, X., Yu, K., Zhang, T., Huang, T.S.: Image classification using super-vector coding of local image descriptors. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 141–154. Springer, Heidelberg (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Foteini Markatopoulou
    • 1
    • 2
  • Nikiforos Pittaras
    • 1
  • Olga Papadopoulou
    • 1
  • Vasileios Mezaris
    • 1
  • Ioannis Patras
    • 2
  1. 1.CERTHInformation Technologies Institute (ITI)ThermiGreece
  2. 2.Queen Mary University of LondonUK

Personalised recommendations