Enhanced VIREO KIS at VBS 2018

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10705)


The VIREO Known-Item Search (KIS) system has joined the Video Browser Showdown (VBS) [1] evaluation benchmark for the first time in year 2017. With experiences learned, the second version of VIREO KIS is presented in this paper. Considering the color-sketch based retrieval, we propose a simple grid-based approach for color query. This method allows the aggregation of color distributions in video frames into a shot representation, and generates the pre-computed rank list for all available queries which reduces computational resources and favors a recommendation module. With focusing on concept based retrieval, we modify our multimedia event detection system at TRECVID 2015 in VIREO KIS 2017. In this year, the concept bank of VIREO KIS has been upgraded to 14K concepts. An adaptive concept selection, combination and expansion mechanism, which assists the user in picking the right concepts and logically combining concepts to form more expressive query, has been developed. In addition, metadata is included for textual query and some interface designs are also revised for providing a flexible view of results to the user.


Video search Known-Item Search Color sketch query Concept query Concept selection Concept combination 



The work described in this paper was supported by two grants from the Research Grants Council of the Hong Kong Special Administrative Region, China (CityU 11210514, 11250716).


  1. 1.
    Cobârzan, C., Schoeffmann, K., Bailer, W., Hürst, W., Blažek, A., Lokoč, J., Vrochidis, S., Barthel, K.U., Rossetto, L.: Interactive video search tools: a detailed analysis of the video browser showdown 2015. Multimed. Tools Appl. 76(4), 5539–5571 (2017)CrossRefGoogle Scholar
  2. 2.
    Lu, Y.-J., Nguyen, P.A., Zhang, H., Ngo, C.-W.: Concept-based interactive search system. In: Amsaleg, L., Guðmundsson, G.Þ., Gurrin, C., Jónsson, B.Þ., Satoh, S. (eds.) MMM 2017. LNCS, vol. 10133, pp. 463–468. Springer, Cham (2017). CrossRefGoogle Scholar
  3. 3.
    Lokoč, J., Blažek, A., Skopal, T.: Signature-based video browser. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MMM 2014. LNCS, vol. 8326, pp. 415–418. Springer, Cham (2014). CrossRefGoogle Scholar
  4. 4.
    Blažek, A., Lokoč, J., Matzner, F., Skopal, T.: Enhanced signature-based video browser. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 243–248. Springer, Cham (2015). Google Scholar
  5. 5.
    Lokoč, J., Phuong, A.N., Vomlelová, M., Ngo, C.-W.: Color-sketch simulator: a guide for color-based visual known-item search. In: Cong, G., Peng, W.-C., Zhang, W.E., Li, C., Sun, A. (eds.) ADMA 2017. LNCS, vol. 10604, pp. 754–763. Springer, Cham (2017). CrossRefGoogle Scholar
  6. 6.
    Ueki, K., Kikuchi, K., Saito, S., Kobayashi, T.: Waseda at TRECVID 2016: ad-hoc video search. In: TRECVID 2016 Workshop, Gaithersburg, MD, USA (2016)Google Scholar
  7. 7.
    Bae, G.Y., Olkkonen, M., Allred, S.R., Flombaum, J.I.: Why some colors appear more memorable than others: a model combining categories and particulars in color working memory. J. Exp. Psychol. Gen. 144(4), 744–763 (2015)CrossRefGoogle Scholar
  8. 8.
    Wang, J., Hua, X.-S.: Interactive image search by color map. ACM Trans. Intell. Syst. Technol. 3(1), Article Id 12 (2011)Google Scholar
  9. 9.
    Lu, Y.J., Zhang, H., de Boer, M., Ngo, C.W.: Event detection with zero example: select the right and suppress the wrong concepts. In: ACM ICMR (2016)Google Scholar
  10. 10.
    Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Advances in Neural Information Processing Systems, pp. 487–495 (2014)Google Scholar
  11. 11.
    Zhang, W., Zhang, H., Yao, T., Lu, Y., Chen, J., Ngo, C.-W.: VIREO @ TRECVID 2014: instance search and semantic indexing. In: NIST TRECVID Workshop (2014)Google Scholar
  12. 12.
    Strassel, S., Morris, A., Fiscus, J., Caruso, C., Lee, H., Over, P., Fiumara, J., Shaw, B., Antonishek, B., Michel, M.: Creating HAVIC: heterogeneous audio-visual internet collection. In: Chair, N.C.C., Choukri, K., Declerck, T., Dogan, M.U., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S. (eds.) LREC, Istanbul, Turkey, May 2012. ELRA (2012)Google Scholar
  13. 13.
    Sigurbjornsson, B., Zwol, R.V.: Flickr tag recommendation based on collective knowledge. In: Proceeding of ACM Intelligent World Wide Web Conference, pp. 327–336 (2008)Google Scholar
  14. 14.
    Smith, R.: An overview of the tesseract OCR engine. In: Proceeding of 9th International Conference on Document Analysis & Recognition (2007)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  1. 1.City University of Hong KongKowloonHong Kong

Personalised recommendations