Concept-Based Interactive Search System

  • Yi-Jie Lu
  • Phuong Anh Nguyen
  • Hao Zhang
  • Chong-Wah Ngo
Conference paper

DOI: 10.1007/978-3-319-51814-5_42

Part of the Lecture Notes in Computer Science book series (LNCS, volume 10133)
Cite this paper as:
Lu YJ., Nguyen P.A., Zhang H., Ngo CW. (2017) Concept-Based Interactive Search System. In: Amsaleg L., Guðmundsson G., Gurrin C., Jónsson B., Satoh S. (eds) MultiMedia Modeling. MMM 2017. Lecture Notes in Computer Science, vol 10133. Springer, Cham

Abstract

Our successful multimedia event detection system at TREC-VID 2015 showed its strength on handling complex concepts in a query. The system was based on a large number of pre-trained concept detectors for textual-to-visual relation. In this paper, we enhance the system by enabling human-in-the-loop. In order to facilitate a user to quickly find an information need, we incorporate concept screening, video reranking by highlighted concepts, relevance feedback and color sketch to refine a coarse retrieval result. The aim is to eventually come up with a system suitable for both Ad-hoc Video Search and Known-Item Search. In addition, as the increasing awareness of difficulty in distinguishing shots of very similar scenes, we also explore the automatic story annotation along the timeline of a video, so that a user can quickly grasp the story happened in the context of a target shot and reject shots with incorrect context. With the story annotation, a user can refine the search result as well by simply adding a few keywords in a special “context field” of a query.

Keywords

Video search Known-Item Search Concept bank Semantic query Video reranking Story annotation 

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yi-Jie Lu
    • 1
  • Phuong Anh Nguyen
    • 1
  • Hao Zhang
    • 1
  • Chong-Wah Ngo
    • 1
  1. 1.Department of Computer ScienceCity University of Hong KongKowloonHong Kong

Personalised recommendations