Trends and Surveys

International Journal of Multimedia Information Retrieval

, Volume 2, Issue 2, pp 73-101

First online:

High-level event recognition in unconstrained videos

  • Yu-Gang JiangAffiliated withSchool of Computer Science, Fudan University Email author 
  • , Subhabrata BhattacharyaAffiliated withComputer Vision Lab, University of Central Florida
  • , Shih-Fu ChangAffiliated withDepartment of Electrical Engineering, Columbia University
  • , Mubarak ShahAffiliated withComputer Vision Lab, University of Central Florida


The goal of high-level event recognition is to automatically detect complex high-level events in a given video sequence. This is a difficult task especially when videos are captured under unconstrained conditions by non-professionals. Such videos depicting complex events have limited quality control, and therefore, may include severe camera motion, poor lighting, heavy background clutter, and occlusion. However, due to the fast growing popularity of such videos, especially on the Web, solutions to this problem are in high demands and have attracted great interest from researchers. In this paper, we review current technologies for complex event recognition in unconstrained videos. While the existing solutions vary, we identify common key modules and provide detailed descriptions along with some insights for each of them, including extraction and representation of low-level features across different modalities, classification strategies, fusion techniques, etc. Publicly available benchmark datasets, performance metrics, and related research forums are also described. Finally, we discuss promising directions for future research.


Video events Recognition Unconstrained videos Multimedia event detection Multimodal features Fusion