Weighting Multiple Features and Double Fusion Method for HMM Based Video Classification

Abstract

In this paper we present an effective and innovative way of classifying videos into different genres based on Hidden Markov Model (HMM) thereby facilitating subsequent analysis like video indexing, retrieval and so on. In particular, this work focuses on weighting Multiple Features and also on the challenging task of fusion technique at two different levels. The multiple features are used based on the observation that no single feature can provide the necessary discriminative information to better characterize the given video content in different aspects for distinguishing large video collections. Hence, the features such as 3D-color Histogram, Wavelet-HOG, and Motion are extracted from each video and a separate HMM is trained for each feature of video class. All the classifiers are grouped into sections such that each section contains classifiers with different features of the same genre. These features are evaluated in terms of weights based on Fuzzy Comprehensive Evaluation (FCE) technique for finding the degree of use of each feature in identifying the class. For classification, Double Fusion strategy is applied in terms of Intra section fusion and Inter section fusion methods. Intra section Fusion i.e. weighted-sum method is applied at the outputs of classifiers within the section of each genre. These weights represent the relative importance which is assigned to each feature vector in finding that particular class. Then an Inter section fusion i.e. Arg-Max method is applied to fuse the scores of all sections to make final decision. We tested our scheme on video database having videos such as Sports, Cartoons, Documentaries and News and the results are compared with other methods. The results show that multiple features, double fusion and also the use of fuzzy logic enhance video classification performance in terms of Accuracy Rate (AR) and Error Rate (ER).