Violence detection in videos using interest frame extraction and 3D convolutional neural network

Abstract

With the rapid development of detecting violent behaviors in surveillance cameras, requests on systems that automatically recognize violent events are expanded. Nowadays, violence detection has become an active research field in image processing and machine learning. The relevant works in such a field are classified into hand-crafted and deep learning methods. Despite the effectiveness of hand-crafted ones, their computational cost may be suppressive for practical applications. Additionally, deep learning techniques usually exploit 3D Convolutional Networks (3D ConvNets) to do this task. To improve the accuracy of these networks, meaningful regions and temporal changes in videos should be considered. Consequently, the performance of a 3D ConvNet can be reinforced by selecting significant temporal information and noticing to special regions in two spatial dimensions. In this work, we propose a novel 3D ConvNet along with a technique for extracting interest frames. The Structural Similarity Index Measure (SSIM) is exploited to extract interest frames as significant temporal information. Indeed, the SSIM uses the statistical features of two consecutive frames for this reason. In this way, sixteen video frames with the smallest SSIM are considered as dominant motion frames, which are then sent to a 3D CNN for classification. Moreover, a spatial attention module is exploited to make attention on the specific regions. Furthermore, three benchmark datasets are employed to evaluate the performance of the proposed method. The results show that in terms of accuracy, our scheme outperforms existing approaches.