Abstract
There has been remarkable progress in the field of Semantic segmentation in recent years. Yet, it remains a challenging problem to apply segmentation to the video-based applications. Videos usually involve significantly larger volume of data compared to images. Particularly, a video contains around 30 frames per second. Segmentation of the similar frames unnecessarily adds to the time required for segmentation of complete video. In this paper, we propose a contour detection-based approach for detection of salient frames for faster semantic segmentation of videos. We propose to detect the salient frames of the video and pass only the salient frames through the segmentation block. Then, the segmented labels of the salient frames are mapped to the non-salient frames. The salient frame is defined by the variation in the pixel values of the background subtracted frames. The background subtraction is done using MOG2 background subtractor algorithm for background subtraction in various lighting conditions. We demonstrate the results using the Pytorch model for semantic segmentation of images. We propose to concatenate the semantic segmentation model to our proposed framework. We evaluate our result by comparing the time taken and the mean Intersection over Union (mIoU) for segmentation of the video with and without passing the video input through our proposed framework. We evaluate the results of Saliency Detection Block using Retention and Condensation ratio as the quality metrics.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)
Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. CoRR (2015). abs/1511.00561
Bao. L., Le, D.-N., Nhu, N., Bhateja, V., Satapathy, S.: Optimizing feature selection in video-based recognition using max-min ant system for the online video contextual advertisement user-oriented system. J. Comput. Sci. 21, 361–370 (2017)
Bhateja, V., Malhotra, C., Rastogi, K., Verma, A.: Improved decision median filter for video sequences corrupted by impulse noise. In: 2014 International Conference on Signal Processing and Integrated Networks (SPIN), February (2014)
de Avila, S.E.F., da_Luz, A., Araújo. A.D., Cord, M.: Vsumm: an approach for automatic video summarization and quantitative evaluation. In: 2008 XXI Brazilian Symposium on Computer Graphics and Image Processing, pp. 103–110, October (2008)
Mundur, P., Rao, Y., Yesha, Y.: Keyframe-based video summarization using delaunay clustering. Int. J. Digit. Libr. 6(2), 219–232 (2006)
Muratov, O., Zontone, P., Boato, G., De Natale, F.G.B.: A segment-based image saliency detection. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1217–1220, May (2011)
Paszke, A., Chaurasia, A., Kim, S., Culurciello, E.: Enet: a deep neural network architecture for real-time semantic segmentation. CoRR (2016). abs/1606.02147
Sujatha, C., Chivate, A.R., Ganihar, S.A., Mudenagudi, U.: Time driven video summarization using gmm. In: 2013 Fourth National Conference on Computer Vision. Pattern Recognition, Image Processing and Graphics (NCVPRIPG) (2013)
Sujatha, C., Mudenagudi, U.: Gaussian mixture model for summarization of surveillance videos. In: 2015 Fifth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG), pp. 1–4, December (2015)
Yi, Y. Su, L., Huang, Q., Wu, Z., Wang, C.: Saliency detection with two-level fully convolutional networks. In: 2017 IEEE International Conference on Multimedia and Expo (ICME), pp. 271–276, July (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Vasudev, H. et al. (2020). Saliency Detection for Semantic Segmentation of Videos. In: Bhateja, V., Satapathy, S., Zhang, YD., Aradhya, V. (eds) Intelligent Computing and Communication. ICICC 2019. Advances in Intelligent Systems and Computing, vol 1034. Springer, Singapore. https://doi.org/10.1007/978-981-15-1084-7_31
Download citation
DOI: https://doi.org/10.1007/978-981-15-1084-7_31
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-1083-0
Online ISBN: 978-981-15-1084-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)