Saliency Subtraction Inspired Automated Event Detection in Underwater Environments
Unmanned underwater exploration in unconstrained environments is a challenging problem. Analysis of the large volumes of images/videos captured by underwater stations/vehicles manually is a major bottleneck for further research. Existing computer vision methods either do not target unconstrained underwater environments or they only aim to detect static or moving entities. In this paper, we present a novel method for analyzing underwater videos and detecting events. Entry/exit of an object in scene is treated as an event independent of the other objects present therein. The method is applied on underwater videos with no prior knowledge, thus aiding in automated underwater exploration. The method is inspired by the fact that saliency of objects in the scene is invariant of the surrounding environment. The proposed method is composed of three main steps: Local Patch Saliency, Adaptive Saliency Subtraction, and event generation for analyzing underwater imagery from the videos. The method is aimed at detecting overlapping events containing man-made as well as natural objects including those containing multiple objects in the unconstrained underwater conditions. The performance of the method is evaluated on publicly available videos obtained from Ocean Networks Canada and Fish4Knowledge datasets. Ground truth for Ocean Networks Canada videos is not available; hence, a method for generating the same for varied sources is also presented. The algorithm achieves a precision of 98% for event detection with 20% misclassification rate. The results show the robustness of the method that performs even in complex and varying underwater conditions.
KeywordsAdaptive Saliency Subtraction Event detection Ground truth generation Local Patch Saliency
The authors are grateful to Dr. Maia Hoeberechts and team for providing the Ocean Networks Canada Dataset. We are also thankful to Akanksha Pathania, Parminder Kaur, Gifty Aggarwal, and Neha for assisting us in generating the ground truth for the underwater videos.
Nitin Kumar is thankful to the Council of Scientific and Industrial Research - Central Scientific Instruments Organisation (CSIR-CSIO), Chandigarh for providing the funding and opportunity to carry out this work at CSIR-CSIO.
Compliance with Ethical Standards
Conflict of interest
The authors declare that they have no conflict of interest.
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent is not necessary for the present study.
- 2.Griffiths G (ed.) Technology and applications of autonomous underwater vehicles. CRC Press; 2002 Nov 28.Google Scholar
- 8.Olmos A, Trucco E. 2002. Detecting man-made objects in unconstrained subsea videos. In: BMVC. p. 1–10.Google Scholar
- 9.Edgington DR, Salamy KA, Risi M, Sherlock RE, Walther D, Koch C. Automated event detection in underwater video. Oceans 2003. Celebrating the past teaming toward the future (IEEE Cat. No. 03CH37492). IEEE; 2003. p. P2749–53.Google Scholar
- 10.Walther D, Edgington DR, Koch C. Detection and tracking of objects in underwater video. Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. IEEE; 2004. p. I–I.Google Scholar
- 11.Kabatek M, Azimi-Sadjadi MR, Tucker JD. An underwater target detection system for electro-optical imagery data. OCEANS. IEEE; 2009. p. 1–8.Google Scholar
- 13.Lipton AJ, Fujiyoshi H, Patil RS. Moving target classification and tracking from real-time video. Proceedings fourth IEEE workshop on applications of computer vision. WACV’98 (Cat. No. 98EX201). IEEE; 1998. p. 8–14.Google Scholar
- 14.Piccardi M. Background subtraction techniques: a review. 2004 IEEE International conference on systems, man and cybernetics (IEEE Cat. No. 04CH37583). IEEE; 2004. p. 3099–104.Google Scholar
- 15.Spampinato C, Palazzo S. Enhancing object detection performance by integrating motion objectness and perceptual organization. Proceedings of the 21st international conference on pattern recognition (ICPR2012). IEEE; 2012. p. 3640–3.Google Scholar
- 16.Palazzo S, Kavasidis I, Spampinato C. Covariance based modeling of underwater scenes for fish detection. 2013 IEEE International conference on image processing. IEEE; 2013. p. 1481–5.Google Scholar
- 19.Liao S, Zhao G, Kellokumpu V, Pietikäinen M, Li SZ. Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes. 2010 IEEE Computer society conference on computer vision and pattern recognition. IEEE; 2010. p. 1301–6.Google Scholar
- 20.Spampinato C, Chen-Burger YH, Nadarajan G, Fisher RB. Detecting, tracking and counting fish in low quality unconstrained underwater videos. VISAPP (2) 2008;2008(514–9):1.Google Scholar
- 24.Rova A, Mori G, Dill LM. One fish, two fish, butterfish, trumpeter: recognizing fish in underwater video. InMVA. 2007; 404–7.Google Scholar
- 27.Oliver K, Hou W, Wang S. 2010. Image feature detection and matching in underwater conditions, Vol. 7678: International Society for Optics and Photonics.Google Scholar
- 28.Kavasidis I, Palazzo S. Quantitative performance analysis of object detection algorithms on underwater video footage. Proceedings of the 1st ACM international workshop on multimedia analysis for ecological data. ACM; 2012. p. 57–60.Google Scholar
- 29.Han KM, Choi HT. Shape context based object recognition and tracking in structured underwater environment. 2011 IEEE International geoscience and remote sensing symposium. IEEE; 2011. p. 617–20.Google Scholar
- 30.Kim D, Lee D, Myung H, Choi HT. Object detection and tracking for autonomous underwater robots using weighted template matching. 2012 Oceans-Yeosu. IEEE; 2012. p. 1–5.Google Scholar
- 31.Leonard I, Arnold-Bos A, Alfalou A. Interest of correlation-based automatic target recognition in underwater optical images: theoretical justification and first results. Ocean sensing and monitoring II. International Society for Optics and Photonics; 2010. p. 76780O.Google Scholar
- 32.Barat C, Rendas MJ. A robust visual attention system for detecting manufactured objects in underwater video. OCEANS 2006. IEEE; 2006. p. 1–6.Google Scholar
- 33.Barat C, Phlypo R. A fully automated method to detect and segment a manufactured object in an underwater color image. EURASIP J Adv Signal Process 2010;1:10.Google Scholar
- 35.Wang HB, Dong X, Shen J, Wu XW, Chen Z. Saliency-based adaptive object extraction for color underwater images. Applied mechanics and materials. Trans Tech Publications; 2013. p. 3964–70.Google Scholar
- 36.Bazeille S, Quidu I, Jaulin L. Identification of underwater man-made object using a colour criterion. Conference on detection and classification of underwater targets; 2007. p. xx.Google Scholar
- 37.Maldonado-Ramírez A, Torres-Méndez LA. 2016. Robotic visual tracking of relevant cues in underwater environments with poor visibility conditions. J Sensors.Google Scholar
- 38.Gebali A, Albu AB, Hoeberechts M. Detection of salient events in large datasets of underwater video. IEEE; 2012. p. 14.Google Scholar
- 39.Oliva A, Torralba A, Castelhano MS, Henderson JM. Top-down control of visual attention in object detection. Proceedings 2003 international conference on image processing (Cat. No. 03CH37429). IEEE; 2003. p. I-253.Google Scholar
- 40.Rutishauser U, Walther D, Koch C, Perona P. Is bottom-up attention useful for object recognition? Proceedings of the 2004 IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. IEEE; 2004. p. II-II.Google Scholar
- 43.Zelnik-Manor L, Irani M. Event-based analysis of video. InCVPR (2) 2001;8:123–30.Google Scholar
- 44.Ke Y, Sukthankar R, Hebert M. Event detection in crowded videos. 2007 IEEE 11th international conference on computer vision. IEEE; 2007. p. 1–8.Google Scholar
- 48.Borji A, Itti L. Exploiting local and global patch rarities for saliency detection. 2012 IEEE conference on computer vision and pattern recognition. IEEE; 2012. p. 478–85.Google Scholar
- 54.Barnes CR, Best MM, Bornhold BD, Juniper SK, Pirenne B, Phibbs P. The NEPTUNE project-a cabled ocean observatory in the NE Pacific: overview, challenges and scientific objectives for the installation and operation of Stage I in Canadian waters. 2007 Symposium on underwater technology and workshop on scientific use of submarine cables and related technologies. IEEE; 2007. p. 308–13.Google Scholar