Abstract
Machine learning based saliency detection methods have achieved better performance than traditional methods. Here, we propose a machine learning based method that utilizes Convex Hull and Extreme Learning Machine (ELM) for detecting salient object(s) in an image. The novelty of the proposed method lies in the generation of training set without using human annotations. Initially, an input image is segmented using SLIC algorithm at different scales to produce multiscale segmented images. This is followed by estimating two different saliency priors viz. (a) Convex Hull center prior and (b) contrast prior for each segmented image. These priors exploit foreground center and spatially weighted contrast respectively. Both of these estimated priors help in computing initial saliency of each segment across all scales. For each scale, the initial saliency map along with the Convex Hull based label map is employed on a segmented image to determine the positive (salient) and negative (background) training set. Distinctive features for each segment belonging to the training set are extracted and then passes to the Extreme Learning Machine for learning the ELM model. Afterwards, multiscale saliency maps of an image are found by applying the learned ELM model on distinctive features extracted from each segment across multiple scales. These multiscale saliency maps are linearly combined to obtain the final saliency map. The effectiveness of the proposed method is supported through extensive experimental results performed on six publicly available datasets viz. MSRA10K, DUT-OMRON, ECSSD, PASCAL-S, SED2, and THUR15K. The performance of the proposed method was compared with 11 state-of-the-art methods in terms of Precision, Recall, F-Measure, Receiver Operating Characteristics (ROC), and Area under the curve (AUC). The proposed method outperforms or comparable with compared methods in terms of all the performance measures.
Similar content being viewed by others
References
Achanta R, Hemami S, Estrada F, Süsstrunk S. (2009) Frequency-tuned salient region detection. In: IEEE international conference on computer vision and pattern recognition (CVPR 2009), CONF, pp 1597–1604
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S., et al. (2010) Slic superpixels. Ecole Polytechnique Fédéral de Lausssanne (EPFL), Tech. Rep 149300, 155–162
Alpert S, Galun M, Basri R, Brandt A (2007) Image segmentation by probabilistic bottom-up aggregation and cue integration. In: Computer vision and pattern recognition, 2007. CVPR0́7. IEEE conference on, IEEE, pp 1–8
Borji A, Cheng MM, Jiang H, Li J (2015) Salient object detection: A benchmark. IEEE Trans Image Process 12(24):5706–5722
Borji A, Sihite DN, Itti L (2012) Probabilistic learning of task-specific visual attention. In: 2012 IEEE conference on computer vision and pattern recognition, IEEE, pp 470–477
Cheng MM, Mitra NJ, Huang SX, Torr PH, Hu SM (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37 (3):569–582
Cheng MM, Mitra NJ, Huang X, Hu SM (2014) Salientshape: group saliency in image collections. Vis Comput 30(4):443–453
Cornia M, Baraldi L, Serra G, Cucchiara R (2018) Predicting human eye fixations via an lstm-based saliency attentive model. IEEE Trans Image Process 27(10):5142–5154
Duan L, Wu C, Miao J, Qing L, Fu Y (2011) Visual saliency detection by spatially weighted dissimilarity. In: Computer vision and pattern recognition (CVPR), 2011 IEEE Conference on, IEEE, pp 473–480
Erdem E, Erdem A (2013) Visual saliency estimation by nonlinearly integrating features using region covariances. J Vis 13(4):11–11
Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The pascal visual object classes (voc) challenge. Int J Comput Vis 88 (2):303–338
Fu Y, Cheng J, Li Z, Lu H (2008) Saliency cuts: An automatic approach to object segmentation. In: 2008 19th international conference on pattern recognition, IEEE, pp 1–4
GOFERMAN S (2010) Context-aware saliency detection. In: Proceedings of IEEE conference on computer vision and pattern recognition, 2010, pp 2376–2383
Hou Q, Cheng MM, Hu X, Borji A, Tu Z, Torr PH (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3203–3212
Hou X, Zhang L (2007) Saliency detection: A spectral residual approach. In: Computer vision and pattern recognition, 2007. CVPR’07. IEEE Conference on, IEEE, pp 1–8
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1-3):489–501
Huang GB, Zhu QY, Siew CK, et al. (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. Neural Netw 2:985–990
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 11:1254–1259
Ji Y, Zhang H, Tseng KK, Chow TW, Wu QJ (2019) Graph model-based salient object detection using objectness and multiple saliency cues. Neurocomputing 323:188–202
Jiang B, Zhang L, Lu H, Yang C, Yang MH (2013) Saliency detection via absorbing markov chain. In: Proceedings of the IEEE international conference on computer vision, pp 1665–1672
Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: A discriminative regional feature integration approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2083–2090
Lee YJ, Ghosh J, Grauman K (2012) Discovering important people and objects for egocentric video summarization. In: 2012 IEEE conference on computer vision and pattern recognition, IEEE, pp 1346–1353
Li J, Levine MD, An X, Xu X, He H (2013) Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans Pattern Anal Mach Intell 35(4):996–1010. https://doi.org/10.1109/TPAMI.2012.147
Li X, Li Y, Shen C, Dick A, Van Den Hengel A (2013) Contextual hypergraph modeling for salient object detection. In: Proceedings of the IEEE international conference on computer vision, pp 3328–3335
Li Y, Hou X, Koch C, Rehg JM, Yuille AL (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 280–287
Liu GH, Yang J (2019) Exploiting color volume and color difference for salient region detection. IEEE Trans Image Process 28(1):6–16
Liu N, Han J (2018) A deep spatial contextual long-term recurrent convolutional network for saliency detection. IEEE Trans Image Process 27(7):3264–3274
Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X, Shum H (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353
Minhas R, Baradarani A, Seifzadeh S, Wu QJ (2010) Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73(10-12):1906–1917
Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. In: Computer vision and pattern recognition (cvpr), 2011 ieee conference on, IEEE, pp 433–440
Pan C, Park DS, Yang Y, Yoo HM (2012) Leukocyte image segmentation by visual attention and extreme learning machine. Neural Comput and Applic 21(6):1217–1227
Qin Y, Lu H, Xu Y, Wang H (2015) Saliency detection via cellular automata. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 110–119
Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New York
Seo HJ, Milanfar P (2009) Static and space-time visual saliency detection by self-resemblance. J Vis 9(12):15–15
Serre D (2002) Matrices volume 216 of graduate texts in mathematics
Singh N, Arya R, Agrawal R (2016) A convex hull approach in conjunction with gaussian mixture model for salient object detection. Digit Signal Process 55:22–31
Wang J, Lu H, Li X, Tong N, Liu W (2015) Saliency detection via background and foreground seed selection. Neurocomputing 152:359–368
Xie Y, Lu H, Yang MH (2012) Bayesian saliency via low and mid level cues. IEEE Trans Image Process 22(5):1689–1698
Yan Q, Xu L, Shi J, Jia J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1155–1162
Yang C, Zhang L, Lu H (2013) Graph-regularized saliency detection with convex-hull-based center prior. IEEE Signal Process Lett 20(7):637–640
Yang C, Zhang L, Lu H, Ruan X, Yang MH (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173
Yang J, Yang MH (2016) Top-down visual saliency via joint crf and dictionary learning. IEEE Trans Pattern Anal Mach Intell 39(3):576–588
Yang Y, Li B, Li P, Liu Q (2018) A two-stage clustering based 3d visual saliency model for dynamic scenarios. IEEE Trans Multimed 21(4):809–820
Zhang J, Sclaroff S (2016) Exploiting surroundedness for saliency detection: a boolean map approach. IEEE Trans Pattern Anal Mach Intell 38(5):889–902
Zhang L, Li J, Lu H (2016) Saliency detection via extreme learning machine. Neurocomputing 218:103–112
Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) Sun: A bayesian framework for saliency using natural statistics. J Vis 8(7):32–32
Zhang L, Zhang D, Sun J, Wei G, Bo H (2019) Salient object detection by local and global manifold regularized svm model. Neurocomputing 340:42–54
Zhang W, Wu QJ, Wang G, Yin H (2010) An adaptive computational model for salient object detection. IEEE Trans Multimed 12(4):300–316
Zhu W, Liang S, Wei Y, Sun J (2014) Saliency optimization from robust background detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2814–2821
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Singh, V.K., Kumar, N. CHELM: Convex Hull based Extreme Learning Machine for salient object detection. Multimed Tools Appl 80, 13535–13558 (2021). https://doi.org/10.1007/s11042-020-10374-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-10374-x