Abstract
Due to the remarkable distinction between human monocular vision and binocular vision, stereoscopic visual attention becomes an emerging question in the study of 3D applications. Some of existing 3D visual saliency detection models take advantage of ground-truth disparity map to compute center-surround differences of the depth features with high computational cost. In some 3D applications, the ground-truth disparity map may not be always available. In this paper, an efficient and simple 3D visual saliency detection model is proposed without using ground-truth disparity map. The proposed model is based on a band-pass filtering method which coincides with the visual perceptual process in human visual system. Firstly, the monocular luminance, color and texture features are extracted from the left view’s image; the binocular depth feature is extracted from the two views’ disparity map. Then, all the feature maps are filtered to generate three types of saliency maps, i.e., 2D saliency map, texture saliency map and depth saliency map. Subsequently, the three saliency maps are fused to one 3D saliency map by a linear pooling strategy. Finally, the final 3D visual saliency map is enhanced by the center-bias factor. Experimental results on a public eye tracking database show that the proposed model achieves better detection performance with low computational cost among the existing 3D visual saliency detection models.
Similar content being viewed by others
References
Anderson SJ, Burr DC (1985) Spatial and temporal selectivity of the human motion detection system. Vis Res 25(8):1147–1154
Borji A, Ahmadabadi MN, Araabi BN, Hamidi M (2010) Online learning of task-driven object-based visual attention control. J Image Vision Comput 28:1130–1145
Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207
Bruce N, Tsotsos J (2005) An attentional framework for stereo vision. In: Proc. 2nd IEEE Canadian Conf. Comput. Robot Vis., pp 88–95
Bruce ND, Tsotsos JK (2006) Saliency based on information maximization. In: Proc. Adv. NIPS, pp 155–162
Chamaret C, Godeffroy S, Lopez P, Le Meur O (2010) Adaptive 3D rendering based on region-of-interest. In: Proc. SPIE, Stereoscopic Displays and Applications XXI, vol. 7524, 75240V
Chamaret C, Le Meur O, Chevet JC (2010) Spatio-temporal combination of saliency maps and eye-tracking assessment of different strategies. In: Proc. ICIP, pp 1077–1080
Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. CVPR, pp 409–416
Cho SH, Kang HB (2014) A new saliency detection method for stereoscopic images using contrast and prior knowledge. Prog Pattern Recogn Image Anal Comput Vis Appl 8827:778–786
Ciptadi A, Hermans T, Rehg JM (2013) An in depth view of saliency. In: Proc. BMVC, pp 112.1–112.11
Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J Opt Soc Am A 2(7):1160–1169
Dittrich T, Kopf S, Schaber P, Guthier B, Effelsberg W (2013) Saliency detection for stereoscopic video. ACM, pp 12–23
Fang Y, Chen Z, Lin W, Lin CW (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process 21(9):3888–3901
Fang YM, Wang JL, Narwaria M, Le Callet P, Lin WS (2014) Saliency detection for stereoscopic images. IEEE Trans Image Process 23(6):2625–2636
Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. J Opt Soc Am A 2379–2394
Gautier J, Le Meur O (2012) A time-dependent saliency model combining center and depth biases for 2D and 3D viewing conditions. Cogn Comput 4(2):141–156
Goferman S, Zelnik-Manor L, Tal A (2010) Context-aware saliency detection. In: Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., pp 2376–2383
Guo C, Zhang L (2010) A novel multi-resolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans Image Process 19(1):185–198
Hakkinen J, Kawai T, Takatalo J, Mitsuya R, Nyman G (2010) What do people look at when they watch stereoscopic movies? In: Proc. SPIE, vol. 7524. Stereoscopic Displays and Applications XXI, pp 75240E
Hirschmuller H (2008) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341
Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., pp 1–8
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259
Jung YJ, Lee S, Sohn H, Park HW, Ro YM (2012) Visual comfort assessment metric based on salient object motion information in stereoscopic video. Proc. SPIE Electronic Imaging 011008
Khaustova D, Fournier J, Wyckens E (2014) An investigation of visual selection priority of objects with texture and crossed and uncrossed disparities. In: Proc. of SPIE-IS&T Electronic Imaging, vol. 9014, 90140D
Lin W, Jay Kuo C-C (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312
Lu Z, Lin W, Yang X, Ong E, Yao S (2005) Modeling visual attention’s modulatory aftereffects on visual sensitivity and quality evaluation. IEEE Trans Image Process 14(11):1928–1942
Maki A, Nordlund P, Eklundh J (1996) A computational model of depth based attention. Proc Pattern Recogn 4:734–739
Marčelja S (1980) Mathematical description of the responses of simple cortical cells. J Opt Soc Am 70(11):1297–1300
Niu Y, Geng Y, Li X, Liu F (2012) Leveraging stereopsis for saliency analysis. In: Proc. IEEE Int. Conf. CVPR, pp 454–461
Ouerhani N, Hugli H (2000) Computing visual attention from scene depth. In: Proc. IEEE 15th Int. Conf. Pattern Recognit., pp 375–378
Potapova E, Zillich M, Vincze M (2011) Learning what matters: combining probabilistic models of 2D and 3D saliency cues. In: Proc. 8th Int. Comput. Vis. Syst., pp 132–142
Potapova E, Zillich M, Vincze M (2011) Learning what matters: combining probabilistic models of 2d and 3d saliency cues. Computer Vision Systems, pp 132–142
Robertson AR (1977) The CIE 1976 color difference formulas. Color Res Appl 2(1):7–11
Roe A, Parker A, Born R, De Angelis G (2007) Disparity channels in early vision. J Neurosci Off J Soc Neurosci 27(44):11820–11831
Rosenholtz R, Dorai A, Freeman R (2011) Do predictions of visual perception aid design? ACM Trans Appl Percept 8(2)
Siagian C, Itti L (2009) Biologically inspired mobile robot vision localization. IEEE Trans Robot 25(4):861–873
Sun D, Roth S, Black M (2010) Secrets of optical flow estimation and their principles. In: CVPR, 2010. Available: http://www.cs.brown.edu/~black/
Treisman AM, Gelade G (1980) A feature integration theory of attention. Cogn Psychol 12:97–136
Wang J, Perreira Da Silva M, Le Callet P, Ricordel V (2013) Computational model of stereoscopic 3D visual saliency. IEEE Trans Image Process 22(6):2151–2165
Wolfe JM, Horowitz TS (2004) What attributes guide the deployment of visual attention and how do they do it? Nat Rev Neurosci 5:1–7
Xing PY, Tian YH, Huang TJ, Gao W (2013) Surveillance video coding with quadtree partition based ROI extraction. In: Proceedings of 30th Picture Coding Symposium, pp 1–4
Yan J, Liu J, Li Y, Niu Z, Liu Y (2010) Visual saliency detection via rank-sparsity decomposition. In: Proc. IEEE 17th ICIP, pp 1089–1092
Zhai G, Kaupy A, Wang J, Yang X (2013) Retina model inspired image quality assessment. VCIP, pp 1–6
Zhang L, Gu ZY, Li HY (2013) SDSP: a novel saliency detection method by combining simple priors. In: Proc. ICIP
Zhang Y, Jiang G, Yu M, Chen K (2010) Stereoscopic visual attention model for 3d video. In: Proc. 16th Int. Conf. Adv. Multimedia Model, pp 314–324
Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386
Zuo WM, Zhang L, Song CW, Zhang D (2013) Texture enhanced image denoising via gradient histogram preservation. CVPR
Acknowledgments
This work was supported in part by the Major State Basic Research Development Program of China 973 Program under Grant 2015CB351804 and in part by the National Science Foundation of China (NSFC) under Grant 61272386, 61472101 and 61390513.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Qi, F., Zhao, D., Liu, S. et al. 3D visual saliency detection model with generated disparity map. Multimed Tools Appl 76, 3087–3103 (2017). https://doi.org/10.1007/s11042-015-3229-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-015-3229-6