Skip to main content
SpringerLink
Log in

3D visual saliency detection model with generated disparity map

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Due to the remarkable distinction between human monocular vision and binocular vision, stereoscopic visual attention becomes an emerging question in the study of 3D applications. Some of existing 3D visual saliency detection models take advantage of ground-truth disparity map to compute center-surround differences of the depth features with high computational cost. In some 3D applications, the ground-truth disparity map may not be always available. In this paper, an efficient and simple 3D visual saliency detection model is proposed without using ground-truth disparity map. The proposed model is based on a band-pass filtering method which coincides with the visual perceptual process in human visual system. Firstly, the monocular luminance, color and texture features are extracted from the left view’s image; the binocular depth feature is extracted from the two views’ disparity map. Then, all the feature maps are filtered to generate three types of saliency maps, i.e., 2D saliency map, texture saliency map and depth saliency map. Subsequently, the three saliency maps are fused to one 3D saliency map by a linear pooling strategy. Finally, the final 3D visual saliency map is enhanced by the center-bias factor. Experimental results on a public eye tracking database show that the proposed model achieves better detection performance with low computational cost among the existing 3D visual saliency detection models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Anderson SJ, Burr DC (1985) Spatial and temporal selectivity of the human motion detection system. Vis Res 25(8):1147–1154

    Article  Google Scholar 

  2. Borji A, Ahmadabadi MN, Araabi BN, Hamidi M (2010) Online learning of task-driven object-based visual attention control. J Image Vision Comput 28:1130–1145

    Article  Google Scholar 

  3. Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207

    Article  Google Scholar 

  4. Bruce N, Tsotsos J (2005) An attentional framework for stereo vision. In: Proc. 2nd IEEE Canadian Conf. Comput. Robot Vis., pp 88–95

  5. Bruce ND, Tsotsos JK (2006) Saliency based on information maximization. In: Proc. Adv. NIPS, pp 155–162

  6. Chamaret C, Godeffroy S, Lopez P, Le Meur O (2010) Adaptive 3D rendering based on region-of-interest. In: Proc. SPIE, Stereoscopic Displays and Applications XXI, vol. 7524, 75240V

  7. Chamaret C, Le Meur O, Chevet JC (2010) Spatio-temporal combination of saliency maps and eye-tracking assessment of different strategies. In: Proc. ICIP, pp 1077–1080

  8. Cheng MM, Zhang GX, Mitra NJ, Huang X, Hu SM (2011) Global contrast based salient region detection. CVPR, pp 409–416

  9. Cho SH, Kang HB (2014) A new saliency detection method for stereoscopic images using contrast and prior knowledge. Prog Pattern Recogn Image Anal Comput Vis Appl 8827:778–786

    Google Scholar 

  10. Ciptadi A, Hermans T, Rehg JM (2013) An in depth view of saliency. In: Proc. BMVC, pp 112.1–112.11

  11. Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J Opt Soc Am A 2(7):1160–1169

    Article  Google Scholar 

  12. Dittrich T, Kopf S, Schaber P, Guthier B, Effelsberg W (2013) Saliency detection for stereoscopic video. ACM, pp 12–23

  13. Fang Y, Chen Z, Lin W, Lin CW (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process 21(9):3888–3901

    Article  MathSciNet  MATH  Google Scholar 

  14. Fang YM, Wang JL, Narwaria M, Le Callet P, Lin WS (2014) Saliency detection for stereoscopic images. IEEE Trans Image Process 23(6):2625–2636

    Article  MathSciNet  MATH  Google Scholar 

  15. Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. J Opt Soc Am A 2379–2394

  16. Gautier J, Le Meur O (2012) A time-dependent saliency model combining center and depth biases for 2D and 3D viewing conditions. Cogn Comput 4(2):141–156

    Article  Google Scholar 

  17. Goferman S, Zelnik-Manor L, Tal A (2010) Context-aware saliency detection. In: Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., pp 2376–2383

  18. Guo C, Zhang L (2010) A novel multi-resolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans Image Process 19(1):185–198

    Article  MathSciNet  MATH  Google Scholar 

  19. Hakkinen J, Kawai T, Takatalo J, Mitsuya R, Nyman G (2010) What do people look at when they watch stereoscopic movies? In: Proc. SPIE, vol. 7524. Stereoscopic Displays and Applications XXI, pp 75240E

  20. Hirschmuller H (2008) Stereo processing by semiglobal matching and mutual information. IEEE Trans Pattern Anal Mach Intell 30(2):328–341

    Article  Google Scholar 

  21. Hou X, Zhang L (2007) Saliency detection: a spectral residual approach. In: Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., pp 1–8

  22. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259

    Article  Google Scholar 

  23. Jung YJ, Lee S, Sohn H, Park HW, Ro YM (2012) Visual comfort assessment metric based on salient object motion information in stereoscopic video. Proc. SPIE Electronic Imaging 011008

  24. Khaustova D, Fournier J, Wyckens E (2014) An investigation of visual selection priority of objects with texture and crossed and uncrossed disparities. In: Proc. of SPIE-IS&T Electronic Imaging, vol. 9014, 90140D

  25. Lin W, Jay Kuo C-C (2011) Perceptual visual quality metrics: a survey. J Vis Commun Image Represent 22(4):297–312

    Article  Google Scholar 

  26. Lu Z, Lin W, Yang X, Ong E, Yao S (2005) Modeling visual attention’s modulatory aftereffects on visual sensitivity and quality evaluation. IEEE Trans Image Process 14(11):1928–1942

    Article  Google Scholar 

  27. Maki A, Nordlund P, Eklundh J (1996) A computational model of depth based attention. Proc Pattern Recogn 4:734–739

    Google Scholar 

  28. Marčelja S (1980) Mathematical description of the responses of simple cortical cells. J Opt Soc Am 70(11):1297–1300

    Article  MathSciNet  Google Scholar 

  29. Niu Y, Geng Y, Li X, Liu F (2012) Leveraging stereopsis for saliency analysis. In: Proc. IEEE Int. Conf. CVPR, pp 454–461

  30. Ouerhani N, Hugli H (2000) Computing visual attention from scene depth. In: Proc. IEEE 15th Int. Conf. Pattern Recognit., pp 375–378

  31. Potapova E, Zillich M, Vincze M (2011) Learning what matters: combining probabilistic models of 2D and 3D saliency cues. In: Proc. 8th Int. Comput. Vis. Syst., pp 132–142

  32. Potapova E, Zillich M, Vincze M (2011) Learning what matters: combining probabilistic models of 2d and 3d saliency cues. Computer Vision Systems, pp 132–142

  33. Robertson AR (1977) The CIE 1976 color difference formulas. Color Res Appl 2(1):7–11

    Article  Google Scholar 

  34. Roe A, Parker A, Born R, De Angelis G (2007) Disparity channels in early vision. J Neurosci Off J Soc Neurosci 27(44):11820–11831

    Article  Google Scholar 

  35. Rosenholtz R, Dorai A, Freeman R (2011) Do predictions of visual perception aid design? ACM Trans Appl Percept 8(2)

  36. Siagian C, Itti L (2009) Biologically inspired mobile robot vision localization. IEEE Trans Robot 25(4):861–873

    Article  Google Scholar 

  37. Sun D, Roth S, Black M (2010) Secrets of optical flow estimation and their principles. In: CVPR, 2010. Available: http://www.cs.brown.edu/~black/

  38. Treisman AM, Gelade G (1980) A feature integration theory of attention. Cogn Psychol 12:97–136

    Article  Google Scholar 

  39. Wang J, Perreira Da Silva M, Le Callet P, Ricordel V (2013) Computational model of stereoscopic 3D visual saliency. IEEE Trans Image Process 22(6):2151–2165

    Article  MathSciNet  MATH  Google Scholar 

  40. Wolfe JM, Horowitz TS (2004) What attributes guide the deployment of visual attention and how do they do it? Nat Rev Neurosci 5:1–7

    Article  Google Scholar 

  41. Xing PY, Tian YH, Huang TJ, Gao W (2013) Surveillance video coding with quadtree partition based ROI extraction. In: Proceedings of 30th Picture Coding Symposium, pp 1–4

  42. Yan J, Liu J, Li Y, Niu Z, Liu Y (2010) Visual saliency detection via rank-sparsity decomposition. In: Proc. IEEE 17th ICIP, pp 1089–1092

  43. Zhai G, Kaupy A, Wang J, Yang X (2013) Retina model inspired image quality assessment. VCIP, pp 1–6

  44. Zhang L, Gu ZY, Li HY (2013) SDSP: a novel saliency detection method by combining simple priors. In: Proc. ICIP

  45. Zhang Y, Jiang G, Yu M, Chen K (2010) Stereoscopic visual attention model for 3d video. In: Proc. 16th Int. Conf. Adv. Multimedia Model, pp 314–324

  46. Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386

    Article  MathSciNet  MATH  Google Scholar 

  47. Zuo WM, Zhang L, Song CW, Zhang D (2013) Texture enhanced image denoising via gradient histogram preservation. CVPR

Download references

Acknowledgments

This work was supported in part by the Major State Basic Research Development Program of China 973 Program under Grant 2015CB351804 and in part by the National Science Foundation of China (NSFC) under Grant 61272386, 61472101 and 61390513.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, China

    Feng Qi, Debin Zhao, Shaohui Liu & Xiaopeng Fan

Authors
  1. Feng Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Debin Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaohui Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaopeng Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feng Qi.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qi, F., Zhao, D., Liu, S. et al. 3D visual saliency detection model with generated disparity map. Multimed Tools Appl 76, 3087–3103 (2017). https://doi.org/10.1007/s11042-015-3229-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3229-6

Keywords

Search

Navigation