Advertisement

Saliency Preservation in Low-Resolution Grayscale Images

  • Shivanthan Yohanandan
  • Andy Song
  • Adrian G. Dyer
  • Dacheng Tao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11210)

Abstract

Visual salience detection originated over 500 million years ago and is one of nature’s most efficient mechanisms. In contrast, many state-of-the-art computational saliency models are complex and inefficient. Most saliency models process high-resolution color images; however, insights into the evolutionary origins of visual salience detection suggest that achromatic low-resolution vision is essential to its speed and efficiency. Previous studies showed that low-resolution color and high-resolution grayscale images preserve saliency information. However, to our knowledge, no one has investigated whether saliency is preserved in low-resolution grayscale (LG) images. In this study, we explain the biological and computational motivation for LG, and show, through a range of human eye-tracking and computational modeling experiments, that saliency information is preserved in LG images. Moreover, we show that using LG images leads to significant speedups in model training and detection times and conclude by proposing LG images for fast and efficient salience detection.

Keywords

Saliency detection Fully convolutional network Peripheral vision 

Notes

Acknowledgements

This research was supported by an Australian Postgraduate Award scholarship, the Professor Robert and Josephine Shanks scholarship, and Australian Research Council grants FL-170100117, DP-180103424, and LP-150100671. The authors wish to thank the eye tracking participants for volunteering their time and Mr Wei Li for helping with the experiments.

References

  1. 1.
    McMains, S.A., Kastner, S.: Visual attention. In: Binder, M.D., Hirokawa, N., Windhorst, U. (eds.) Encyclopedia of Neuroscience, pp. 4296–4302. Springer, Heidelberg (2009).  https://doi.org/10.1007/978-3-540-29678-2CrossRefGoogle Scholar
  2. 2.
    Morawetz, L., Spaethe, J.: Visual attention in a complex search task differs between honeybees and bumblebees. J. Exp. Biol. 215(Pt 14), 2515–2523 (2012)CrossRefGoogle Scholar
  3. 3.
    Avarguès-Weber, A., Dyer, A.G., Ferrah, N., Giurfa, M.: The forest or the trees: preference for global over local image processing is reversed by prior experience in honeybees. Proc. Roy. Soc. B Biol. Sci. 282(1799), 20142384 (2015)CrossRefGoogle Scholar
  4. 4.
    Morawetz, L., Svoboda, A., Spaethe, J., Dyer, A.G.: Blue colour preference in honeybees distracts visual attention for learning closed shapes. J. Comp. Physiol. A Neuroethol. Sens. Neural Behav. Physiol. 199(10), 817–827 (2013)CrossRefGoogle Scholar
  5. 5.
    Hou, W., Gao, X., Tao, D., Li, X.: Visual saliency detection using information divergence. Pattern Recogn. 46(10), 2658–2669 (2013)CrossRefGoogle Scholar
  6. 6.
    Kümmerer, M., Wallis, T.S.A., Bethge, M.: DeepGaze II: reading fixations from deep features trained on object recognition. arXiv:1610.01563 [cs, q-bio, stat], October 2016
  7. 7.
    Huang, X., Shen, C., Boix, X., Zhao, Q.: SALICON: reducing the semantic gap in saliency prediction by adapting deep neural networks. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 262–270, December 2015Google Scholar
  8. 8.
    Kruthiventi, S.S.S., Ayush, K., Babu, R.V.: DeepFix: a fully convolutional neural network for predicting human eye fixations. IEEE Trans. Image Process. 26(9), 4446–4456 (2017)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Cornia, M., Baraldi, L., Serra, G., Cucchiara, R.: Predicting human eye fixations via an LSTM-based saliency attentive model. arXiv:1611.09571 [cs], November 2016
  10. 10.
    Rajankar, O.S., Kolekar, U.D.: Scale space reduction with interpolation to speed up visual saliency detection. Int. J. Image Graph. Sig. Process. (IJIGSP) 7(8), 58 (2015)CrossRefGoogle Scholar
  11. 11.
    Wang, W., Shen, J.: Deep visual attention prediction. arXiv:1705.02544 [cs], May 2017
  12. 12.
    Vo, A.V., et al.: Processing of extremely high resolution LiDAR and RGB data. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 9(12), 5560–5575 (2016)CrossRefGoogle Scholar
  13. 13.
    Lamb, T.D.: Evolution of phototransduction, vertebrate photoreceptors and retina. In: Kolb, H., Fernandez, E., Nelson, R. (eds.) Webvision: The Organization of the Retina and Visual System. University of Utah Health Sciences Center, Salt Lake City (UT) (1995)Google Scholar
  14. 14.
    Judd, T., Durand, F., Torralba, A.: Fixations on low-resolution images. J. Vis. 11(4), 1–20 (2011)CrossRefGoogle Scholar
  15. 15.
    Shen, C., Huang, X., Zhao, Q.: Learning of proto-object representations via fixations on low resolution. ArXiv e-prints, December 2014. arXiv:1412.7242
  16. 16.
    Ho-Phuoc, T., Guyader, N., Landragin, F., Guérin-Dugué, A.: When viewing natural scenes, do abnormal colors impact on spatial or temporal parameters of eye movements? J. Vis. 12(2), 4 (2012)CrossRefGoogle Scholar
  17. 17.
    Hamel, S., Guyader, N., Pellerin, D., Houzet, D.: Contribution of color information in visual saliency model for videos. In: Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D. (eds.) ICISP 2014. LNCS, vol. 8509, pp. 213–221. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-07998-1_24CrossRefGoogle Scholar
  18. 18.
    Hamel, S., Guyader, N., Pellerin, D., Houzet, D.: Contribution of color in saliency model for videos. Sig. Image Video Process. 10(3), 423–429 (2016)CrossRefGoogle Scholar
  19. 19.
    Hamel, S., Houzet, D., Pellerin, D., Guyader, N.: Does color influence eye movements while exploring videos? J. Eye Mov. Res. 8(1) (2015)Google Scholar
  20. 20.
    Frey, H.P., Honey, C., König, P.: What’s color got to do with it? The influence of color on visual attention in different categories. J. Vis. 8(14), 6 (2008)CrossRefGoogle Scholar
  21. 21.
    Baddeley, R.J., Tatler, B.W.: High frequency edges (but not contrast) predict where we fixate: a Bayesian system identification analysis. Vis. Res. 46(18), 2824–2833 (2006)CrossRefGoogle Scholar
  22. 22.
    Dorr, M., Vig, E., Barth, E.: Colour saliency on video. In: Suzuki, J., Nakano, T. (eds.) BIONETICS 2010. LNICST, vol. 87, pp. 601–606. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-32615-8_59CrossRefGoogle Scholar
  23. 23.
    Xu, Y., Xiao, T., Zhang, J., Yang, K., Zhang, Z.: Scale-invariant convolutional neural networks. arXiv:1411.6369 [cs], November 2014
  24. 24.
    Advani, S., Sustersic, J., Irick, K., Narayanan, V.: A multi-resolution saliency framework to drive foveation. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2596–2600, May 2013Google Scholar
  25. 25.
    Lindeberg, T., Florack, L.: Foveal Scale-space and the Linear Increase of Receptive Field Size as a Function of Eccentricity. KTH Royal Institute of Technology, Stockholm (1994)Google Scholar
  26. 26.
    Koenderink, J.J., Doorn, A.J.V.: Visual detection of spatial contrast; Influence of location in the visual field, target extent and illuminance level. Biol. Cybern. 30(3), 157–167 (1978)CrossRefGoogle Scholar
  27. 27.
    Romeny, B.M.H.: Front-End Vision and Multi-Scale Image Analysis: Multi-scale Computer Vision Theory and Applications, Written in Mathematica. Springer, Heidelberg (2008)Google Scholar
  28. 28.
    Treisman, A.M., Gelade, G.: A feature-integration theory of attention. Cogn. Psychol. 12(1), 97–136 (1980)CrossRefGoogle Scholar
  29. 29.
    Humphrey, G.K., Goodale, M.A., Jakobson, L.S., Servos, P.: The role of surface information in object recognition: studies of a visual form agnosic and normal subjects. Perception 23(12), 1457–1481 (1994)CrossRefGoogle Scholar
  30. 30.
    Gegenfurtner, K.R., Rieger, J.: Sensory and cognitive contributions of color to the recognition of natural scenes. Curr. Biol. 10(13), 805–808 (2000)CrossRefGoogle Scholar
  31. 31.
    Lennie, P.: Color vision: putting it together. Curr. Biol. 10(16), R589–R591 (2000)CrossRefGoogle Scholar
  32. 32.
    Yang, Y., Song, M., Bu, J., Chen, C., Jin, C.: Color to gray: attention preservation. In: 2010 Fourth Pacific-Rim Symposium on Image and Video Technology, pp. 337–342, November 2010Google Scholar
  33. 33.
    Pope, B., Druyan, A., Soter, S., deGrasse Tyson, N., Hanich, L., Holtzman, S.: Cosmos: A Spacetime Odyssey (episode 2: some of the things that molecules do) (2014)Google Scholar
  34. 34.
    Nilsson, D.E.: The evolution of eyes and visually guided behaviour. Philos. Trans. Roy. Soc. Lond. B: Biol. Sci. 364(1531), 2833–2847 (2009)CrossRefGoogle Scholar
  35. 35.
    Potter, M.C., Wyble, B., Hagmann, C.E., McCourt, E.S.: Detecting meaning in RSVP at 13 ms per picture. Atten. Percept. Psychophys. 76(2), 270–279 (2014)CrossRefGoogle Scholar
  36. 36.
    Stojcev, M., Radtke, N., D’Amaro, D., Dyer, A.G., Neumeyer, C.: General principles in motion vision: color blindness of object motion depends on pattern velocity in honeybee and goldfish. Vis. Neurosci. 28(4), 361–370 (2011)CrossRefGoogle Scholar
  37. 37.
    Wandell, B.A.: Foundations of Vision. Sinauer Associates, Sunderland (1995)Google Scholar
  38. 38.
    Veale, R., Hafed, Z.M., Yoshida, M.: How is visual salience computed in the brain? Insights from behaviour, neurobiology and modelling. Philos. Trans. Roy. Soc. B: Biol. Sci. 372(1714), 20160113 (2017)CrossRefGoogle Scholar
  39. 39.
    Okawa, H., Sampath, A.P.: Optimization of single-photon response transmission at the rod-to-rod bipolar synapse. Physiology (Bethesda, Md.) 22, 279–286 (2007)Google Scholar
  40. 40.
    White, B.J., Kan, J.Y., Levy, R., Itti, L., Munoz, D.P.: Superior colliculus encodes visual saliency before the primary visual cortex. Proc. Nat. Acad. Sci. 114(35), 9451–9456 (2017)CrossRefGoogle Scholar
  41. 41.
    Krauzlis, R.J., Lovejoy, L.P., Zénon, A.: Superior colliculus and visual spatial attention. Ann. Rev. Neurosci. 36(1), 165–182 (2013)CrossRefGoogle Scholar
  42. 42.
    Poynton, C.A.: Rehabilitation of gamma. Int. Soc. Opt. Photonics 3299, 232–250 (1998)Google Scholar
  43. 43.
    Poynton, C., Funt, B.: Perceptual uniformity in digital image representation and display. Color Res. Appl. 39(1), 6–15 (2014)CrossRefGoogle Scholar
  44. 44.
    Torralba, A.: How many pixels make an image? Vis. Neurosci. 26(1), 123–131 (2009)MathSciNetCrossRefGoogle Scholar
  45. 45.
    Wooding, D.S.: Fixation maps: quantifying eye-movement traces. In: Proceedings of the 2002 Symposium on Eye Tracking Research & Applications, pp. 31–36 (2002)Google Scholar
  46. 46.
    Tavakoli, H.R., Ahmed, F., Borji, A., Laaksonen, J.: Saliency revisited: analysis of mouse movements versus fixations. arXiv:1705.10546 [cs], May 2017
  47. 47.
    Borji, A., Itti, L.: CAT2000: a large scale fixation dataset for boosting saliency research. arXiv:1505.03581 [cs], May 2015
  48. 48.
    Bylinskii, Z., Judd, T., Oliva, A., Torralba, A., Durand, F.: What do different evaluation metrics tell us about saliency models? arXiv:1604.03605 [cs], April 2016
  49. 49.
    Peters, R.J., Iyer, A., Itti, L., Koch, C.: Components of bottom-up gaze allocation in natural images. Vis. Res. 45(18), 2397–2416 (2005)CrossRefGoogle Scholar
  50. 50.
    Liang, J., Zhang, Y.: Top down saliency detection via Kullback-Leibler divergence for object recognition. In: 2015 International Symposium on Bioelectronics and Bioinformatics (ISBB), pp. 200–203, October 2015Google Scholar
  51. 51.
    Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2106–2113, September 2009Google Scholar
  52. 52.
    Borji, A., Tavakoli, H.R., Sihite, D.N., Itti, L.: Analysis of scores, datasets, and models in visual saliency prediction. In: 2013 IEEE International Conference on Computer Vision, pp. 921–928, December 2013Google Scholar
  53. 53.
    Le Meur, O., Le Callet, P., Barba, D.: Predicting visual fixations on video based on low-level visual features. Vis. Res. 47(19), 2483–2498 (2007)CrossRefGoogle Scholar
  54. 54.
    Koch, C., Ullman, S.: Shifts in selective visual attention: towards the underlying neural circuitry. Hum. Neurobiol. 4(4), 219–227 (1985)Google Scholar
  55. 55.
    Engelke, U., et al.: A comparative study of fixation density maps. IEEE Trans. Image Process. 22(3), 1121–1133 (2013)MathSciNetCrossRefGoogle Scholar
  56. 56.
    Wang, W., Shen, J., Shao, L.: Video salient object detection via fully convolutional networks. IEEE Trans. Image Process. 27(1), 38–49 (2018)MathSciNetCrossRefGoogle Scholar
  57. 57.
    Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. J. Vis. 7(14), 4.1–17 (2007)CrossRefGoogle Scholar
  58. 58.
    Judd, T., Durand, F., Torralba, A.: A benchmark of computational models of saliency to predict human fixationsGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Shivanthan Yohanandan
    • 1
  • Andy Song
    • 1
  • Adrian G. Dyer
    • 1
  • Dacheng Tao
    • 2
  1. 1.RMIT UniversityMelbourneAustralia
  2. 2.UBTECH Sydney AI Centre, SIT, FEITThe University of SydneySydneyAustralia

Personalised recommendations