Biologically Motivated Local Contextual Modulation Improves Low-Level Visual Feature Representations

  • Xun Shi
  • Neil D. B. Bruce
  • John K. Tsotsos
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7324)


This paper describes a biologically motivated local context operator to improve low-level visual feature representations. The computation borrows the idea from the primate visual system that different visual features are computed with different speeds in the visual system and thus they can positively affect each other via early recurrent modulations. The modulation improves visual representation by suppressing responses with respect to background pixels, cluttered scene parts and image noise. The proposed local contextual computation is fundamentally different from exiting approaches that involve “whole scene” perspectives. Context-modulated visual feature representations are tested in a variety of existing saliency algorithms. Using real images and videos, we quantitatively compare output saliency representations between modulated and non-modulated architectures with respect to human experimental data. Results clearly demonstrate that local contextual modulation has a positive and consistent impact on the saliency computation.


local context visual saliency recurrent modulation 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Anderson, C.H., Van Essen, D.C.: Shifter circuits: a computational strategy for dynamic aspects of visual processing. PNAS 84(17), 6297–6301 (1987)CrossRefGoogle Scholar
  2. 2.
    Bar, M.: Visual objects in context. Nat. Rev. Neurosci. 5(8), 617–629 (2004)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Bradley, D.C., Goyal, M.S.: Velocity computation in the primate visual system. Nature Reviews Neuroscience 9(9), 686–695 (2008)CrossRefGoogle Scholar
  4. 4.
    Bruce, N.D., Tsotsos, J.K.: Spatiotemporal saliency: Towards a hierarchical representation of visual saliency. In: Paletta, L., Tsotsos, J.K. (eds.) Attention in Cognitive Systems, pp. 98–111. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  5. 5.
    Bruce, N.D.B., Tsotsos, J.K.: Saliency, attention, and visual search: An information theoretic approach. Journal of Vision 9(3) (2009)Google Scholar
  6. 6.
    Bullier, J.: Integrated model of visual processing. Brain Research Reviews 36(2-3), 96–107 (2001)CrossRefGoogle Scholar
  7. 7.
    Derrington, A.M., Lennie, P.: Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of macaque. The Journal of Physiology 357(1), 219–240 (1984)Google Scholar
  8. 8.
    Field, D.J.: Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4(12), 2379–2394 (1987)CrossRefGoogle Scholar
  9. 9.
    Gabbiani, F., Krapp, H.G., Koch, C., Laurent, G.: Multiplicative computation in a visual neuron sensitive to looming. Nature 420(6913), 320–324+ (2002)Google Scholar
  10. 10.
    Henderson, J.M., Hollingworth, A.: High-level scene perception. Annual Review of Psychology 50(1), 243–271 (1999)CrossRefGoogle Scholar
  11. 11.
    Hupé, J.M., James, A.C., Payne, B.R., Lomber, S.G., Girard, P., Bullier, J.: Cortical feedback improves discrimination between figure and background by V1, V2 and V3 neurons. Nature 394(6695), 784–787 (1998)CrossRefGoogle Scholar
  12. 12.
    Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(11), 1254–1259 (1998)CrossRefGoogle Scholar
  13. 13.
    Kaplan, E., Marcus, S., So, Y.T.: Effects of dark adaptation on spatial and temporal properties of receptive fields in cat lateral geniculate nucleus. The Journal of Physiology 294(1), 561–580 (1979)Google Scholar
  14. 14.
    McAdams, C.J., Maunsell, J.H.R.: Effects of attention on orientation-tuning functions of single neurons in macaque cortical area v4. The Journal of Neuroscience 19(1), 431–441 (1999)Google Scholar
  15. 15.
    Nowak, L., Munk, M., Girard, P., Bullier, J.: Visual latencies in areas v1 and v2 of the macaque monkey. Visual Neuroscience 12(02), 371–384 (1995)CrossRefGoogle Scholar
  16. 16.
    Oliva, A.: Gist of the scene. In: Itti, L., Rees, G., Tsotsos, J.K. (eds.) The Encyclopedia of Neurobiology of Attention, pp. 251–256. Elsevier, CA (2005)Google Scholar
  17. 17.
    Oliva, A., Torralba, A.: Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope. Int. J. Comput. Vision 42(3), 145–175 (2001)zbMATHCrossRefGoogle Scholar
  18. 18.
    Schyns, P.G., Oliva, A.: From blobs to boundary edges: Evidence for time and spatial scale dependent scene recognition. Psychological Science 5, 195–200 (1994)CrossRefGoogle Scholar
  19. 19.
    Shi, X., Bruce, N., Tsotsos, J.: Fast, recurrent, attentional modulation improves saliency representation and scene recognition. In: 2011 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1–8 (June 2011)Google Scholar
  20. 20.
    Ungerleider, L.G., Mishkin, M.: Two Cortical Visual Systems, ch. 18, pp. 549–586 (1982)Google Scholar
  21. 21.
    Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: Sun: A bayesian framework for saliency using natural statistics. Journal of Vision 8(7) (2008)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Xun Shi
    • 1
  • Neil D. B. Bruce
    • 1
  • John K. Tsotsos
    • 1
  1. 1.Department of Computer Science & Engineering, and, Centre for Vision ResearchYork UniversityTorontoCanada

Personalised recommendations