A Real-Time Event-Based Selective Attention System for Active Vision

Abstract

In real world scenarios, guiding vision to focus on salient parts of the visual space is a computationally demanding tasks. Selective attention is a biologically inspired strategy to cope with this problem, that can be used in engineered systems with limited resources. In active vision systems however, the stringent realtime requirements limit the space of solutions that can be achieved with conventional machine vision techniques and systems.We propose a hybrid approach where we combine a custom neuromorphic VLSI saliency-map based attention system with a conventional machine vision system, to implement both fast contrast-based saccadic eye movements in parallel with conventional visual attention models that use high-resolution color input images. We describe the system and characterize its response properties with experiments using both basic control visual stimuli and natural scenes.

Keywords

Selective Attention Vision Sensor Active Vision Salient Region Very Large Scale Integra 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bartolozzi, C., Indiveri, G.: Selective attention in multi-chip address-event systems. Sensors 9(7), 5076–5098 (2009), http://www.mdpi.com/1424-8220/9/6/5076, doi:10.3390/s90705076CrossRefGoogle Scholar
  2. 2.
    Behrmann, M., Haimson, C.: The cognitive neuroscience of visual attention. Current Opinion in Neurobiology 9, 158–163 (1999)CrossRefGoogle Scholar
  3. 3.
    Bernays, E.: Selective attention and host-plant specialization. Entomologia Experimentalis et Applicata 80(1), 125–131 (1996)CrossRefGoogle Scholar
  4. 4.
    Boahen, K.A.: Point-to-point connectivity between neuromorphic chips using address-events. IEEE Transactions on Circuits and Systems II 47(5), 416–434 (2000)MATHCrossRefGoogle Scholar
  5. 5.
    Chan, V., Liu, S.C., van Schaik, A.: AER EAR: A matched silicon cochlea pair with address event representation interface. IEEE Transactions on Circuits and Systems I 54(1), 48–59 (2007), Special Issue on SensorsCrossRefGoogle Scholar
  6. 6.
    Culham, J., Brandt, S., Cavanagh, P., Kanwisher, N., Dale, A., Tootell, R.: Cortical fMRI activation produced by attentive tracking of moving targets. J. Neurophysiol. 81, 388–393 (1999)Google Scholar
  7. 7.
    Deiss, S., Douglas, R., Whatley, A.: A pulse-coded communications infrastructure for neuromorphic systems. In: Maass, W., Bishop, C. (eds.) Pulsed Neural Networks, ch. 6, pp. 157–178. MIT Press (1998)Google Scholar
  8. 8.
    Delbrück, T.: Frame-free dynamic digital vision. In: Hotate, K., et al. (eds.) Proc. of the Intl. Symp. on Secure-Life Electronics, University of Tokyo, vol. 1, pp. 21–26 (2008)Google Scholar
  9. 9.
    Desimone, R., Duncan, J.: Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995)CrossRefGoogle Scholar
  10. 10.
    Fasnacht, D., Indiveri, G.: A PCI based high-fanout AER mapper with 2 GiB RAM look-up table, 0.8 μs latency and 66 mhz output event-rate. In: Conference on Information Sciences and Systems, CISS 2011, Johns Hopkins University, pp. 1–6 (2011), http://ncs.ethz.ch/pubs/pdf/Fasnacht_Indiveri11.pdf, doi:10.1109/CISS.2011.5766102
  11. 11.
    Fasnacht, D., Whatley, A., Indiveri, G.: A serial communication infrastructure for multi-chip address event system. In: International Symposium on Circuits and Systems, ISCAS 2008, pp. 648–651. IEEE (2008), http://ncs.ethz.ch/pubs/pdf/Fasnacht_etal08.pdf, doi: http://dx.doi.org/10.1109/ISCAS.2008.4541501
  12. 12.
    Frintrop, S., Rome, E., Christensen, H.: Computational visual attention systems and their cognitive foundation: A survey. ACM Transactions on Applied Perception 7(1), 1–46 (2010)CrossRefGoogle Scholar
  13. 13.
    Hoffman, J., Subramaniam, B.: The role of visual attention in saccadic eye movements. Perception and Psychophysics 57(6), 787–795 (1995)CrossRefGoogle Scholar
  14. 14.
    Indiveri, G.: A current-mode hysteretic winner-take-all network, with excitatory and inhibitory coupling. Analog Integrated Circuits and Signal Processing 28(3), 279–291 (2001), http://ncs.ethz.ch/pubs/pdf/Indiveri01.pdf CrossRefGoogle Scholar
  15. 15.
    Indiveri, G., Chicca, E., Douglas, R.: A VLSI array of low-power spiking neurons and bistable synapses with spike–timing dependent plasticity. IEEE Transactions on Neural Networks 17(1), 211–221 (2006), http://ncs.ethz.ch/pubs/pdf/Indiveri_etal06.pdf, doi:10.1109/TNN.2005.860850CrossRefGoogle Scholar
  16. 16.
    Indiveri, G., Linares-Barranco, B., Hamilton, T., van Schaik, A., Etienne-Cummings, R., Delbruck, T., Liu, S.C., Dudek, P., Häfliger, P., Renaud, S., Schemmel, J., Cauwenberghs, G., Arthur, J., Hynna, K., Folowossele, F., Saighi, S., Serrano-Gotarredona, T., Wijekoon, J., Wang, Y., Boahen, K.: Neuromorphic silicon neuron circuits. Frontiers in Neuroscience 5, 1–23 (2011), http://www.frontiersin.org/Neuromorphic_Engineering/10.3389/fnins.2011.00073/abstract, doi:10.3389/fnins.2011.00073Google Scholar
  17. 17.
    Itti, L.: Quantifying the contribution of low-level saliency to human eye movements in dynamic scenes. Visual Cognition 12(6), 1093–1123 (2005)CrossRefGoogle Scholar
  18. 18.
    Itti, L., Koch, C.: Computational modeling of visual attention. Nature Reviews Neuroscience 2(3), 194–203 (2001)CrossRefGoogle Scholar
  19. 19.
    Kastner, S., De Weerd, P., Desimone, R., Ungerleider, L.: Mechanisms of directed attention in the human extrastriate cortex as revealed by functional MRI. Science 282(2), 108–111 (1998)CrossRefGoogle Scholar
  20. 20.
    Lichtsteiner, P., Posch, C., Delbruck, T.: An 128x128 120dB 15μs-latency temporal contrast vision sensor. IEEE J. Solid State Circuits 43(2), 566–576 (2008)CrossRefGoogle Scholar
  21. 21.
    Miller, M.J., Bockisch, C.: Where are the things we see? Nature 386(10), 550–551 (1997)CrossRefGoogle Scholar
  22. 22.
    Mozer, M., Sitton, M.: Computational modeling of spatial attention. In: Pashler, H. (ed.) Attention, pp. 341–395. Psychology Press, East Sussex (1998)Google Scholar
  23. 23.
    Pollack, G.: Selective attention in an insect auditory neuron. Jour. Neurosci. 8, 2635–2639 (1988)Google Scholar
  24. 24.
    Sonnleithner, D., Indiveri, G.: Active vision driven by a neuromorphic selective attention system. Proc. of International Symposium on Autonomous Minirobots for Research and Edutainment, AMiRE 2011, 1–10 (2011), http://ncs.ethz.ch/pubs/pdf/Sonnleithner_Indiveri11b.pdf Google Scholar
  25. 25.
    Sonnleithner, D., Indiveri, G.: A neuromorphic saliency-map based active vision system. In: Conference on Information Sciences and Systems, CISS, Johns Hopkins University, pp. 1–6 (2011), http://ncs.ethz.ch/pubs/pdf/Sonnleithner_Indiveri11.pdf, doi:10.1109/CISS.2011.5766145

Copyright information

© Springer-Verlag GmbH Berlin Heidelberg 2012

Authors and Affiliations

  1. 1.Institute of NeuroinformaticsUniversity of Zurich and ETH ZurichZurichSwitzerland

Personalised recommendations