Eye Tracking in Coloured Image Scenes Represented by Ambisonic Fields of Musical Instrument Sounds

  • Guido Bologna
  • Michel Vinckenbosch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3561)


We present our recent project on visual substitution by Ambisonic 3D-sound fields. Ideally, our system should be used by blind or visually impaired subjects having already seen. The original idea behind our targeted prototype is the use of an eye tracker and musical instrument sounds encoding coloured pixels. The role of the eye tracker is to activate the process of attention inherent in the vision and to restore by simulation the mechanisms of central and peripheral vision. Moreover, we advocate the view that cerebral areas devoted to the integration of information will play a role by rebuilding a global image of the environment. Finally, the role of colour itself is to help subjects distinguishing coloured objects or perceiving textures, such as sky, walls, grass and trees, etc ...


Plane Wave Sound Source Musical Instrument Blind Person Spatial Hearing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Algazi, V.R., Duda, R.O., Thompson, D.P., Avendano: The CIPIC HRTF Database. In: IEEE Proc. WASPAA 2001, New Paltz, NY (2001)Google Scholar
  2. 2.
    Andersen, R.A., Snyder, L.H., Bradley, D.C., Xing, J.: Multimodal Representation of Space in the Posterior Parietal Cortex and its Use in Planning. Annu. Rev. Neurosci. 20, 303–330 (1997)CrossRefGoogle Scholar
  3. 3.
    Bamford, J.S.: An Analysis of Ambisonic Sound Systems of First and Second Order. Master Thesis, Waterloo, Ontario, Canada (1995)Google Scholar
  4. 4.
    Begault, R.: 3-D Sound for Virtual Reality and Multimedia. Boston A.P. Professional (1994) ISBN: 0120847353Google Scholar
  5. 5.
    Brown, C.P., Duda, R.O.: A Structural Model for Binaural Sound Synthesis. IEEE Trans. Speech and Audio Processing 6(5) (1998)Google Scholar
  6. 6.
    Capelle, C., Trullemans, C., Arno, P., Veraart, C.: A Real Time Experimental Prototype for Enhancement of Vision Rehabilitation Using Auditory Substitution. IEEE T. Bio-Med. Eng. 45, 1279–1293 (1998)CrossRefGoogle Scholar
  7. 7.
    Colby, C.L., Goldberg, M.E.: Space and Attention in Parietal Cortex. Annu. Rev. Neurosci. 22, 319–349 (1999)CrossRefGoogle Scholar
  8. 8.
    Cronly-Dillon, J., Persaud, K., Gregory, R.P.F.: The Perception of Visual Images Encoded in Musical Form: a Study in Cross-Modality Information. Proc. Biological Sciences 266, 2427–2433 (1999)CrossRefGoogle Scholar
  9. 9.
    Daniel, J.: Acoustic Field Representation, Application to the Transmission and the Reproduction of Complex Sound Environments in a Multimedia Context. PhD thesis, University of Paris 6 (in French, English abstract available) (2000)Google Scholar
  10. 10.
    Gerzon, M.A.: Design of Ambisonic Decoders for Multispeaker Surround Sound. Journal of the Audio Engineering Society (Abstracts) 25, 1064 (1977)Google Scholar
  11. 11.
    Gonzalez-Mora, J.L., Rodriguez-Hernandez, A., Rodriguez-Ramos, L.F., Dfaz-Saco, L., Sosa, N.: Development of a New Space Perception System for Blind People, Based on the Creation of a Virtual Acoustic Space. In: Mira, J. (ed.) IWANN 1999. LNCS, vol. 1606, pp. 321–330. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  12. 12.
    Kay, L.: A Sonar Aid to Enhance Spatial Perception of the Blind: Engineering Design and Evaluation. The Radio and Electronic Engineer 44, 605–627 (1974)CrossRefGoogle Scholar
  13. 13.
    Malham, D.G., Myatt, A.: 3-D Sound Spatialisation using Ambisonic Techniques. Computer Music Journal 19(4), 58–70 (1995)CrossRefGoogle Scholar
  14. 14.
    Meijer, P.B.L.: An Experimental System for Auditory Image Representations. IEEE Transactions on Biomedical Engineering 39(2), 112–121 (1992)CrossRefGoogle Scholar
  15. 15.
    Wong, F.H.T., Nagarajan, R., Yaacob, S., Chekima, A., Belkhamza, N.E.: A Stereo Auditory Display for Visually Impaired. In: Proc. TENCON, pp. 377–383 (2000)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Guido Bologna
    • 1
  • Michel Vinckenbosch
    • 1
  1. 1.Laboratoire d’Informatique IndustrielleUniversity of Applied Science HES-SOGenevaSwitzerland

Personalised recommendations