Identifying Major Components of Pictures by Audio Encoding of Colours

  • Guido Bologna
  • Benoît Deville
  • Thierry Pun
  • Michel Vinckenbosch
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4528)


The goal of the See ColOr project is to achieve a non-invasive mobility aid for blind users that will use the auditory pathway to represent in real-time frontal image scenes. More particularly, we have developed a prototype which transforms HSL coloured pixels into spatialized classical instrument sounds lasting for 300 ms. Hue is sonified by the timbre of a musical instrument, saturation is one of four possible notes, and luminosity is represented by bass when luminosity is rather dark and singing voice when it is relatively bright. Our first experiments are devoted to static images on the computer screen. Six participants with their eyes covered by a dark tissue were trained to associate colours with musical instruments and then asked to determine on several pictures, objects with specific shapes and colours. In order to simplify the protocol of experiments, we used a tactile tablet, which took the place of the camera. Overall, experiment participants found that colour was helpful for the interpretation of image scenes.


Musical Instrument Image Scene Blind User Major Colour ColOr Project 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Algazi, V.R., Duda, R.O., Thompson, D.P.: Avendano. The CIPIC HRTF Database. In: IEEE Proc. Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA’01), Mohonk Mountain House, New Paltz, NY (2001)Google Scholar
  2. 2.
    Bologna, G., Vinckenbosch, M.: Eye Tracking in Coloured Image Scenes Represented by Ambisonic Fields of Musical Instrument Sounds. In: Mira, J., Álvarez, J.R. (eds.) IWINAC 2005. LNCS, vol. 3561, pp. 327–337. Springer, Heidelberg (2005)Google Scholar
  3. 3.
    Bamford, J.S.: An Analysis of Ambisonic Sound Systems of First and Second Order. Master Thesis, Waterloo, Ontario, Canada (1995)Google Scholar
  4. 4.
    Capelle, C., Trullemans, C., Arno, P., Veraart, C.: A Real Time Experimental Prototype for Enhancement of Vision Rehabilitation Using Auditory Substitution. IEEE T. Bio-Med Eng. 45, 1279–1293 (1998)CrossRefGoogle Scholar
  5. 5.
    Cronly-Dillon, J., Persaud, K., Gregory, R.P.F.: The Perception of Visual Images Encoded in Musical Form: a Study in Cross-Modality Information. Proc. Biological Sciences 266, 2427–2433 (1999)CrossRefGoogle Scholar
  6. 6.
    Daniel, J.: Acoustic Field Representation, Application to the Transmission and the Reproduction of Complex Sound Environments in a Multimedia Context. PhD thesis, University of Paris 6 (2000)Google Scholar
  7. 7.
    Gerzon, M.A.: Design of Ambisonic Decoders for Multispeaker Surround Sound. Journal of the Audio Engineering Society 25, 1064 (1977)Google Scholar
  8. 8.
    Gonzalez-Mora, J.L., Rodriguez-Hernandez, A., Rodriguez-Ramos, L.F., Dfaz-Saco, L., Sosa, N.: Development of a New Space Perception System for Blind People, Based on the Creation of a Virtual Acoustic Space. In: Mira, J. (ed.) IWANN 1999. LNCS, vol. 1607, pp. 321–330. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  9. 9.
    Kay, L.: A Sonar Aid to Enhance Spatial Perception of the Blind: Engineering Design and Evaluation. The Radio and Electronic Engineer 44, 605–627 (1974)CrossRefGoogle Scholar
  10. 10.
    Lakatos, S.: Recognition of Complex Auditory-Spatial Patterns. Perception 22, 363–374 (1993)CrossRefGoogle Scholar
  11. 11.
    Malham, D.G., Myatt, A.: 3-D Sound Spatialisation using Ambisonic Techniques. Computer Music Journal 19(4), 58–70 (1995)CrossRefGoogle Scholar
  12. 12.
    Meijer, P.B.L.: An Experimental System for Auditory Image Representations. IEEE Transactions on Biomedical Engineering 39(2), 112–121 (1992)CrossRefGoogle Scholar
  13. 13.
    Ruff, R.M., Perret, E.: Auditory Spatial Pattern Perception Aided by Visual Choices. Psychological Research 38, 369–377 (1976)CrossRefGoogle Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Guido Bologna
    • 1
  • Benoît Deville
    • 2
  • Thierry Pun
    • 2
  • Michel Vinckenbosch
    • 1
  1. 1.Laboratoire d’Informatique Industrielle, University of Applied Science HES-SO, Rue de la Prairie 4, 1202 GenevaSwitzerland
  2. 2.Computer Science Center, University of Geneva, Rue Général Dufour 24, 1211 GenevaSwitzerland

Personalised recommendations