Identifying Major Components of Pictures by Audio Encoding of Colours
The goal of the See ColOr project is to achieve a non-invasive mobility aid for blind users that will use the auditory pathway to represent in real-time frontal image scenes. More particularly, we have developed a prototype which transforms HSL coloured pixels into spatialized classical instrument sounds lasting for 300 ms. Hue is sonified by the timbre of a musical instrument, saturation is one of four possible notes, and luminosity is represented by bass when luminosity is rather dark and singing voice when it is relatively bright. Our first experiments are devoted to static images on the computer screen. Six participants with their eyes covered by a dark tissue were trained to associate colours with musical instruments and then asked to determine on several pictures, objects with specific shapes and colours. In order to simplify the protocol of experiments, we used a tactile tablet, which took the place of the camera. Overall, experiment participants found that colour was helpful for the interpretation of image scenes.
Unable to display preview. Download preview PDF.
- 1.Algazi, V.R., Duda, R.O., Thompson, D.P.: Avendano. The CIPIC HRTF Database. In: IEEE Proc. Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA’01), Mohonk Mountain House, New Paltz, NY (2001)Google Scholar
- 2.Bologna, G., Vinckenbosch, M.: Eye Tracking in Coloured Image Scenes Represented by Ambisonic Fields of Musical Instrument Sounds. In: Mira, J., Álvarez, J.R. (eds.) IWINAC 2005. LNCS, vol. 3561, pp. 327–337. Springer, Heidelberg (2005)Google Scholar
- 3.Bamford, J.S.: An Analysis of Ambisonic Sound Systems of First and Second Order. Master Thesis, Waterloo, Ontario, Canada (1995)Google Scholar
- 6.Daniel, J.: Acoustic Field Representation, Application to the Transmission and the Reproduction of Complex Sound Environments in a Multimedia Context. PhD thesis, University of Paris 6 (2000)Google Scholar
- 7.Gerzon, M.A.: Design of Ambisonic Decoders for Multispeaker Surround Sound. Journal of the Audio Engineering Society 25, 1064 (1977)Google Scholar
- 8.Gonzalez-Mora, J.L., Rodriguez-Hernandez, A., Rodriguez-Ramos, L.F., Dfaz-Saco, L., Sosa, N.: Development of a New Space Perception System for Blind People, Based on the Creation of a Virtual Acoustic Space. In: Mira, J. (ed.) IWANN 1999. LNCS, vol. 1607, pp. 321–330. Springer, Heidelberg (1999)CrossRefGoogle Scholar