Abstract
A design for an implemented, prototype wearable artificial sensory system is presented, which uses data sonification to compensate for normal limitations in the human visual system. The system gives insight into the complete visible‐light spectra from objects being seen by the user. Long‐term wear and consequent training might lead to identification of various visually‐indistinguishable materials based on the sounds of their spectra. A detailed system design and results of user testing are presented, and many possible extensions to both the sonification and the sensor package are discussed.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
H. Anderson, ed., A Physicist's Desk Reference (Am. Inst. of Phys., 1989).
T. Cornsweet, Visual Perception (Harcourt Brace Jovanovich, 1970).
R.E. Cytowic, Synesthesia: A Union of the Senses, Springer Series in Neuropsychology (Springer-Verlag, 1989).
E. Fubini, A. De Bono and G. Ruspa, System for monitoring and indicating acoustically the operating conditions of a motor vehicle, US Patent No. 4,785,280, US Patent and Trademark Office.
R.H. Gilkey and T.R. Anderson, eds., Binaural and Spatial Hearing in Real and Virtual Environments (Erlbaum, 1997).
S. Handel, Listening (MIT Press, 1989).
J.E. Harrison and S. Baron-Cohen, eds., Synaesthesia: Classic and Contemporary Readings (Blackwell, 1996).
C. Hertzman, S.D. Walter, L. From and A. Alison, Observer perception of skin color in a study of malignant melanoma, American J. Epidemiology, The Johns Hopkins University School of Hygiene and Public Health, 126(5) (1987).
G. Kramer, ed., Auditory Display: Sonification, Audification, and Auditory Interfaces (Addison–Wesley, 1994).
G. Kramer, An introduction to auditory display, in: Auditory Display: Sonification, Audification, and Auditory Interfaces, ed. G. Kramer (Addison–Wesley, 1994).
D.R. Lide, editor-in-chief, The CRC Handbook of Chemistry and Physics, 76th edn. (CRC Press, 1996).
D. Lunney and R. Morrison, High technology laboratory aids for visually handicapped chemistry students, J. Chemical Education 58 (1981) 228–231.
R. Marchesini, M. Brambilla, C. Clemente, M. Maniezzo, A.E. Sichirollo, A. Testori, D.R. Venturoli and N. Cascinelli, In vivo spectrophotometric evaluation of neoplastic and non-neoplastic skin pigmented lesions – I. Reflectance measurements, Photochemistry and Photobiology 53(1) (1991) 77–84.
R.D. Patterson, Guidelines for auditory warning systems on civil aircraft, Civil Aviation Authority, London (1982).
I. Pollack and L. Ficks, Information of elementary multidimensional displays, J. Acoustical Society of America 26(2) (1954) 155–158.
S. Pentland, T. Starner and K. Pipe, Personal communication.
E. Wenzel, Spatial sounds and sonification, in: Auditory Display: Sonification, Audification, and Auditory Interfaces, ed. G. Kramer (Addison–Wesley, 1994).
M. Winston, The Biology of the Honey Bee (Harvard Univ. Press, 1987).
W.A. Yost and D.W. Nielsen, Fundamentals of Hearing, 2nd edn. (Holt, Rinehart & Winston, 1985).
L. Young and D. Sheena, Survey of eye movement recording methods, Behavior Research Methods & Instrumentation 7(5) (1975) 397–429.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Foner, L.N. Artificial synesthesia via sonification: A wearable augmented sensory system. Mobile Networks and Applications 4, 75–81 (1999). https://doi.org/10.1023/A:1019178210975
Issue Date:
DOI: https://doi.org/10.1023/A:1019178210975