Personal and Ubiquitous Computing

, Volume 10, Issue 4, pp 241–254 | Cite as

Acoustic environment as an indicator of social and physical context

  • Dan SmithEmail author
  • Ling Ma
  • Nick Ryan
Original Article


Acoustic environments provide many valuable cues for context-aware computing applications. From the acoustic environment we can infer the types of activity, communication modes and other actors involved in the activity. Environmental or background noise can be classified with a high degree of accuracy using recordings from microphones commonly found in PDAs and other consumer devices. We describe an acoustic environment recognition system incorporating an adaptive learning mechanism and its use in a noise tracker. We show how this information is exploited in a mobile context framework. To illustrate our approach we describe a context-aware multimodal weather forecasting service, which accepts spoken or written queries and presents forecast information in several forms, including email, voice and sign languages.


Acoustic environment Context awareness Classification Machine learning Adaptive feedback Mobile computing 



We wish to thank, first, the VisiCast project team at UEA, particularly Judy Tryggvason, for the signing output, and Mike Lincoln for discussions on the language model, second, Steve Dorling and others from WeatherQuest, and, third, the anonymous reviewers whose comments have been much appreciated in revising this paper.


  1. 1.
    Ma L, Smith DJ, Milner BP (2003) Context awareness using environmental noise classification. In: Proceedings of the Eurospeech 2003, Geneva, Switzerland, pp 2237–2240Google Scholar
  2. 2.
    Ma L, Smith DJ, Milner BP (2003) Environmental noise classification for context-aware applications. In: Proceedings of the DEXA 2003, (LNCS 2736), pp 360–370Google Scholar
  3. 3.
    Moran TP, Dourish P (2001) Introduction to special issue on context-aware computing. Human-Comput Interact 16(2–4):87–94CrossRefGoogle Scholar
  4. 4.
    Dey AK, Salber D, Abowd GD (2001) A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Human-Comput Interact 16(2–4):97–166CrossRefGoogle Scholar
  5. 5.
    Want A, Hopper A, Falcao V, Gibbons J (1992) The active badge location system. ACM TOIS 10(1):91–102CrossRefGoogle Scholar
  6. 6.
    Schilit BN, Adams NI, Want R (1994) Context-aware computing applications, workshop on mobile computing systems and applications. Santa Cruz, pp 85–90Google Scholar
  7. 7.
    Dey AK (2001) Understanding and using context. Pers Ubiquit Comput 5(1):4–7CrossRefGoogle Scholar
  8. 8.
    Dey AK, Abowd GD (2000) Towards a better understanding of context and context-awareness. In: CHI 2000 workshop on the what, who, where, when, and how of context-awarenessGoogle Scholar
  9. 9.
    Anhalt J, Smailagic A, Siewiorek DP, Gemperle F, Salber D, Weber SM, Beck J, Jennings J (2001) Toward context aware computing: experiences and lessons. IEEE Intell Syst 16(3):38–46CrossRefGoogle Scholar
  10. 10.
    Brown PJ, Bovey JD, Xian C (1997) Context-aware applications: from the laboratory to the marketplace. IEEE Pers Commun 4(5):58–64CrossRefGoogle Scholar
  11. 11.
    Abowd GD, Dey AK, Orr RJ, Brotherton J (1998) Context-awareness in wearable and ubiquitous computing. Virtual Real Soc Int J 3:200–211CrossRefGoogle Scholar
  12. 12.
    Chen G, Kotz D (2000) A survey of context-aware mobile computing research, Department of Computer Science, Dartmouth College–381/Google Scholar
  13. 13.
    Lieberman H, Selker T (2000) Out of context: computer systems that adapt to, and learn from, context. IBM Syst J 39(3–4):617–632Google Scholar
  14. 14.
    Dey AK, Salber D, Abowd GD, Futakawa M (1999) The conference assistant: combining context-awareness with wearable computing. In: Proceedings third international symposium on wearable computersGoogle Scholar
  15. 15.
    Yan H, Selker T (2000) A context-aware office assistant. In: ACM international conference on intelligent user interfaces (IUI-2000), New Orleans, pp 276–279Google Scholar
  16. 16.
    Abowd GD, Atkeson CG, Hong J, Long S, Kooper R, Pinkerton M (1997) Cyberguide: a mobile context-aware tour guide. Wireless Networ 3(5):421–433CrossRefGoogle Scholar
  17. 17.
    Randell C, Muller H (2000) The shopping jacket: wearable computing for the consumer. Pers Technol 4(4):241–244CrossRefGoogle Scholar
  18. 18.
    Sumi Y, Etani T, Fels S, Simone N, Kobayashi K, Mase K (1998) C-MAP: building a context-aware mobile assistant for exhibition tours. Social interaction and communityware, Kyoto, Japan (LNCS 1519), pp 137–154Google Scholar
  19. 19.
    Brown PJ (1996) STICK-E NOTES: changing notes and contexts—the SeSensor module and the loading of notes, EP-oddGoogle Scholar
  20. 20.
    Beigl M (2000) MemoClip: a location based remembrance appliance. Pers Technol 4(4):230–233CrossRefGoogle Scholar
  21. 21.
    Dey AK, Abowd GD (2000) CybreMinder: a context-aware system for supporting reminders. In: Proceedings of the second international symposium on handheld and ubiquitous computing (HUC), pp 172–186Google Scholar
  22. 22.
    Rhodes B.(2003) Using physical context for just-in-time information retrieval. IEEE Trans Comput 52(8):1011–1014CrossRefGoogle Scholar
  23. 23.
    Marmasse N,. Schmandt C (2000) Location-aware information delivery with comMotion. In: Proceedings of the second international symposium on handheld and ubiquitous computing (HUC), Bristol, pp 157–171Google Scholar
  24. 24.
    Sawhney N, Schmandt C (2000) Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments. ACM ToCHI 7(3):353–383CrossRefGoogle Scholar
  25. 25.
    Gerasimov V, Bender W (2000) Things that talk: using sound for device-to-device and device-to-human communication. IBM Syst J 39(3–4):530–546Google Scholar
  26. 26.
    Mynatt ED, Back M, Want R, Baer M, Ellis JB (1998) Designing audio aura. In: Proceedings of the CHI’98, New York, pp 566–573Google Scholar
  27. 27.
    Schmandt C, Marmasse N, Marti S, Shawhney N, Wheeler S (2000) Everywhere messaging. IBM Syst J 39(3–4):660–677CrossRefGoogle Scholar
  28. 28.
    Hindus D, Schmandt C (1992) Ubiquitous audio: capturing spontaneous collaboration. In: Proceedings of the computer-supported cooperative work (CSCW). Toronto, pp 210—217Google Scholar
  29. 29.
    Huang X, Weng J, Zhang Z (2004) Office presence detection using multimodal context information. In: Proceedings of the international conference on acoustics, speech, and signal processing, MontrealGoogle Scholar
  30. 30.
    Foote J (1999) An overview of audio information retrieval. Multimedia Syst 7(1):2–11CrossRefGoogle Scholar
  31. 31.
    Wold E, Blum T, Keslar D, Wheaton J (1996) Content-based classification search and retrieval of audio. IEEE Multimedia 3(3):27–36CrossRefGoogle Scholar
  32. 32.
    Brown GJ, Cooke MP (1994) Computational auditory scene analysis. Comput Speech Lang 8:297–336CrossRefGoogle Scholar
  33. 33.
    Couvreur C (1997) Environmental sound recognition: a statistical approach. PhD thesis, Faculté Polytechnique de Mons, Belgium Google Scholar
  34. 34.
    Gaunard P, Mubikangiey CG, Couvreur C, Fontaine V (1998) Automatic classification of environmental noise events by Hidden Markov Models. Appl Acoust 187–206Google Scholar
  35. 35.
    Nishiura T, Nakamura S, Miki K, Shikano K (2003) Environment sound source identification based on Hidden Markov Model for robust speech recognition. EuroSpeech 2157–2160Google Scholar
  36. 36.
    Peltonen V, Tuomi J, Klapuri A, Huopaniemi J, Sorsa T (2002) Computational auditory scene recognition. In: Proceedings of the international conference on acoustic, speech and signal processing, OrlandoGoogle Scholar
  37. 37.
    Sawhney N (1997) Situational awareness from environmental sounds technical report for modeling adaptive behavior (MAS 738), Pattie Maes, MIT Media LabGoogle Scholar
  38. 38.
    Clarkson B, Sawhney N, Pentland A (1998) Auditory context awareness via wearable computing. In: Workshop on perceptual user interfaces, pp 37–42Google Scholar
  39. 39.
    Clarkson B, Pentland A (1999) Unsupervised clustering of ambulatory audio and video. In: Proceedings of the international conference on acoustics, speech, and signal processing, PhoenixGoogle Scholar
  40. 40.
    Ellis DPW (1999) Using knowledge to organize sound: the prediction-driven approach to computational auditory scene analysis and its application to speech/nonspeech mixtures. Speech Commun 27(3–4):281–298CrossRefGoogle Scholar
  41. 41.
    The HTK Book (2001) Version 3.1, Cambridge University Engineering Department, Scholar
  42. 42.
    Vinsensius Vega SB, Bressan S (2003) Continuous naive Bayesian classifications, ICADL, pp 279–289Google Scholar
  43. 43.
    Java Sound API Scholar
  44. 44.
    Ryan NS, Osbakk P (2003) The MobiComp Infrastructure, Scholar
  45. 45.
    Pascoe J, Morse DR, Ryan NS (1998) Developing personal technology for the field. Pers Technol 2:28–36CrossRefGoogle Scholar
  46. 46.
    Ryan NS, Pascoe J, Morse DR (1999) Fieldnote: a handheld information system for the field. In: Laurini R (ed) Proceedings of the TeleGeo’99, 1st international workshop on TeleGeoProcessing, Lyon, pp 156–163Google Scholar
  47. 47.
    van Leusen M, Ryan NS (2001) Educating the digital fieldwork assistant. In: Burenhult G (ed) Proceedings of computer applications and quantitative methods in archeology conference (CAA 2001:), Gotland, pp 401–412Google Scholar
  48. 48.
    Ahuja S, Carriero N, Gelertner D (1986) Linda and friends. IEEE Comput 19(8):26–34Google Scholar
  49. 49.
    Gelertner D, Carriero N (1992) Coordination languages and their significance. CACM 32(2):96–107Google Scholar
  50. 50.
    Johanson B, Fox A (2002) The event heap: a coordination infrastructure for interactive workspaces. In: Proceedings of the 4th IEEE workshop on mobile computer systems and applications (WMCSA-2002), Callicoon, pp 83–93Google Scholar
  51. 51.
    XQuery (2004) 1.0 An XML query language, W3C working draft, 23 July 2004, Scholar
  52. 52.
    IBM WebSphere Voice Server Administrator’s Guide, for information on deploying VoiceXML applications in a Voice over IP environmentGoogle Scholar
  53. 53.
    Java speech grammar format (JSGF) Specification (2001) Scholar
  54. 54.
    Ozmutlu S, Ozmutlu HC, Spink A (2002) Trends in multimedia web searching: excite queries. In: Proceedings of the ITCC 2002, pp 40–45Google Scholar
  55. 55.
    Ozmutlu S, Spink A, Ozmutlu HC (2004) A day in the life of web searching: an exploratory study. Inform Process Manage 40(2):319–345CrossRefGoogle Scholar
  56. 56. Scholar
  57. 57.
    Verlinden M, Tijsseling C, Frowen H (2002) A signing avatar on the WWW, Gesture and sign languages in human-computer interaction, pp 169–172Google Scholar

Copyright information

© Springer-Verlag London Limited 2005

Authors and Affiliations

  1. 1.School of Computing SciencesUniversity of East AngliaNorwichUK
  2. 2.Computing LaboratoryUniversity of KentCanterburyUK

Personalised recommendations