Wearable sensor-based human activity recognition from environmental background sounds

Original Research


Understanding individual’s activities, social interaction, and group dynamics of a certain society is one of fundamental problems that the social and community intelligence (SCI) research faces. Environmental background sound is a rich information source for identifying individual and social behaviors. Therefore, many power-aware wearable devices with sound recognition function are widely used to trace and understand human activities. The design of these sound recognition algorithms has two major challenges: limited computation resources and a strict power consumption requirement. In this paper, a new method for recognizing environmental background sounds with a power-aware wearable sensor is presented. By employing a novel low calculation one-dimensional (1-D) Haar-like sound feature with hidden Markov model (HMM) classification, this method can achieve high recognition accuracy while still meeting the wearable sensor’s power requirement. Our experimental results indicate an average recognition accuracy of 96.9 % has been achieved when testing with 22 typical environmental sounds related to personal and social activities. It outperforms other commonly used sound recognition algorithms in terms of both accuracy and power consumption. This is very helpful and promising for future integration with other sensor(s) to provide more trustworthy activity recognition results for the SCI system.


Social and community intelligence Digital footprint WSNs Sound recognition Haar-like feature HMM 


  1. Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: Pervasive 2004, LNCS 3001, pp 1–7Google Scholar
  2. Bharatula NB, Ossevoort S, Stager M, Troster G (2004) Towards wearable autonomous microsystems. In: PERVASIVE 2004, LNCS 3001, pp 225–237Google Scholar
  3. Bharatula NB, Stager M, Lukowics P, Troster G (2005) Empirical study of design choices in multi-sensor context recognition systems. In: The 2nd international forum on applied wearable computing (IFAWC’05), pp 79–93Google Scholar
  4. Bonfiglio A, Rossi DR (2011) Wearable monitoring systems. Springer, NYCrossRefGoogle Scholar
  5. Chen J, Zhang JA, Kam H, Shue L (2005) Bathroom activity monitoring based on sound. In: PERVASIVE 2005, LNCS 3468, pp 47–61Google Scholar
  6. Choudhury T (2004) Sensing and modeling human networks. PhD Dissertation, MIT. http://hd.media.mit.edu
  7. Chu S, Narayanan S, Kuo C-CJ (2009) Environmental sound recognition with time-frequency audio features. IEEE Trans Audio Speech Lang Process 17:1142–1158CrossRefGoogle Scholar
  8. Cowling M, Sitte R (2003) Comparison of techniques for environmental sound recognition. Pattern recognit lett 24:2895–2907CrossRefGoogle Scholar
  9. Culler D, Estrin D, Srivastava M (2004) Overview of sensor networks. Computer 37:41–49CrossRefGoogle Scholar
  10. Davis SB, Mermelstein P (1980) Comparison of parametric representations of monosyllabic word recognition in continuously spoken sentences. IEEE Trans Speech Audio Process 28:357–366CrossRefGoogle Scholar
  11. Doherty L, Warneke BA, Boser BE, Pister KSJ (2001) Energy and performance considerations for smart dust. Int J Parallel Distrib Syst Netw 4:121–133Google Scholar
  12. Dong R, Hermann D, Cornu E, Chau E (2007) Low-power implementation of an HMM-based sound environment classification algorithm for hearing aid application. In: Proceedings of EUSIPCO 2007Google Scholar
  13. Duda RO, Hart PE, Stork DG (2001) Pattern classification, 2nd edn. Wiley, NYMATHGoogle Scholar
  14. Eronen AJ, Peltonen VT et al (2006) Audio-based context recognition. IEEE Trans Audio Speech Lang Process 14:321–329CrossRefGoogle Scholar
  15. Gold B, Morgan N (2000) Speech and audio signal processing. Wiley, NYGoogle Scholar
  16. Goldhor RS (1993) Recognition of environmental sounds. In: IEEE ICASSP. pp 149–152Google Scholar
  17. Guo B, Zhang D, Imai M (2011a) Toward a cooperative programming framework for context-aware applications. J Pers Ubiquitous Comput 15:221–233CrossRefGoogle Scholar
  18. Guo B, Zhang D, Wang Z (2011b) Living with internet of things: the emergence of embedded intelligence. In: IEEE international conference on cyber, physical and social computing (CPSCom), pp 297–304Google Scholar
  19. Hanai Y, Nishimura J, Kuroda T (2009) Haar-like filtering for human activity recognition using 3D accelerometer. In: IEEE 13th digital signal processing workshop and 5th IEEE signal processing education workshop, pp 675–678Google Scholar
  20. Krause A et al (2005) Trading off prediction accuracy and power consumption for context-aware wearable computing. In: Proceeding of the 9th IEEE international symposium on wearable computers (ISWC’05), pp 20–26Google Scholar
  21. Laibowitz M, Gips J, Aylward R, Pentland A, Paradiso J (2006) A sensor network for social dynamic. In: IEEE IPSN’06, pp 483–491Google Scholar
  22. Linde Y, Buzo A, Gray RM (1980) An algorithm for vector quantizer design. IEEE Trans Commun 28:84–95CrossRefGoogle Scholar
  23. Lynch JP, Loh KJ (2006) A summary review of wireless sensors and sensor networks for structural health monitoring. Shock Vib Dig 38:91–128CrossRefGoogle Scholar
  24. Ma L, Milner B, Smith D (2006) Acoustic environment classification. ACM Trans Speech Lang Process 3:1–22CrossRefGoogle Scholar
  25. Nishimura J, Kuroda T (2008a) Haar-like filtering based speech detection using Integral Signal for sensornet. In: International conference on sensing technology, pp 52–56 Google Scholar
  26. Nishimura J, Kuroda T (2008b) Low cost speech detection using Haar-like filtering for sensornet. In: 9th international conference on signal processing, pp 2608–2611Google Scholar
  27. Nishimura J, Sato N, Kuroda T (2008) Speech “siglet” detection for business microscope. In: IEEE international conference on pervasive computing and communications (PerCom08), pp 147–152Google Scholar
  28. Peltonen V, Tuomi J, Klapuri A, Huopaniemi J, Sorsa T (2002) Computational auditory scene recognition. In: IEEE ICASSP, pp 1941–1944Google Scholar
  29. Pentland A (2005) Socially aware computation and communication. IEEE Comput 38:33–40CrossRefGoogle Scholar
  30. Rabiner LR (1989) A tutorial on Hidden Markov Models and selected applications in speech recognition. Proc IEEE 77:257–286CrossRefGoogle Scholar
  31. Rabiner LR, Juang BH (1993) Fundamentals of speech recognition. Prentice-Hall, Englewood CliffGoogle Scholar
  32. Rota N, Thonnat M (2000) Activity recognition from video sequences using declarative models. In: Proceedings of the 14th European conference on artificial intelligence, pp 673–680Google Scholar
  33. Veitch R, Aubert LM, Woods R, Fischaber S (2011) FPGA implementation of a pipelined Gaussian calculation for HMM-based large vocabulary speech recognition. Int J Reconfi Comput. doi:10.1155/2011/697080
  34. Viola P, Jones M (2004) Rapid object detection using a boosted cascade of simple features. In: Computer society conference on computer vision and pattern recognition, pp 511–518Google Scholar
  35. Welch LR (2003) Hidden Markov models and the Baum–Welch algorithm. IEEE Inf Theory Soc Newslett 53:9–13Google Scholar
  36. Wiki_k-means (2012) Wikipedia introduction of the k-means algorithm. http://en.wikipedia.org/wiki/K-means_clustering. Accessed Jan 2012
  37. Yamashita S, Shimura T, Aiki K, Ara K, Ogata Y, Shimokawa I, Tanaka T, Kuriyama H, Shimada K, Yano K (2006) A 15 × 15 mm, 1 μA, reliable sensor-net module: enabling application-specific nodes. In: The fifth international conference on information processing in sensor networks (IPSN 2006), pp 383–390Google Scholar
  38. Yano K, Sato N, Wakisaka Y, Tsuji S, Ohkubo N, Hayakawa M, Moriwaki N (2008) Life thermoscope: integrated microelectronics for visualizing hidden life rhythm. In: IEEE ISSCC digests of technical papers, pp 136–137Google Scholar
  39. Yano K, Ara K, Moriwaki N, Kuriyama H (2009) Measurement of human behavior: creating a society for discovering opportunities. Hitachi Rev 58:139–144Google Scholar
  40. Yin J, Yang Q, Pan JJ (2008) Sensor-based abnormal human-activity detection. IEEE Trans Knowl Data Eng 20:1082–1090CrossRefGoogle Scholar
  41. Zhang D, Guo B, Yu Z (2011) The emergence of social and community intelligence. IEEE Comput 44:21–28CrossRefGoogle Scholar

Copyright information

© Springer-Verlag 2012

Authors and Affiliations

  1. 1.Department of Electronics and Electrical EngineeringKeio UniversityYokohama-shiJapan

Personalised recommendations