Sensor data fusion for activity monitoring in the PERSONA ambient assisted living project

  • Michele Amoretti
  • Sergio Copelli
  • Folker Wientapper
  • Francesco Furfari
  • Stefano Lenzi
  • Stefano Chessa
Original Research


User activity monitoring is a major problem in ambient assisted living, since it requires to infer new knowledge from collected and fused sensor data while dealing with highly dynamic environments, where devices continuously change their availability and (or) physical location. In the context of the European project PERSONA, we have developed an activity monitoring sub-system characterized by high modularity, little invasiveness of the environment and good responsiveness. In this paper we first illustrate the functional architecture of the proposed solution from a general point of view, discussing the motivations of the design. Then we describe in details the software components—sensor abstraction and integration layer, human posture classification, activity monitor—and the resulting activity monitoring application, presenting also a performance evaluation.


Applications for pervasive computing Context awareness Ambient assisted living Sensor data fusion Activity monitoring 


  1. Amoretti M, Wientapper F, Furfari F, Lenzi S, Chessa S (2009) Sensor data fusion for activity monitoring in ambient assisted living environments. In: Proceedings of the 1st International Conference on Sensor Systems and Software (S-CUBE). Springer, PisaGoogle Scholar
  2. Azizyan M, Constandache I, Choudhury RR (2009) Surroundsense: mobile phone localization via ambience fingerprinting. In: Proceedings of the 15th annual international conference on mobile computing and networking (MobiCom09), ACM, Beijing, ChinaGoogle Scholar
  3. Bao L, Intille SS (2004) Activity recognition from user-annotated acceleration data. In: Pervasive computing. Springer, Berlin, pp 1–17Google Scholar
  4. Barger T, Brown D, Alwan M (2005) Health-status monitoring through analysis of behavioral patterns. IEEE Trans Syst Man Cybern Part A Syst Hum 35(1):22–27CrossRefGoogle Scholar
  5. Baronti P, Pillai P, Chook VWC, Chessa S, Gotta A, Hu YF (2007) Wireless sensor networks: a survey on the state of the art and the 802.15.4 and ZigBee standards. Comput Commun 30(7):1655–1695. doi: 10.1016/j.comcom.2006.12.020 Google Scholar
  6. Bauer G, Lukowicz P (2008) Developing a sub room level indoor location system for wide scale deployment in assisted living systems. In: Proceedings of the 11th international conference on computers helping people with special needs. Springer, ViennaGoogle Scholar
  7. Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720. doi: 10.1109/34.598228 Google Scholar
  8. Bieber G, Hoffmeyer A, Gutzeit E, Peter C, Urban B (2009) Activity monitoring by fusion of optical and mechanical tracking technologies for user behavior analysis. In: Proceedings of the 2nd international conference on PErvasive technologies related to assistive environments, ACMGoogle Scholar
  9. Bobick A (1997) Movement, activity and action: the role of knowledge in the perception of motion. Phil Trans R Soc Lond 352:1257–1265CrossRefGoogle Scholar
  10. Bolla R, Rapuzzi R, Repetto M, Barsocchi P, Chessa S, Lenzi S (2009) Automatic multimedia session migration by means of a context-aware mobility framework. In: Proceedings of the 6th international conference on mobile technology, applications, and systems (mobility conference), pp 1–7Google Scholar
  11. Bouckaert R (2008) Bayesian network classifiers in Weka, v.3.5.8.Google Scholar
  12. Chen R, Hou Y, Huang Z, He J (2009) Modeling the ambient intelligence application system: Concept, software, data, and network. IEEE Trans Syst Man Cybern Part C Appl Rev 39(3):299–314CrossRefGoogle Scholar
  13. CNR-ISTI, ITACA-TSB (2011) ZB4OSGi.
  14. Cozman FG (1998) The interchange format for bayesian networks.
  15. Cucchiara R, Grana C, Prati A, Vezzani R (2005a) Probabilistic posture classification for human-behavior analysis. IEEE Trans Syst Man Cybern Part A 35(1):42–54. doi: 10.1109/TSMCA.2004.838501 Google Scholar
  16. Cucchiara R, Prati A, Vezzani R (2005b) Posture classification in a multi-camera indoor environment. In: Proceedings of the IEEE international conference on image processing (ICIP), vol 1, pp 11–14. doi: 10.1109/ICIP.2005.1529853
  17. Cucchiara R, Prati A, Vezzani R (2007) A multi-camera vision system for fall detection and alarm generation. Exp Syst 24:334–345. doi: 10.1111/j.1468-0394.2007.00438.x Google Scholar
  18. Fides-Valero Á, Freddi M, Furfari F, Tazari MR (2008) The PERSONA framework for supporting context-awareness in open distributed systems. In: Proceedings of the european conference on ambient intelligence (AmI08), LNCS, vol 5355, pp 91–108Google Scholar
  19. Frank E, Hall M, Trigg L (2008) Weka 3: data mining software in Java.
  20. Girolami M, Lenzi S, Furfari F, Chessa S (2008) SAIL: a sensor abstraction and integration layer for context aware architectures. In: Proceedings of the 34th EUROMICRO conference on software engineering and advanced applications (SEAA 2008), IEEE, Parma, Italy, pp 374–381Google Scholar
  21. He X, Niyogi P (2003) Locality preserving projections. In: Advances in neural information processing systems, vol 16. MIT Press, CambridgeGoogle Scholar
  22. Heckerman D (1999) A tutorial on learning with Bayesian networks, chapt. 11. MIT Press, CambridgeGoogle Scholar
  23. Huynh T, Fritz M, Schiele B (2008) Discovery of activity patterns using topic models. In: Proceedings of the 10th international conference on ubiquitous computing (Ubi-Comp)Google Scholar
  24. IST Advisory Group EC (2011) Scenarios for ambient intelligence in 2010Google Scholar
  25. Jensen F (2001) Bayesian networks and decision graphs. Springer, BerlinGoogle Scholar
  26. Kotsiantis S (2007) Supervised machine learning: a review of classification techniques. Informatica 31:249–268Google Scholar
  27. Lu H, Pan W, Lane ND, Choudhury T, Campbell AT (2009) Soundsense: scalable sound sensing for people-centric applications on mobile phones. In: Proceedings of the 7th annual international conference on mobile systems, applications and services (MobiSys09), ACMGoogle Scholar
  28. Madden M (2003) The performance of bayesian network classifiers constructed using different techniques. In: Workshop on probabilistic graphical models for classification. Proceedings of the european conference on machine learningGoogle Scholar
  29. MESA Imaging (2009)
  30. Moeslund TB, Hilton A, Krueger V (2006) A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Underst 104(2):90–126. doi: 10.1016/j.cviu.2006.08.002
  31. Molla MM, Ahamed SI (2006) A survey of middleware for sensor network and challenges. In: Proceedings of the international conference on parallel processing workshop (ICPP 2006)Google Scholar
  32. Norsys (2009) Netica-j: Java version of the netica api.
  33. OASIS (2011) Oasis project home page.
  34. Padovitz A, Loke S, Zaslavsky A (2008) Multiple-agent perspectives in reasoning about situations for context-aware pervasive computing systems. IEEE Trans Syst Man Cybern Part A Syst Hum 38(4):729–742CrossRefGoogle Scholar
  35. Pallikonda M, Radhakrishnan S, Subbaraj P (2009) Elderly patient monitoring system using a wireless sensor network. Telemed e-Health 15(1):73–79CrossRefGoogle Scholar
  36. Pellegrini S, Iocchi L (2008) Human posture tracking and classification through stereo vision and 3d model matching. J Image Video Process 8(2):1–12. doi: 10.1155/2008/476151 Google Scholar
  37. PERSONA (2010) PERSONA project web site.
  38. Piccardi M (2004) Background subtraction techniques: a review. In: Proceedings of the IEEE international conference on systems, man and cybernetics, pp 3099–3104Google Scholar
  39. Pioggia G, Ricci G, Bonfiglio S, Bekiaris E, Siciliano G, De Rossi D (2009) An ontology-driven multisensorial platform to enable unobtrusive human monitoring and independent living. In: Proceedings of the 9th international conference on intelligent systems design and applicationsGoogle Scholar
  40. Poppe R (2007) Vision-based human motion analysis: an overview. Comput Vis Image Underst 108(1–2):4–18. doi: 10.1016/j.cviu.2006.10.016 Google Scholar
  41. Porikli F, Tuzel O (2005) Bayesian background modeling for foreground detection. In: Proceedings of the 3rd ACM international workshop on videosurveillance & sensor networks (VSSN), New York, NY, USA, pp 55–58. doi: 10.1145/1099396.1099407
  42. Ramos C, Augusto JC, Shapiro D (2008) Ambient intelligence—the next step for artificial intelligence. IEEE Intell Syst 23(2):15–18CrossRefGoogle Scholar
  43. Rumelhart DE, Hinton GE, Williams RJ (1997) Learning dynamic bayesian networks. In: Adaptive processing of temporal information. Lecture notes in artificial intelligence. Springer, Berlin, pp 168–197Google Scholar
  44. SOPRANO (2011) Soprano project home page.
  45. Turk M, Pentland A (1991) Face recognition using eigenfaces. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 586–591. doi: 10.1109/CVPR.1991.139758
  46. Virone G, Sixsmith A (2008) Monitoring activity patterns and trends of older adults. In: 30th annual international IEEE EMBS conferenceGoogle Scholar
  47. Wang H, Chen S, Hu Z, Zheng W (2008) Locality-preserved maximum information projection. IEEE Trans Neural Netw 19(4):571–585. doi: 10.1109/TNN.2007.910733 Google Scholar
  48. Wang L, Suter D (2006) Analyzing human movements from silhouettes using manifold learning. In: Proceedings of the IEEE international conference on video and signal based surveillance (AVSS), pp 1–7. doi: 10.1109/AVSS.2006.25
  49. Wang L, Suter D (2007) Learning and matching of dynamic shape manifolds for human action recognition. IEEE Trans Image Process 16(6):1646–1661. doi: 10.1109/TIP.2007.896661 Google Scholar
  50. Wartena F, Muskens J, Schmitt L, Petkovic M (2010) Continua: the reference architecture of a personal telehealth ecosystem. In: WASP (Healthcom accompaining workshop)Google Scholar
  51. Wientapper F, Ahrens K, Wuest H, Bockholt U (2009) Linear-projection-based classification of human postures in time-of-flight data. In: Proceedings of the IEEE international conference on systems, man and cybernetics, pp 559–564. doi: 10.1109/ICSMC.2009.5346892
  52. Wood A, Virone G, Doan T, Cao Q, Selavo L, Wu Y, Fang L, He Z, Lin S, Stankovic J (2006) Alarm-net: wireless sensor networks for assisted-living and residential monitoring. Tech. Rep. CS-2006-13, Wireless Sensor Network Research Group, Department of Computer Science, University of VirginiaGoogle Scholar
  53. Zhou F, Jiao J, Chen S, Zhang D (2010) A case-driven ambient intelligence system for elderly in-home assistance applications. IEEE Trans Syst Man Cybern Part C Appl Rev 99:1–11. doi: 10.1109/TSMCC.2010.2052456 Google Scholar

Copyright information

© Springer-Verlag 2011

Authors and Affiliations

  • Michele Amoretti
    • 1
  • Sergio Copelli
    • 1
  • Folker Wientapper
    • 2
  • Francesco Furfari
    • 3
  • Stefano Lenzi
    • 3
  • Stefano Chessa
    • 3
    • 4
  1. 1.R&S INFOParmaItaly
  2. 2.Fraunhofer IGDDarmstadtGermany
  3. 3.CNR-ISTIPisaItaly
  4. 4.Computer Science DepartmentUniversity of PisaPisaItaly

Personalised recommendations