Culture as a Sensor? A Novel Perspective on Human Activity Recognition

  • Ting-Chia Chiang
  • Barbara BrunoEmail author
  • Roberto Menicatti
  • Carmine Tommaso Recchiuto
  • Antonio Sgorbissa


Human Activity Recognition (HAR) systems are devoted to identifying, amidst the sensory stream provided by one or more sensors located so that they can monitor the actions of a person, portions related to the execution of a number of a-priori defined activities of interest. Improving the performance of systems for Human Activity Recognition is a long-standing research goal: solutions include more accurate sensors, more sophisticated algorithms for the extraction and analysis of relevant information from the sensory data, and the enhancement of the sensory analysis with general or person-specific knowledge about the execution of the activities of interest. Following the latter trend, in this article we propose the association and enhancement of the sensory data analysis with cultural information, that can be seen as an estimate of person-specific information, relieved of the burden of a long/complex setup phase. We propose a culture-aware Human Activity Recognition system which associates the recognition response provided by a state-of-the-art, culture-unaware HAR system with culture-specific information about where and when activities are most likely performed in different cultures, encoded in an ontology. The merging of the cultural information with the culture-unaware responses is done by a Bayesian Network, whose probabilistic approach allows for avoiding stereotypical representations. Experiments performed offline and online, using images acquired by a mobile robot in an apartment, show that the culture-aware HAR system consistently outperforms the culture-unaware HAR system.


Culture-aware robotics Human Activity Recognition Ontology Bayesian network Vision-based HAR 



This work has been partially supported by the European Commission Horizon2020 Research and Innovation Programme under Grant Agreement No. 737858 (CARESSES), and by the Erasmus+ programme under Grant Agreement No. 2014-2616/001-001 (EMARO+).

Compliance with Ethical Standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. 1.
    Agarwal P, Verma R, Mallik A (2016) Ontology based disease diagnosis system with probabilistic inference. In: 2016 1st India international conference on information processing (IICIP). IEEE, pp 1–5Google Scholar
  2. 2.
    Aggarwal JK, Xia L (2014) Human activity recognition from 3D data: a review. Pattern Recognit Lett 48:70–80CrossRefGoogle Scholar
  3. 3.
    Bakar U, Ghayvat H, Hasanm S, Mukhopadhyay S (2016) Activity and anomaly detection in smart home: a survey. In: Next Generation Sensors and Systems. Springer, pp 191–220Google Scholar
  4. 4.
    Banerjee T, Keller JM, Popescu M, Skubic M (2015) Recognizing complex instrumental activities of daily living using scene information and fuzzy logic. Comput Vis Image Underst 140:68–82. CrossRefGoogle Scholar
  5. 5.
    Bruno B, Mastrogiovanni F, Sgorbissa A, Vernazza T, Zaccaria R (2013) Analysis of human behavior recognition algorithms based on acceleration data. ICRA 2013:2293–2299Google Scholar
  6. 6.
    Bruno B, Mastrogiovanni F, Sgorbissa A (2014) A public domain dataset for ADL recognition using wrist-placed accelerometers. In: 23rd IEEE international symposium on robot and human interactive communication (IEEE RO-MAN 2014)Google Scholar
  7. 7.
    Bruno B, Mastrogiovanni F, Pecora F, Sgorbissa A, Saffiotti A (2017) A framework for culture-aware robots based on fuzzy logic. In: 2017 IEEE international conference on fuzzy systems (FUZZ-IEEE). IEEE, pp 1–6Google Scholar
  8. 8.
    Bruno B, Recchiuto CT, Papadopoulos I, Saffiotti A, Koulouglioti C, Menicatti R, Mastrogiovanni F, Zaccaria R, Sgorbissa A (2019) Knowledge representation for culturally competent personal robots: requirements, design principles, implementation, and assessment. Int J Soc Robot. CrossRefGoogle Scholar
  9. 9.
    Bulling A, Blanke U, Schiele B (2014) A tutorial on human activity recognition using body-worn inertial sensors. ACM Comput Surv (CSUR) 46(3):33CrossRefGoogle Scholar
  10. 10.
    Carvalho RN, Laskey KB, Costa PC (2017) PR-OWL—a language for defining probabilistic ontologies. Int J Approx Reason 91:56–79MathSciNetCrossRefGoogle Scholar
  11. 11.
    Chen L, Nugent CD, Wang H (2012) A knowledge-driven approach to activity recognition in smart homes. IEEE Trans Knowl Data Eng 24(6):961–974CrossRefGoogle Scholar
  12. 12.
    Cook DJ, Crandall AS, Thomas BL, Krishnan NC (2013) CASAS: a smart home in a box. Computer 46(7):62–69CrossRefGoogle Scholar
  13. 13.
    Coppola C, Krajńík T, Duckett T, Bellotto N (2016) Learning temporal context for activity recognition. Front Artif Intell Appl 285:107–115Google Scholar
  14. 14.
    Crispim-Junior CF, Buso V, Avgerinakis K, Meditskos G, Briassouli A, Benois-Pineau J, Kompatsiaris I, Bremond F (2016) Semantic event fusion of different visual modality concepts for activity recognition. IEEE Trans Pattern Anal Mach Intell 38(8):1598–1611. CrossRefGoogle Scholar
  15. 15.
    Faria DR, Vieira M, Premebida C, Nunes U (2015) Probabilistic human daily activity recognition towards robot-assisted living. In: 2015 24th IEEE international symposium on robot and human interactive communication (RO-MAN). IEEE, pp 582–587Google Scholar
  16. 16.
    Fjellstrm C (2004) Mealtime and meal patterns from a cultural perspective. Scand J Nutr 48(4):161–164. CrossRefGoogle Scholar
  17. 17.
    Froehlich JE, Larson E, Campbell T, Haggerty C, Fogarty J, Patel SN (2009) Hydrosense: infrastructure-mediated single-point sensing of whole-home water activity. In: Proceedings of the 11th international conference on Ubiquitous computing. ACM, pp 235–244Google Scholar
  18. 18.
    Gayathri K, Easwarakumar K, Elias S (2017) Probabilistic ontology based activity recognition in smart homes using Markov Logic Network. Knowl Based Syst 121:173–184. CrossRefGoogle Scholar
  19. 19.
    Guarino N, et al (1998) Formal ontology and information systems. In: Proceedings of FOIS, pp 81–97Google Scholar
  20. 20.
    Guptill AE, Copelton DA, Lucal B (2017) Food and society: principles and paradoxes. Wiley, HobokenGoogle Scholar
  21. 21.
    Hofstede G, Hofstede GJ, Minkov M (1991) Cultures and organizations: software of the mind, vol 2. CiteseerGoogle Scholar
  22. 22.
    Katz S, Chinn A, Cordrey L (1959) Multidisciplinary studies of illness in aged persons: a new classification of functional status in activities of daily living. J Chronic Dis 9(1):55–62CrossRefGoogle Scholar
  23. 23.
    Kim E, Helal S, Cook D (2010) Human activity recognition and pattern discovery. IEEE Pervasive Comput IEEE Comput Soc IEEE Commun Soc 9(1):48CrossRefGoogle Scholar
  24. 24.
    Latfi F, Lefebvre B, Descheneaux C (2007) Ontology-based management of the telehealth smart home, dedicated to elderly in loss of cognitive autonomy. In: OWLED, vol 258Google Scholar
  25. 25.
    Law M (1993) Evaluating activities of daily living: directions for the future. Am J Occup Ther 47:233–237CrossRefGoogle Scholar
  26. 26.
    Lawton M, Brody E (1969) Assessment of older people: self-maintaining and instrumental activities of daily living. The Gerontologist 9:179–186CrossRefGoogle Scholar
  27. 27.
    Lugrin B, Frommel J, André E (2015) Modeling and evaluating a bayesian network of culture-dependent behaviors. Cult Comput 2015:33–40Google Scholar
  28. 28.
    Menicatti R, Bruno B, Sgorbissa A (2017) Modelling the influence of cultural information on vision-based human home activity recognition. In: 2017 14th International conference on ubiquitous robots and ambient intelligence (URAI), pp 32–38Google Scholar
  29. 29.
    Okeyo G, Chen L, Wang H, Sterritt R (2014) Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive Mob Comput 10(PART B):155–172. CrossRefGoogle Scholar
  30. 30.
    Onofri L, Soda P, Pechenizkiy M, Iannello G (2016) A survey on using domain and contextual knowledge for human activity recognition in video streams. Expert Syst Appl 63:97–111. CrossRefGoogle Scholar
  31. 31.
    Papadopoulos I (2006) Transcultural health and social care: development of culturally competent practitioners. Elsevier, AmsterdamGoogle Scholar
  32. 32.
    Poppe R (2010) A survey on vision-based human action recognition. Image Vis Comput 28(6):976–990CrossRefGoogle Scholar
  33. 33.
    Rehm M, Bee N, Endrass B, Wissner M, André E (2007) Too close for comfort? Adapting to the user’s cultural background. HCM 2007:85–94CrossRefGoogle Scholar
  34. 34.
    Scalmato A, Sgorbissa A, Zaccaria R (2013) Describing and recognizing patterns of events in smart environments with description logic. IEEE Trans Cybern 43(6):1882–1897. CrossRefGoogle Scholar
  35. 35.
    Shoaib M, Bosch S, Incel O, Scholten H, Havinga P (2015) A survey of online activity recognition using mobile phones. Sensors 15(1):2059–2085CrossRefGoogle Scholar
  36. 36.
    Soo-Hoo F (2016) How women around the world get clean.
  37. 37.
    Trovato G, Ham JR, Hashimoto K, Ishii H, Takanishi A (2015) Investigating the effect of relative cultural distance on the acceptance of robots. ICSR 2016:664–673Google Scholar
  38. 38.
    W3C Owl Working Group and others (2009) OWL 2 web ontology language document overviewGoogle Scholar
  39. 39.
    Weiss GM, Timko JL, Gallagher CM, Yoneda K, Schreiber AJ (2016) Smartwatch-based activity recognition: a machine learning approach. In: 2016 IEEE-EMBS international conference on biomedical and health informatics (BHI). IEEE, pp 426–429Google Scholar
  40. 40.
    Ye J, Stevenson G, Dobson S (2015) KCAR: a knowledge-driven approach for concurrent activity recognition. Pervasive Mob Comput 19(2):47–70. CrossRefGoogle Scholar

Copyright information

© Springer Nature B.V. 2019

Authors and Affiliations

  1. 1.Department of Informatics, Bioengineering, Robotics and Systems EngineeringUniversity of GenoaGenoaItaly

Personalised recommendations