A Hybrid Reasoning Approach for Activity Recognition Based on Answer Set Programming and Dempster–Shafer Theory

  • Fadi Al MachotEmail author
  • Heinrich C. Mayr
  • Suneth Ranasinghe
Part of the Studies in Systems, Decision and Control book series (SSDC, volume 109)


This chapter discusses a promising approach for multisensor-based activity recognition in smart homes. The research originated in the domain of active and assisted living, particularly in the field of supporting people in mastering their daily life activities. The chapter proposes (a) a reasoning method based on answer set programming that uses different types of features for selecting the optimal sensor set, and (b) a fusion approach to combine the beliefs of the selected sensors using an advanced evidence combination rule of Dempster–Shafer theory. In order to check the overall performance, this approach was tested with the HBMS dataset on an embedded platform. The results demonstrated a highly promising accuracy compared to other approaches.



This work was funded the by Klaus Tschira Stiftung GmbH, Heidelberg


  1. 1.
    Akdemir, U., Turaga, P., Chellappa, R.: An ontology based approach for activity recognition from video. In: Proceedings of the 16th ACM international conference on Multimedia, pp. 709–712. ACM (2008)Google Scholar
  2. 2.
    Al Machot, F., Kyamakya, K., Dieber, B., Rinner, B.: Real time complex event detection for resource-limited multimedia sensor networks. In: 2011 8th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), pp. 468–473. IEEE (2011)Google Scholar
  3. 3.
    Al Machot, F., Tasso, C., Dieber, B., Kyamakya, K., Piciarelli, C., Micheloni, C., Londero, S., Valotto, M., Omero, P., Rinner, B.: Smart resource-aware multimedia sensor network for automatic detection of complex events (2011)Google Scholar
  4. 4.
    Ali, T., Dutta, P., Boruah, H.: A new combination rule for conflict problem of dempster-shafer evidence theory. Int. J. Energy Inf. Commun. 3(1), 35–40 (2012)Google Scholar
  5. 5.
    Baral, C.: Knowledge Representation, Reasoning and Declarative Problem Solving. Cambridge University Press, Cambridge (2003)CrossRefzbMATHGoogle Scholar
  6. 6.
    Baral, C., Gelfond, G., Son, T.C., Pontelli, E.: Using answer set programming to model multi-agent scenarios involving agents’ knowledge about other’s knowledge. In: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1-Volume 1, pp. 259–266. International Foundation for Autonomous Agents and Multiagent Systems (2010)Google Scholar
  7. 7.
    Beal, M.J.: Variational algorithms for approximate Bayesian inference. Ph.D. thesis, University of London (2003)Google Scholar
  8. 8.
    Bobillo, F., Straccia, U.: fuzzydl: an expressive fuzzy description logic reasoner. In: FUZZ-IEEE, pp. 923–930 (2008)Google Scholar
  9. 9.
    Brain, M., De Vos, M.: Answer set programming–a domain in need of explanation. In: Exact08: International Workshop on Explanation-aware Computing (2008)Google Scholar
  10. 10.
    Brendel, W., Fern, A., Todorovic, S.: Probabilistic event logic for interval-based event recognition. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3329–3336. IEEE (2011)Google Scholar
  11. 11.
    Chiang, S.-Y., Kan, Y.-C., Tu, Y.-C., Lin, H.-C.: Activity recognition by fuzzy logic system in wireless sensor network for physical therapy. In: Intelligent Decision Technologies, pp. 191–200. Springer (2012)Google Scholar
  12. 12.
    Dezert, J.: Foundations for a new theory of plausible and paradoxical reasoning. Inf. Secur. 9, 13–57 (2002)Google Scholar
  13. 13.
    Dietterich, T.G., Ashenfelter, A., Bulatov, Y.: Training conditional random fields via gradient tree boosting. In: Proceedings of the twenty-first international conference on Machine learning, p. 28. ACM (2004)Google Scholar
  14. 14.
    Dubois, D., Prade, H.: Rough fuzzy sets and fuzzy rough sets*. Int. J. Gen. Syst. 17(2–3), 191–209 (1990)CrossRefzbMATHGoogle Scholar
  15. 15.
    Grau, B.C., Horrocks, I., Motik, B., Parsia, B., Patel-Schneider, P., Sattler, U.: Owl 2: the next step for owl. Web Semant. Sci. Serv. Agents World Wide Web 6(4), 309–322 (2008)CrossRefGoogle Scholar
  16. 16.
    Guyon, I., Weston, J., Barnhill, S., Vapnik, V.: Gene selection for cancer classification using support vector machines. Mach. Learn. 46(1–3), 389–422 (2002)CrossRefzbMATHGoogle Scholar
  17. 17.
    Jones, R.E., Connors, E.S., Endsley, M.R.: Incorporating the human analyst into the data fusion process by modeling situation awareness using fuzzy cognitive maps. In: 12th International Conference on Information Fusion, 2009. FUSION’09, pp. 1265–1271. IEEE (2009)Google Scholar
  18. 18.
    Katz, S.: Assessing self-maintenance: activities of daily living, mobility, and instrumental activities of daily living. J. Am. Geriatr. Soc. 31(12), 721–727 (1983)CrossRefGoogle Scholar
  19. 19.
    Khaleghi, B., Khamis, A., Karray, F.O., Razavi, S.N.: Multisensor data fusion: a review of the state-of-the-art. Inf. Fusion 14(1), 28–44 (2013)CrossRefGoogle Scholar
  20. 20.
    Klein, L.A.: Sensor and Data Fusion Concepts and Applications. Society of Photo-Optical Instrumentation Engineers (SPIE), Bellingham (1999)Google Scholar
  21. 21.
    Klein, L.A.: Sensor and Data Fusion: A Tool for Information Assessment and Decision Making, vol. 324. Spie Press, Bellingham (2004)CrossRefGoogle Scholar
  22. 22.
    Kofod-Petersen, A., Cassens, J.: Using activity theory to model context awareness. In: Modeling and Retrieval of Context, pp. 1–17. Springer (2006)Google Scholar
  23. 23.
    Kohlas, J., Monney, P.-A.: A Mathematical Theory of Hints: An Approach to the Dempster-Shafer Theory of Evidence, vol. 425. Springer Science & Business Media, Berlin (2013)zbMATHGoogle Scholar
  24. 24.
    Liang-zhou, C., Wen-kang, S., Yong, D., Zhen-fu, Z.: A new fusion approach based on distance of evidences. J. Zhejiang Univ. Sci. A 6(5), 476–482 (2005)CrossRefGoogle Scholar
  25. 25.
    Lifschitz, V.: Answer set programming and plan generation. Artif. Intell. 138(1), 39–54 (2002)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Lowrance, J.D., Garvey, T.D., Strat, T.M.: A framework for evidential-reasoning systems. In: Classic Works of the Dempster-Shafer Theory of Belief Functions, pp. 419–434. Springer (2008)Google Scholar
  27. 27.
    Lukasiewicz, T., Straccia, U.: Managing uncertainty and vagueness in description logics for the semantic web. Web Semant. Sci. Serv. Agents World Wide Web 6(4), 291–308 (2008)CrossRefGoogle Scholar
  28. 28.
    McCallum, A.: Efficiently inducing features of conditional random fields. In: Proceedings of the Nineteenth conference on Uncertainty in Artificial Intelligence, pp. 403–410. Morgan Kaufmann Publishers Inc. (2002)Google Scholar
  29. 29.
    McKeever, S.: Recognising situations using extended Dempster-Shafer theory. Ph.D. thesis, National University of Ireland, Dublin (2011)Google Scholar
  30. 30.
    Meditskos, G., Dasiopoulou, S., Kompatsiaris, I.: Metaq: a knowledge-driven framework for context-aware activity recognition combining sparql and owl 2 activity patterns. Pervasive Mob. Comput. (2015)Google Scholar
  31. 31.
    Menschner, P., Prinz, A., Koene, P., Köbler, F., Altmann, M., Krcmar, H., Leimeister, J.M.: Reaching into patients homes-participatory designed aal services. Electron. Mark. 21(1), 63–76 (2011)CrossRefGoogle Scholar
  32. 32.
    Michael, J., Grießer, A., Strobl, T., Mayr, H.C.: Cognitive modeling and support for ambient assistance. In: Information Systems: Methods, Models, and Applications, pp. 96–107. Springer (2012)Google Scholar
  33. 33.
    Murphy, K.P.: Dynamic bayesian networks: representation, inference and learning. Ph.D. thesis, University of California (2002)Google Scholar
  34. 34.
    Novakovic, J.: Using information gain attribute evaluation to classify sonar targets. In: 17th Telecommunications Forum TELFOR, pp. 24–26 (2009)Google Scholar
  35. 35.
    Okeyo, G., Chen, L., Wang, H., Sterritt, R.: Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive Mob. Comput. 10, 155–172 (2014)CrossRefGoogle Scholar
  36. 36.
    Piyathilaka, L., Kodagoda, S.: Gaussian mixture based hmm for human daily activity recognition using 3d skeleton features. In: 2013 8th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 567–572. IEEE (2013)Google Scholar
  37. 37.
    Pruteanu-Malinici, I., Carin, L.: Infinite hidden markov models for unusual-event detection in video. IEEE Trans. Image Process. 17(5), 811–822 (2008)CrossRefMathSciNetGoogle Scholar
  38. 38.
    Rice, J.: Mathematical Statistics and Data Analysis. Cengage learning, Boston (2006)Google Scholar
  39. 39.
    Sentz, K., Ferson, S.: Combination of Evidence in Dempster-Shafer Theory, vol. 4015. Citeseer (2002)Google Scholar
  40. 40.
    Smets, P.: The combination of evidence in the transferable belief model. IEEE Trans. Pattern Anal. Mach. Intell. 12(5), 447–458 (1990)CrossRefGoogle Scholar
  41. 41.
    Soininen, T.: An Approach to Knowledge Representation and Reasoning for Product Configuration Tasks. Finnish Academies of Technology, Espoo (2000)Google Scholar
  42. 42.
    Stoilos, G., Simou, N., Stamou, G., Kollias, S.: Uncertainty and the semantic web. Intell. Syst. IEEE 21(5), 84–87 (2006)CrossRefGoogle Scholar
  43. 43.
    Vail, D.L., Veloso, M.M., Lafferty, J.D.: Conditional random fields for activity recognition. In: Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems, pp. 235. ACM (2007)Google Scholar
  44. 44.
    Van Kasteren, T., Noulas, A., Englebienne, G., Kröse, B.: Accurate activity recognition in a home setting. In: Proceedings of the 10th international conference on Ubiquitous computing, pp. 1–9. ACM (2008)Google Scholar
  45. 45.
    Walczak, B., Massart, D.: Rough sets theory. Chemometr. Intell. Lab. Syst. 47(1), 1–16 (1999)CrossRefGoogle Scholar
  46. 46.
    Wang, X., Ji, Q.: Incorporating contextual knowledge to dynamic bayesian networks for event recognition. In: Pattern Recognition (ICPR), 2012 21st International Conference on, pp. 3378–3381. IEEE (2012)Google Scholar
  47. 47.
    Welch, G., Bishop, G.: An introduction to the kalman filter (1995)Google Scholar
  48. 48.
    Yen, J.: Generalizing the dempster–shafer theory to fuzzy sets. In: Classic Works of the Dempster-Shafer Theory of Belief Functions, pp. 529–554. Springer (2008)Google Scholar
  49. 49.
    Zhang, W., Chen, F., Xu, W., Cao, Z.: Decomposition in hidden markov models for activity recognition. In: Multimedia Content Analysis and Mining, pp. 232–241. Springer (2007)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Fadi Al Machot
    • 1
    Email author
  • Heinrich C. Mayr
    • 1
  • Suneth Ranasinghe
    • 1
  1. 1.Institute for Applied Informatics, Application EngineeringAlpen-Adria-UniversitätKlagenfurtAustria

Personalised recommendations