An Ontology for Modelling Human Machine Interaction in Smart Environments

  • Norman KösterEmail author
  • Sebastian Wrede
  • Philipp Cimiano
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 16)


In many smart environments including smart homes, human machine interactions are crucial. Being able to describe, store and query interaction data is an important feature. This especially gains importance when artificial embodied agents (i.e. robotic companions) are present. In order to support querying interaction data at a conceptual level, abstracting from specific and heterogeneous data schemata, this paper presents an ontology for the domain of interaction. To model the interaction domain, the ontology imports two pre-existing ontologies: the Semantic Sensor Network Ontology and the Time Ontology. The ontology further includes interaction-relevant concepts as they occur in smart homes but also in other embodied interactive smart environments. Also, a reference application is described for a data management and query system for interaction data that builds on this ontology. The goal is to support the easy querying of interaction concepts both by machine agents but also by developers of applications running in smart environments. Therefore, further exemplary queries are provided to show the retrieval capabilities of a system that uses the ontology as a proof-of-concept.


Smart environments Human machine interaction Interaction data Ontology Model driven engineering 



This research/work was supported by the Cluster of Excellence Cognitive Interaction Technology ‘CITEC’ (EXC 277) at Bielefeld University, which is funded by the German Research Foundation (DFG).


  1. 1.
    Compton, M., Barnaghi, P., Bermudez, L., GarcíA-Castro, R., Corcho, O., Cox, S., et al.: The SSN ontology of the W3C semantic sensor network incubator group. Web Semant. Sci. Serv. Agents World Wide Web 17, 25–32 (2012)CrossRefGoogle Scholar
  2. 2.
    Hobbs, J.R., Pan, F.: Time ontology in OWL. W3C working draft, vol. 27, p. 133 (2006)Google Scholar
  3. 3.
    Yanco, H.A., Dury, J.: Classifying Human-Robot Interaction: An Updated Taxonomy (2004)Google Scholar
  4. 4.
    Gruber, T.R.: A translation approach to portable ontology specifications. Knowl. Acquisition 5, 199–220 (1993)CrossRefGoogle Scholar
  5. 5.
    Masolo, C., Borgo, S., Gangemi, A., Guarino, N., Oltramari, A.: WonderWeb deliverable D18. Communities 2003, 343 (2003)Google Scholar
  6. 6.
    Pease, A., Niles, I., Li, J.: The suggested upper merged ontology: a large ontology for the semantic web and its applications. Imagine 28, 7–10 (2002)Google Scholar
  7. 7.
    Guha, R.: Introducing search engines come together for a richer web. Google Official Blog (2011)Google Scholar
  8. 8.
    McAvoy, L., Chen, L., Donnelly, M.: An Ontology-based Context Management System for Smart Environments, pp. 18–23 (2012)Google Scholar
  9. 9.
    Mallik, A., Tripathi, A., Kumar, R., Chaudhury, S., Sinha, K.: Ontology based context aware situation tracking, pp. 687–692. IEEE (2015)Google Scholar
  10. 10.
    Wongpatikaseree, K., Ikeda, M., Buranarach, M., Supnithi, T., Lim, A.O., Tan, Y.: Activity recognition using context-aware infrastructure ontology in smart home domain. In: 7th International Conference on Knowledge, Information and Creativity Support Systems, pp. 50–57 (2012)Google Scholar
  11. 11.
    Tenorth, M., Beetz, M.: KnowRob: a knowledge processing infrastructure for cognition-enabled robots. Int. J. Robot. Res. 32(5), 566–590 (2013)CrossRefGoogle Scholar
  12. 12.
    Beetz, M., Tenorth, M., Winkler, J.: Open-EASE. In: IEEE International Conference on Robotics and Automation, pp. 1983–1990 (2015)Google Scholar
  13. 13.
    Lim, G.H., Suh, I.H., Suh, H.: Ontology-based unified robot knowledge for service robots in indoor environments. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 41(3), 492–509 (2011)CrossRefGoogle Scholar
  14. 14.
    Lemaignan, S., Ros, R., Mösenlechner, L., Alami, R., Beetz, M.: ORO, a knowledge management platform for cognitive architectures in robotics. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3548–3553. IEEE (2010)Google Scholar
  15. 15.
    Gómez-Pérez, A.: Ontological engineering: a state of the art. Expert Update: Knowl. Based Syst. Appl. Artif. Intell. 2(3), 33–43 (1999)Google Scholar
  16. 16.
    Sure, Y., Staab, S., Studer, R.: Ontology engineering methodology. In: Handbook on Ontologies, pp. 135–152. Springer (2009)Google Scholar
  17. 17.
    Fensel, D., Harmelen, F.V., Klein, M., Akkermans, H., Schnurr, H.-P., Studer, R., et al.: On-To-Knowledge: ontology-based tools for knowledge management. In: Proceedings of the eBusiness and eWork, pp. 18–20 (2000)Google Scholar
  18. 18.
    Noy, N.F., Sintek, M., Decker, S., Crubézy, M., Fergerson, R.W., Musen, M.A.: Creating semantic web contents with Protege-2000. IEEE Intell. Syst. 16(2), 60–71 (2001)CrossRefGoogle Scholar
  19. 19.
    Köster, N., Wrede, S., Cimiano, P.: Requirements for domain-specific data access of long-term interaction data in smart environments (2015)Google Scholar
  20. 20.
    Beeson, P., Kortenkamp, D., Bonasso, R.P., Persson, A., Loutfi, A., Bona, J.P.: An ontology-based symbol grounding system for human-robot interaction. In: AAAI Fall Symposium Series (2014)Google Scholar
  21. 21.
    Riboni, D., Pareschi, L., Radaelli, L., Bettini, C.: Is ontology-based activity recognition really effective? In: 2011 IEEE International Conference on Pervasive Computing and Communications Workshops, PERCOM Workshops 2011, pp. 427–431 (2011)Google Scholar
  22. 22.
    Wienke, J., Wrede, S.: A middleware for collaborative research in experimental robotics. In: IEEE/SICE International Symposium on System Integration (SII), vol. 2011, pp. 1183–1190. IEEE (2011)Google Scholar
  23. 23.
    Holthaus, P., Leichsenring, C., Richter, V., Pohling, M., Carlmeyer, B., Köster, N., et al.: How to address smart homes with a social robot? a multi-modal corpus of user interactions with an intelligent environment. In: International Conference on Language Resources and Evaluation, European Language Resources Association (ELRA) (2016). (in press)Google Scholar
  24. 24.
    Modoni, G.E., Sacco, M., Terkaj, W.: A survey of RDF store solutions. In: 2014 International Conference on Engineering, Technology and Innovation: Engineering Responsible Innovation in Products and Services, ICE 2014, pp. 1–7 (2014)Google Scholar
  25. 25.
    Neo Technology Inc., openCypher, 2015. Accessed 28 Feb 2016

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Norman Köster
    • 1
    Email author
  • Sebastian Wrede
    • 1
    • 2
  • Philipp Cimiano
    • 1
  1. 1.Cluster of Excellence Center in Cognitive Interactive Technology (CITEC)Bielefeld UniversityBielefeldGermany
  2. 2.Research Institute for Cognition and Robotics (CoR-Lab)Bielefeld UniversityBielefeldGermany

Personalised recommendations