KI - Künstliche Intelligenz

, Volume 30, Issue 1, pp 21–27 | Cite as

Perception for Everyday Human Robot Interaction

  • Jan-Hendrik Worch
  • Ferenc Bálint-Benczédi
  • Michael Beetz
Technical Contribution


The ability to build robotic agents that can perform everyday tasks heavily depends on understanding how humans perform them. In order to achieve close to human understanding of a task and generate a formal representation of it, it is important to jointly reason about the human actions and the objects that are being acted on. We present a robotic perception framework for perceiving actions performed by a human in a household environment that can be used to answer questions such as “which object did the human act on?” or “which actions did the human perform?”. To do this we extend the RoboSherlock framework with the capabilities of detecting humans and objects at the same time, while simultaneously reasoning about the possible actions that are being performed.


Object Detection Action Recognition Belief State Object Label Robotic Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



This work was supported in part by the EU FP7 Projects RoboHow (Grant Agreement Number 288533), SAPHARI (Grant Agreement Number 287513) and ACAT (Grant Agreement Number 600578) and by the German Research Foundation (DFG) as part of the Project MeMoMan2.


  1. 1.
    Yazdani F, Brieber B, Beetz M (2014) Cognition-enabled robot control for mixed human-robot rescue teams. In: Proceedings of the 13th International Conference on Intelligent Autonomous Systems (IAS-13)Google Scholar
  2. 2.
    Beetz M, Bartels G, Albu-Schäffer A, Bálint-Benczédi F, Belder R, Beßler D, Haddadin S, Maldonado A, Mansfeld N, Wiedemeyer T, Weitschat R, Worch JH (2015) Robotic agents capable of natural and safe physical interaction with human co-workers. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, GermanyGoogle Scholar
  3. 3.
    Beetz M, Jain D, Mosenlechner L, Tenorth M, Kunze L, Blodow N, Pangercic D (2012) Cognition-enabled autonomous robot control for the realization of home chore task intelligence. Proc IEEE 100(8):2454–2471CrossRefGoogle Scholar
  4. 4.
    Beetz M, Tenorth M, Winkler J (2015) Open-EASE—a knowledge processing service for robots and robotics/ai researchers. In: IEEE International Conference on Robotics and Automation (ICRA). Seattle, WashingtonGoogle Scholar
  5. 5.
    Beetz M, Balint-Benczedi F, Blodow N, Nyga D, Wiedemeyer T, Marton ZC (20150 RoboSherlock: unstructured information processing for robot perception. In: IEEE International Conference on Robotics and Automation (ICRA). Seattle, WashingtonGoogle Scholar
  6. 6.
    Ferrucci D, Lally A (2004) UIMA: an architectural approach to unstructured information processing in the corporate research environment. In: Natural Language Engineering, vol. 10, pp. 3–4. Cambridge University PressGoogle Scholar
  7. 7.
    Ferrucci D, Brown E, Chu-Carroll J, Fan J, Gondek D, Kalyanpur AA, Lally A, Murdock JW, Nyberg E, Prager J, Schlaefer N, Welty C (2010) Building Watson: an overview of the DeepQA project. AI Mag 31(3):59–79Google Scholar
  8. 8.
    Nyga D, Balint-Benczedi F, Beetz M (2014) PR2 Looking at things: ensemble learning for unstructured information processing with Markov logic networks. In: IEEE International Conference on Robotics and Automation (ICRA). Hong Kong, ChinaGoogle Scholar
  9. 9.
    Tenorth M, Beetz M (2009) KnowRob—knowledge processing for autonomous personal robots. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4261–4266Google Scholar
  10. 10.
    Wiedemeyer T, Balint-Benczedi F, Beetz M (2015) Pervasive ’calm’ perception for autonomous robotic agents. In: Proceedings of the 2015 International Conference on Autonomous Agents and Multiagen Systems. ACM, IstanbulGoogle Scholar
  11. 11.
    Patterson D, Fox D, Kautz H, Philipose M (2005) Fine-grained activity recognition by aggregating abstract object usage. In: Proc. of the International Symposium on Wearable ComputersGoogle Scholar
  12. 12.
    Billard A, Calinon S, Dillmann R, Schaal S (2008) Robot programming by demonstration. In: Springer Hand-book of Robotics, Springer, ch. 59Google Scholar
  13. 13.
    Kim S, Shukla A, Billard A (2014) Catching objects in flight. EEE Trans Robot 30(5):1049–1065CrossRefGoogle Scholar
  14. 14.
    Nguyen H, Jain A, Anderson C, Kemp C (2008) A clickable world: behavior selection through pointing and context for mobile manipulation. In: Intelligent robots and systems. IROS 2008. IEEE/RSJ International Conference on, pp. 787–793Google Scholar
  15. 15.
    Blodow N, Marton ZC, Pangercic D, Rühr T, Tenorth M, Beetz M (2011) Inferring generalized pick-and-place tasks from pointing gestures. In: IEEE International Conference on Robotics and Automation (ICRA), Workshop on Semantic Perception, Mapping and ExplorationGoogle Scholar
  16. 16.
    Nyga D, Tenorth M, Beetz M (2011) How-models of human reaching movements in the context of everyday manipulation activities. In: IEEE International Conference on Robotics and Automation (ICRA). Shanghai, ChinaGoogle Scholar
  17. 17.
    Haidu A, Kohlsdorf D, Beetz M (2014) Learning task outcome prediction for robot control from interactive environments. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Chicago, USAGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Jan-Hendrik Worch
    • 1
  • Ferenc Bálint-Benczédi
    • 1
  • Michael Beetz
    • 1
  1. 1.Institute for Artificial IntelligenceUniversität BremenBremenGermany

Personalised recommendations