Autonomous Robots

, Volume 39, Issue 4, pp 469–485 | Cite as

Cognitive robots learning failure contexts through real-world experimentation

  • Sertac Karapinar
  • Sanem Sariel


Learning is essential for cognitive robots as humans to gain experience and to adapt to the real world. We propose an experiential learning method for robots to build their experience online and to transfer knowledge among appropriate contexts. Experience gained through learning is used as a guide to future decisions of the robot for both efficiency and robustness. We use Inductive Logic Programming (ILP) learning paradigm to frame hypotheses represented in first-order logic that are useful for further reasoning and planning processes. Furthermore, incorporation of background knowledge is also possible to generalize the framed hypotheses. Partially specified world states can also be easily represented by these hypotheses. All these advantages of ILP make this approach superior to the other supervised learning methods. We have analyzed the performance of the learning method on our autonomous mobile robot and on our robot arm both building their experience on action executions online. It has been observed in both domains that our experience-based learning and learning-based guidance methods frame sound hypotheses that are useful for constraining and guiding the future tasks of the robots. This learning paradigm is promising especially for the contexts where abstraction is useful for efficient transfer of knowledge.


Robot learning Learning from experience Experiential learning in robots Planning and plan execution Action execution monitoring Real-world experimentation Failure contexts 



This research is funded by a grant from the Scientific and Technological Research Council of Turkey, Grant No. 111E-286. We thank Petek Yildiz for her contribution in the planning guidance implementations, Burak Topal and Abdullah Cihan Ak for their contributions in the controller design of the robot arm and help in the experiments, Dogan Altan for ground robot experiments, Mustafa Ersen, Melis Kapotoglu, Melodi Deniz Ozturk, Cagatay Koc and Arda Inceoglu for their contributions in different components of the framework.


  1. Aler, R., Borrajo, D., & Isasi, P. (2002). Using genetic programming to learn and improve control knowledge. Artificial Intelligence, 141, 141–1.CrossRefGoogle Scholar
  2. Aoude, G. S., Luders, B. D., Levine, D. S., & How, J. P. (2010) Threat-aware path planning in uncertain urban environments. In IEEE/RSJ international conference on intelligent robots and systems (IROS).Google Scholar
  3. Bacchus, F., & Ady, M. (2001). Planning with resources and concurrency a forward chaining approach. In Proceedings of the 17th international joint conference on artificial intelligence (Vol. 1, pp. 417–424).Google Scholar
  4. Bacchus, F., & Kabanza, F. (2000). Using temporal logics to express search control knowledge for planning. Artificial Intelligence, 116(1), 123–191.zbMATHMathSciNetCrossRefGoogle Scholar
  5. Borrajo, D., & Veloso, M. (1994). Incremental learning of control knowledge for improvement of planning efficiency. In AAAI-94 fall symposium on planning and learning (pp. 5–9).Google Scholar
  6. Borrajo, D., & Veloso, M. (1996). Lazy incremental learning of control knowledge for efficiently obtaining quality plans. AI Review Journal. Special Issue on Lazy Learning 11, 371–405.Google Scholar
  7. Botea, A., Enzenberger, M., Müller, M., & Schaeffer, J. (2005). Macro-FF: Improving AI planning with automatically learned macro-operators. Journal of Artificial Intelligence Research, 24, 581–621.zbMATHGoogle Scholar
  8. Chow, Y. L., Pavone, M., Sadler, B. M., & Carpin, S. (2014). Trading safety versus performance: Rapid deployment of robotic swarms with robust performance constraints. Journal of Dynamic Systems, Measurement and Control, 137(3), 031005.CrossRefGoogle Scholar
  9. Cohen, W. W. (1990). Learning approximate control rules of high utility. In Proceedings of the seventh international conference on machine learning, Morgan Kaufmann, Burlington (pp. 268–276).Google Scholar
  10. Dearden, R., & Burbridge, C. (2014). Manipulation planning using learned symbolic state abstractions. Robotics and Autonomous Systems, 62(3), 355–365.CrossRefGoogle Scholar
  11. Duran, G. (2006) Integrating macro-operators and control-rules learning. In Proceedings of the international conference on automated planning and scheduling (ICAPS), Cumbria, UK.Google Scholar
  12. Ersen, M., Ozturk, M. D., Biberci, M., Sariel, S., & Yalcin, H. (2014) Scene interpretation for lifelong robot learning. In Proceedings of the 9th international workshop on cognitive robotics (CogRob 2014) held in conjunction with ECAI-2014, Prague, Czech Republic.Google Scholar
  13. Ersen, M., Sariel-Talay, S., & Yalcin, H. (2013). Extracting spatial relations among objects for failure detection. In Procedings of the KI 2013 workshop on visual and spatial cognition (pp. 13–20).Google Scholar
  14. Estlin, T. A., & Mooney, R. J. (1997) Learning to improve both efficiency and quality of planning. In Proceedings of the fifteenth international joint conference on artificial intelligence, Morgan Kaufmann, Burlington (pp. 1227–1232).Google Scholar
  15. Fernandez, S., Aler, R., & Borrajo, D. (2004). Using previous experience for learning planning control knowledge. In Proceedings of the seventeen international Florida artificial intelligence symposium (FLAIRS04), AAAI Press.Google Scholar
  16. Feyzabadi, S., Carpin, S. (2014). Risk-aware path planning using hirerachical constrained markov decision processes. In Proceedings of the IEEE international conference on automation science and engineering (CASE).Google Scholar
  17. Gspandl, S., Pill, I., Reip, M., Steinbauer, G., & Ferrein, A. (2011). Belief management for high-level robot programs. In Proceedings of the international joint conference on artificial intelligence (IJCAI).Google Scholar
  18. Haigh, K. Z., & Veloso, M. M. (1999). Learning situation-dependent costs: Improving planning from probabilistic robot execution. Robotics and Autonomous Systems, 29, 145–174.CrossRefGoogle Scholar
  19. Hermans, T. R. (2014). Representing and learning affordance-based behaviors. Ph.D. thesis, Georgia Institute of Technology.Google Scholar
  20. Hinterstoisser, S., Cagniart, C., Ilic, S., Sturm, P. F., Navab, N., Fua, P., et al. (2012). Gradient response maps for real-time detection of textureless objects. IEEE Transaction on Pattern Analysis and Machine Intelligence, 34(5), 876–888.CrossRefGoogle Scholar
  21. Horn, A. (1951). On sentences which are true of direct unions of algebras. Journal of Symbolic Logic, 16(1), 14–21.zbMATHMathSciNetCrossRefGoogle Scholar
  22. Kabanza, F., Barbeau, M., & St-Denis, R. (1997). Planning control rules for reactive agents. Artificial Intelligence, 95(1), 67–113.zbMATHMathSciNetCrossRefGoogle Scholar
  23. Kapotoglu, M., Koc, C., Sariel, S., & Ince, G. (2014) Action monitoring in cognitive robots. In Proceedings of the IEEE 22nd signal processing and communications applications conference (SIU) (pp. 2154–2157).Google Scholar
  24. Karapinar, S., Altan, D., & Sariel-Talay, S. (2012) A robust planning framework for cognitive robots. In Proceedings of the AAAI-12 workshop on cognitive robotics (CogRob) (pp. 102–108).Google Scholar
  25. Karapinar, S., Ersen, M., Kapotoglu, M., Yildiz, P., Sariel-Talay, S., & Yalcin, H. (2013). Bilissel robotlarda deneyimsel ogrenme. In Proceedings of the 21st IEEE signal processing and communications applications conference (SIU), Girne, KKTC.Google Scholar
  26. Karapinar, S., & Sariel, S. (2015). Cognitive robots learning failure contexts through experimentation. In Proceedings of the 14th international conference on autonomous agents & multiagent systems (AAMAS).Google Scholar
  27. Karapinar, S., & Sariel-Talay, S. (2012). Failure handling in a planning framework. In Proceedings of the AAAI-12 student workshop, AAAI Press, Toronto, ON, Canada (pp. 2431–2432).Google Scholar
  28. Karapinar, S., Sariel-Talay, S., Yildiz, P., & Ersen, M. (2013). Learning guided planning for robust task execution in cognitive robotics. In Proceedings of the AAAI-13 workshop on intelligent robotic systems, Bellevue, USA (pp. 26–31).Google Scholar
  29. Katukam, S., & Kambhampati, S. (1994). Learning explanation-based search control rules for partial order planning. In Proceedings of the 12th national conference on artificial intelligence (AAAI) (pp. 582–587).Google Scholar
  30. Khardon, R. (1997). Learning action strategies for planning domains. Artificial Intelligence, 113, 125–148.MathSciNetCrossRefGoogle Scholar
  31. Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs: Prentice-Hall P T R.Google Scholar
  32. Kvarnström, J., Heintz, F., & Doherty, P. (2008). A temporal logic-based planning and execution monitoring system. In Proceedings of the 18th international conference on automated planning and scheduling (ICAPS) (pp. 198–205).Google Scholar
  33. Lang, T., & Toussaint, M. (2010). Planning with noisy probabilistic relational rules. Journal of Artificial Intelligence Research, 39, 1–49.zbMATHGoogle Scholar
  34. Leckie, C., & Zukerman, I. (1998). Inductive learning of search control rules for planning. Artificial Intelligence, 101(1–2), 63–98.zbMATHCrossRefGoogle Scholar
  35. López, B., & Plaza, E. (1991) Case-based learning of strategic knowledge. In Proceedings of the European working session on learning porto (EWSL) (pp. 398–411)Google Scholar
  36. Magnenat, S., Chappelier, J. C., Mondada, F. (2012). Integration of online learning into htn planning for robotic tasks. In Proceedings of the AAAI spring symposium: Designing intelligent robots, AAAI technical report, AAAI (Vol. SS-12-02).Google Scholar
  37. Morisset, B., & Ghallab, M. (2008). Learning how to combine sensory-motor functions into a robust behavior. Artificial Intelligence, 172(4), 392–412.CrossRefGoogle Scholar
  38. Muggleton, S. (1995). Inverse entailment and Progol. New Generation Computing, 13(3–4), 245–286. Special issue on Inductive Logic Programming.CrossRefGoogle Scholar
  39. Ozturk, M. D., Ersen, M., Kapotoglu, M., Koc, C., Sariel-Talay, S., & Yalcin, H. (2014). Scene interpretation for self-aware cognitive robots. In Proceedings of the AAAI spring symposia qualitative representations for robots, California, USA.Google Scholar
  40. Pasula, H., Zettlemoyer, L. S., & Kaelbling, L. P. (2007). Learning symbolic models of stochastic domains. Journal of Artificial Intelligence Research, 29, 309–352.Google Scholar
  41. Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. (2009) ROS: an open-source robot operating system. In Proceedings of the ICRA-09 workshop on open source software.Google Scholar
  42. Rintanen, J. (2000). Incorporation of temporal logic control into plan operators. In Proceedings of the European conference on artificial intelligence (ECAI), IOS Press (pp. 526–530).Google Scholar
  43. Sariel, S., Yildiz, P., Karapinar, S., Altan, D., & Kapotoglu, M. (2015). Robust task execution through experience-based guidance for cognitive robots. In Proceedings of the 17th international conference on advanced robotics (ICAR).Google Scholar
  44. Saxena, A., Jain, A., Sener, O., Jami, A., Misra, D. K., & Koppula, H. S. (2014). RoboBrain: Large-scale knowledge engine for robots. Computing Research Repository (CoRR) abs/1412.0691Google Scholar
  45. Stansbury, R. S., & Agah, A. (2012). A robot decision making framework using constraint programming. Artificial Intelligence Review, 38, 67–83.CrossRefGoogle Scholar
  46. Tenorth, M., & Beetz, M. (2013). KnowRob: A knowledge processing infrastructure for cognition-enabled robots. International Journal of Robotics Research, 32(5), 566–590.CrossRefGoogle Scholar
  47. Trevor, A. J. B., Gedikli, S., Rusu, R. B., & Christensen, H. I. (2013) Efficient organized point cloud segmentation with connected components. In Proceedings of the 3rd workshop on semantic perception, mapping and exploration (SPME).Google Scholar
  48. Usug, U. C., Altan, D., & Sariel-Talay, S. (2012). Robots that create alternative plans against failures. In Proceedings of the 10th IFAC symposium on robot control.Google Scholar
  49. Usug, U. C., & Sariel-Talay, S. (2011). Dynamic temporal planning for multirobot systems. In Proceedings of the AAAI-11 workshop on automated action planning for autonomous mobile robots (PAMR).Google Scholar
  50. Waibel, M., Beetz, M., Civera, J., DÁndrea, R., Elfring, J., Galvez-Lopez, D., et al. (2011). Roboearth—A world wide web for robots. IEEE Robotics Automation Magazine, 18(2), 69–82.Google Scholar
  51. Wang, X., Simon, H. A., Lehman, J. F., & Fisher, D. H. (1996). Learning planning operators by observation and practice. In Proceedings of the second international conference on AI planning systems, AIPS-94 (pp. 335–340).Google Scholar
  52. Yildiz, P., Karapinar, S., & Sariel-Talay, S. (2013). Learning guided symbolic planning for cognitive robots. In Proceedings of the IEEE international conference on robotics and automation (ICRA), autonomous learning workshop, Karlsruhe, Germany.Google Scholar

Copyright information

© Springer Science+Business Media New York 2015

Authors and Affiliations

  1. 1.Artificial Intelligence and Robotics Laboratory, Computer Engineering DepartmentIstanbul Technical UniversityIstanbulTurkey

Personalised recommendations