Healthcare robots will be the next big advance in humans’ domestic welfare, with robots able to assist elderly people and users with disabilities. However, each user has his/her own preferences, needs and abilities. Therefore, robotic assistants will need to adapt to them, behaving accordingly. Towards this goal, we propose a method to perform behavior adaptation to the user preferences, using symbolic task planning. A user model is built from the user’s answers to simple questions with a fuzzy inference system, and it is then integrated into the planning domain. We describe an adaptation method based on both the user satisfaction and the execution outcome, depending on which penalizations are applied to the planner’s rules. We demonstrate the application of the adaptation method in a simple shoe-fitting scenario, with experiments performed in a simulated user environment. The results show quick behavior adaptation, even when the user behavior changes, as well as robustness to wrong inference of the initial user model. Finally, some insights in a non-simulated world shoe-fitting setup are also provided.
This is a preview of subscription content, access via your institution.
Buy single article
Instant access to the full article PDF.
Tax calculation will be finalised during checkout.
For simplicity, we consider only one successful outcome, though it can be easily extended to several successful outcomes.
We define costs as negative rewards, thus subtracting to the previous cost we are worsening it.
This would happen if the user changes his/her behavior from the adapted model, always providing a positive feedback score.
Note that the feedback update modifies the reward, thus the reward plots of the methods using it keep improving its reward because of this.
Shoe grasping is out of the scope of the paper.
Alili, S., Warnier, M., Ali, M., & Alami, R. (2009). Planning and plan-execution for human–robot cooperative task achievement. In 19th international conference on automated planning and scheduling (pp. 19–23).
Canal, G., Alenyà, G., & Torras, C. (2016). Personalization framework for adaptive robotic feeding assistance. In 8th international conference on social robotics (ICSR) (pp. 22–31).
Canal, G., Alenyà, G., & Torras, C. (2017). A taxonomy of preferences for physically assistive robots. In IEEE international symposium on robot and human interactive communication (RO-MAN) (pp. 292–297).
Castellano, G., Leite, I., & Paiva, A. (2016). Detecting perceived quality of interaction with a robot using contextual features. Autonomous Robots, 41, 1245–1261.
Chance, G., Camilleri, A., Winstone, B., Caleb-Solly, P., & Dogramadzi, S. (2016). An assistive robot to support dressing-strategies for planning and error handling. In Proceedings of the 6th IEEE RAS/EMBS international conference on biomedical robotics and biomechatronics. IEEE.
Chen, T. L., Ciocarlie, M., Cousins, S., Grice, P. M., Hawkins, K., Hsiao, K., et al. (2013). Robots for humanity: Using assistive robotics to empower people with disabilities. IEEE Robotics Automation Magazine, 20(1), 30–39.
de Silva, L., Lallement, R., & Alami, R. (2015). The HATP hierarchical planner: Formalisation and an initial study of its usability and practicality. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 6465–6472).
Erol, K., Hendler, J., & Nau, D. S. (1994). HTN planning: Complexity and expressivity. AAAI, 94, 1123–1128.
Fiore, M., Clodic, A., & Alami, R. (2016). On planning and task achievement modalities for human–robot collaboration. Springer Tracts in Advanced Robotics, 109, 293–306.
Gao, Y., Chang, H. J., & Demiris, Y. (2015). User modelling for personalised dressing assistance by humanoid robots. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 1840–1845). IEEE.
Gao, Y., Chang, H. J., & Demiris, Y. (2016). Iterative path optimisation for personalised dressing assistance using vision and force information. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 4398–4403).
Griffith, S., Subramanian, K., Scholz, J., Isbell, C., & Thomaz, A. L. (2013). Policy shaping: Integrating human feedback with reinforcement learning. In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, & K. Q. Weinberger (Eds.), Advances in neural information processing systems (Vol. 26, pp. 2625–2633). La Jolla: Curran Associates, Inc.
Heerink, M., Krose, B., Evers, V., & Wielinga, B. (2009). Measuring acceptance of an assistive social robot: A suggested toolkit. In RO-MAN 2009—The 18th IEEE international symposium on robot and human interactive communication (pp. 528–533).
Klee, S. D., Ferreira, B. Q., Silva, R., Costeira, J. P., Melo, F. S., & Veloso, M. (2015). Personalized assistance for dressing users. In Social robotics: 7th international conference, ICSR 2015 (pp. 359–369). Springer.
Knox, W. B., & Stone, P. (2009). Interactively shaping agents via human reinforcement: The tamer framework. In The fifth international conference on knowledge capture.
Kühnlenz, B., Sosnowski, S., Buß, M., Wollherr, D., Kühnlenz, K., & Buss, M. (2013). Increasing helpfulness towards a robot by emotional adaption to the user. International Journal of Social Robotics, 5(4), 457–476.
Lallement, R., De Silva, L., & Alami, R. (2014). HATP: An HTN planner for robotics. In 2nd ICAPS workshop on planning and robotics.
Mamdani, E., & Assilian, S. (1975). An experiment in linguistic synthesis with a fuzzy logic controller. International Journal of Man-Machine Studies, 7(1), 1–13. https://doi.org/10.1016/S0020-7373(75)80002-2.
Martínez, D., Alenyà, G., & Torras, C. (2015). Planning robot manipulation to clean planar surfaces. Engineering Applications of Artificial Intelligence, 39, 23–32.
McLafferty, E., & Farley, A. (2008). Assessing pain in patients. Nursing Standard, 22(25), 42–46.
Muir, B. M. (1987). Trust between humans and machines, and the design of decision aids. International Journal of Man-Machine Studies, 27(5), 527–539.
Pasula, H. M., Zettlemoyer, L. S., & Kaelbling, L. P. (2007). Learning symbolic models of stochastic domains. Journal of Artificial Intelligence Research, 29, 309–352.
Pignat, E., & Calinon, S. (2017). Learning adaptive dressing assistance from human demonstration. Robotics and Autonomous Systems, 93, 61–75.
Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., & Ng, A. Y. (2009). ROS: An open-source robot operating system. In ICRA workshop on open source software (Vol. 3, p. 5).
Rada-Vilela, J. (2014). fuzzylite: A fuzzy logic control library. http://www.fuzzylite.com. Accessed 26 Oct 2016.
Robinson, H., MacDonald, B., & Broadbent, E. (2014). The role of healthcare robots for older people at home: A review. International Journal of Social Robotics, 6(4), 575–591.
Rozo, L., Calinon, S., Caldwell, D. G., Jimenez, P., & Torras, C. (2016). Learning physical collaborative robot behaviors from human demonstrations. IEEE Transactions on Robotics, 32(3), 513–527.
Tamei, T., Matsubara, T., Rai, A., & Shibata, T. (2011). Reinforcement learning of clothing assistance with a dual-arm robot. In 2011 11th IEEE-RAS international conference on humanoid robots (humanoids) (pp. 733–738). IEEE.
Thomaz, A. L., & Breazeal, C. (2006). Reinforcement learning with human teachers: Evidence of feedback and guidance with implications for learning performance. In Proceedings of the 21st national conference on artificial intelligence (Vol. 1, pp. 1000–1005). AAAI Press, AAAI’06.
Vahrenkamp, N., Wächter, M., Kröhnert, M., Welke, K., & Asfour, T. (2015). The robot software framework armarx. Information Technology, 57(2), 99–111.
Yamazaki, K., Oya, R., Nagahama, K., Okada, K., & Inaba, M. (2014). Bottom dressing by a life-sized humanoid robot provided failure detection and recovery functions. In 2014 IEEE/SICE international symposium on system integration (pp. 564–570).
The authors thank Clea Parcerisas, Sergi Foix and Adrià Colomé for their help in the realization of the experiments, pictures and video.
This is one of the several papers published in Autonomous Robots comprising the Special Issue on Learning for Human–Robot Collaboration.
This work has been partially supported by the CHIST-ERA Project I-DRESS PCIN-2015-147, by the Spanish Ministry of Science and Innovation HuMoUR TIN2017-90086-R and the AEI through the María de Maeztu Seal of Excellence to IRI (MDM-2016-0656). Gerard Canal is also supported by the Spanish Ministry of Education, Culture and Sport via a doctoral grant FPU15/00504.
Electronic supplementary material
Below is the link to the electronic supplementary material.
About this article
Cite this article
Canal, G., Alenyà, G. & Torras, C. Adapting robot task planning to user preferences: an assistive shoe dressing example. Auton Robot 43, 1343–1356 (2019). https://doi.org/10.1007/s10514-018-9737-2
- Planning with preferences
- Behavior adaptation
- Task personalization
- Shoe fitting