Abstract
A soft robot is a kind of robot that is constructed with soft, deformable and elastic materials. Control of soft robots presents complex modeling and planning challenges. We introduce a new approach to accomplish that, making two key contributions: designing an abstract representation of the state of soft robots, and developing a reinforcement learning method to derive effective control policies. The reinforcement learning process can be trained quickly by ignoring the specific materials and structural properties of the soft robot. We apply the approach to the Honeycomb PneuNets Soft Robot and demonstrate the effectiveness of the training method and its ability to produce good control policies under different conditions.
Notes
References
Bai, A., Wu, F., Chen, X.: Bayesian mixture modelling and inference based thompson sampling in monte-carlo tree search. In: Proceedings of NIPS, pp. 1646–1654 (2013)
Bai, A., Wu, F., Chen, X.: Online planning for large markov decision processes with hierarchical decomposition. ACM Trans. Intell. Syst. Technol. 6(4) (2015). Article No. 45
Baird, L., et al.: Residual algorithms: Reinforcement learning with function approximation. In: Proceedings of ICML, pp. 30–37 (1995)
Chen, Y., Wu, F., Shuai, W., Wang, N., Chen, R., Chen, X.: Kejia robot - an attractive shopping mall guider. In: Proceedings of ICSR, pp. 145–154 (2015)
Chen, Y., Wu, F., Wang, N., Tang, K., Cheng, M., Chen, X.: KeJia-LC: a low-cost mobile robot platform - champion of demo challenge on benchmarking service robots at RoboCup 2015. In: Proceedings of RoboCup, vol. 9513, pp. 60–71 (2015)
Cheng, B., Sun, H., Chen, X.: Evolving honeycomb pneumatic finger in bullet physics engine. Robot Intell. Tech. App. 3, 411–423 (2015)
Cheng, M., Chen, X., Tang, K., Wu, F., Kupcsik, A., Iocchi, L., Chen, Y., Hsu, D.: Synthetical benchmarking of service robots: a first effort on domestic mobile platforms. In: Almeida, L., Ji, J., Steinbauer, G., Luke, S. (eds.) RoboCup 2015. LNCS (LNAI), vol. 9513, pp. 377–388. Springer, Cham (2015). doi:10.1007/978-3-319-29339-4_32
Duriez, C.: Control of elastic soft robots based on real-time finite element method. In: Proceedings of ICRA, pp. 3982–3987 (2013)
Giorelli, M., Renda, F., Ferri, G., Laschi, C.: A feed-forward neural network learning the inverse kinetics of a soft cable-driven manipulator moving in three-dimensional space. In: Proceedings of IROS, pp. 5033–5039 (2013)
Hannan, M.W., Walker, I.D.: Kinematics and the implementation of an elephant’s trunk manipulator and other continuum style robots. J. Robotic Syst. 20(2), 45–63 (2003)
Inoue, T., Hirai, S.: Modeling of soft fingertip for object manipulation using tactile sensing. In: Proceedings of IROS, pp. 2654–2659 (2003)
Kober, J., Peters, J.: Reinforcement learning in robotics: a survey. In: Reinforcement Learning, pp. 579–610 (2012)
Laschi, C., Cianchetti, M., Mazzolai, B., Margheri, L., Follador, M., Dario, P.: Soft robot arm inspired by the octopus. Adv. Robot. 26(7), 709–727 (2012)
Laschi, C., Mazzolai, B., Cianchetti, M.: Soft robotics: technologies and systems pushing the boundaries of robot abilities. Sci. Robot. (2016)
Lu, D., Zhou, Y., Wu, F., Zhang, Z., Chen, X.: Integrating answer set programming with semantic dictionaries for robot task planning. In: Proceedings of IJCAI (2017)
Luo, M., Pan, Y., Skorina, E.H., Tao, W., Chen, F., Ozel, S., Onal, C.D.: Slithering towards autonomy: a self-contained soft robotic snake platform with integrated curvature sensing. Bioinspir. Biomim. 10(5), 055001 (2015)
Mazzolai, B., Margheri, L., Cianchetti, M., Dario, P., Laschi, C.: Soft-robotic arm inspired by the octopus: II. From artificial requirements to innovative technological solutions. Bioinspir. Biomim. 7(2), 025005 (2012)
McMahan, W., Chitrakaran, V., Csencsits, M., Dawson, D., Walker, I.D., Jones, B.A., Pritts, M., Dienno, D., Grissom, M., Rahn, C.D.: Field trials and testing of the OctArm continuum manipulator. In: Proceedings of ICRA, pp. 2336–2341 (2006)
Pfeifer, R.: soft robotics - the next generation of intelligent machines. Invited talk on IJCAI (2013)
Ramchurn, S.D., Huynh, T.D., Wu, F., Ikuno, Y., Flann, J., Moreau, L., Fischer, J.E., Jiang, W., Rodden, T., Simpson, E., Reece, S., Roberts, S., Jennings, N.R.: A disaster response system based on human-agent collectives. J. Artif. Intell. Res. 57, 661–708 (2016)
Renda, F., Giorelli, M., Calisti, M., Cianchetti, M., Laschi, C.: Dynamic model of a multibending soft robot arm driven by cables. IEEE Trans. Robot. 30(5), 1109–1122 (2014)
Rus, D., Tolley, M.T.: Design, fabrication and control of soft robots. Nature 521(7553), 467–475 (2015)
Shibata, M., Hirai, S.: Soft object manipulation by simultaneous control of motion and deformation. In: Proceedings ICRA, pp. 2460–2465 (2006)
Sun, H., Chen, X.-P.: Towards honeycomb pneunets robots. In: Kim, J.-H., Matson, E.T., Myung, H., Xu, P., Karray, F. (eds.) Robot Intelligence Technology and Applications 2. AISC, vol. 274, pp. 331–340. Springer, Cham (2014). doi:10.1007/978-3-319-05582-4_28
Tolley, M.T., Shepherd, R.F., Mosadegh, B., Galloway, K.C., Wehner, M., Karpelson, M., Wood, R.J., Whitesides, G.M.: A resilient, untethered soft robot. Soft Robot. 1(3), 213–223 (2014)
Webster, R.J., Jones, B.: Design and kinematic modeling of constant curvature continuum robots: a review. Int. J. Robot. Res. 29(13), 1661–1683 (2010)
Wu, F., Chen, X.: Solving large-scale and sparse-reward DEC-POMDPs with correlation-MDPs. In: Proceedings of RoboCup, pp. 208–219 (2007)
Wu, F., Ramchurn, S., Chen, X.: Coordinating human-UAV teams in disaster response. In: Proceedings of IJCAI, pp. 524–530 (2016)
Acknowledgments
Feng Wu was supported in part by National Natural Science Foundation of China (No. 61603368), the Youth Innovation Promotion Association of CAS (No. 2015373), and Natural Science Foundation of Anhui Province (No. 1608085QF134).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Zhang, H., Cao, R., Zilberstein, S., Wu, F., Chen, X. (2017). Toward Effective Soft Robot Control via Reinforcement Learning. In: Huang, Y., Wu, H., Liu, H., Yin, Z. (eds) Intelligent Robotics and Applications. ICIRA 2017. Lecture Notes in Computer Science(), vol 10462. Springer, Cham. https://doi.org/10.1007/978-3-319-65289-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-65289-4_17
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-65288-7
Online ISBN: 978-3-319-65289-4
eBook Packages: Computer ScienceComputer Science (R0)