Traditional joint-link robots have been widely used in production lines because of their high precision for single tasks. With the development of the manufacturing and service industries, the requirement for the comprehensive performance of robotics is growing. Numerous types of bio-inspired robotics have been investigated to realize human-like motion control and manipulation. A study route from inner mechanisms to external structures is proposed to imitate humans and animals better. With this idea, a brain-inspired intelligent robotic system is constructed that contains visual cognition, decision-making, motion control, and musculoskeletal structures. This paper reviews cutting-edge research in brain-inspired visual cognition, decision-making, motion control, and musculoskeletal systems. Two software systems and a corresponding hardware system are established, aiming at the verification and applications of next-generation brain-inspired musculoskeletal robots.
H. Qiao, P. J. Yin, R. Li, P. Wang. What is the meaning for the interdisciplinary research of robot and neuroscience? — Thoughts on the future development of intelligent robot. Bulletin of Chinese Academy of Sciences, vol. 30, no. 6, pp. 762–771, 2015. DOI: https://doi.org/10.16418/j.issn.1000-3045.2015.06.007. (in Chinese)
Y. Sakagami, R. Watanabe, C. Aoyama, S. Matsunaga, N. Higaki, K. Fujimura. The intelligent ASIMO: System overview and integration. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, Lausanne, Switzerland, pp. 2478–2483, 2002. DOI: https://doi.org/10.1109/IRDS.2002.1041641.
Q. Huang, K. Yokoi, S. Kajita, K. Kaneko, H. Arai, N. Koyachi, K. Tanie. Planning walking patterns for a biped robot. IEEE Transactions on Robotics and Automation, vol. 17, no. 3, pp. 280–289, 2001. DOI: https://doi.org/10.1109/70.938385.
Q. Huang, G. G. Dong, Z. G. Yu, X. G. Chen, Q. Q. Li, H. Z. Chen, H. X. Liu. Resistant compliance control for biped robot inspired by humanlike behavior. IEEE/ASME Transactions on Mechatronics, vol. 27, no. 5, pp. 3463–3473, 2022. DOI: https://doi.org/10.1109/TMECH.2021.3139332.
G. Metta, L. Natale, F. Nori, G. Sandini, D. Vernon, L. Fadiga, C. Von Hofsten, K. Rosander, M. Lopes, J. Santos-Victor, A. Bernardino, L. Montesano. The iCub humanoid robot: An open-systems platform for research in cognitive development. Neural Networks, vol. 23, no. 8–9, pp. 1125–1134, 2010. DOI: https://doi.org/10.1016/j.neunet.2010.08.010.
S. Kuindersma, R. Deits, M. Fallon, A. Valenzuela, H. K. Dai, F. Permenter, T. Koolen, P. Marion, R. Tedrake. Optimization-based locomotion planning, estimation, and control design for the atlas humanoid robot. Autonomous Robots, vol. 40, no. 3, pp. 429–455, 2016. DOI: https://doi.org/10.1007/s10514-015-9479-3.
M. Vukobratovic, A. A. Frank, D. Juricic. On the stability of biped locomotion. IEEE Transactions on Biomedical Engineering, vol. BME-17, no. 1, pp. 25–36, 1970. DOI: https://doi.org/10.1109/TBME.1970.4502681.
C. L. Fu, K. Chen. Gait synthesis and sensory control of stair climbing for a humanoid robot. IEEE Transactions on Industrial Electronics, vol. 55, no. 5, pp. 2111–2120, 2008. DOI: https://doi.org/10.1109/TIE.2008.921205.
M. Vukobratović, B. Borovac. Zero-moment point-thirty five years of its life. International Journal of Humanoid Robotics, vol. 1, no. 1, pp. 157–173, 2004. DOI: https://doi.org/10.1142/S0219843604000083.
J. Pratt, J. Carff, S. Drakunov, A. Goswami. Capture point: A step toward humanoid push recovery. In Proceedings of the 6th IEEE-RAS International Conference on Humanoid Robots, IEEE, Genova, Italy, pp. 200–207, 2006. DOI: https://doi.org/10.1109/ICHR.2006.321385.
T. Koolen, T. De Boer, J. Rebula, A. Goswami, J. Pratt. Capturability-based analysis and control of legged locomotion, Part 1: Theory and application to three simple gait models. International Journal of Robotics Research, vol. 31, no. 9, pp. 1094–1113, 2012. DOI: https://doi.org/10.1177/0278364912452673.
J. Pratt, T. Koolen, T. De Boer, J. Rebula, S. Cotton, J. Carff, M. Johnson, P. Neuhaus. Capturability-based analysis and control of legged locomotion, Part 2: Application to M2V2, a lower-body humanoid. International Journal of Robotics Research, vol. 31, no. 10, pp. 1117–1133, 2012. DOI: https://doi.org/10.1177/0278364912452762.
M. Thor, T. Kulvicius, P. Manoonpong. Generic neural locomotion control framework for legged robots. IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 9, pp. 4013–4025, 2021. DOI: https://doi.org/10.1109/TNNLS.2020.3016523.
A. Miguel-Blanco, P. Manoonpong. General distributed neural control and sensory adaptation for self-organized locomotion and fast adaptation to damage of walking robots. Frontiers in Neural Circuits, vol. 14, Article number 46, 2020. DOI: https://doi.org/10.3389/fncir.2020.00046.
S. Levine, C. Finn, T. Darrell, P. Abbeel. End-to-end training of deep visuomotor policies. Journal of Machine Learning Research, vol. 17, no. 1, pp. 1334–1373, 2016. DOI: https://doi.org/10.5555/2946645.2946684.
F. Zhang, Y. Demiris. Learning garment manipulation policies toward robot-assisted dressing. Science Robotics, vol. 7, no. 65, Article number eabm6010, 2022. DOI: https://doi.org/10.1126/scirobotics.abm6010.
J. Silvério, S. Calinon, L. Rozo, D. J. Caldwell. Learning task priorities from demonstrations. IEEE Transactions on Robotics, vol. 35, no. 1, pp. 78–94, 2019. DOI: https://doi.org/10.1109/TRO.2018.2878355.
B. Fang, S. D. Jia, D. Guo, M. H. Xu, S. H. Wen, F. C. Sun. Survey of imitation learning for robotic manipulation. International Journal of Intelligent Robotics and Applications, vol. 3, no. 4, pp. 362–369, 2019. DOI: https://doi.org/10.1007/s41315-019-00103-5.
H. Qiao, J. H. Chen, X. Huang. A survey of brain-inspired intelligent robots: Integration of vision, decision, motion control, and musculoskeletal systems. IEEE Transactions on Cybernetics, vol. 52, no. 10, pp. 11267–11280, 2022. DOI: https://doi.org/10.1109/TCYB.2021.3071312.
X. Y. Xi, P. J. Yin, H. Qiao, Y. L. Li, W. S. Feng. A biologically inspired model mimicking the memory and two distinct pathways of face perception. Neurocomputing, vol. 205, pp. 349–359, 2016. DOI: https://doi.org/10.1016/j.neucom.2016.04.032.
H. Qiao, Y. L. Li, T. Tang, P. Wang. Introducing memory and association mechanism into a biologically inspired visual model. IEEE Transactions on Cybernetics, vol. 44, no. 9, pp. 1485–1496, 2014. DOI: https://doi.org/10.1109/TCYB.2013.2287014.
H. Qiao, Y. L. Li, F. F. Li, X. Y. Xi, W. Wu. Biologically inspired model for visual cognition achieving unsupervised episodic and semantic feature learning. IEEE Transactions on Cybernetics, vol. 46, no. 10, pp. 2335–2347, 2016. DOI: https://doi.org/10.1109/TCYB.2015.2476706.
H. Qiao, X. Y. Xi, Y. L. Li, W. Wu, F. F. Li. Biologically inspired visual model with preliminary cognition and active attention adjustment. IEEE Transactions on Cybernetics, vol. 45, no. 11, pp. 2612–2624, 2015. DOI: https://doi.org/10.1109/TCYB.2014.2377196.
P. J. Yin, H. Qiao, W. Wu, L. Qi, Y. L. Li, S. L. Zhong, B. Zhang. A novel biologically inspired visual cognition model: Automatic extraction of semantics, formation of integrated concepts, and reselection features for ambiguity. IEEE Transactions on Cognitive and Developmental Systems, vol. 10, no. 2, pp. 420–431, 2018. DOI: https://doi.org/10.1109/TCDS.2017.2749978.
J. Pei, L. Deng, S. Song, M. G. Zhao, Y. H. Zhang, S. Wu, G. R. Wang, Z. Zou, Z. Z. Wu, W. He, F. Chen, N. Deng, S. Wu, Y. Wang, Y. J. Wu, Z. Y. Yang, C. Ma, G. Q. Li, W. T. Han, H. L. Li, H. Q. Wu, R. Zhao, Y. Xie, L. P. Shi. Towards artificial general intelligence with hybrid Tianjic chip architecture. Nature, vol. 572, no. 7767, pp. 106–111, 2019. DOI: https://doi.org/10.1038/s41586-019-1424-8.
C. Wang, S. J. Liang, C. Y. Wang, Z. Z. Yang, Y. M. Ge, C. Pan, X. Shen, W. Wei, Y. C. Zhao, Z. C. Zhang, B. Cheng, C. Zhang, F. Miao. Scalable massively parallel computing using continuous-time data representation in nanoscale crossbar array. Nature Nanotechnology, vol. 16, no. 10, pp. 1079–1085, 2021. DOI: https://doi.org/10.1038/S41565-021-00943-Y.
S. Wittmeier, C. Alessandro, N. Bascarevic, K. Dalamagkidis, D. Devereux, A. Diamond, M. Jäntsch, K. Jovanovic, R. Knight, H. G. Marques, P. Milosavljevic, B. Mitra, B. Svetozarevic, V. Potkonjak, R. Pfeifer, A. Knoll, O. Holland. Toward anthropomimetic robotics: Development, simulation, and control of a musculoskeletal torso. Artificial Life, vol. 19, no. 1, pp. 171–193, 2013. DOI: https://doi.org/10.1162/ARTL_a_00088.
Y. Asano, K. Okada, M. Inaba. Design principles of a human mimetic humanoid: Humanoid platform to study human intelligence and internal body system. Science Robotics, vol. 2, no. 13, Article number eaaq0899, 2017. DOI: https://doi.org/10.1126/scirobotics.aaq0899.
S. L. Zhong, J. H. Chen, X. Y. Niu, H. Fu, H. Qiao. Reducing redundancy of musculoskeletal robot with convex hull vertexes selection. IEEE Transactions on Cognitive and Developmental Systems, vol. 12, no. 3, pp. 601–617, 2020. DOI: https://doi.org/10.1109/TCDS.2019.2953642.
Z. J. Li, K. K. Zhao, L. B. Zhang, X. Y. Wu, T. Zhang, Q. J. Li, X. Li, C. Y. Su. Human-in-the-loop control of a wearable lower limb exoskeleton for stable dynamic walking. IEEE/ASME Transactions on Mechatronics, vol. 26, no. 5, pp. 2700–2711, 2021. DOI: https://doi.org/10.1109/TMECH.2020.3044289.
Z. J. Li, C. J. Deng, K. K. Zhao. Human-cooperative control of a wearable walking exoskeleton for enhancing climbing stair activities. IEEE Transactions on Industrial Electronics, vol. 67, no. 4, pp. 3086–3095, 2020. DOI: https://doi.org/10.1109/TIE.2019.2914573.
X. Y. Wu, Z. J. Li. Cooperative manipulation of wearable dual-arm exoskeletons using force communication between partners. IEEE Transactions on Industrial Electronics, vol. 67, no. 8, pp. 6629–6638, 2020. DOI: https://doi.org/10.1109/tie.2019.2937036.
Z. J. Li, C. C. Xu, Q. Wei, C. Shi, C. Y. Su. Human-inspired control of dual-arm exoskeleton robots with force and impedance adaptation. IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 50, no. 12, pp. 5296–5305, 2020. DOI: https://doi.org/10.1109/tsmc.2018.2871196.
G. X. Li, Z. J. Li, Z. Kan. Assimilation control of a robotic exoskeleton for physical human-robot interaction. IEEE Robotics and Automation Letters, vol. 7, no. 2, pp. 2977–2984, 2022. DOI: https://doi.org/10.1109/LRA.2022.3144537.
J. J. Li, Z. J. Li, F. Chen, A. Bicchi, Y. Sun, T. Fukuda. Combined sensing, cognition, learning, and control for developing future neuro-robotics systems: A survey. IEEE Transactions on Cognitive and Developmental Systems, vol. 11, no. 2, pp. 148–161, 2019. DOI: https://doi.org/10.1109/TCDS.2019.2897618.
M. Donk, W. Van Zoest. Effects of salience are short-lived. Psychological Science, vol. 19, no. 7, pp. 733–739, 2008. DOI: https://doi.org/10.1111/j.1467-9280.2008.02149.x.
L. Itti, C. Koch. Computational modelling of visual attention. Nature Reviews Neuroscience, vol. 2, no. 3, pp. 194–203, 2001. DOI: https://doi.org/10.1038/35058500.
D. M. Beck, S. Kastner. Stimulus context modulates competition in human extrastriate cortex. Nature Neuroscience, vol. 8, no. 8, pp. 1110–1116, 2005. DOI: https://doi.org/10.1038/nn1501.
T. Töllner, M. Zehetleitner, K. Gramann, H. J. Müller. Stimulus saliency modulates pre-attentive processing speed in human visual cortex. PLoS One, vol. 6, no. 1, Article number e16276, 2011. DOI: https://doi.org/10.1371/journal.pone.0016276.
T. Töllner, M. Zehetleitner, J. Krummenacher, H. J. Müller. Perceptual basis of redundancy gains in visual pop-out search. Journal of Cognitive Neuroscience, vol. 23, no. 1, pp. 137–150, 2011. DOI: https://doi.org/10.1162/jocn.2010.21422.
E. Rosch. Principles of categorization. Concepts: Core Readings, E. Margolis, S. Laurence, Eds., Cambridge, USA: MIT Press, pp. 189–206, 1999.
K. Grill-Spector, N. Kanwisher. Visual recognition: As soon as you know it is there, you know what it is. Psychological Science, vol. 16, no. 2, pp. 152–160, 2005. DOI: https://doi.org/10.1111/j.0956-7976.2005.00796.X.
K. Grill-Spector, N. Kanwisher. The functional organization of human ventral temporal cortex is based on stimulus selectivity, not recognition task. Soc Neurosci Abs, vol. 122, Article number 10, 2001.
C. Keysers, D. K. Xiao, P. Földiák, D. I. Perrett. The speed of sight. Journal of Cognitive Neuroscience, vol. 13, no. 1, pp. 90–101, 2001. DOI: https://doi.org/10.1162/089892901564199.
L. K. Tyler, S. Chiu, J. Zhuang, B. Randall, B. J. Devereux, P. Wright, A. Clarke, K. I. Taylor. Objects and categories: Feature statistics and object processing in the ventral stream. Journal of Cognitive Neuroscience, vol. 25, no. 10, pp. 1723–1735, 2013. DOI: https://doi.org/10.1162/jocn_a_00419.
M. Spiridon, B. Fischl, N. Kanwisher. Location and spatial profile of category-specific regions in human extrastriate cortex. Human Brain Mapping, vol. 27, no. 1, pp. 77–89, 2006. DOI: https://doi.org/10.1002/hbm.20169.
L. T. McIntosh, N. Maheswaranathan, A. Nayebi, S. Ganguli, S. A. Baccus. Deep learning models of the retinal response to natural scenes. In Proceedings of the 30th International Conference on Neural Information Processing Systems, ACM, Barcelona, Spain, pp. 1369–1377, 2016. DOI: https://doi.org/10.5555/3157096.3157249.
D. A. Klindt, A. S. Ecker, T. Euler, M. Bethge. Neural system identification for large populations separating what and where. In Proceedings of the 31st International Conference on Neural Information Processing Systems, ACM, Long Beach, USA, pp. 3506–3516, 2017. DOI: https://doi.org/10.5555/3294996.3295109.
M. Riesenhuber, T. Poggio. Hierarchical models of object recognition in cortex. Nature Neuroscience, vol. 2, no. 11, pp. 1019–1025, 1999. DOI: https://doi.org/10.1038/14819.
G. Azzopardi, A. Rodríguez-Sánchez, J. Piater, N. Petkov. A push-pull CORF model of a simple cell with antiphase inhibition improves SNR and contour detection. PLoS One, vol. 9, no. 7, Article number e98424, 2014. DOI: https://doi.org/10.1371/journal.pone.0098424.
T. Serre, L. Wolf, S. Bileschi, M. Riesenhuber, T. Poggio. Robust object recognition with cortex-like mechanisms. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 3, pp. 411–426, 2007. DOI: https://doi.org/10.1109/TPAMI.2007.56.
S. Dura-Bernal, T. Wennekers, S. L. Denham. Top-down feedback in an HMAX-like cortical model of object perception based on hierarchical Bayesian networks and belief propagation. PLoS One, vol. 7, no. 11, Article number e48216, 2012. DOI: https://doi.org/10.1371/journal.pone.0048216.
C. Liu, F. C. Sun. HMAX model: A survey. In Proceedings of the International Joint Conference on Neural Networks, IEEE, Killarney, Ireland, 2015. DOI: https://doi.org/10.1109/IJCNN.2015.7280677.
Y. L. Li, W. Wu, B. Zhang, F. F. Li. Enhanced HMAX model with feedforward feature learning for multiclass categorization. Frontiers in Computational Neuroscience, vol. 9, Article number 123, 2015. DOI: https://doi.org/10.3389/fncom.2015.00123.
E. T. Rolls. Invariant visual object and face recognition: Neural and computational bases, and a model, VisNet. Frontiers in Computational Neuroscience, vol. 6, Article number 35, 2012. DOI: https://doi.org/10.3389/fncom.2012.00035.
K. Benchenane, A. Peyrache, M. Khamassi, P. L. Tierney, Y. Gioanni, F. P. Battaglia, S. I. Wiener. Coherent theta oscillations and reorganization of spike timing in the hippocampal-prefrontal network upon learning. Neuron, vol. 66, no. 6, pp. 921–936, 2010. DOI: https://doi.org/10.1016/j.neuron.2010.05.013.
D. Kumaran, J. J. Summerfield, D. Hassabis, E. A. Maguire. Tracking the emergence of conceptual knowledge during human decision making. Neuron, vol. 63, no. 6, pp. 889–901, 2009. DOI: https://doi.org/10.1016/j.neuron.2009.07.030.
M. T. R. Van Kesteren, G. Fernández, D. G. Norris, E. J. Hermans. Persistent schema-dependent hippocampal-neocortical connectivity during memory encoding and postencoding rest in humans. Proceedings of the National Academy of Sciences of the United States of America, vol. 107, no. 16, pp. 7550–7555, 2010. DOI: https://doi.org/10.1073/pnas.0914892107.
A. Pasupathy, C. E. Connor. Shape representation in area v4: Position-specific tuning for boundary conformation. Journal of Neurophysiology, vol. 86, no. 5, pp. 2505–2519, 2001. DOI: https://doi.org/10.1152/jn.2001.86.5.2505.
A. E. Green, M. R. Munafò, C. G. DeYoung, J. A. Fossella, J. Fan, J. R. Gray. Using genetic data in cognitive neuroscience: From growing pains to genuine insights. Nature Reviews Neuroscience, vol. 9, no. 9, pp. 710–720, 2008. DOI: https://doi.org/10.1038/nrn2461.
A. Martin, L. L. Chao. Semantic memory and the brain: Structure and processes. Current Opinion in Neurobiology, vol. 11, no. 2, pp. 194–201, 2001. DOI: https://doi.org/10.1016/S0959-4388(00)00196-3.
M. W. Brown, J. P. Aggleton. Recognition memory: What are the roles of the perirhinal cortex and hippocampus? Nature Reviews Neuroscience, vol. 2, no. 1, pp. 51–61, 2001. DOI: https://doi.org/10.1038/35049064.
L. R. Squire, J. T. Wixted, R. E. Clark. Recognition memory and the medial temporal lobe: A new perspective. Nature Reviews Neuroscience, vol. 8, no. 11, pp. 872–883, 2007. DOI: https://doi.org/10.1038/nrn2154.
M. Scheutz. Useful roles of emotions in artificial agents: A case study from artificial life. In Proceedings of the 19th National Conference on Artificial Intelligence, ACM, San Jose, USA, pp. 42–47, 2004.
T. M. Moerland, J. Broekens, C. M. Jonker. Emotion in reinforcement learning agents and robots: A survey. Machine Learning, vol. 107, no. 2, pp. 443–480, 2018. DOI: https://doi.org/10.1007/s10994-017-5666-0.
N. Goerke. EMOBOT: A robot control architecture based on emotion-like internal values. Mobile Robotics, Moving Intelligence, J. Buchli, Ed., Mammendorf, Germany: Advanced Robotic Systems International, 2006. DOI: https://doi.org/10.5772/4715.
Á. Castro-González, M. Malfaz, M. Á. Salichs. An autonomous social robot in fear. IEEE Transactions on Autonomous Mental Development, vol. 5, no. 2, pp. 135–151, 2013. DOI: https://doi.org/10.1109/TAMD.2012.2234120.
J. Broekens, W. A. Kosters, F. J. Verbeek. Affect, anticipation, and adaptation: Affect-controlled selection of anticipatory simulation in artificial adaptive agents. Adaptive Behavior, vol. 15, no. 4, pp. 397–422, 2007. DOI: https://doi.org/10.1177/1059712307084686.
J. G. Taylor, N. F. Fragopanagos. The interaction of attention and emotion. Neural Networks, vol. 18, no. 4, pp. 353–369, 2005. DOI: https://doi.org/10.1016/j.neunet.2005.03.005.
C. Balkenius, J. Morén. Emotional learning: A computational model of the amygdala. Cybernetics and Systems, vol. 32, no. 6, pp. 611–636, 2001. DOI: https://doi.org/10.1080/01969720118947.
M. A. Sharbafi, C. Lucas, R. Daneshvar. Motion control of omni-directional three-wheel robots by brain-emotional-learning-based intelligent controller. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), vol. 40, no. 6, pp. 630–638, 2010. DOI: https://doi.org/10.1109/TSMCC.2010.2049104.
X. Huang, W. Wu, H. Qiao, Y. D. Ji. Brain-inspired motion learning in recurrent neural network with emotion modulation. IEEE Transactions on Cognitive and Developmental Systems, vol. 10, no. 4, pp. 1153–1164, 2018. DOI: https://doi.org/10.1109/TCDS.2018.2843563.
X. Huang, W. Wu, H. Qiao. Connecting model-based and model-free control with emotion modulation in learning systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 8, pp. 4624–4638, 2021. DOI: https://doi.org/10.1109/TSMC.2019.2933152.
X. Huang, W. Wu, H. Qiao. Computational modeling of emotion-motivated decisions for continuous control of mobile robots. IEEE Transactions on Cognitive and Developmental Systems, vol. 13, no. 1, pp. 31–44, 2021. DOI: https://doi.org/10.1109/TCDS.2019.2963545.
S. Boonphoapichart, S. Komada, T. Hori, W. A. Gruver. Robot motion decision-making system in unknown environments. In Proceedings of IEEE International Conference on Robotics and Automation, Taipei, China, pp. 4197–4202, 2003. DOI: https://doi.org/10.1109/ROBOT.2003.1242248.
S. H. Tseng, J. H. Hua, S. P. Ma, L. E. Fu. Human awareness based robot performance learning in a social environment. In Proceedings of IEEE International Conference on Robotics and Automation, Karlsruhe, Germany, pp. 4291–4296, 2013. DOI: https://doi.org/10.1109/ICRA.2013.6631184.
J. T. Ebert, M. Gauci, F. Mallmann-Trenn, R. Nagpal. Bayes bots: Collective Bayesian decision-making in decentralized robot swarms. In Proceedings of IEEE International Conference on Robotics and Automation, Paris, France, pp. 7186–7192, 2020. DOI: https://doi.org/10.1109/ICRA40945.2020.9196584.
M. Riedmiller, T. Gabel, R. Hafner, S. Lange. Reinforcement learning for robot soccer. Autonomous Robots, vol. 27, no. 1, pp. 55–73, 2009. DOI: https://doi.org/10.1007/s10514-009-9120-4.
K. Doya. Metalearning and neuromodulation. Neural Networks, vol. 151, no. 4–6, pp. 495–506, 2002. DOI: https://doi.org/10.1016/S0893-6080(02)00044-8.
D. G. Thelen, F. C. Anderson, S. L. Delp. Generating dynamic simulations of movement using computed muscle control. Journal of Biomechanics, vol. 36, no. 3, pp. 321–328, 2003. DOI: https://doi.org/10.1016/S0021-9290(02)00432-3.
K. M. Jagodnik, A. J. Van Den Bogert. Optimization and evaluation of a proportional derivative controller for planar arm movement. Journal of Biomechanics, vol. 43, no. 6, pp. 1086–1091, 2010. DOI: https://doi.org/10.1016/j.jbiomech.2009.12.017.
K. Tahara, Y. Kuboyama, R. Kurazume. Iterative learning control for a musculoskeletal arm: Utilizing multiple space variables to improve the robustness. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal, pp. 4620–4625, 2012. DOI: https://doi.org/10.1109/IROS.2012.6385628.
H. Dong, N. Figueroa, A. El Saddik. Muscle force control of a kinematically redundant bionic arm with real-time parameter update. In Proceedings of IEEE International Conference on Systems, Man, and Cybernetics, Manchester, UK, pp. 1640–1647, 2013. DOI: https://doi.org/10.1109/SMC.2013.283.
K. Tahara, H. Kino. Iterative learning scheme for a redundant musculoskeletal arm: Task space learning with joint and muscle redundancies. In Proceedings of International Conference on Broadband, Wireless Computing, Communication and Applications, IEEE, Fukuoka, Japan, pp. 760–765, 2010. DOI: https://doi.org/10.1109/BWCCA.2010.168.
M. Jäntsch, C. Schmaler, S. Wittmeier, K. Dalamagkidis, A. Knoll. A scalable joint-space controller for musculoskeletal robots with spherical joints. In Proceedings of IEEE International Conference on Robotics and Biomimetics, Karon Beach, Thailand, pp. 2211–2216, 2011. DOI: https://doi.org/10.1109/ROBIO.2011.6181620.
K. Kawaharazuka, M. Kawamura, S. Makino, Y. Asano, K. Okada, M. Inaba. Antagonist inhibition control in redundant tendon-driven structures based on human reciprocal innervation for wide range limb motion of musculoskeletal humanoids. IEEE Robotics and Automation Letters, vol. 2, no. 4, pp. 2119–2126, 2017. DOI: https://doi.org/10.1109/LRA.2017.2720854.
M. Nakada, T. Zhou, H. L. Chen, T. Weiss, D. Terzopoulos. Deep learning of biomimetic sensorimotor control for biomechanical human animation. ACM Transactions on Graphics, vol. 37, no. 4, Article number 56, 2018. DOI: https://doi.org/10.1145/3197517.3201305.
L. Rane, Z. Y. Ding, A. H. McGregor, A. M. J. Bull. Deep learning for musculoskeletal force prediction. Annals of Biomedical Engineering, vol. 47, no. 3, pp. 778–789, 2019. DOI: https://doi.org/10.1007/s10439-018-02190-0.
Ł. Kidziński, S. P. Mohanty, C. F. Ong, Z. W. Huang, S. C. Zhou, A. Pechenko, A. Stelmaszczyk, P. Jarosik, M. Pavlov, S. Kolesnikov, S. Plis, Z. B. Chen, Z. Z. Zhang, J. L. Chen, J. Shi, Z. B. Zheng, C. Yuan, Z. H. Lin, H. Michalewski, P. Milos, B. Osinski, A. Melnik, M. Schilling, H. Ritter, S. F. Carroll, J. Hicks, S. Levine, M. Salathé, S. Delp. Learning to run challenge solutions: Adapting reinforcement learning methods for neuromusculoskeletal environments. In Proceedings of NIPS’17 Competition: Building Intelligent Systems, Springer, Cham, Germany, pp. 121–153, 2018. DOI: https://doi.org/10.1007/978-3-319-94042-7_7.
V. Caggiano, H. W. Wang, G. Durandau, M. Sartori, V. Kumar. Myosuite: A contact-rich simulation suite for musculoskeletal motor control. In Proceedings of Learning for Dynamics and Control Conference, Stanford, USA, pp. 492–507, 2022.
E. Tahami, A. H. Jafari, A. Fallah. Learning to control the three-link musculoskeletal arm using actor-critic reinforcement learning algorithm during reaching movement. Biomedical Engineering: Applications, Basis and Communications, vol. 26, no. 5, Article number 1450064, 2014. DOI: https://doi.org/10.4015/S1016237214500641.
J. Izawa, T. Kondo, K. Ito. Biological robot arm motion through reinforcement learning. In Proceedings of IEEE International Conference on Robotics and Automation, Washington DC, USA, vol. 4, pp. 3398–3403, 2002. DOI: https://doi.org/10.1109/ROBOT.2002.1014236.
D. C. Crowder, J. Abreu, R. F. Kirsch. Hindsight experience replay improves reinforcement learning for control of a MIMO musculoskeletal model of the human arm. IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 29, pp. 1016–1025, 2021. DOI: https://doi.org/10.1109/TNSRE.2021.3081056.
H. R. Xu, X. Ma, L. Y. Xu, Q. Wang. Training musculoskeletal arm play taichi with deep reinforcement learning. In Proceedings of IEEE International Instrumentation and Measurement Technology Conference, Dubrovnik, Croatia, 2020, pp. 1–6. DOI: https://doi.org/10.1109/I2MTC43012.2020.9129264.
J. H. Chen, H. Qiao. Muscle-synergies-based neuromuscular control for motion learning and generalization of a musculoskeletal system. IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 6, pp. 3993–4006, 2021. DOI: https://doi.org/10.1109/TSMC.2020.2966818.
J. H. Chen, S. L. Zhong, E. L. Kang, H. Qiao. Realizing human-like manipulation with a musculoskeletal system and biologically inspired control scheme. Neurocomputing, vol. 339, pp. 116–129, 2019. DOI: https://doi.org/10.1016/j.neucom.2018.12.069.
J. H. Chen, H. Qiao. Motor-cortex-like recurrent neural network and multitask learning for the control of musculoskeletal systems. IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 2, pp. 424–436, 2022. DOI: https://doi.org/10.1109/TCDS.2020.3045574.
X. N. Wang, J. H. Chen, H. Qiao. Motion learning and rapid generalization for musculoskeletal systems based on recurrent neural network modulated by initial states. IEEE Transactions on Cognitive and Developmental Systems, vol. 14, no. 4, pp. 1691–1704, 2022. DOI: https://doi.org/10.1109/TCDS.2021.3136854.
J. J. Zhou, J. H. Chen, H. Deng, H. Qiao. From rough to precise: Human-inspired phased target learning framework for redundant musculoskeletal systems. Frontiers in Neurorobotics, vol. 13, Article number 61, 2019. DOI: https://doi.org/10.3389/fnbot.2019.00061.
J. J. Zhou, S. L. Zhong, W. Wu. Hierarchical motion learning for goal-oriented movements with speed-accuracy tradeoff of a musculoskeletal system. IEEE Transactions on Cybernetics, vol. 52, no. 11, pp. 11453–11466, 2022. DOI: https://doi.org/10.1109/TCYB.2021.3109021.
J. H. Zhang, J. H. Chen, W. Wu, H. Qiao. A cerebellum-inspired prediction and correction model for motion control of a musculoskeletal robot. IEEE Transactions on Cognitive and Developmental Systems, to be published. DOI: https://doi.org/10.1109/TCDS.2022.3200839.
H. Qiao, C. Li, P. J. Yin, W. Wu, Z. Y. Liu. Human-inspired motion model of upper-limb with fast response and learning ability-a promising direction for robot system and control. Assembly Automation, vol. 36, no. 1, pp. 97–107, 2016. DOI: https://doi.org/10.1108/AA-11-2015-099.
A. A. Russo, S. R. Bittner, S. M. Perkins, J. S. Seely, B. M. London, A. H. Lara, A. Miri, N. J. Marshall, A. Kohn, T. M. Jessell, L. F. Abbott, J. P. Cunningham, M. M. Churchland. Motor cortex embeds muscle-like commands in an untangled population response. Neuron, vol. 97, no. 4, pp. 953–966.e8, 2018. DOI: https://doi.org/10.1016/j.neuron.2018.01.004.
D. Sussillo, M. M. Churchland, M. T. Kaufman, K. V. Shenoy. A neural network that finds a naturalistic solution for the production of muscle activity. Nature Neuroscience, vol. 18, no. 7, pp. 1025–1033, 2015. DOI: https://doi.org/10.1038/nn.4042.
T. C. Kao, M. S. Sadabadi, G. Hennequin. Optimal anticipatory control as a theory of motor preparation: A thalamo-cortical circuit model. Neuron, vol. 109, no. 9, pp. 1567–1581.e12, 2021. DOI: https://doi.org/10.1016/j.neuron.2021.03.009.
A. D’Avella, P. Saltiel, E. Bizzi. Combinations of muscle synergies in the construction of a natural motor behavior. Nature Neuroscience, vol. 6, no. 3, pp. 300–308, 2003. DOI: https://doi.org/10.1038/nn1010.
H. Hatze. A myocybernetic control model of skeletal muscle. Biological Cybernetics, vol. 25, no. 2, pp. 103–119, 1977. DOI: https://doi.org/10.1007/BF00337268.
A. J. Van Soest, M. F. Bobbert. The contribution of muscle properties in the control of explosive movements. Biological Cybernetics, vol. 69, no. 3, pp. 195–204, 1993. DOI: https://doi.org/10.1007/BF00198959.
D. G. Thelen. Adjustment of muscle mechanics model parameters to simulate dynamic contractions in older adults. Journal of Biomechanical Engineering, vol. 125, no. 1, pp. 70–77, 2003. DOI: https://doi.org/10.1115/1.1531112.
A. V. Hill. The heat of shortening and the dynamic constants of muscle. Proceedings of the Royal Society B: Biological Sciences, vol. 126, no. 843, pp. 136–195, 1938. DOI: https://doi.org/10.1098/rspb.1938.0050.
A. V. Hill. The heat of activation and the heat of shortening in a muscle twitch. Proceedings of the Royal Society B: Biological Sciences, vol. 136, no. 883, pp. 195–211, 1949. DOI: https://doi.org/10.1098/rspb.1949.0019.
B. C. Abbott, D. R. Wilkie. The relation between velocity of shortening and the tension-length curve of skeletal muscle. Journal of Physiology, vol. 120, no. 1–2, pp. 214–223, 1953. DOI: https://doi.org/10.1113/jphysiol.1953.sp004886.
F. X. Zajac. Muscle and tendon: Properties, models, scaling, and application to biomechanics and motor control. Critical Reviews in Biomedical Engineering, vol. 17, no. 4, pp. 359–411, 1989.
M. Millard, T. Uchida, A. Seth, S. L. Delp. Flexing computational muscle: Modeling and simulation of musculotendon dynamics. Journal of Biomechanical Engineering, vol. 135, no. 2, Article number 021005, 2013. DOI: https://doi.org/10.1115/1.4023390.
F. Romero, F. J. Alonso. A comparison among different Hill-type contraction dynamics formulations for muscle force estimation. Mechanical Sciences, vol. 7, no. 1, pp. 19–29, 2016. DOI: https://doi.org/10.5194/ms-7-19-2016.
K. Hosoda, Y. Sakaguchi, H. Takayama, T. Takuma. Pneumatic-driven jumping robot with anthropomorphic muscular skeleton structure. Autonomous Robots, vol. 28, no. 3, pp. 307–316, 2010. DOI: https://doi.org/10.1007/s10514-009-9171-6.
E. Acome, S. K. Mitchell, T. G. Morrissey, M. B. Emmett, C. Benjamin, M. King, M. Radakovitz, C. Keplinger. Hydraulically amplified self-healing electrostatic actuators with muscle-like performance. Science, vol. 359, no. 6371, pp. 61–65, 2018. DOI: https://doi.org/10.1126/science.aao6139.
N. Kellaris, V. Gopaluni Venkata, G. M. Smith, S. K. Mitchell, C. Keplinger. Peano-HASEL actuators: Muscle-mimetic, electrohydraulic transducers that linearly contract on activation. Science Robotics, vol. 3, no. 14, Article number eaar3276, 2018. DOI: https://doi.org/10.1126/scirobotics.aar3276.
D. R. Higueras-Ruiz, M. W. Shafer, H. P. Feigenbaum. Cavatappi artificial muscles from drawing, twisting, and coiling polymer tubes. Science Robotics, vol. 6, no. 53, Article number eabd5383, 2021. DOI: https://doi.org/10.1126/scirobotics.abd5383.
D. F. B. Haeufle, M. Günther, A. Bayer, S. Schmitt. Hill-type muscle model with serial damping and eccentric force-velocity relation. Journal of Biomechanics, vol. 47, no. 6, pp. 1531–1536, 2014. DOI: https://doi.org/10.1016/j.jbiomech.2014.02.009.
Y. X. Wu, J. H. Chen, H. Qiao. Anti-interference analysis of bio-inspired musculoskeletal robotic system. Neurocomputing, vol. 436, pp. 114–125, 2021. DOI: https://doi.org/10.1016/j.neucom.2021.01.054.
B. Liggett, N. P. Psuty, E. Goray. Kinesiology of the Musculoskeletal System, St. Louis, USA: Mosby/Elsevier, 2010.
M. S. Gazzaniga, R. B. Ivry, G. R. Mangun. Cognitive Neuroscience: The Biology of the Mind, 2nd ed., St. Louis, USA: Mosby/Elsevier, 2009.
S. F. Giszter, F. A. Mussa-Ivaldi, E. Bizzi. Convergent force fields organized in the frog’s spinal cord. Journal of Neuroscience, vol. 13, no. 2, pp. 467–491, 1993. DOI: https://doi.org/10.1523/JNEUROSCI.13-02-00467.1993.
E. Bizzi, N. Accornero, W. Chappie, N. Hogan. Arm trajectory formation in monkeys. Experimental Brain Research, vol. 46, no. 1, pp. 139–143, 1982. DOI: https://doi.org/10.1007/BF00238107.
W. J. Kargo, S. F. Giszter. Rapid correction of aimed movements by summation of force-field primitives. Journal of Neuroscience, vol. 20, no. 1, pp. 409–426, 2000. DOI: https://doi.org/10.1523/JNEUROSCI.20-01-00409.2000.
C. B. Hart, S. F. Giszter. A neural basis for motor primitives in the spinal cord. Journal of Neuroscience, vol. 30, no. 4, pp. 1322–1336, 2010. DOI: https://doi.org/10.1523/JNEUROSCL5894-08.2010.
S. L. Zhong, Z. Y. Chen, J. J. Zhou. Structure transforming for constructing constraint force field in musculoskeletal robot. Assembly Automation, vol. 42, no. 2, pp. 169–180, 2022. DOI: https://doi.org/10.1108/AA-07-2021-0093.
S. L. Zhong, J. J. Zhou, W. Wu. Constructing constraint force field in musculoskeletal robot by co-optimizing muscle arrangements and constant activations. In Proceedings of International Joint Conference on Neural Networks, IEEE, Padua, Italy, 2022. DOI: https://doi.org/10.1109/IJCNN55064.2022.9892969.
H. Qiao, M. Wang, J. H. Su, S. X. Jia, R. Li. The concept of “attractive region in environment” and its application in high-precision tasks with low-precision systems. IEEE/ASME Transactions on Mechatronics, vol. 20, no. 5, pp. 2311–2327, 2015. DOI: https://doi.org/10.1109/TMECH.2014.2375638.
S. L. Delp, F. C. Anderson, A. S. Arnold, P. Loan, A. Habib, C. T. John, E. Guendelman, D. G. Thelen. OpenSim: Open-source software to create and analyze dynamic simulations of movement. IEEE Transactions on Biomedical Engineering, vol. 54, no. 11, pp. 1940–1950, 2007. DOI: https://doi.org/10.1109/TBME.2007.901024.
F. Lacquaniti, J. F. Soechting. Coordination of arm and wrist motion during a reaching task. Journal of Neuroscience, vol. 2, no. 4, pp. 399–408, 1982. DOI: https://doi.org/10.1523/JNEUROSCI.02-04-00399.1982.
P. Ramkumar, B. Dekleva, S. Cooler, L. Miller, K. Kording. Premotor and motor cortices encode reward. PLoS One, vol. 11, no. 8, Article number e0160851, 2016. DOI: https://doi.org/10.1371/journal.pone.0160851.
This work was supported by National Natural Science Foundation of China (Nos. 91948303, 62203443 and 62203439), the Major Project of Science and Technology Innovation 2030 C Brain Science and Brain-inspired Intelligence (No. 2021ZD0200408), the Strategic Priority Research Program of Chinese Academy of Science (No. XDB 32050100), the Science Foundation for Youth of the State Key Laboratory of Management and Control for Complex System (No. 2022QN09).
Qiao Hong received the B. Eng. degree in hydraulics and control and the M. Eng. degree in robotics from Xi’an Jiaotong University, China in 1986 and 1989, respectively, the M. Phil. degree in robotics control from the Industrial Control Center, University of Strathclyde, UK in 1992, and the Ph.D. degree in robotics and artificial intelligence from De Montfort University, UK in 1995. She was a university research fellow with De Montfort University from 1995 to 1997. She was a research assistant professor with Department of Manufacturing Engineering and Engineering Management, City University of Hong Kong, China, from 1997 to 2000, where she was an assistant professor from 2000 to 2002. Since 2002, she has been a lecturer with School of Informatics, University of Manchester, UK. She is currently a professor with State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, China. She first proposed the concept of the attractive region in strategy investigation, which has successfully been applied by herself in robot assembly, robot grasping, and part recognition. She has authored the book entitled Advanced Manufacturing Alert (Wiley, 1999).
Prof. Qiao is currently an Associate Editor of the IEEE Transactions on Cybernetics and the IEEE Transactions on Automation Science and Engineering. She is the Editor-in-Chief of the Assembly Automation. She is currently a Member of the Administrative Committee of the IEEE Robotics and Automation Society, the IEEE Medal for Environmental and Safety Technologies Committee, the Early Career Award Nomination Committee, the Most Active Technical Committee Award Nomination Committee, and the Industrial Activities Board for RAS.
Her research interests include information-based strategy investigation, robotics and intelligent agents, animation, machine learning, and pattern recognition.
Ya-Xiong Wu received the B.Eng. degree in mechanical engineering from University of Science and Technology Beijing, China in 2019. He is currently a Ph. D. degree candidate in mechanical engineering at University of Science and Technology Beijing, China.
His research interests include the robustness analysis and controller design of bio-inspired musculoskeletal robotic systems.
Shan-Lin Zhong received the B. Eng. degree in control theory and control engineering from North China Electric Power University, China in 2016, and the Ph.D. degree in control theory and control engineering from Institute of Automation, Chinese Academy of Sciences, China in 2022. He is currently an assistant professor with State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, China.
His research interests include brain-like intelligent robot, robotic manipulation, and machine learning.
Pei-Jie Yin received the B.Sc. degree in statistics from University of Science and Technology of China, China in 2013, and the Ph.D. degree in applied mathematics from Institute of Applied Mathematics, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, China in 2018. He is now an assistant professor with Institute of Automation, Chinese Academy of Sciences, China.
His research interests include biologically inspired visual algorithms, dynamic environment understanding and humanoid motion learning.
Jia-Hao Chen received the B.Eng. degree in control theory and control engineering from China Agricultural University, China in 2016, and the Ph.D. degree in control theory and control engineering from Institute of Automation, Chinese Academy of Sciences, China in 2021. He also serves as an editorial assistant of Assembly Automation and review editor of Frontiers in Neurorobotics and Frontiers in Neuroscience.
His research interests include musculoskeletal robots, brain-inspired motion learning, reinforcement learning, and multi-task continual learning.
About this article
Cite this article
Qiao, H., Wu, YX., Zhong, SL. et al. Brain-inspired Intelligent Robotics: Theoretical Analysis and Systematic Application. Mach. Intell. Res. 20, 1–18 (2023). https://doi.org/10.1007/s11633-022-1390-8
- Brain-inspired intelligent robot
- software and hardware
- decision making
- muscle control
- cognitive intelligence