Reinforcement Learning to Run… Fast

Conference paper
Part of the The Springer Series on Challenges in Machine Learning book series (SSCML)


This paper describes the approach taken by the NNAISENSE Intelligent Automation team to win the NIPS ’17 “Learning to Run” challenge involving a biomechanically realistic model of the human lower musculoskeletal system.



The authors would like to thank Jürgen Schmidhuber, Jonathan Masci, Rupesh Srivastava, Christian Osendorfer, and Marco Gallieri for fruitful and inspiring discussions during the work on this competition.


  1. S. L. Delp, F. C. Anderson, A. S. Arnold, P. Loan, A. Habib, C. T. John, E. Guendelman, and D. G. Thelen. OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE Transactions on Biomedical Engineering, 54(11):1940–1950, 2007.CrossRefGoogle Scholar
  2. Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel. Benchmarking deep reinforcement learning for continuous control. In International Conference on Machine Learning, pages 1329–1338, 2016.Google Scholar
  3. Ł. Kidziński, S. P. Mohanty, C. Ong, J. Hicks, S. Francis, S. Levine, M. Salathé, and S. Delp. Learning to run challenge: Synthesizing physiologically accurate motion using deep reinforcement learning. In S. Escalera and M. Weimer, editors, NIPS 2017 Competition Book. Springer, Springer, 2018.Google Scholar
  4. J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015.Google Scholar
  5. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. CoRR, 2017.Google Scholar
  6. R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction, volume 1. MIT press Cambridge, 1998.Google Scholar
  7. D. G. Thelen et al. Adjustment of muscle mechanics model parameters to simulate dynamic contractions in older adults. Transactions-American Society Of Mechanical Engineers Journal Of Biomechanical Engineering, 125(1):70–77, 2003.Google Scholar
  8. E. Todorov, T. Erez, and Y. Tassa. MuJoCo: A physics engine for model-based control. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5026–5033. IEEE, 2012.Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.NNAISENSE SALuganoSwitzerland

Personalised recommendations