Reinforcement Learning for Active Damping of Harmonically Excited Pendulum with Highly Nonlinear Actuator
Active vibration dampers can reduce or eliminate unwanted vibrations, but determining a good control policy can be challenging for highly nonlinear systems. For these types of systems, reinforcement learning is one method to optimize a control policy with only limited prior knowledge of the system dynamics. An experimental system was constructed by attaching a permanent magnet to the end of a pendulum and positioning an electromagnetic actuator below the resting position of the pendulum. The pendulum was excited with a sinusoidal force applied horizontally at the pivot point, and the control input was the applied voltage across the electromagnet. Due to the geometric arrangement and the strong dependence of magnetic force on distance, the relationship between the position of the pendulum and the actuation torque for any control input was highly nonlinear. A generalized version of the PILCO reinforcement learning algorithm was used to optimize a control policy for the electromagnet with the objective of minimizing the distance between the end of the pendulum and the downward position. After 16 s of interaction with the experimental system, the resulting learned policy was able to substantially reduce the amplitude of oscillation. This experiment illustrates the applicability of reinforcement learning to highly nonlinear active vibration damping problems.
KeywordsReinforcement learning Active damping Nonlinear dynamical system Nonlinear control Vibration
Funding was provided by Army Research Office (ARO) grant W911NF-17-0047 and the National Defense Science & Engineering Graduate (NDSEG) Fellowship.
- 1.Takács, G., Rohal’-Ilkiv, B.: Model Predictive Vibration Control: Efficient Constrained MPC Vibration Control for Lightly Damped Mechanical Structures. Springer, London (2012). https://doi.org/10.1007/978-1-4471-2333-0
- 2.Deisenroth, M.P., Rasmussen, C.E.: PILCO: a model-based and data-efficient approach to policy search. In: Proceedings of the International Conference on Machine Learning (2011)Google Scholar
- 3.Deisenroth, M.P., Rasmussen, C.E.: A practical and conceptual framework for learning in control. Technical report. UW-CSE-10-06-01. Department of Computer Science and Engineering, University of Washington, June 2010Google Scholar