基于强化学习的随机振动主动控制策略

A active vibration control strategy based on reinforcement learning

针对被控系统的不确定性和非线性特征,提出了一种基于强化学习的随机振动主动控制策略。采用深度确定性策略梯度(DDPG)的强化学习算法设计振动控制器,该过程不涉及专家经验,完全由算法和数据自主交互学习完成。控制器是一个多层神经网络模型,这种由强化学习算法设计的控制器称为RL-NN控制器。通过两个数值仿真算例验证了RL-NN控制器的性能,其中考虑不确定性的单自由度系统主动控制效果达97%,考虑不确定性和非线性的车辆1/4悬架系统半主动控制效果达74%,结果表明RL-NN控制器对不确定性和非线性系统具有优异的振动控制能力。强化学习算法仅仅花费数小时设计的随机振动主动控制策略便优于专家经验数年来设计的控制策略,这为复杂系统振动主动/半主动控制器的设计提供了一种新的实现途径。

Concerning the uncertainty and nonlinearity of a controlled system,an active control strategy for random vibration based on reinforcement learning was proposed.The vibration controller was designed by a reinforcement learning algorithm-deep deterministic policy gradient(DDPG).This process does not involve expert experience and was entirely completed by the autonomous interactive learning of DDPG algorithms and data.The controller is a multi-layer neural network model,and this kind of controller designed by reinforcement learning algorithm is called neural network controller designed by reinforcement learning(RL-NN)controller.The performance of the RL-NN controller was verified through two numerical simulation examples:the active control effect of a single degree of freedom system with uncertainty reaches 97%.The semi-active control effect of the 1/4 vehicle suspension system with uncertainty and nonlinearity reaches 74%.The results show that the RL-NN controller has excellent vibration control capabilities for systems with uncertain and nonlinear.The random vibration active control strategy designed by the reinforcement learning algorithm in only a few hours is better than the control strategy designed by experts over few years.This provides a new approach to design active/semi-active controllers for complex systems.