基于强化学习的非线性主动悬架系统的最优控制

Optimal Control of Nonlinear Active Suspension System Based on Reinforcement Learning

为了改善车辆性能和乘客舒适性,针对带有非线性阻尼的1/4汽车主动悬架系统,提出了一种基于在线迭代算法的最优控制策略。首先根据所建立的非线性模型给出了合理的代价函数,并利用最优控制理论设计了初始最优控制策略。然后为了处理求解HJB方程的困难,借助于强化学习的Actor-Critic框架构造了一种新颖的策略在线学习HJB方程的近似解,同时为提高系统的鲁棒性,在Actor-Critic更新率中增加了泄漏项。最后通过稳定性分析表明所提策略可通过调节参数使状态收敛到零的充分小邻域,同时使代价函数达到最优。

In order to improve vehicle performance and passenger comfort,an optimal control strategy is proposed based on online policy iteration algorithm for 1/4 vehicle active suspension system with nonlinear damping.Firstly,the reasonable cost function is given according to the established nonlinear model,and the initial optimal control strategy is designed by using the optimal control theory.Then,in order to deal with the difficulty of solving the HJB equation,a novel strategy is constructed to learn the approximate solution of the HJB equation online with the help of the Actor-Critic framework of reinforcement learning,and a leakage term is added to the update rate of the Actor-Critic to improve the robustness of the system.Finally,by using the stability analysis,it is demonstrated that the proposed strategy can make the state converge to a sufficiently small neighborhood of zero by adjusting the parameters,and the cost function can be optimized at the same time.