结构拓扑优化的Q学习-元胞方法研究

Study on structural topology optimization of Q-learning cell method

传统的拓扑优化算法均基于灵敏度分析的方式求解,如渐进结构优化法(Evolutionary Structural Optimization, ESO)和变密度法(Solid Isotropic Material with Penalization, SIMP)等,灵敏度分析依赖于严谨的数学模型,结果可信度高,但面对不同的结构和约束条件都需要反复重新推导单元灵敏度,对使用人员的数学能力有较高要求,而且也导致了收敛速度慢、迭代步数多的问题。针对现有优化方法中存在的缺陷,结合强化学习Q学习理论和元胞自动机原理,提出一种新的拓扑优化方法:Q学习-元胞法(Q-learning-Cellular Automaton, QCA),尝试为工程构件的优化设计提供一种新思路。这种方法以有限元单元作为元胞,将所有元胞的智能行为集成为一个Q-learning智能体。训练过程中,各个元胞首先完成对自身环境的感知,然后调用智能体进行决策并通过环境交互得到反馈,智能体也借此得到大量数据来学习更新,整个过程不涉及数学模型推导,通过智能体和元胞的不断探索即可完成优化。在此基础上,探讨元胞的选择及其邻域和状态的描述方式,针对元胞的动作空间及收益函数进行比选,进而编制相关拓扑优化软件。优化算例表明,QCA方法优化后的拓扑构型与传统优化方法的构型基本一致,迭代步数较SIMP法降低了64%,且柔顺度更低。Q学习-元胞法在结构拓扑优化中具备良好的可行性,计算效率高且具有迁移优化能力,在结构拓扑优化领域极具潜力。

Conventional topology optimization algorithms are based on the solution of sensitivity analysis,such as evolutionary structural optimization(ESO)and solid isotropic material with penalization(SIMP).Sensitivity analysis relies on rigorous mathematical models and the results are highly credible.However,it requires repeated re-derivation of unit sensitivity for diverse structures and constraints,which requires high level of mathematical proficiency of the user.This drawback also leads to slow convergence and high number of iteration steps.In response to the limitations in existing optimization methods,a new topology optimization approach by merging Q-learning theory with Cellular Automata principles:Q-learning-Cellular Automation(QCA)was proposed.This approach attempted to provide a new idea for the optimal design of engineering components.Finite elements were treated as cells,and the intelligent behaviors of all cells were integrated into a Q-learning Agent.Throughout the learning process,each cell initially completed the perception of its own environment,and subsequently called on the Agent to complete the decision-making.In turn,the Agent was learnt and updated based on feedback from the environment.The entire process circumvented the need for mathematical model derivations,allowing optimization through continuous exploration by the Agent and cells.On this basis,the selection of cells and the description of their neighborhoods and states were discussed.A comparison was made on the action space and benefit function of cells,and relevant topology optimization software was developed.The optimization example illustrated that the topological configuration of the QCA method is basically similar to that of the traditional optimization method.Notably,its iterative steps were 64%lower than SIMP with a lower degree of compliance.In summary,the QCA method has good feasibility in structural topological optimization.With high computational efficiency and adept migration optimization capability,it holds great potential in the field of structural topological optimization.