基于强化学习的阈值电压分配漏功耗优化方法研究

Research on reinforcement learning based leakage power optimization framework(RL-LPO)

随着先进工艺的演进,泄漏功耗在集成电路总功耗中的占比不断增大,已逐渐成为制约电路功耗降低的重要因素之一。在已有的漏功耗优化方法中,基于阈值电压分配的方法具有指数关系的功耗优化效果,并且对已进行的布局布线不产生影响,因而被广泛采用。然而,在商用签核工具中,为了保持伪线性复杂度而限制了底层算法所做的全局搜索,使得难以获得最优结果。本文提出一种基于图神经网络和强化学习的联合优化框架RL LPO,实现高效的门单元阈值电压分配。在RL LPO中,采用图神经网络GraphSAGE编码电路的时序和物理信息对目标单元及其局部邻域信息进行聚合;采用深度确定性策略梯度(Deep Deterministic Policy Gradient,DDPG)强化学习算法,在奖励函数的指导下,考虑漏功耗和时序变化进行阈值电压的分配。本文提出的门单元阈值电压分配框架RL LPO在28 nm工艺下由IWLS2005和Opencores基准电路进行验证,与商用签核工具相比,在不增加时序违例的前提下,RL LPO降低了至少2.1%的额外漏功耗,并实现了至少4.2倍的加速。

With the evolution of advanced technology,the proportion of leakage power consumption in the total power consumption of integrated circuits continues to increase,which has gradually become one of the important factors restricting the reduction of circuit power consumption.Among the existing leakage power optimization methods,the method based on threshold voltage allocation has an exponential power optimization effect and has no influence on the layout and routing,so it is widely adopted.However,in commercial signoff tools,in order to maintain pseudolinear complexity,the global search made by the underlying algorithm is limited,which makes it difficult to obtain optimal results.In this paper,a joint optimization framework RL LPO based on graph neural network and reinforcement learning is proposed to achieve efficient gate unit threshold voltage distribution.In RL LPO,the timing and physical information of the graph neural network GraphSAGE encoding circuit are used to aggregate the target unit and its local neighborhood information.Using the Deep Deterministic Policy Gradient(DDPG)reinforcement learning algorithm,the threshold voltage allocation is carried out considering the leakage power consumption and timing variation under the guidance of the reward function.The gate unit threshold voltage distribution framework RL LPO proposed in this paper is verified by IWLS2005 and Opencores reference circuits under the 28 nm process,and compared with commercial signoff tools,RL LPO reduces the additional leakage power consumption by at least 2.1%and achieves at least 4.2 times acceleration without adding timing violations.