基于深度强化学习的5G基站储能调度策略

Dispatching Strategy of Energy Storage for 5G Base Stations Based on Deep Reinforcement Learning

为提高5G基站闲置储能的利用率,设计了一种5G基站储能参与电网调度的多基站储能系统,采用参与电网需求响应和低储高放的协同调度机制,建立了考虑过充过放惩罚和调度成本等多因素的多基站储能系统经济模型。以经济效益最大化为目标,提出了一种基于深度强化学习的5G基站储能充放电行为调控方法。该方法使用深度确定性策略梯度对系统环境信息进行学习,并对5G基站储能进行分组以降低学习维度。仿真结果表明,该方法能够保证5G基站储能单元后续可调容量,有效控制基站储能荷电状态变化范围,并最大化基站储能参与电网调度所得收益。

In order to improve the utilization rate of idle energy storage of 5G base stations,this paper designs a multi-base-station energy storage system(MBSESS)that uses the energy storage of 5G base station to participate in power grid dispatching.By adopting the coordinated dispatching mechanism of participating in power grid demand response and charging during off-peak tariff and discharging during peak tariff,an economic model of MBSESS is established,which takes into account multiple factors such as overcharge and overdischarge penalties and dispatching costs.With the goal of maximizing economic benefits,a deep reinforcement learning based dispatching method is proposed for regulating the charging and discharging behavior of energy storage of 5G base station.The method uses deep deterministic policy gradient to learn system environment information,and groups energy storage of 5G base station to reduce the learning dimension.Simulation results show that the method can ensure the subsequent adjustable capacity of energy storage of 5G base station,effectively control the variation range of the state of charge(SOC)of the base station energy storage,and maximize the revenue of energy storage of 5G base station participating in power grid dispatching.