基于A3C的有序充电算法

Orderly Charging Algorithm Based on A3C

由于电动汽车的日益普及,其充电问题已成为电力系统的新的用电挑战。实际生活中,充电站一般都被认为是电动汽车有序充电行为的调度主体。为解决传统模型驱动的充电算法无法应用于电动汽车随机进站的问题,提出将数据驱动的无模型深度强化学习算法A3C(Asynchronous Advantage Actor-critic,异步演员评论家算法)应用于有序充电。该算法利用特征函数来近似模型所需要的价值函数和策略函数,解决因随机进站而引起的空间维度变化的问题。通过需求响应机制关联充电费用和需求,实现两者的动态调度。为避免因为经验回放而导致的数据相关性过强,利用多线程实现模型与多个环境进行互动,提高了模型的收敛性。最后以某地区充电站实测数据为例进行仿真分析。结果表明,该算法在只基于历史充电数据的情况下能优化充电行为,较大程度地抑制充电负荷方差,实现削峰填谷,同时在满足用户需求的基础上提高充电站收益。

Due to the increasing popularity of electric vehicles(EV),the charging problem has been a new challenge of electrical system. In particular, charging stations are always considered as an important role who schedule the orderly charging behavior of EV. In order to solve the problem that conventional model-driven charging algorithms cannot be applied to the situation where electric EV enter the station randomly, propose to apply a data-driven model-free reinforcement learning algorithms A3 C(Asynchronous Advantage Actor-critic) for orderly charging. The algorithm deal with the varying state spaces caused by random EV arrivals by approximating the state function and policy function with feature function. The demand response mechanism is applied to associate the charging price with the charging demands and dynamic scheduling them. To avoid the strong correlation caused by experience replay, multiprocessing is used to implement the effect that the model interact with multiple environments through which can improve the convergence of the algorithm. Finally, the simulation analysis is conducted by the measured data of charging stations in a certain area. The results show that the purposed algorithm can optimize the charging behavior even the only known is the previous charging data, reduce the charging load variance greatly and realize the peak load shifting of the grid. Beside satisfy the EVs demand, it can also increase charging station’s profits.