基于强化学习的含电动汽车虚拟电厂优化调度

Optimal scheduling of virtual power plants integrating electric vehicles based on reinforcement learning

大量电动汽车(EV)用户的无序充电可能造成电网负荷剧烈波动,危及电网的安全稳定。随着EV入网(V2G)技术的应用,将EV充电站及其周边的分布式新能源发电聚合为虚拟电厂(VPP)后进行优化调度,有助于改善EV用户充放电的经济性及满意度,同时提高分布式新能源的利用率,平抑电网负荷波动,但EV充电站的整体充放电负荷是大量个体EV用户随机行为的聚合,难以用数学模型精确描述。针对包含EV的VPP,提出一种基于深度强化学习的交互式调度框架,以最大化VPP内EV用户的总效益。VPP控制中心作为智能体决策EV个体的充放电动作,无需掌握个体详细模型,而是通过与区域电网环境的交互,不断学习和更新动作策略,从而克服集中式优化方法的局限性。该优化调度框架采用深度确定性策略梯度(DDPG)算法进行求解。仿真结果表明,与集中式优化方法相比,该优化算法提高了各EV用户的效益,并使EV充放电负荷与分布式新能源发电协调配合实现削峰填谷,改善了VPP的整体运行性能。

Disorderly charging behaviors of massive electric vehicles(EVs)may cause violent fluctuations in power loads,affecting the security and stability of the power grid.With the application of vehicle to grid(V2G)technology,the scheduling method can be optimized by aggregating EV charging stations and surrounded distributed renewable energy generators into a virtual power plant(VPP).The aggregation can improve the economy of charging behaviors and satisfaction of EV users,raise the utilization rate of distributed renewable energy,and mitigate load fluctuations in the grid.However,the overall charging or discharging load is the aggregation result of random charging or discharging behaviors of massive individual EVs,which is difficult to be accurately described by mathematical models.Thus,an interactive optimal scheduling framework based on deep reinforcement learning is presented for a VPP including EVs,with the objective of maximizing the benefit of all EV users in the VPP.The VPP control center,serving as an intelligent agent,can decide the charging and discharging of individual EVs without their detailed models.The agent continuously learns and updates its strategies through interactions with regional grids,overcoming the limitations of centralized optimal scheduling.The framework is solved by Deep Deterministic Policy Gradient(DDPG)algorithm.Simulation results show that,compared with the centralized scheduling,the proposed method improves the benefits of individual EV users,and the coordinative scheduling of EV charging/discharging loads and renewable energy outputs shaves the peak loads in the grid,and boosts the overall performance of the VPP.