集群电动汽车充电行为的深度强化学习优化方法

Deep Reinforcement Learning Based Optimization for Charging of Aggregated Electric Vehicles

随着用电信息采集系统的推广,数据驱动的机器学习方法在用户侧用电行为优化领域的应用已引起广泛关注。利用深度强化学习方法(deep reinforcement learning,DRL),基于充电监测系统实时反馈的数据与分时电价信号,从负荷聚合商层面优化电动汽车(electric vehicles,EVs)充电行为。通过双延迟深度确定性策略梯度算法(twin delay deep deterministic policy gradient,TD3)对单辆电动汽车充电过程进行建模。通过在训练智能体时向其状态中引入随机噪声,该模型获得了对不同状态下的电动汽车充电行为的泛化控制能力。通过将训练得到的智能体进行分布式部署,该方法实现了对集群电动汽车充电行为的快速实时优化,其效果在算例中得到了验证。

With the popularization of electricity data acquisition systems,the application of data-driven machine learning methods has played a significant role on optimal decision-making in demand response.In this paper,based on the real-time feedback data from the charging monitoring system and TOU tariff,a deep reinforcement learning(DRL)method is proposed to optimize the charging strategy of the electric vehicles(EVs)from the perspective of aggregator.A twin delay deep deterministic policy gradient(TD3)algorithm is implemented to model the charging process of a single vehicle.By adding randomly the noises in the states of the trained agent,our model attained generalized abilities to control EV charging strategies under divergent states.With the distributed deployment of the well-trained agents,this method realizes the real-time optimization of aggregated EVs’charging strategy,which is proved with examples.