基于强化学习的电动汽车集群实时优化调度策略

Real-time Optimal Scheduling Strategy for Electric Vehicle Clusters Based on Reinforcement Learning

针对大规模电动汽车的实时调度存在维度高和随机性强等问题,提出基于强化学习的电动汽车集群实时优化调度策略。首先,以最小化综合成本(机组发电成本和补贴成本)为目标,建立电动汽车集群参与的电网机组经济调度模型。将实时阶段下的该模型构建为一个马尔可夫决策过程,利用基于最大熵的深度强化学习算法对马尔可夫决策过程进行模型训练和求解。此外,融合强化学习不依赖预测信息和运筹优化算法保证物理约束的优势,将电动汽车充电和机组出力分开优化调度。最后,通过算例验证所提策略在降低成本和削峰填谷方面的可行性和有效性。

Targeting the high dimension and strong randomness in the real-time scheduling of large-scale electric vehicles(EVs),this paper proposes a real-time optimal scheduling strategy for EV clusters based on reinforcement learning(RL).Firstly,the economic dispatch model of the unit in power grid intergrating the EV clusters is established,with the goal of minimizing the overall costs(unit generation costs and subsidy costs).Then the model is formulated as a Markov decision process(MDP)model,and the maximum entropy based RL is used to train and solve the MDP model.In addition,making use of advantages of not relying on predictive information with RL algorithm and ensuring physical constraints with traditional optimization algorithm,the EV charging power and unit output power are optimized separately.Finally,the feasibility and effectiveness of the proposed strategy in reducing costs as well as peak-shaving and valley-filling are verified through case studies.