基于深度强化学习的园区综合能源系统低碳经济调度

Deep Reinforcement Learning-based Low-carbon Economic Dispatch of Park Integrated Energy System

为降低园区综合能源系统的运行成本和碳排放量,同时应对系统不确定性带来的随机波动,提出一种考虑阶梯式碳交易的园区综合能源系统低碳经济调度模型,并采用深度强化学习方法求解。首先构建园区阶梯式碳交易模型,将碳交易成本考虑在内对园区综合能源系统低碳经济调度问题进行数学描述;其次将该调度问题表述为马尔可夫决策过程框架,定义系统的观测状态、调度动作和奖励函数;继而采用近端策略优化算法进行低碳经济调度决策。所提方法无需进行负荷预测或不确定性建模,能够对源和荷的随机波动做出实时响应。最后基于多场景多算法进行算例仿真,结果表明所提方法提高系统运行经济性的同时降低了系统碳排放量。

To reduce the park-integrated energy system's operating costs and carbon emissions while solving the random fluctuations caused by the system uncertainty,a low-carbon economic dispatch model of the park-integrated energy system considering ladder-type carbon trading is proposed and solved by the deep reinforcement learning method.Firstly,the ladder-type carbon trading model is proposed,and the low carbon economic dispatch problem of the park-integrated energy system is mathematically described by taking carbon trading costs into account;secondly,the dispatch problem is formulated as a Markov decision process framework,defining the observation state,dispatch action and reward function of the system;then the proximal policy optimization algorithm is used to make low carbon economic dispatch decisions.The proposed method does not need to predict load or model the uncertainty,and the network is trained to respond to the system state in real-time.Finally,the simulation is based on multiple scenarios and algorithms,and the results show that the proposed method improves the system operation economy while reducing the system's carbon emissions.