基于深度强化学习的电力市场量价组合竞价策略

Strategic Bidding of Price-quantity Pairs in Electricity Market Based on Deep Reinforcement Learning

目前国内电力市场普遍采用“优先消纳、保障收购”的市场机制应对新能源消纳需求,因此传统能源将面对具有高不确定性的净负荷市场情形进行竞争,并通过策略报价使自身收益最大化。然而,现有策略报价的相关研究仅考虑了发电商报价而不报量、且忽略了竞争对手博弈主动性,导致难以反映策略发电商真实的市场竞价行为。对此,提出了一种基于深度强化学习的电力市场量价组合竞价策略分析方法。首先,针对现有策略竞价研究仅报价不报量的缺陷,研究了考虑量价组合申报的发电商双层双线性竞价模型。然后,为了考虑竞争对手行为的不确定性,构建了基于K-Medoids聚类方法与深度神经网络的发电商典型报价与净负荷间的概率映射,旨在为策略发电商提供贴近真实市场的竞价环境。最后,为高效求解策略报价的双层双线性模型,探讨了考虑不完全信息博弈与净负荷不确定性的深度确定性策略梯度强化学习方法。算例研究结果验证了所提量价组合申报模型的有效性以及所提方法应对电力市场净负荷和对手行为变化的鲁棒性,并能够提高发电商的竞价收益。

The domestic electricity market generally adopts the mechanism of“priority consumption and guaranteed procurement”to meet the requirements of renewable energy accommodations.Therefore,traditional energy will compete in the market with high uncertainty in the net load and maximize its profits through strategic bidding.However,existing research on strategic bidding only considers generators’bidding prices without bidding quantities.It ignores the initiative of competitors in the game,making it difficult to reflect the real market bidding behaviors of strategic generators.This paper proposes a deep reinforcement learning-based method for analyzing the bidding strategy of price-quantity pairs in electricity markets.First,to overcome the shortcomings of existing strategic bidding studies that only considered bidding prices without bidding quantities,a two-level bilinear bidding model for a generator considering price-quantity pairs is studied.Then,to consider the uncertainties of competitors’behaviors,probability mappings between typical bids of generators and net loads based on the K-Medoids clustering method and deep neural network are constructed,aiming to provide a bidding environment close to the real market for the strategic generator.Finally,to solve the two-level bilinear model of strategic bidding efficiently,a deep deterministic policy gradient reinforcement learning method considering incomplete information games and net load uncertainty is explored.Case studies verify the effectiveness of our model of price-quantity pairs,as well as the robustness of the proposed method to cope with changes in net load and competitors’behaviors in the electricity market.They can improve the bidding profits of the generator.