基于两阶段深度强化学习算法的多智能体自由合谋竞价机理研究

Study on Free Joint Bidding Mechanism in Multi-agent Environment Based on Two-stage Deep Reinforcement Learning Algorithm

电力市场建设初期,不完善的监管机制为发电商提供了暗中交流,联合竞价的机会。然而,如何找到潜在的发电商合谋组合是相对困难的事情。针对这一问题,该文建立一种允许发电商自由联合的竞价模型,并提出全新的两阶段深度强化学习算法,来求解由离散的合谋对象选择和连续的报价系数确定组合形成的离散、连续动作混合决策问题。在不同阻塞情况下,对发电商联合策略形成过程进行分析,并在大算例中验证了算法的有效性。仿真结果表明,所提出的方法可以对市场主体的自由联合行为进行有效模拟,发现潜在的合谋组合。

In the early stages of power market development,the imperfect regulatory mechanism provides opportunities for power generators to secretly communicate and jointly bid.Detecting such collusive behaviors is challenging.In this paper,a bidding model considering the association among generators is established and a novel two-stage deep reinforcement learning algorithm is proposed to tackle the discrete-and-continuous decision problem of choosing collusion partners and bidding coefficients.The joint strategies of generators under different congestion scenarios are analyzed,and the effectiveness of the algorithm is validated in a large-scale case study.The simulation results demonstrate that the proposed method can effectively emulate the free association of generators and identify potential collusion.