基于Q强化学习的综合能源服务商现货市场申报策略研究

A Novel Declaration Strategy for Integrated Energy Servicer Based on Q-Learning Algorithm in Power Spot Market

随着综合能源系统建设和电力市场改革推进,综合能源服务商有望成为新的市场交易成员。为解决申报阶段有限的决策参考信息制约申报策略制定的问题,文章提出了一种基于Q强化学习的综合能源服务商现货市场申报策略,以提升申报策略的理想度。该方法的主要特点在于充分利用庞大的历史运行信息,通过人工智能算法训练申报策略智能体,建立综合能源服务商所掌握的有限参考信息与最优申报策略之间的内在关系。智能体以市场公开信息、社会公共信息及服务商私有信息为环境变量,能够实现申报策略的自动生成和智能改进。最后,基于某省电网实际数据构造算例表明,该方法能较好地拟合合作博弈下的申报策略,具有收敛速度快、理想度高、计算效率高等特点,更符合综合能源服务商决策需求。

With the development of integrated energy system and power market reform,integrated energy servicer is expected to become a new market member in power market transaction.In order to solve the problem that the limited reference information in the declaration stage restricts the formulation of the declaration strategy,a declaration strategy based on Q-learning for integrated energy servicer is proposed to improve the ideal degree of the declaration strategy.The core idea of the proposed strategy is to make full use of the huge historical operation information and train the declaration strategy agent by artificial intelligence algorithms to establish the inherent relationship between the limited reference information grasped by integrated energy servicer during the market bidding process and its optimal declaration strategy.The declaration agent can realize automatic generation and intelligent improvement of declaration policies,which takes energy market public information,social public information and enterprise private information as environment variables.Finally,a case study based on the actual data of a provincial power grid shows that the proposed method can better match the declaration strategy under the cooperative game and has the characteristics of fast convergence,high ideal degree and high computational efficiency,which is more suitable for the actual needs of integrated energy servicer.