基于循环扩张机制的ConvGRU-Transformer短期电力负荷预测方法

A Short-term Load Forecasting Method Based on Recurrent and Dilated Mechanism of ConvGRU-transformer

Transformer作为一种建立在自注意力机制上的新颖神经网络模型,其所具有的高度并行化计算结构和有效捕捉序列长期依赖性的能力为短期电力负荷预测带来了新的发展空间。舍弃递归与卷积结构为提取序列关联性提供便利,同时也导致信息碎片化。为充分挖掘注意力模型潜力,提出了一种基于循环扩张机制的卷积门控循环单元(Convolutional Gated Recurrent Unit,ConvGRU)-Transformer短期电力负荷预测方法。针对输入序列分割影响长期特征提取的问题,提出一种循环扩张注意力机制,在提高计算效率的同时扩大了学习视野。为保证注意力视野扩大后信息位置的一致性,建立了一种基于ConvGRU的全局位置编码方法。实验结果表明与常规方法相比,所提方法有更高的预测精度和良好的可解释性。

As a novel neural network model based on the self-attention mechanism,Transformer has highly parallelized computing structure and the ability to capture the long-term dependence of the sequence effectively,which brings new development scope for short-term power load forecasting.Abandoning the recurrent and convolutional structure facilitates the sequence relevance extraction,and also leads to information fragmentation.In order to tap the potential of the attention model,a short-term load forecasting method is proposed based on convolutional gated recurrent unit(ConvGRU)-Transformer network and the dilated mechanism.To cope with the problem that input sequence segmentation affects long-term feature extraction,a recurrent and dilated attention mechanism is proposed which expands the learning horizon and improves computational efficiency.Then a relative position encoding method based on ConvGRU is proposed to ensure the consistency of the information and its position after the attention field expansion.The experimental results show that compared with conventional methods,the proposed method has higher forecast precision and better interpretability.