量子自注意力神经网络的时间序列预测

Research on Time Series Prediction via Quantum Self-Attention Neural Networks

在“量子-经典”混合模式下,设计了多头量子自注意力神经网络预测模型(MQSAPN)用以进行时间序列预测,模型包括多头量子自注意力模块以及变分量子线路预测模块两部分。通过对输入数据按时间步长分别进行量子态编码以及K、Q、V的计算,借鉴已有研究使用高斯函数进行自注意力系数的估计方式,将量子自注意力特征提取后的数据再次编码到变分预测线路中,经过线路演化及测量,最终获取预测结果。完整流程与模型搭建均采用VQNet框架实现。在天气学变量的时间序列预测任务中,该模型表现出与经典多头自注意力模型预测模型以及长短期记忆单元网络模型相当的预测精度。此外,相对于同样是量子机器学习的data-reuploading变分线路而言,在近乎同等规模线路深度与参数量的前提下,表现出更高的预测精度,这也进一步验证了引入量子自注意力机制的有效性。值得指出的是,作为预测部分的变分线路会随着输入数据量的增多(如时间窗加长、特征变量规模增加等),其参数量与线路深度也会显著增加,尽管多层QSA能够较好地进行特征表达,但依然有可能因遇到“贫瘠高原”困难而成为整个网络的瓶颈。

A Multi-head Quantum Self-Attention Predict Network(MQSAPN)is designed in hybrid manner,which could be used in time-series forecasting.MQSAPN comprises two components,one is the Multi-head Quantum Self-Attention(MQSA)model,and the other is the predicting Variational Quantum Circuits(pVQC).When fed with sequential inputs,the MQSA firstly computes the key,query,and value vectors corresponding to all time steps through the variational circuits,and then according to exist studies,the attention is estimated via Gaussian function.With residual link on input and multi-head features,the output of MQSA were pushed to pVQC part,which was encoded into quantum circuit again,and the prediction would be ultimately calculated out by measurements on observables.The prediction results of MQSAPN numerical experiments on atmospheric variables indicate the effectiveness of quantum self-attention,by comparison with the results of a data-reuploading VQC model with almost same amount of parameters.The accuracy of predicting is close to classic multi-head transformer model and LSTM net.To be noted,as input time window extends or the more features are adopted,the number of parameters of pVQC will also increases correspondingly,which makes the pVQC part become the bottleneck of the whole model due to‘barren plateau’problems during training process.