基于强化学习的非线性时间序列智能预测模型

Intelligent prediction model for nonlinear time series based on reinforcement learning

针对时间序列预测中非线性、噪声高等特点,提出时间序列向前多步混合智能预测模型.首先,在模型训练过程中,提出一种将强化学习与隐马尔可夫模型相结合的新方法,强化学习运用TD(λ)方法,采用历史观测数据作为报酬回报,强调远近期历史数据的不同影响程度并用以迭代增强历史观测数据在模型中的作用;进一步,在向前多步预测过程中,提出一种以强化学习为桥梁、将神经网络与隐马尔可夫模型相结合的方法,用以充分发挥神经网络数据拟合优势和隐马尔可夫模型减小系统随机误差方面的优势.利用稀土期货交易数据进行预测实验,结果表明:智能预测模型显著降低了预测的平均绝对误差、百分比绝对平均误差、均方根误差,提高了预测的准确性和效果.

Aiming at the nonlinear and high noise issues in time series prediction, an hybrid intelligent model for time se- ries forward multistep prediction was proposed. Firstly, for training the model, a method of combining reinforcement learning and hidden Markov model was proposed, TD （λ） approach was used to reinforce learning, and historical observed data was adopted as reward returns and emphasized the different degree of influence in far more recent to iteratively improve the roles of historical observed data played in model training processes. Further, in the process of forward multistep prediction, a method of combining reinforcement learning with neural network and hidden Markov model was proposed, which took full advantages of data fitting by neural network and reducing systematic random error in predicting processes by hidden Markov model. Experimental results of rare earth futures trade data predictions show that the intelligent hybrid model obviously reduces mean absolute error, mean absolute percentage error and root mean square error, which improves the prediction precision and effect.