基于深度强化学习的自适应股指预测研究

Adaptive stock index prediction based on deep reinforcement learning

基于股指成分股基本面和技术面数据构建了时序股票关联网络,然后利用深度图神经网络学习股票关联网络层次化表征,以端到端的方式获得候选预测信号.在此基础上,提出了一种考虑动作评估反馈的深度强化学习方法(Action Evaluation Feedback based Deep Q-Learn-ing,AEF-DQN),旨在将不同的候选预测信号融入智能体的动作空间,并基于股票关联网络层次化表征、股票市场整体运行状态和历史动作评估反馈学习环境状态;借鉴前景理论中的参照依赖特性估计奖励值函数,从而建立状态、动作与奖励值之间的映射关系.最后,采用沪深300指数、标普500指数、英国富时100指数和日经225指数的成分股历史数据,构造了股指期货交易模拟器,在投资胜率、最大回撤率、阿尔法比率和夏普比率4个回测指标上对股指预测模型展开实证分析.研究结果表明:1)通过层次化聚合股票关联网络的节点属性信息可以动态捕捉不同行业对股指价格波动的影响,进而可提升预测方法的准确率;2)考虑动作评估反馈的深度强化学习结构可智能化选择适用于当前股票市场环境的最优模型结构,进而可提升预测方法的鲁棒性.

Based on the fundamental and technical data of constituent stocks,this paper constructs the sequential stock correlation networks,and utilizes the deep graph neural network to learn the hierarchical representations of stock correlation networks for obtaining the candidate prediction signals in an end-to-end fashion.On this basis,an Action Evaluation Feedback based Deep Q-Learning(AEF-DQN)is proposed to integrate different candidate prediction signals into the agent's action space.Specifically,AEF-DQN learns the environmental state using the hierarchical representation of stock correlation network,overall movement state of stock market and evaluation feedback of historical actions.By employing the reference dependency property in the prospect theory to estimate the reward function,AEF-DQN establishes the mapping relationship between state,action,and reward value.Finally,using the historical constituent stock data of SHSZ 300 index,S&P 500 index,FTSE 100 index and Nikkei 225 index,this paper constructs a stock index futures trading simulator to empirically analyze the proposed stock index prediction model from the profitable rate,maximum drawdown rate,Alpha ratio and Sharp ratio measures.The results show that:1)by hierarchically aggregating the node attributes of stock correlation networks,the proposed network representation model can dynamically capture the potential impacts of different industry sectors on the price fluctuation of stock index,so as to improve the accuracy of the prediction model;2)by considering the evaluation feedback of historical actions,the proposed deep reinforcement learning structure can intelligently select the optimal model structure suitable for the current stock market environment,so as to improve the robustness of the prediction method.