基于动态选择预测器的深度强化学习投资组合模型

Deep Reinforcement Learning Portfolio Model Based on Dynamic Selectors

近年来,投资组合管理问题在人工智能领域得到了广泛的研究,但现有的基于深度学习的量化交易方法还存在一些问题。首先,对股票的预测模式单一,通常一个模型只能训练出一个交易专家,交易决策也仅根据模型预测结果作出;其次,模型使用的数据源相对单一,只考虑了股票自身数据,忽略了整个市场风险对股票的影响。针对上述问题,提出了基于动态选择预测器的强化学习模型(DSDRL)。该模型分为3部分,首先提取股票数据的特征并传入多个预测器中,针对不同的投资策略训练多个预测模型,用动态选择器得到当前最优预测结果;其次,利用市场环境评价模块对当前市场风险进行量化,得到合适的投资金额比例;最后,在前两个模块的基础上建立了一种深度强化学习模型模拟真实的交易环境,基于预测的结果和投资金额比例得到实际投资组合策略。文中使用中证500和标普500的日k线数据进行测试验证,结果表明,此模型在夏普率等指标上均优于其他参照模型。

In recent years,portfolio management problems have been extensively studied in the field of artificial intelligence,but there are some improvements in the existing quantitative trading methods based on deep learning.First of all,the prediction model of stocks is single,usually a model only trains a trading expert,and the decision of trading is only based on the prediction results of the model.Secondly,the data source used in the model is relatively single,only considering the stock’s own data,ignoring the impact of the entire market risk on the stock.Aiming at the above problems,a reinforcement learning model based on dynamic selection predictor(DSDRL)is proposed.The model is divided into three parts.Firstly,the characteristics of stock data are extracted and introduced into multiple predictors.Multiple prediction models are trained for different investment strategies,and the current optimal prediction results are obtained by dynamic selector.Secondly,the market environment evaluation module is used to quantify the current market risk and obtain the appropriate proportion of investment amount.Finally,based on the first two mo-dules,a deep reinforcement learning model is established to simulate the real trading environment,and the actual portfolio strategy is obtained based on the predicted results and the proportion of investment amount.In this paper,the daily k-line data of China Securities 500 and S&P 500 are used for test verification.The results show that the proposed model is superior to other refe-rence models in Sharpe rate and other indicators.