基于深度强化学习的投资组合管理问题

Portfolio Management Problem Based on Deep Reinforcement Learning

投资组合管理任务一直是金融领域的热点问题之一,随着人工智能技术的发展,已经有越来越多的工作将人工智能技术应用于投资组合管理领域。而其中最为主要的则是强化学习技术,强化学习技术是机器学习的一个分支,不断地根据环境反馈来调整自己的策略而不需要预先给定标签。同时,深度学习具有高阶特征抽取能力,因此,本文将使用深度强化学习来解决投资组合管理问题。针对金融序列存在着大量噪音的问题,使用EWT经验小波变换来对股价序列进行去噪,然后使用去噪后的序列构建科技指标,并输入到模型之中。使用TCN时间卷积网络来提取股票的时序特征,然后使用多头注意力网络来提取股票之间的空间关系,最后输入到全连接层之中,再经过sigmoid函数和softmax函数来得到投资组合的权重。使用基本的策略梯度方法,并使用夏普比率作为目标函数,分别在DJIA、HSI和DAX三种数据集上对本文构建的模型进行实验,使用了六种指标来评价不同策略的优劣,实验证实本文提出的模型具有一定的优势,能够实现较低的风险以及较高的回报。

The task of portfolio management has been a hot issue in the financial field. With the development of artificial intelligence technology, more and more work has been applied artificial intelligence technology to the field of portfolio management. The most important of these is reinforcement learning, which is a branch of machine learning that continuously adjusts its strategy based on environmental feedback without prespecifying labels. At the same time, deep learning has high-order feature extraction capabilities, so this article will use deep reinforcement learning to solve the problem of portfolio management. Aiming at the problem of a large amount of noise in the financial data, the EWT is used to denoise the stock price sequence, and the denoised sequence is used to construct technical indicators and input into the model. And then the TCN is used to extract the time series features of stocks, multi-head attention is used to extract the spatial relationship between stocks, and finally input it into the fully connected layer and get the portfolio through the sigmoid function and softmax function. Using the basic strategy gradient method and using the Sharpe ratio as the objective function, the model constructed in this paper was tested on three data sets of DJIA, HSI and DAX respectively, and six indicators were used to evaluate different strategies, and the experiments confirmed that the model proposed in this paper has advantages and can achieve lower risk and higher return.