基于深度强化学习的投资组合管理研究

Deep Reinforcement Learning-Based Portfolio Management Research

投资组合管理是金融投资领域最常遇到的问题之一,在给定一组投资组合资产下,投资者把资金按一定比例分别投资于不同资产上,以实现分散风险、提高收益的目的。深度强化学习是一门新兴的研究领域,目前已经成功达到或超过人类在玩Atari游戏时的水平。深度强化学习的成功引起了金融界的广泛兴趣,我们考虑这些技术是否可以用于金融投资组合管理问题中。实现两种流行的深度强化学习算法——双延迟深度确定性策略梯度算法(TD3)和策略梯度算法(PG),并应用于中国市场龙头企业中成交量较大的5支股票和国债组成的资产包中。实验结果表示,TD3和PG算法在测试集上年利率分别可达84.71%和55.06%,明显高于其他对照组,充分证实深度强化学习在金融投资组合管理问题中的有效性。

For financial investment,portfolio management is one of the most common problems.For example,given a set of portfolio assets,investors invest funds in different assets at a certain percentage to achieve the purpose of diversifying risks and increasing revenues.At present,deep reinforcement learning is an emerging industry which applied into Atari games has surpassed human beings a lot.Therefore,it has success⁃fully aroused broad interests in the finance,and we will consider whether these technologies are able to be used in financial portfolio man⁃agement.In this article,we have implemented two popular deep reinforcement learning algorithms:Twin Delayed Deep Deterministic Poli⁃cy Gradient algorithm(TD3)and Policy Gradient algorithm(PG),and applied them to the asset package which are composed by 5 stocks with large turnover in leading enterprises and national bonds in the Chinese market.Finally,on the test set,the experimental results show that the Annual Percentage Rates of TD3 and PG algorithms can reach 84.71%and 55.06%respectively.It is significantly higher than oth⁃er control groups,which fully confirms the effectiveness of deep reinforcement learning in portfolio management.