基于梯度优化的多任务混合学习方法

An Approach of Mixed Multi-task Learning Based on Gradient Optimization

多任务学习作为深度学习的一个分支,得到了广泛关注与深入研究,但仍然存在网络结构复杂、任务区分困难的问题。据此,基于硬参数共享神经网络给出梯度优化的多任务混合学习方法。首先,无需区分不同任务,将多任务训练数据一同送入网络进行混合训练,所有任务共用一个损失函数,前一次训练所得的网络共享层参数作为下次训练的共享层初始化参数;其次,根据不同共享层提取特征的差异和任务在深层梯度变化的不同,调节相应的激活值,优化网络参数,既保持了硬参数共享神经网络结构的简洁性特点,又利于解决多任务训练过程中数据的非平衡问题;最后,通过在UCI公开数据集中的鸢尾花和天平秤数据上的实际应用,以及与传统的硬参数共享神经网络的纵向对比,验证了该学习方法的可行性与有效性。

As a branch of deep learning,multi-task learning has received extensive attention and in-depth research.However,there are still some difficult problems such as overly complex network structure and indistinguishable multiple related tasks.An approach of mixed multi-task learning based on gradient optimization is proposed,that is established for the hard parameter sharing neural network.Firstly,the multi-task training data is fed into the network together for mixed training without distinguishing between different tasks,and all tasks share one loss function,during which the network sharing layer parameters obtained from the previous training are used as the initialization parameters of the sharing layer for the next training.Secondly,according to the difference of features extracted from different sharing layers and the different gradient changes of tasks in deep layers,the corresponding activation values are adjusted and the network parameters are optimized,which not only maintain the conciseness of the hard parameter sharing network structure,but also help to solve the non-equilibrium problem of data during the multi-task training.Finally,through the practical application of iris and balance data in the UCI open data set,as well as the longitudinal comparison with the traditional hard parameter sharing neural network,the feasibility and effectiveness of the proposed learning method is verified.