基于逻辑回归的多任务域快速分类学习算法

Multi-task coupled logistic regression and its fast implementation for large multi-task datasets

多任务学习通过寻找并共享不同任务域之间的共性特征来完成学习,利用知识迁移加速不同任务域的学习为每个任务域构建一个分类器。提出了一种基于罗杰斯特回归模型的多任务学习方法 MTC-LR(Multi-task Coupled Logistic Regression)。"罗杰斯特回归模型"已经被成功应用于单任务分类器上,该模型被众多实验证明是有效的,正是这种方法给人们带来了启示。从理论上证明了通过构造多任务分类器的"开销函数"和"差异性度量函数",MTC-LR算法可以提高多任务分类器的各自分类精度。相比传统的基于SVM的多任务学习方法,MTC-LR并不依赖于核方法而是通过共轭梯度下降法寻找各个分类器的最优参数。同时MTC-LR与采用"罗杰斯特回归模型"的快速算法CDdual更容易结合,可扩展至大样本的多任务分类学习。正是基于上述发现,为了充分高效利用大样本的多任务域数据,满足大样本的快速运算,在MTC-LR算法的基础上,结合最新的CDdual(The Dual Coordinate Descent Method)算法,提出了MTC-LR的快速算法MTC-LR-CDdual,并对该算法进行了相关的理论分析。将该算法在人工数据集和真实数据集上进行了验证,实验结果表明该算法有着较高的识别率、快速的识别速度和较好的鲁棒性。

When facing multi-task learning problems,it is desirable that the learning method can find the correct inputoutputfeatures and share the commonality among multiple domains and also scale up for large multi-task datasets.Thispaper introduces the multi-task coupled logistic regression framework called MTC-LR,which is a new method for generatingeach classifier for each task,capable of sharing the commonality among multi-task domains.The basic idea of MTCLRis to use all individual logistic regression based classifiers,each one appropriate for each task domain,but in contrastto other SVM based proposals,learning all the parameter vectors of all individual classifiers by using the conjugate gradientmethod,in a global way and without the use of kernel trick,and being easily extended into its scaled version.This papertheoretically shows that the addition of a new term in the cost function of the set of logistic regressions(that penalizes thediversity among multiple tasks)produces a coupling of multiple tasks that allows MTC-LR to improve the learning performancein a logistic-regression way.This finding can make us easily integrate it with a state-of-the-art fast logistic regressionalgorithm called CDdual to develop its fast version MTC-LR-CDdual for large multi-task datasets.The proposedalgorithm MTC-LR-CDdual is also theoretically analyzed.The experimental results on artificial and real datasets indicatethe effectiveness of the proposed algorithm MTC-LR-CDdual in classification accuracy,speed and robustness.