分布式网络的多任务扩散算法综述

A Review of Multitask Diffusion Strategies in Distributed Networks

在分布式网络中,如何从带噪的实时数据流中估计潜在的模型参数是一个非常重要而极具挑战的问题。应对该问题的一种有效途径是设计扩散算法,通过网络局部信息交换和自组织协同方式来估计网络中所有节点的模型参数。相关方面早期的研究主要关注的是单任务问题,即所有节点估计同一个模型参数。但实际应用中不同节点往往需要应对不同的估计任务,同时多个待估计的模型参数又可能具备一定的关联性,所以近年来人们将研究重点聚焦到了多任务问题,包括多任务关系建模以及参数优化问题的设计与求解等,并取得了一些重要进展。本文对分布式网络中多任务问题的建模以及当前常用的多任务扩散算法进行简要综述,内容涵盖无监督类型的扩散算法和有监督类型的扩散算法等。具体而言,针对使用无监督方式设计多任务扩散算法,介绍了静态组合矩阵的选取、自适应组合矩阵的设计、两个组合矩阵的组合结构,并简单讨论了三者的适用场景。针对使用有监督方式设计多任务扩散算法,讨论了基于梯度投影法和正则化架构的两种设计方式,并借助文献中的一些典型算法对这些设计方式进行说明。论文侧重于算法原理、算法架构和求解思路等方面的探讨,具体的算法实现细节感兴趣的读者可以参考相应的文献。

In a distributed network,it is important yet challenging to estimate model parameters from noisy streaming data.To cope with this problem,diffusion strategies have been intensively studied over the past decade,which attempt to estimate the model parameters cooperatively by authorizing a local exchange of data within neighboring nodes in a selforganized manner.Within the literature of diffusion strategies,the earliest such effort can trace back to the so-called single-task problem,in which all the nodes in a distributed network estimate the same parameter.In real applications,however,different nodes may have to estimate different but related parameters.By taking these relations into consideration,the more general and flexible multitask problem has been proposed recently.The keys to solving multitask problems include a proper modelling and formulation of the real problem,as well as designing effective diffusion strategies to achieve a useful solution.Over the recent years,a great amount of multitask diffusion strategies have been proposed.In this paper,we discuss the basic principles underlying those multitask diffusion strategies,covering the topics of both the unsupervised learning and the supervised learning,which are two dominant routines of designing multitask diffusion strategies.Specifically,on the one hand,for multitask diffusion strategies based on the unsupervised learning,we introduce some details on the selection of both static combination matrices and adaptive combination matrices,along with a novel combination scheme for two combination matrices.The differences among them are further highlighted,so as to make them clear for the interested reader.On the other hand,for multitask diffusion strategies based on the supervised learning,the so-called projected gradient descent method and several regularization based algorithms have been discussed,as well as several typical references about these algorithms are introduced.In addition,to make the above algorithms to be clear,several examples are given at the end of each algorithm.This paper mainly focus on the principles and structures of each algorithm,and due to space limit,we will not present much detail on the derivation of every individual algorithm.The interested reader is therefore encouraged to follow the respective references listed in the paper.