Distributionally Robust Learning Based on Dirichlet Process Prior in Edge Networks

In order to meet the real-time performance requirements,intelligent decisions in Internet of things applications must take place right here right now at the network edge.Pushing the artificial intelligence frontier to achieve edge intelligence is nontrivial due to the constrained computing resources and limited training data at the network edge.To tackle these challenges,we develop a distributionally robust optimization(DRO)-based edge learning algorithm,where the uncertainty model is constructed to foster the synergy of cloud knowledge and local training.Specifically,the cloud transferred knowledge is in the form of a Dirichlet process prior distribution for the edge model parameters,and the edge device further constructs an uncertainty set centered around the empirical distribution of its local samples.The edge learning DRO problem,subject to these two distributional uncertainty constraints,is recast as a single-layer optimization problem using a duality approach.We then use an Expectation-Maximization algorithm-inspired method to derive a convex relaxation,based on which we devise algorithms to learn the edge model.Furthermore,we illustrate that the meta-learning fast adaptation procedure is equivalent to our proposed Dirichlet process prior-based approach.Finally,extensive experiments are implemented to showcase the performance gain over standard approaches using edge data only.

基于一般Dirichlet过程的非参数贝叶斯分析

基于Jayaram Sethuraman在1994提出的扩展Dirichlet过程先验,将其推广到更一般的情形,使Dirichlet过程和扩展Dirichlet过程都成为一般Dirichlet过程的特例,并对非参数的贝叶斯进行了研究和讨论,给出了一般Dirichlet分布的期望、二阶矩等,证明了一般Dirichlet过程的支撑是足够大的,说明一般Dirichlet过程的构造是合理而又恰当的.

采用自适应先验表观模型的目标跟踪方法

为有效解决可变目标在跟踪过程中的"漂移"问题,提出一种基于自适应先验表观模型的目标跟踪方法。该方法首先在一致架构内融合HDP-EVO演化聚类模型和在线Boosting学习。以Dirichlet过程为先验分布,对总体表观示例进行聚类分析,获得随时间自适应演化的表观类先验知识,进而利用共享的表观类混合比例的权重平滑约束各时刻的表观模型。改进Gibbs抽样过程,使之能融入目标示例的分类误差,并交替迭代地从数据中自主学习聚类和表观分类器。最后,根据表观模型中各表观类的权重系数组合它们的分类评分去定位目标位置。仿真实验表明新方法学习的表观模型能较鲁棒地自适应于目标的表观变化,提高了跟踪精度。

Dirichlet Process Gaussian Mixture Models:Choice of the Base Distribution

In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mixture models sidestep the problem of finding the ＂correct＂ number of mixture components by assuming infinitely many components. In this paper Dirichlet process mixture （DPM） models are cast as infinite mixture models and inference using Markov chain Monte Carlo is described. The specification of the priors on the model parameters is often guided by mathematical and practical convenience. The primary goal of this paper is to compare the choice of conjugate and non-conjugate base distributions on a particular class of DPM models which is widely used in applications, the Dirichlet process Gaussian mixture model （DPGMM）. We compare computational efficiency and modeling performance of DPGMM defined using a conjugate and a conditionally conjugate base distribution. We show that better density models can result from using a wider class of priors with no or only a modest increase in computational effort.

面向图像先验建模的可扩展高斯混合模型

针对使用高斯混合模型的图像先验建模中分量数目难以扩展的问题,构建基于狄利克雷过程的可扩展高斯混合模型.通过聚类分量的新增及归并机制,使模型复杂度根据数据规模自适应变化,从而增强先验模型结构的紧密度,以提升其可解释性.此外,对高斯混合模型的推理过程进行优化,给出一种基于批次处理方式的可扩展变分推理算法,求解图像去噪中所有隐变量的变分后验分布,实现先验学习.实验结果表明,该模型在图像去噪任务中较EPLL等传统去噪模型能够取得更高的峰值信噪比,去噪效果更佳,验证了该模型的有效性.

Dirichlet process and its developments: a survey

The core of the nonparametric/semiparametric Bayesian analysis is to relax the particular parametric assumptions on the distributions of interest to be unknown and random,and assign them a prior.Selecting a suitable prior therefore is especially critical in the nonparametric Bayesian fitting.As the distribution of distribution,Dirichlet process(DP)is the most appreciated nonparametric prior due to its nice theoretical proprieties,modeling flexibility and computational feasibility.In this paper,we review and summarize some developments of DP during the past decades.Our focus is mainly concentrated upon its theoretical properties,various extensions,statistical modeling and applications to the latent variable models.

Effective Frameworks Based on Infinite Mixture Model for Real-World Applications

Interest in automated data classification and identification systems has increased over the past years in conjunction with the high demand for artificial intelligence and security applications.In particular,recognizing human activities with accurate results have become a topic of high interest.Although the current tools have reached remarkable successes,it is still a challenging problem due to various uncontrolled environments and conditions.In this paper two statistical frameworks based on nonparametric hierarchical Bayesian models and Gamma distribution are proposed to solve some realworld applications.In particular,two nonparametric hierarchical Bayesian models based on Dirichlet process and Pitman-Yor process are developed.These models are then applied to address the problem of modelling grouped data where observations are organized into groups and these groups are statistically linked by sharing mixture components.The choice of the Gamma mixtures is motivated by its flexibility for modelling heavy-tailed distributions.In addition,deploying the Dirichlet process prior is justified by its advantage of automatically finding the right number of components and providing nice properties.Moreover,a learning step via variational Bayesian setting is presented in a flexible way.The priors over the parameters are selected appropriately and the posteriors are approximated effectively in a closed form.Experimental results based on a real-life applications that concerns texture classification and human actions recognition show the capabilities and effectiveness of the proposed framework.

Topic model for graph mining based on hierarchical Dirichlet process

In this paper,a nonparametric Bayesian graph topic model(GTM)based on hierarchical Dirichlet process(HDP)is proposed.The HDP makes the number of topics selected flexibly,which breaks the limitation that the number of topics need to be given in advance.Moreover,theGTMreleases the assumption of‘bag of words’and considers the graph structure of the text.The combination of HDP and GTM takes advantage of both which is named as HDP–GTM.The variational inference algorithm is used for the posterior inference and the convergence of the algorithm is analysed.We apply the proposed model in text categorisation,comparing to three related topic models,latent Dirichlet allocation(LDA),GTM and HDP.

基于狄利克雷过程混合模型的内外先验融合

近年来,使用高斯混合模型作为块先验的贝叶斯方法取得了优秀的图像复原性能,针对这类模型分量固定及主要依赖外部学习的缺点,提出了一种新的基于狄利克雷过程混合模型的图像先验模型。该模型从干净图像数据库中学习外部通用先验,从退化图像中学习内部先验,借助模型中统计量的可累加性自然实现内外部先验融合。通过聚类的新增及归并机制,模型的复杂度随着数据的增大或缩小而自适应地变化,可以学习到可解释及紧凑的模型。为了求解所有隐变量的变分后验分布,提出了一种结合新增及归并机制的批次更新可扩展变分算法,解决了传统坐标上升算法在大数据集下效率较低、容易陷入局部最优解的问题。在图像去噪及填充实验中,相比传统方法,所提模型无论在客观质量评价还是视觉观感上都更有优势,验证了该模型的有效性。

TRANSFORMATION OF SYMMETRIC MULTIPLICATIVE FUNCTIONALS

In the present paper the transformation of symmetric Markov processes by symmetric martingale multiplicative functionals is studied and the corresponding Dirichlet form is formulated.

Predictive Analysis of Microarray Data

Microarray gene expression data are analyzed by means of a Bayesian nonparametric model, with emphasis on prediction of future observables, yielding a method for selection of differentially expressed genes and the corresponding classifier.

The Dirichlet Problem of a Discontinuous Markov Process

Given a Markov process satisfying certain general type conditions,whose paths are notassumed to be continuous. Let D by an open subset of the state space E. Any bounded function defined on thecomplement of D extends to be a function on E (?)uch that it is harmonic in D and satisfies the Dirichletboundary condition at any regular boundary point of D. The relation between harmonic functions and theebaracteristic operator of the given process is discussed.

Gamma-Dirichlet algebra and applications

The Gamma-Dirichlet algebra corresponds to the decomposition of the gamma process into the independent product of a gamma random variable and a Dirichlet process. This structure allows us to study the properties of the Dirichlet process through the gamma process and vice versa. In this article, we begin with a brief survey of several existing results concerning this structure. New results are then obtained for the large deviations of the jump sizes of the gamma process and the quasi-invariance of the two-parameter Poisson-Dirichlet distribution. We finish the paper with the derivation of the transition function of the Fleming-Viot process with parent independent mutation from the transition function of the measure-valued branching diffusion with immigration by exploring the Gamma-Dirichlet algebra embedded in these processes. This last result is motivated by an open R. C. Gritfiths. problem proposed by S. N. Ethier and

Heterogeneous clustering via adversarial deep Bayesian generative model

This paper aims to study the deep clustering problem with heterogeneous features and unknown cluster number.To address this issue,a novel deep Bayesian clustering framework is proposed.In particular,a heterogeneous feature metric is first constructed to measure the similarity between different types of features.Then,a feature metric-restricted hierarchical sample generation process is established,in which sample with heterogeneous features is clustered by generating it from a similarity constraint hidden space.When estimating the model parameters and posterior probability,the corresponding variational inference algorithm is derived and implemented.To verify our model capability,we demonstrate our model on the synthetic dataset and show the superiority of the proposed method on some real datasets.Our source code is released on the website:Github.com/yexlwh/Heterogeneousclustering.

基于MCMC抽样的金融贝叶斯半参数GARCH模型研究

GARCH模型是研究金融资产收益的重要模型,然而现有参数GARCH模型依然不能有效刻画金融资产收益偏态厚尾特性且存在模型设定风险。本文在非参数分布和GARCH模型基础上,建立半参数GARCH模型以提高模型的有效性;同时在贝叶斯框架内发展有效MCMC抽样解决模型的参数估计难问题,并利用DIC4研究模型比较问题;最后通过模拟研究和实证研究考察MCMC抽样的有效性,检验半参数GARCH模型在刻画金融资产收益特性和风险价值预测方面的实际效果。

半参数贝叶斯分层分位回归模型及其在保险公司成本分析中的应用

本文建立了一种半参数贝叶斯分层分位回归模型,并基于美国NAIC提供的多个保险公司连续多年期的非平衡纵向成本观测数据进行了实证分析.本文主要贡献包括三个方面:一是首次在有限正态混合误差假定下,对具有右偏厚尾性的成本数据建立半参数分层分位回归模型,并考虑到保险公司的聚类性,选用狄利克雷过程先验进行模型非参数部分的估计,进一步推广了分位回归模型在保险精算领域中的应用;二是通过模拟数据研究,系统比较了在非对称拉普拉斯误差假定下和有限正态混合误差假定下,半参数分层分位回归模型对复杂数据的拟合精度及参数估计的精确性,结果表明,有限正态混合误差更能充分捕捉数据的复杂性;三是通过实际观测的保险公司成本数据进行分析,选出了对成本具有较强效应的解释变量,并发现在不同分位数水平下各个解释变量对响应变量的效应具有较大区别.

评价多水平因子分析模型的异质性:半参数贝叶斯方法

数据的异质性因相关的解释变量被排除在既定模型.然而,目前经典统计方法难以处理的异质参数的依赖关系或"结".为了解决非均质参数之间的依赖关系,本文基于多水平因子分析模型提出一个半参数贝叶斯分析程序.对于模型的截距和/或协方差结构参数的分布赋予折断的Dirichlet过程先验.在贝叶斯马尔可夫链蒙特卡罗框架内,分块Gibbs抽样器被用来执行后验分析.统计推断的基础上进行这些观察的经验分布.仿真研究表明,忽略了非均质参数之间的联系会导致不变参数估计产生严重的偏差.

Incorporating travel time reliability in predicting the likelihood of severe crashes on arterial highways using non-parametric random-effect regression

Travel time reliability(TTR) modeling has gain attention among researchers’ due to its ability to represent road user satisfaction as well as providing a predictability of a trip travel time.Despite this significant effort,its impact on the severity of a crash is not well explored.This study analyzes the effect of TTR and other variables on the probability of the crash severity occurring on arterial roads.To address the unobserved heterogeneity problem,two random-effect regressions were applied;the Dirichlet random-effect(DRE)and the traditional random-effect(TRE) logistic regression.The difference between the two models is that the random-effect in the DRE is non-parametrically specified while in the TRE model is parametrically specified.The Markov Chain Monte Carlo simulations were adopted to infer the parameters’ posterior distributions of the two developed models.Using four-year police-reported crash data and travel speeds from Northeast Florida,the analysis of goodness-of-fit found the DRE model to best fit the data.Hence,it was used in studying the influence of TTR and other variables on crash severity.The DRE model findings suggest that TTR is statistically significant,at 95 percent credible intervals,influencing the severity level of a crash.A unit increases in TTR reduces the likelihood of a severe crash occurrence by 25 percent.Moreover,among the significant variables,alcohol/drug impairment was found to have the highest impact in influencing the occurrence of severe crashes.Other significant factors included traffic volume,weekends,speed,work-zone,land use,visibility,seatbelt usage,segment length,undivided/divided highway,and age.

One Dimensional Stochastic Differential Equations with Distributional Drifts

In this paper we study the existence, pathwise uniqueness and homeomorphism flow of strong solutions to a class of one dimensional SDEs driven by infinitely many Brownian motions, and with Yamada- Watanabe diffusion coefficients and distributional drift coefficients.

Semiparametric Bayesian Inference for Mean-Covariance Regression Models

In this paper, we propose a Bayesian semiparametric mean-covariance regression model with known covariance structures. A mixture model is used to describe the potential non-normal distribution of the regression errors. Moreover, an empirical likelihood adjusted mixture of Dirichlet process model is constructed to produce distributions with given mean and variance constraints. We illustrate through simulation studies that the proposed method provides better estimations in some non-normal cases. We also demonstrate the implementation of our method by analyzing the data set from a sleep deprivation study.