Combined UAMP and MF Message Passing Algorithm for Multi-Target Wideband DOA Estimation with Dirichlet Process Prior

When estimating the direction of arrival (DOA) of wideband signals from multiple sources, the performance of sparse Bayesian methods is influenced by the frequency bands occupied by signals in different directions. This is particularly true when multiple signal frequency bands overlap. Message passing algorithms (MPA) with Dirichlet process (DP) prior can be employed in a sparse Bayesian learning (SBL) framework with high precision. However, existing methods suffer from either high complexity or low precision. To address this, we propose a low-complexity DOA estimation algorithm based on a factor graph. This approach introduces two strong constraints via a stretching transformation of the factor graph. The first constraint separates the observation from the DP prior, enabling the application of the unitary approximate message passing (UAMP) algorithm for simplified inference and mitigation of divergence issues. The second constraint compensates for the deviation in estimation angle caused by the grid mismatch problem. Compared to state-of-the-art algorithms, our proposed method offers higher estimation accuracy and lower complexity.

Distributionally Robust Learning Based on Dirichlet Process Prior in Edge Networks

In order to meet the real-time performance requirements,intelligent decisions in Internet of things applications must take place right here right now at the network edge.Pushing the artificial intelligence frontier to achieve edge intelligence is nontrivial due to the constrained computing resources and limited training data at the network edge.To tackle these challenges,we develop a distributionally robust optimization(DRO)-based edge learning algorithm,where the uncertainty model is constructed to foster the synergy of cloud knowledge and local training.Specifically,the cloud transferred knowledge is in the form of a Dirichlet process prior distribution for the edge model parameters,and the edge device further constructs an uncertainty set centered around the empirical distribution of its local samples.The edge learning DRO problem,subject to these two distributional uncertainty constraints,is recast as a single-layer optimization problem using a duality approach.We then use an Expectation-Maximization algorithm-inspired method to derive a convex relaxation,based on which we devise algorithms to learn the edge model.Furthermore,we illustrate that the meta-learning fast adaptation procedure is equivalent to our proposed Dirichlet process prior-based approach.Finally,extensive experiments are implemented to showcase the performance gain over standard approaches using edge data only.

基于一般Dirichlet过程的非参数贝叶斯分析

基于Jayaram Sethuraman在1994提出的扩展Dirichlet过程先验,将其推广到更一般的情形,使Dirichlet过程和扩展Dirichlet过程都成为一般Dirichlet过程的特例,并对非参数的贝叶斯进行了研究和讨论,给出了一般Dirichlet分布的期望、二阶矩等,证明了一般Dirichlet过程的支撑是足够大的,说明一般Dirichlet过程的构造是合理而又恰当的.

Modulation classification of MPSK signals based on nonparametric Bayesian inference

A nonparametric Bayesian method is presented to classify the MPSK （M-ary phase shift keying） signals. The MPSK signals with unknown signal noise ratios （SNRs） are modeled as a Gaussian mixture model with unknown means and covariances in the constellation plane, and a clustering method is proposed to estimate the probability density of the MPSK signals. The method is based on the nonparametric Bayesian inference, which introduces the Dirichlet process as the prior probability of the mixture coefficient, and applies a normal inverse Wishart （NIW） distribution as the prior probability of the unknown mean and covariance. Then, according to the received signals, the parameters are adjusted by the Monte Carlo Markov chain （MCMC） random sampling algorithm. By iterations, the density estimation of the MPSK signals can be estimated. Simulation results show that the correct recognition ratio of 2/4/8PSK is greater than 95% under the condition that SNR 〉5 dB and 1 600 symbols are used in this method.

采用自适应先验表观模型的目标跟踪方法

为有效解决可变目标在跟踪过程中的"漂移"问题,提出一种基于自适应先验表观模型的目标跟踪方法。该方法首先在一致架构内融合HDP-EVO演化聚类模型和在线Boosting学习。以Dirichlet过程为先验分布,对总体表观示例进行聚类分析,获得随时间自适应演化的表观类先验知识,进而利用共享的表观类混合比例的权重平滑约束各时刻的表观模型。改进Gibbs抽样过程,使之能融入目标示例的分类误差,并交替迭代地从数据中自主学习聚类和表观分类器。最后,根据表观模型中各表观类的权重系数组合它们的分类评分去定位目标位置。仿真实验表明新方法学习的表观模型能较鲁棒地自适应于目标的表观变化,提高了跟踪精度。

Dirichlet Process Gaussian Mixture Models:Choice of the Base Distribution

In the Bayesian mixture modeling framework it is possible to infer the necessary number of components to model the data and therefore it is unnecessary to explicitly restrict the number of components. Nonparametric mixture models sidestep the problem of finding the ＂correct＂ number of mixture components by assuming infinitely many components. In this paper Dirichlet process mixture （DPM） models are cast as infinite mixture models and inference using Markov chain Monte Carlo is described. The specification of the priors on the model parameters is often guided by mathematical and practical convenience. The primary goal of this paper is to compare the choice of conjugate and non-conjugate base distributions on a particular class of DPM models which is widely used in applications, the Dirichlet process Gaussian mixture model （DPGMM）. We compare computational efficiency and modeling performance of DPGMM defined using a conjugate and a conditionally conjugate base distribution. We show that better density models can result from using a wider class of priors with no or only a modest increase in computational effort.

Topic model for graph mining based on hierarchical Dirichlet process

In this paper,a nonparametric Bayesian graph topic model(GTM)based on hierarchical Dirichlet process(HDP)is proposed.The HDP makes the number of topics selected flexibly,which breaks the limitation that the number of topics need to be given in advance.Moreover,theGTMreleases the assumption of‘bag of words’and considers the graph structure of the text.The combination of HDP and GTM takes advantage of both which is named as HDP–GTM.The variational inference algorithm is used for the posterior inference and the convergence of the algorithm is analysed.We apply the proposed model in text categorisation,comparing to three related topic models,latent Dirichlet allocation(LDA),GTM and HDP.

面向图像先验建模的可扩展高斯混合模型

针对使用高斯混合模型的图像先验建模中分量数目难以扩展的问题,构建基于狄利克雷过程的可扩展高斯混合模型.通过聚类分量的新增及归并机制,使模型复杂度根据数据规模自适应变化,从而增强先验模型结构的紧密度,以提升其可解释性.此外,对高斯混合模型的推理过程进行优化,给出一种基于批次处理方式的可扩展变分推理算法,求解图像去噪中所有隐变量的变分后验分布,实现先验学习.实验结果表明,该模型在图像去噪任务中较EPLL等传统去噪模型能够取得更高的峰值信噪比,去噪效果更佳,验证了该模型的有效性.

Self-Adaptive Topic Model： A Solution to the Problem of ＂Rich Topics Get Richer＂

The problem of "rich topics get richer"(RTGR) is popular to the topic models,which will bring the wrong topic distribution if the distributing process has not been intervened.In standard LDA(Latent Dirichlet Allocation) model,each word in all the documents has the same statistical ability.In fact,the words have different impact towards different topics.Under the guidance of this thought,we extend ILDA(Infinite LDA) by considering the bias role of words to divide the topics.We propose a self-adaptive topic model to overcome the RTGR problem specifically.The model proposed in this paper is adapted to three questions:(1) the topic number is changeable with the collection of the documents,which is suitable for the dynamic data;(2) the words have discriminating attributes to topic distribution;(3) a selfadaptive method is used to realize the automatic re-sampling.To verify our model,we design a topic evolution analysis system which can realize the following functions:the topic classification in each cycle,the topic correlation in the adjacent cycles and the strength calculation of the sub topics in the order.The experiment both on NIPS corpus and our self-built news collections showed that the system could meet the given demand,the result was feasible.

Dirichlet process and its developments: a survey

The core of the nonparametric/semiparametric Bayesian analysis is to relax the particular parametric assumptions on the distributions of interest to be unknown and random,and assign them a prior.Selecting a suitable prior therefore is especially critical in the nonparametric Bayesian fitting.As the distribution of distribution,Dirichlet process(DP)is the most appreciated nonparametric prior due to its nice theoretical proprieties,modeling flexibility and computational feasibility.In this paper,we review and summarize some developments of DP during the past decades.Our focus is mainly concentrated upon its theoretical properties,various extensions,statistical modeling and applications to the latent variable models.

Hierarchical topic modeling with nested hierarchical Dirichlet process

This paper deals with the statistical modeling of latent topic hierarchies in text corpora. The height of the topic tree is assumed as fixed, while the number of topics on each level as unknown a priori and to be inferred from data. Taking a nonpara-metric Bayesian approach to this problem, we propose a new probabilistic generative model based on the nested hierarchical Dirichlet process (nHDP) and present a Markov chain Monte Carlo sampling algorithm for the inference of the topic tree structure as well as the word distribution of each topic and topic distribution of each document. Our theoretical analysis and experiment results show that this model can produce a more compact hierarchical topic structure and captures more fine-grained topic rela-tionships compared to the hierarchical latent Dirichlet allocation model.

Effective Frameworks Based on Infinite Mixture Model for Real-World Applications

Interest in automated data classification and identification systems has increased over the past years in conjunction with the high demand for artificial intelligence and security applications.In particular,recognizing human activities with accurate results have become a topic of high interest.Although the current tools have reached remarkable successes,it is still a challenging problem due to various uncontrolled environments and conditions.In this paper two statistical frameworks based on nonparametric hierarchical Bayesian models and Gamma distribution are proposed to solve some realworld applications.In particular,two nonparametric hierarchical Bayesian models based on Dirichlet process and Pitman-Yor process are developed.These models are then applied to address the problem of modelling grouped data where observations are organized into groups and these groups are statistically linked by sharing mixture components.The choice of the Gamma mixtures is motivated by its flexibility for modelling heavy-tailed distributions.In addition,deploying the Dirichlet process prior is justified by its advantage of automatically finding the right number of components and providing nice properties.Moreover,a learning step via variational Bayesian setting is presented in a flexible way.The priors over the parameters are selected appropriately and the posteriors are approximated effectively in a closed form.Experimental results based on a real-life applications that concerns texture classification and human actions recognition show the capabilities and effectiveness of the proposed framework.

基于狄利克雷过程混合模型的内外先验融合

近年来,使用高斯混合模型作为块先验的贝叶斯方法取得了优秀的图像复原性能,针对这类模型分量固定及主要依赖外部学习的缺点,提出了一种新的基于狄利克雷过程混合模型的图像先验模型。该模型从干净图像数据库中学习外部通用先验,从退化图像中学习内部先验,借助模型中统计量的可累加性自然实现内外部先验融合。通过聚类的新增及归并机制,模型的复杂度随着数据的增大或缩小而自适应地变化,可以学习到可解释及紧凑的模型。为了求解所有隐变量的变分后验分布,提出了一种结合新增及归并机制的批次更新可扩展变分算法,解决了传统坐标上升算法在大数据集下效率较低、容易陷入局部最优解的问题。在图像去噪及填充实验中,相比传统方法,所提模型无论在客观质量评价还是视觉观感上都更有优势,验证了该模型的有效性。

Predictive Analysis of Microarray Data

Microarray gene expression data are analyzed by means of a Bayesian nonparametric model, with emphasis on prediction of future observables, yielding a method for selection of differentially expressed genes and the corresponding classifier.

The Dirichlet Problem of a Discontinuous Markov Process

Given a Markov process satisfying certain general type conditions,whose paths are notassumed to be continuous. Let D by an open subset of the state space E. Any bounded function defined on thecomplement of D extends to be a function on E (?)uch that it is harmonic in D and satisfies the Dirichletboundary condition at any regular boundary point of D. The relation between harmonic functions and theebaracteristic operator of the given process is discussed.

Gamma-Dirichlet algebra and applications

The Gamma-Dirichlet algebra corresponds to the decomposition of the gamma process into the independent product of a gamma random variable and a Dirichlet process. This structure allows us to study the properties of the Dirichlet process through the gamma process and vice versa. In this article, we begin with a brief survey of several existing results concerning this structure. New results are then obtained for the large deviations of the jump sizes of the gamma process and the quasi-invariance of the two-parameter Poisson-Dirichlet distribution. We finish the paper with the derivation of the transition function of the Fleming-Viot process with parent independent mutation from the transition function of the measure-valued branching diffusion with immigration by exploring the Gamma-Dirichlet algebra embedded in these processes. This last result is motivated by an open R. C. Gritfiths. problem proposed by S. N. Ethier and

Heterogeneous clustering via adversarial deep Bayesian generative model

This paper aims to study the deep clustering problem with heterogeneous features and unknown cluster number.To address this issue,a novel deep Bayesian clustering framework is proposed.In particular,a heterogeneous feature metric is first constructed to measure the similarity between different types of features.Then,a feature metric-restricted hierarchical sample generation process is established,in which sample with heterogeneous features is clustered by generating it from a similarity constraint hidden space.When estimating the model parameters and posterior probability,the corresponding variational inference algorithm is derived and implemented.To verify our model capability,we demonstrate our model on the synthetic dataset and show the superiority of the proposed method on some real datasets.Our source code is released on the website:Github.com/yexlwh/Heterogeneousclustering.

基于MCMC抽样的金融贝叶斯半参数GARCH模型研究

GARCH模型是研究金融资产收益的重要模型,然而现有参数GARCH模型依然不能有效刻画金融资产收益偏态厚尾特性且存在模型设定风险。本文在非参数分布和GARCH模型基础上,建立半参数GARCH模型以提高模型的有效性;同时在贝叶斯框架内发展有效MCMC抽样解决模型的参数估计难问题,并利用DIC4研究模型比较问题;最后通过模拟研究和实证研究考察MCMC抽样的有效性,检验半参数GARCH模型在刻画金融资产收益特性和风险价值预测方面的实际效果。

半参数贝叶斯分层分位回归模型及其在保险公司成本分析中的应用

本文建立了一种半参数贝叶斯分层分位回归模型,并基于美国NAIC提供的多个保险公司连续多年期的非平衡纵向成本观测数据进行了实证分析.本文主要贡献包括三个方面:一是首次在有限正态混合误差假定下,对具有右偏厚尾性的成本数据建立半参数分层分位回归模型,并考虑到保险公司的聚类性,选用狄利克雷过程先验进行模型非参数部分的估计,进一步推广了分位回归模型在保险精算领域中的应用;二是通过模拟数据研究,系统比较了在非对称拉普拉斯误差假定下和有限正态混合误差假定下,半参数分层分位回归模型对复杂数据的拟合精度及参数估计的精确性,结果表明,有限正态混合误差更能充分捕捉数据的复杂性;三是通过实际观测的保险公司成本数据进行分析,选出了对成本具有较强效应的解释变量,并发现在不同分位数水平下各个解释变量对响应变量的效应具有较大区别.

评价多水平因子分析模型的异质性:半参数贝叶斯方法

数据的异质性因相关的解释变量被排除在既定模型.然而,目前经典统计方法难以处理的异质参数的依赖关系或"结".为了解决非均质参数之间的依赖关系,本文基于多水平因子分析模型提出一个半参数贝叶斯分析程序.对于模型的截距和/或协方差结构参数的分布赋予折断的Dirichlet过程先验.在贝叶斯马尔可夫链蒙特卡罗框架内,分块Gibbs抽样器被用来执行后验分析.统计推断的基础上进行这些观察的经验分布.仿真研究表明,忽略了非均质参数之间的联系会导致不变参数估计产生严重的偏差.