Radar HRRP statistical recognition with temporal factor analysis by automatic Bayesian Ying-Yang harmony learning

Radar high-resolution range profiles(HRRPs)are typical high-dimensional and interdimension dependently distributed data,the statistical modeling of which is a challenging task for HRRP-based target recognition.Supposing that HRRP samples are independent and jointly Gaussian distributed,a recent work[Du L,Liu H W,Bao Z.IEEE Transactions on Signal Processing,2008,56(5):1931–1944]applied factor analysis(FA)to model HRRP data with a two-phase approach for model selection,which achieved satisfactory recognition performance.The theoretical analysis and experimental results reveal that there exists high temporal correlation among adjacent HRRPs.This paper is thus motivated to model the spatial and temporal structure of HRRP data simultaneously by employing temporal factor analysis(TFA)model.For a limited size of high-dimensional HRRP data,the two-phase approach for parameter learning and model selection suffers from intensive computation burden and deteriorated evaluation.To tackle these problems,this work adopts the Bayesian Ying-Yang(BYY)harmony learning that has automatic model selection ability during parameter learning.Experimental results show stepwise improved recognition and rejection performances from the twophase learning based FA,to the two-phase learning based TFA and to the BYY harmony learning based TFA with automatic model selection.In addition,adding many extra free parameters to the classic FA model and thus becoming even worse in identifiability,the model of a general linear dynamical system is even inferior to the classic FA model.

Optimization of fuzzy CMAC using evolutionary Bayesian Ying-Yang learning

Cerebellar model articulation controller(CMAC)is a popular associative memory neural network that imitates human’s cerebellum,which allows it to learn fast and carry out local generalization efficiently.This research aims to integrate evolutionary computation into fuzzy CMAC Bayesian Ying-Yang(FCMACBYY)learning,which is referred to as FCMAC-EBYY,to achieve a synergetic development in the search for optimal fuzzy sets and connection weights.Traditional evolutionary approaches are limited to small populations of short binary string length and as such are not suitable for neural network training,which involves a large searching space due to complex connections as well as real values.The methodology employed by FCMACEBYY is coevolution,in which a complex solution is decomposed into some pieces to be optimized in different populations/species and then assembled.The developed FCMAC-EBYY is compared with various neuro-fuzzy systems using a real application of traffic flow prediction.

On essential topics of BYY harmony learning： Current status, challenging issues, and gene analysis applications

As a supplementary of [Xu L. Front. Electr. Electron. Eng. China, 2010, 5（3）： 281-328], this paper outlines current status of efforts made on Bayesian Ying- Yang （BYY） harmony learning, plus gene analysis appli- cations. At the beginning, a bird＇s-eye view is provided via Gaussian mixture in comparison with typical learn- ing algorithms and model selection criteria. Particularly, semi-supervised learning is covered simply via choosing a scalar parameter. Then, essential topics and demand- ing issues about BYY system design and BYY harmony learning are systematically outlined, with a modern per- spective on Yin-Yang viewpoint discussed, another Yang factorization addressed, and coordinations across and within Ying-Yang summarized. The BYY system acts as a unified framework to accommodate unsupervised, su- pervised, and semi-supervised learning all in one formu- lation, while the best harmony learning provides novelty and strength to automatic model selection. Also, mathe- matical formulation of harmony functional has been ad- dressed as a unified scheme for measuring the proximity to be considered in a BYY system, and used as the best choice among others. Moreover, efforts are made on a number of learning tasks, including a mode-switching factor analysis proposed as a semi-blind learning frame- work for several types of independent factor analysis, a hidden Markov model （HMM） gated temporal fac- tor analysis suggested for modeling piecewise stationary temporal dependence, and a two-level hierarchical Gaus- sian mixture extended to cover semi-supervised learning, as well as a manifold learning modified to facilitate au- tomatic model selection. Finally, studies are applied to the problems of gene analysis, such as genome-wide asso- ciation, exome sequencing analysis, and gene transcrip- tional regulation.

IA-pix2seq:一个实现简笔画可控生成的深度双向学习方法

构建一个高斯混合模型(GMM)分布的编码空间是一种可辅助实现简笔画可控生成的编码方法.每种特定风格和类别的简笔画经过编码,被集中投影到GMM中的一个高斯区域中.通过选取不同高斯中的编码,可以可控地生成具有指定特征的简笔画.然而,现有方法在处理形态相似的简笔画时,所构建的GMM空间中,高斯区域间存在较大重叠.这降低了简笔画生成符合预期特征的准确率,即可控生成性能较差.本文以贝叶斯阴阳和谐学习算法为指导提出了IA-pix2seq深度双向学习模型.模型的双向互逆映射在和谐学习原理指导下,以最默契的方式达到最大共识,将同一高斯成分区域内的编码集中到相应的高斯中心,同时进一步约束了各简笔画在编码空间中的投影范围,从而扩大高斯成分间的边界并降低彼此间的重叠率.实验表明IA-pix2seq能有效降低不同类别简笔画因相似造成的编码重叠,以提高简笔画的可控生成性能.给定插值编码、将含像素缺失的简笔画作为约束,模型生成的简笔画仍能保留更多的预期特征.

Learning Gaussian mixture with automatic model selection:A comparative study on three Bayesian related approaches

Three Bayesian related approaches,namely,variational Bayesian(VB),minimum message length(MML)and Bayesian Ying-Yang(BYY)harmony learning,have been applied to automatically determining an appropriate number of components during learning Gaussian mixture model(GMM).This paper aims to provide a comparative investigation on these approaches with not only a Jeffreys prior but also a conjugate Dirichlet-Normal-Wishart(DNW)prior on GMM.In addition to adopting the existing algorithms either directly or with some modifications,the algorithm for VB with Jeffreys prior and the algorithm for BYY with DNW prior are developed in this paper to fill the missing gap.The performances of automatic model selection are evaluated through extensive experiments,with several empirical findings:1)Considering priors merely on the mixing weights,each of three approaches makes biased mistakes,while considering priors on all the parameters of GMM makes each approach reduce its bias and also improve its performance.2)As Jeffreys prior is replaced by the DNW prior,all the three approaches improve their performances.Moreover,Jeffreys prior makes MML slightly better than VB,while the DNW prior makes VB better than MML.3)As the hyperparameters of DNW prior are further optimized by each of its own learning principle,BYY improves its performances while VB and MML deteriorate their performances when there are too many free hyper-parameters.Actually,VB and MML lack a good guide for optimizing the hyper-parameters of DNW prior.4)BYY considerably outperforms both VB and MML for any type of priors and whether hyper-parameters are optimized.Being different from VB and MML that rely on appropriate priors to perform model selection,BYY does not highly depend on the type of priors.It has model selection ability even without priors and performs already very well with Jeffreys prior,and incrementally improves as Jeffreys prior is replaced by the DNW prior.Finally,all algorithms are applied on the Berkeley segmentation database of real world images.Again,BYY considerably outperforms both VB and MML,especially in detecting the objects of interest from a confusing background.

Codimensional matrix pairing perspective of BYY harmony learning:hierarchy of bilinear systems,joint decomposition of data-covariance,and applications of network biology

One paper in a preceding issue of this journal has introduced the Bayesian Ying-Yang(BYY)harmony learning from a perspective of problem solving,parameter learning,and model selection.In a complementary role,the paper provides further insights from another perspective that a co-dimensional matrix pair(shortly co-dim matrix pair)forms a building unit and a hierarchy of such building units sets up the BYY system.The BYY harmony learning is re-examined via exploring the nature of a co-dim matrix pair,which leads to improved learning performance with refined model selection criteria and a modified mechanism that coordinates automatic model selection and sparse learning.Besides updating typical algorithms of factor analysis(FA),binary FA(BFA),binary matrix factorization(BMF),and nonnegative matrix factorization(NMF)to share such a mechanism,we are also led to(a)a new parametrization that embeds a de-noise nature to Gaussian mixture and local FA(LFA);(b)an alternative formulation of graph Laplacian based linear manifold learning;(c)a codecomposition of data and covariance for learning regularization and data integration;and(d)a co-dim matrix pair based generalization of temporal FA and state space model.Moreover,with help of a co-dim matrix pair in Hadamard product,we are led to a semi-supervised formation for regression analysis and a semi-blind learning formation for temporal FA and state space model.Furthermore,we address that these advances provide with new tools for network biology studies,including learning transcriptional regulatory,Protein-Protein Interaction network alignment,and network integration.

Discriminative training of GMM-HMM acoustic model by RPCL learning

This paper presents a new discriminative approach for training Gaussian mixture models(GMMs)of hidden Markov models(HMMs)based acoustic model in a large vocabulary continuous speech recognition(LVCSR)system.This approach is featured by embedding a rival penalized competitive learning(RPCL)mechanism on the level of hidden Markov states.For every input,the correct identity state,called winner and obtained by the Viterbi force alignment,is enhanced to describe this input while its most competitive rival is penalized by de-learning,which makes GMMs-based states become more discriminative.Without the extensive computing burden required by typical discriminative learning methods for one-pass recognition of the training set,the new approach saves computing costs considerably.Experiments show that the proposed method has a good convergence with better performances than the classical maximum likelihood estimation(MLE)based method.Comparing with two conventional discriminative methods,the proposed method demonstrates improved generalization ability,especially when the test set is not well matched with the training set.

有限混合体模型上的自动模型选择:一种崭新的数据建模方式(英文)

在数据建模和分析中,有限混合体模型被广泛地使用着。然而,如何仅仅针对一组来自于某个有限混合体模型的数据选择出分量或聚类的个数则依然是一个非常困难的问题。由于分量个数是混合体模型的规模度量,其选择问题被称为有限混合体的模型选择问题。最近,针对有限混合体模型,特别是高斯混合模型,一种自动模型选择学习机制逐步发展成熟起来。这种新的机制能够在学习参数的过程中自动地完成模型选择,为数据的建模与分析提供了一种新的思路与途径。本文将对于高斯混合模型或一般有限混合体模型的自动模型选择学习算法及其典型应用进行综述与总结。首先,我们综述了基于贝叶斯阴阳机和谐学习原则的自动模型选择学习算法。然后,我们描述了另一种基于熵惩罚的自动模型选择学习算法。最后,我们给出了自动模型选择学习算法的一些典型的应用。

基于贝叶斯和谐度的层次聚类

层次聚类是一种重要的数据分析技术.传统的层次聚类方法大都采用欧式距离度量类之间相似度,不能有效处理类之间重合和类密度变化大的情况.文中提出一种基于贝叶斯和谐度的层次聚类方法,采用和谐度增幅代替传统层次聚类方法采用的欧式距离.贝叶斯和谐度取自于贝叶斯阴阳和谐学习理论,能衡量整个数据的分布情况和指导选择合适的类别数.文中方法根据和谐度的变化来度量类之间的相似度,能克服传统层次聚类的缺点;同时更易选择阈值终止层次聚类的合并,从而产生合适的类别数.最后通过两个实验验证文中方法的有效性.

采用概率混合模型的圆周曲线识别方法

曲线识别是图像识别和机器视觉的一个重要研究课题。建立圆周曲线的概率混合模型,并分别利用EM算法和贝叶斯阴阳和谐学习(BYY)算法,在曲线条数已知和未知的情况下实现模型选择和参数估计,从而完成对圆周曲线的识别以及数据点的聚类。试验结果表明:用这两种算法处理平面曲线的混合模型可以准确地估计出曲线条数并同时完成参数估计,较好地完成曲线识别。

Adaptive Electric Load Forecaster

In this paper, a methodology, Self-Developing and Self-Adaptive Fuzzy Neural Networks using Type-2 Fuzzy Bayesian Ying-Yang Learning （SDSA-FNN-T2FBYYL） algorithm and multi-objective optimization is proposed. The features of this methodology are as follows： （1） A Bayesian Ying-Yang Learning （BYYL） algorithm is used to construct a compact but high-performance system automatically. （2） A novel multi-objective T2FBYYL is presented that integrates the T2 fuzzy theory with BYYL to automatically construct its best structure and better tackle various data uncertainty problems simultaneously. （3） The weighted sum multi-objective optimization technique with combinations of different weightings is implemented to achieve the best trade-off among multiple objectives in the T2FBYYL. The proposed methods are applied to electric load forecast using a real operational dataset collected from Macao electric utility. The test results reveal that the proposed method is superior to other existing relevant techniques.