Mutual Information-Based Modified Randomized Weights Neural Networks

Randomized weights neural networks have fast learning speed and good generalization performance with one single hidden layer structure. Input weighs of the hidden layer are produced randomly. By employing certain activation function, outputs of the hidden layer are calculated with some randomization. Output weights are computed using pseudo inverse. Mutual information can be used to measure mutual dependence of two variables quantitatively based on the probability theory. In this paper, these hidden layer’s outputs that relate to prediction variable closely are selected with the simple mutual information based feature selection method. These hidden nodes with high mutual information values are maintained as a new hidden layer. Thus, the size of the hidden layer is reduced. The new hidden layer’s output weights are learned with the pseudo inverse method. The proposed method is compared with the original randomized algorithms using concrete compressive strength benchmark dataset.

Extending Qualitative Probabilistic Network with Mutual Information Weights

Bayesian network (BN) is a well-accepted framework for representing and inferring uncertain knowledge. As the qualitative abstraction of BN, qualitative probabilistic network (QPN) is introduced for probabilistic inferences in a qualitative way. With much higher efficiency of inferences, QPNs are more suitable for real-time applications than BNs. However, the high abstraction level brings some inference conflicts and tends to pose a major obstacle to their applications. In order to eliminate the inference conflicts of QPN, in this paper, we begin by extending the QPN by adding a mutual-information-based weight (MI weight) to each qualitative influence in the QPN. The extended QPN is called MI-QPN. After obtaining the MI weights from the corresponding BN, we discuss the symmetry, transitivity and composition properties of the qualitative influences. Then we extend the general inference algorithm to implement the conflict-free inferences of MI-QPN. The feasibility of our method is verified by the results of the experiment.

Weighted-averaging-based classification of laser-induced breakdown spectroscopy measurements using most informative spectral lines

In this study,efficient spectral line selection and wcightcd-avcraging-bascd processing schemes are proposed for the classification of laser-induced breakdown spectroscopy(UBS)measurements.For fast on-line classification,a set of representative spectral lines arc selected ami processed relying on the information metric,instead of the time consuming full spectrum based analysis.I he most informative spectral line sets arc investigated by the joint mutual information estimation(MIR)evaluated with the Gaussian kernel density,where dominant intensity peaks associated with the concentrated components arc not necessarily most valuable for classification.In order to further distinguish the characteristic patterns of die LIBS measured spectrum,two-dimensional spectral images are synthesized through column-wise concatenation of the peaks along with their neighbors.For fast classification while preserv ing die effect of distinctive peak patterns,column-wise Gaussian weighted averaging is applied to die synthesized images,yielding a favorable trade off between classification performance and computational complexity.To explore the applicability of the proposed schemes,two applications of alloy classification and skin cancer detection arc investigated with the multi-class and binary support vector machines classifiers,respectively.Ihc MIE measures associated with selected spectral lines in bodi applications show a strong correlation to the actual classification or detection accuracy,which enables to find out meaningful combinations of spectral lines.In addition,the peak patterns of the selected lines and their Gaussian weighted averaging with nciehbors of the selected peaks efficiently distineuish different classes of LIBS measured spectrum.

Mutual-information based weighted fusion for target tracking in underwater wireless sensor networks

Underwater wireless sensor networks(UWSNs)can provide a promising solution to underwater target tracking.Due to limited energy and bandwidth resources,only a small number of nodes are selected to track a target at each interval.Because all measurements are fused together to provide information in a fusion center,fusion weights of all selected nodes may affect the performance of target tracking.As far as we know,almost all existing tracking schemes neglect this problem.We study a weighted fusion scheme for target tracking in UWSNs.First,because the mutual information(MI)between a node’s measurement and the target state can quantify target information provided by the node,it is calculated to determine proper fusion weights.Second,we design a novel multi-sensor weighted particle filter(MSWPF)using fusion weights determined by MI.Third,we present a local node selection scheme based on posterior Cramer-Rao lower bound(PCRLB)to improve tracking efficiency.Finally,simulation results are presented to verify the performance improvement of our scheme with proper fusion weights.

Spatial weight matrix in dimensionality reduction reconstruction for microelectromechanical system-based photoacoustic microscopy

A micro-electromechanical system(MEMS)scanning mirror accelerates the raster scanning of optical-resolution photoacoustic microscopy(OR-PAM).However,the nonlinear tilt angular-voltage characteristic of a MEMS mirror introduces distortion into the maximum back-projection image.Moreover,the size of the airy disk,ultrasonic sensor properties,and thermal effects decrease the resolution.Thus,in this study,we proposed a spatial weight matrix(SWM)with a dimensionality reduction for image reconstruction.The three-layer SWM contains the invariable information of the system,which includes a spatial dependent distortion correction and 3D deconvolution.We employed an ordinal-valued Markov random field and the Harris Stephen algorithm,as well as a modified delay-and-sum method during a time reversal.The results from the experiments and a quantitative analysis demonstrate that images can be effectively reconstructed using an SWM;this is also true for severely distorted images.The index of the mutual information between the reference images and registered images was 70.33 times higher than the initial index,on average.Moreover,the peak signal-to-noise ratio was increased by 17.08%after 3D deconvolution.This accomplishment offers a practical approach to image reconstruction and a promising method to achieve a real-time distortion correction for MEMS-based OR-PAM.

Low-Complexity Rate Compatible Modulation with Hybrid Weight Set

Rate Compatible Modulation(RCM)is an efficient technique for high spectral efficiency seamless transmission over highly dynamic wireless channels.However,its high decoding complexity at the receiver prevents it from being applied in scenarios where computation resources are limited.To alleviate this problem,an RCM with variable weight sets(RCM-VWS)was presented in the literature to significantly reduce its complexity by employing weight sets of different complexities for channels at different signal-tonoise-ratio(SNR).However,RCM-VWS has to introduce an undesired feedback channel for the transmission of SNR information that is estimated at the receiver and transmitted back to the transmitter for weight set selection.To achieve a low computational complexity while avoiding feedback transmission at the same time,a novel RCM scheme with hybrid weight set(RCM-HWS)is introduced in this paper.A low complexity of decoding is allowed by gradually reducing the complexity of weight sets based on the number symbols already sent.It also avoids the feedback of SNRs since we can deduce SNRs from the number of the symbols transmitted and use a hybrid weight set we have designed.The theoretical analysisand simulation results show that the proposed scheme has the advantages of low demodulation complexity and not requiring feedback channel while maintaining the same transmission throughput as that of the conventional RCM.Therefore,the proposed scheme has a wider range of applications,especially in the case that feedback channel is not available.

采用动态相关度权重的特征选择算法

基于互信息的特征选择算法在考虑候选特征提供的新分类信息时,通常忽略了候选特征的加入会使得已选特征和类标签的相关性发生变化而带来额外的新增信息量,以及在计算冗余信息时采用累加求和的形式可能导致低估候选特征的冗余程度。针对以上问题,提出动态相关度权重的定义,以更全面地考虑候选特征带来的新信息量成分;提出改进冗余项的定义,采用取最大值和归一化策略,以解决传统算法存在的低估冗余问题;在此基础上提出一种采用动态相关度权重的特征选择算法(feature selection using dynamic relevance weight,FSDRW)。选取五种当前主流的基于互信息的过滤式特征选择算法进行对比实验,在来自加州大学尔湾分校UCI和亚利桑那州立大学ASU的机器学习测试数据集上的实验表明,所提出的算法在分类准确率及综合性能方面具有较好的表现。最后将所提出算法应用于广西某水库工程的微震、爆破信号识别中,算法选取出的特征用于微震信号识别可达到98.86%的分类准确率,验证了算法在实际应用中的有效性。

基于邻域互信息与K-means特征聚类的特征选择

针对多数邻域系统通过人工调试很难搜索到最佳邻域半径,以及传统的K-means聚类需要随机选取簇中心和指定簇的数目等问题,提出了一种基于邻域互信息与K-means特征聚类的特征选择方法。首先,将样本在各特征下与其他样本距离的平均值作为自适应邻域半径,确定样本的邻域集,并由此构建自适应邻域熵、邻域互信息、归一化邻域互信息等度量,反映特征之间的相关性;然后,基于归一化邻域互信息构建自适应K近邻集合,利用Pearson相关系数表示特征的权重定义加权K近邻密度,实现自动选取K-means算法的簇中心,进而完成K-means特征聚类;最后,给出加权平均冗余度,选出每个特征簇中加权平均冗余度最大的特征构成最优特征子集。实验结果表明所提算法不仅可以有效提升特征选择的分类结果而且可以获得更好的聚类效果。

基于课程学习权重集成的贝叶斯结构学习算法研究

从大量复杂的数据中学习贝叶斯网络(Bayesian network,BN)一直是一个难点问题,本文借鉴课程学习的思想,引入了一种适合于BN中节点之间互相影响程度的测量,然后划分课程阶段,分阶段构造无向图骨架,并利用优化函数对骨架进行优化;通过集成策略,将各个集成学习结果所得到的课程权重进行集合,并通过边过滤来减少错误边的出现;最后,通过爬山搜索构建BN结构。实验结果表明,在4个标准数据集上,本文所提方法具有较高的精确度和稳定性。与多种传统贝叶斯结构学习(Bayesian network structure learning,BNSL)方法相比,本文所提方法性能平均提高了37.18%。本文分析结果可为BNSL的增量学习过程进一步提供参考。

基于互信息的通信网络节点重要性度量方法

信息化条件下的复杂网络对节点的蓄意攻击非常脆弱,因此准确发掘出网络中的核心节点并进行重点保护对提高网络抗毁性至关重要.在分析特殊条件下通信网络特征属性的基础上,借鉴通信系统中关于"信息量"的定义方法,提出了改进的适用于有向加权网络的节点重要性评估方法.在小规模混合加权网络中对该方法和已有方法进行了对比分析,验证了本文方法的有效性和优势性.构建了一种基于BBV(Barrat-Barthelemy-Vespignani)的混合加权网络演化模型,并对生成的大规模通信网络进行了节点重要性评估仿真,实验结果表明:与现有评估方法相比,本文方法能够更加简单、有效地评估网络节点的重要性.

改进的加权网络节点重要性评估的互信息方法

现有的复杂网络节点重要性评估研究主要集中在无向无权网络上,不能全面客观反映某些真实复杂网络的情况。针对无向加权和有向加权网络中评估指标适用范围有限、评估结果不够全面等问题,借鉴应用于无向无权网络的基于互信息的节点重要性评估方法,提出适用于无向加权网络和有向加权网络的互信息评估方法。该方法将网络中的每条边看作信息流,结合相应复杂网络的结构特点和"信息量"的定义方法,以求出的节点信息量作为节点的重要性评估指标。对实例网络进行分析可知,所提算法在保证评估准确性前提下,能更加细致刻画有向加权网络节点之间的差异性。在对ARPA网络的节点评估中,所提算法与以往指标所评估出的前5个最重要节点的节点编号尤其相近,凸显出该算法快速发掘核心节点的能力,为快速、准确评估无向加权和有向加权网络核心节点,提高网络抗毁性提供一定理论帮助。

基于互信息的粒化特征加权多标签学习k近邻算法

传统基于k近邻的多标签学习算法,在寻找近邻度量样本间的距离时,对所有特征给予同等的重要度.这些算法大多采用分解策略,对单个标签独立预测,忽略了标签间的相关性.多标签学习算法的分类效果跟输入的特征有很大的关系,不同的特征含有的标签分类信息不同,故不同特征的重要度也不同.互信息是常用的度量2个变量间关联度的重要方法之一,能够有效度量特征含有标签分类的知识量.因此,根据特征含有标签分类知识量的大小,赋予相应的权重系数,提出一种基于互信息的粒化特征加权多标签学习k近邻算法(granular feature weighted k-nearest neighbors algorithm for multi-label learning,GFWML-kNN),该算法将标签空间粒化成多个标签粒,对每个标签粒计算特征的权重系数,以解决上述问题和标签组合爆炸问题.在计算特征权重时,考虑到了标签间可能的组合,把标签间的相关性融合进特征的权重系数.实验表明:相较于若干经典的多标签学习算法,所提算法GFWML-kNN整体上能取得较好的效果.

权重融合深度图像与骨骼关键帧的行为识别

针对2D信息量不足导致人体行为识别率不高的问题,提出融合多种深度信息的行为识别方法.首先利用深度图像捕捉行为线索,提取梯度及相关方向特征;然后利用互信息提取骨骼图的关键帧,提出基于关键帧的静态姿态模型、当前运动模型和动态偏移模型表征人体行为底层特征;最后通过权重投票机制为不同种类特征分配权重,实现多类特征下的多权重融合.在MSR_Action3D深度数据集上的实验结果表明,该方法的识别率比其他方法提高1.5%.

Web文档中词语权重计算方法的改进

以向量空间模型作为Web文本的表示方法,对传统的TF*IDF公式进行了改进。首先,结合Web文本中HTML标签的修饰功能,体现了特征词在Web文本结构中的位置信息;其次,以广义信息论为理论基础,引入了基于二次熵的互信息作为权重计算公式的一项,体现了单词的类区分能力。实验验证了该方法的可行性和有效性。

基于改进随机森林算法的工业过程运行状态评价

运行状态评价是指在过程正常生产的前提下,进一步判断生产过程运行状态的优劣.针对复杂工业过程定量信息与定性信息共存的情况,本文提出了一种基于随机森林的工业过程运行状态评价方法.针对随机森林中决策树信息存在冗余的问题,基于互信息将传统随机森林中的决策树进行分组,并选出每组中最优的决策树组成新的随机森林.同时为了强化评价精度高的决策树和弱化评价精度低的决策树对最终评价结果的影响,使用加权投票机制取代传统众数投票方法,最终构成一种基于互信息的加权随机森林算法(Mutual information weighted random forest,MIWRF).对于在线评价,本文通过计算在线数据处于各个等级的概率,并且结合提出的在线评价策略,判定当前样本运行状态等级.为了验证所提算法的有效性,将所提方法应用于湿法冶金浸出过程,实验结果表明,相对于传统随机森林算法,MIWRF降低了模型的复杂度,同时提高了运行状态评价精度.

基于粒子群优化的朴素贝叶斯改进算法

针对朴素贝叶斯(NB)算法因条件独立性的理想式假设引起分类性能降低的问题,提出一种改进的粒子群优化-朴素贝叶斯(PSO-NB)算法。在文本预处理时,引入权重因子、类内和类间离散因子进行属性约简,基于NB加权模型,将条件属性的词频比率作为其初始权值,利用PSO算法迭代寻找全局最优特征权向量,并以此权向量作为加权模型中各个特征词的权值生成分类器。运用经典数据集对PSO-NB算法进行性能分析,结果表明,改进算法可有效减少冗余属性,降低计算复杂度,具有较高的准确率和召回率。

基于特征加权的多关系朴素贝叶斯分类模型

为进一步提高多关系朴素贝叶斯方法的分类准确率,分析了已有的特征加权方法,并在将特征加权方法扩展到多关系的情况下结合元组ID传播方法和面向元组的统计计数方法,建立了基于特征加权的多关系朴素贝叶斯分类模型(MRNBC-W)。标准数据集上的实验结果显示,新方法可以在不增加算法时间复杂度的前提下,有效提高金融数据集的分类准确率。文中也给出了结合扩展互信息标准对属性进行过滤后,加权方法和不加权方法的分类比较。

基于互信息的文本特征加权方法

特征加权是文本分类中的重要环节,通过考察传统的特征选择函数,发现互信息方法在特征加权过程中表现尤为突出。为了提高互信息方法在特征加权时的性能,加入了词频信息、文档频率信息以及类别相关度因子,提出了一种基于改进的互信息特征加权方法。实验结果表明,该方法比传统的特征加权方法具有更好的分类性能。

连续属性朴素贝叶斯分类器的依赖扩展研究

针对朴素贝叶斯分类器不能有效利用属性之间依赖信息的问题,在将连续属性条件互信息计算、条件密度计算与通过建立类约束属性最大权重跨度树的父结点选择相结合的基础上,提出了连续属性朴素贝叶斯分类器选择性树结构依赖扩展方法.通过对比实验和分析,证实了扩展后分类器的分类准确率得到明显的改进.

基于互信息的加权朴素贝叶斯文本分类算法

文本分类是信息检索和文本挖掘的重要基础,朴素贝叶斯是一种简单而高效的分类算法,可以应用于文本分类.但是其属性独立性和属性重要性相等的假设并不符合客观实际,这也影响了它的分类效果.如何克服这种假设,进一步提高其分类效果是朴素贝叶斯文本分类算法的一个难题.根据文本分类的特点,基于文本互信息的相关理论,提出了基于互信息的特征项加权朴素贝叶斯文本分类方法,该方法使用互信息对不同类别中的特征项进行分别赋权,部分消除了假设对分类效果的影响.通过在UCIKDD数据集上的仿真实验,验证了该方法的有效性.