Multimodal Medical Image Fusion Based on Parameter Adaptive PCNN and Latent Low-rank Representation

Medical image fusion has been developed as an efficient assistive technology in various clinical applications such as medical diagnosis and treatment planning.Aiming at the problem of insufficient protection of image contour and detail information by traditional image fusion methods,a new multimodal medical image fusion method is proposed.This method first uses non-subsampled shearlet transform to decompose the source image to obtain high and low frequency subband coefficients,then uses the latent low rank representation algorithm to fuse the low frequency subband coefficients,and applies the improved PAPCNN algorithm to fuse the high frequency subband coefficients.Finally,based on the automatic setting of parameters,the optimization method configuration of the time decay factorαe is carried out.The experimental results show that the proposed method solves the problems of difficult parameter setting and insufficient detail protection ability in traditional PCNN algorithm fusion images,and at the same time,it has achieved great improvement in visual quality and objective evaluation indicators.

Learning Dual-Layer User Representation for Enhanced Item Recommendation

User representation learning is crucial for capturing different user preferences,but it is also critical challenging because user intentions are latent and dispersed in complex and different patterns of user-generated data,and thus cannot be measured directly.Text-based data models can learn user representations by mining latent semantics,which is beneficial to enhancing the semantic function of user representations.However,these technologies only extract common features in historical records and cannot represent changes in user intentions.However,sequential feature can express the user’s interests and intentions that change time by time.But the sequential recommendation results based on the user representation of the item lack the interpretability of preference factors.To address these issues,we propose in this paper a novel model with Dual-Layer User Representation,named DLUR,where the user’s intention is learned based on two different layer representations.Specifically,the latent semantic layer adds an interactive layer based on Transformer to extract keywords and key sentences in the text and serve as a basis for interpretation.The sequence layer uses the Transformer model to encode the user’s preference intention to clarify changes in the user’s intention.Therefore,this dual-layer user mode is more comprehensive than a single text mode or sequence mode and can effectually improve the performance of recommendations.Our extensive experiments on five benchmark datasets demonstrate DLUR’s performance over state-of-the-art recommendation models.In addition,DLUR’s ability to explain recommendation results is also demonstrated through some specific cases.

Low-Rank and Sparse Representation with Adaptive Neighborhood Regularization for Hyperspectral Image Classification

Low-Rank and Sparse Representation(LRSR)method has gained popularity in Hyperspectral Image(HSI)processing.However,existing LRSR models rarely exploited spectral-spatial classification of HSI.In this paper,we proposed a novel Low-Rank and Sparse Representation with Adaptive Neighborhood Regularization(LRSR-ANR)method for HSI classification.In the proposed method,we first represent the hyperspectral data via LRSR since it combines both sparsity and low-rankness to maintain global and local data structures simultaneously.The LRSR is optimized by using a mixed Gauss-Seidel and Jacobian Alternating Direction Method of Multipliers(M-ADMM),which converges faster than ADMM.Then to incorporate the spatial information,an ANR scheme is designed by combining Euclidean and Cosine distance metrics to reduce the mixed pixels within a neighborhood.Lastly,the predicted labels are determined by jointly considering the homogeneous pixels in the classification rule of the minimum reconstruction error.Experimental results based on three popular hyperspectral images demonstrate that the proposed method outperforms other related methods in terms of classification accuracy and generalization performance.

Extraction method of typical IEQ spatial distributions based on low-rank sparse representation and multi-step clustering

Indoor environment quality(IEQ)is one of the most concerned building performances during the operation stage.The non-uniform spatial distribution of various IEQ parameters in large-scale public buildings has been demonstrated to be an essential factor affecting occupant comfort and building energy consumption.Currently,IEQ sensors have been widely employed in buildings to monitor thermal,visual,acoustic and air quality.However,there is a lack of effective methods for exploring the typical spatial distribution of indoor environmental quality parameters,which is crucial for assessing and controlling non-uniform indoor environments.In this study,a novel clustering method for extracting IEQ spatial distribution patterns is proposed.Firstly,representation vectors reflecting IEQ distributions in the concerned space are generated based on the low-rank sparse representation.Secondly,a multi-step clustering method,which addressed the problems of the“curse of dimensionality”,is designed to obtain typical IEQ distribution patterns of the entire indoor space.The proposed method was applied to the analysis of indoor thermal environment in Beijing Daxing international airport terminal.As a result,four typical temperature spatial distribution patterns of the terminal were extracted from a four-month monitoring,which had been validated for their good representativeness.These typical patterns revealed typical environmental issues in the terminal,such as long-term localized overheating and temperature increases due to a sudden influx of people.The extracted typical IEQ spatial distribution patterns could assist building operators in effectively assessing the uneven distribution of IEQ space under current environmental conditions,facilitating targeted environmental improvements,optimization of thermal comfort levels,and application of energy-saving measures.

Weighted Sparse Image Classification Based on Low Rank Representation

The conventional sparse representation-based image classification usually codes the samples independently,which will ignore the correlation information existed in the data.Hence,if we can explore the correlation information hidden in the data,the classification result will be improved significantly.To this end,in this paper,a novel weighted supervised spare coding method is proposed to address the image classification problem.The proposed method firstly explores the structural information sufficiently hidden in the data based on the low rank representation.And then,it introduced the extracted structural information to a novel weighted sparse representation model to code the samples in a supervised way.Experimental results show that the proposed method is superiority to many conventional image classification methods.

结构相似度优化的混合多尺度医学图像融合

现有的多模态医学图像融合方法存在结构信息以及相位特征保存不完整的问题,为此,提出一种基于混合多尺度分解和结构相似度优化的医学图像融合方法。首先,针对单一滤波器在保留图像结构和细节方面的局限性,提出一种多尺度分解潜在低秩表示(MDLat LRR)和非下采样轮廓波变换(NSCT)相结合的混合多尺度分解方法,利用MDLat LRR分解源图像获取低秩层和显著层,使用NSCT对低秩层做进一步分解;其次,在基础层上使用基于局部拉普拉斯能量和的融合规则,使融合图像具有更好的视觉效果,对于细节层,通过脉冲耦合神经网络(PCNN)计算全局耦合以获得融合权重,从而融合细节层;最后,考虑到空间一致性,由初始融合图像获取线性调整图像,利用加权局部结构相似度进行测量从而得到修正系数,并对初始融合图像进行修正,提高融合图像中信息的准确性。实验结果表明,相比于MSMG、EMFusion、CFL等9种方法,该方法在归一化互信息、空间频率误差比等10个客观评价指标上评估性能更高,特别在相位一致性、余弦特征互信息以及差异相关和指标上,分别比次优方法平均提升了13.89%、19.62%和35.8%,所提方法的融合图像具有更丰富、更准确的细节信息和良好的视觉效果。

基于分层潜在语义驱动网络的事件检测

事件检测旨在检测句子中的触发词并将其分类为预定义的事件类型.如何有效地表示触发词是实现该任务的核心要素.目前基于表示的方法通过复杂的深度神经网络来学习候选触发词的语义表示,以提升模型性能.然而,其忽略了2个问题:1)受句子语境的影响,同一个触发词会触发不同的事件类型;2)受自然语言表达多样性的影响,不同的触发词会触发同一个事件类型.受变分自编码器中隐变量及其他自然语言处理(natural language processing,NLP)任务中分层结构的启发,提出基于分层潜在语义驱动网络(hierarchical latent semantic-driven network,HLSD)的事件检测方法,通过句子和单词的潜在语义信息来辅助缓解以上2个问题.模型从文本表示空间中分层降维到新的潜在语义空间,探索事件宏微观语境中更本质的影响信息.首先,通过BERT对候选句子进行编码,得到句子的表示和句子中单词的表示;其次,设计一个双重的潜在语义机制,并采用VAE挖掘句子和单词级潜在语义;最后,从不同粒度的上下文角度,提出采用一个由粗到细的分层结构来充分使用句子和单词的潜在信息,从而提升模型的性能.ACE2005英文语料库上的实验结果表明,所提方法的F1值在事件检测任务上达到了77.9%.此外,在实验部分对以上2个问题进行了定量分析,证明了所提方法的有效性.

解耦表征学习视角下认知图像属性特征的图像生成方法

在生成式人工智能领域,解耦表征学习的研究进一步推动图像生成方法的发展,但现有的解耦方法更多地关注图像生成的低维表示,忽略目标变化图像内在的可解释因素,导致生成的图像容易受到其它不相关属性特征的影响.为此,文中提出解耦表征学习视角下认知图像属性特征的图像生成方法.首先,从生成模型的潜在空间出发,通过训练获得关于目标变化图像的候选遍历方向.然后,构建无监督语义分解策略,并基于候选遍历的方向联合发现嵌入在潜在空间中的可解释方向.最后,利用解耦编码器和对比学习构建对比模拟器和变化空间,进而由可解释方向提取目标变化图像的解耦表征并生成图像.在5个解耦数据集上的实验表明文中方法性能较优.

孪生Swin Transformer的红外与可见光图像融合算法

为解决基于卷积神经网络(convolutional neural networks,CNN)的红外与可见光图像融合算法融合过程中未考虑原图像远程依赖关系的问题,提出了一种孪生Swin Transformer的红外与可见光图像融合算法,构建了孪生网络模型。使用潜在低秩表示(latent low-rank representation,LatLRR)分解方法将原图像分解,再分别融合以提升图像融合的精度;使用Swin Transformer分别提取分解后图像的特征,获取图像的远程依赖关系;使用l1-norm正则化方法求解特征范数,利用Softmax分别获得分解后图像的权重图;利用线性加权方法重构融合图像,通过简单的线性加和获得最终融合图像。在评测基准VIFB上对所提算法进行评测,实验结果表明,所提算法与20种融合算法相比,定性性能优良;定量分析,在由13种评价指标组成的评价体系中,该算法能够取得3个最优值,超过大部分融合算法;在运行时间方面,该算法的运行时间成本低于大部分融合算法。所提融合算法相较于其他融合算法,在主观和客观方面均表现出更好的融合性能。

潜在空间中深度强化学习方法研究综述

深度强化学习(DRL)是实现通用人工智能的一种有效学习范式,已在一系列实际应用中取得了显著成果。然而,DRL存在泛化性能差、样本效率低等问题。基于深度神经网络的表示学习通过学习环境的底层结构,能够有效缓解上述问题。因此,基于潜在空间的深度强化学习成为该领域的主流方法。系统地综述了基于潜在空间的表示学习在深度强化学习中的研究进展,分析并总结了现有基于潜在空间的深度强化学习的方法,将其分为潜在空间中的状态表示、动作表示以及动力学模型进行详细阐述。其中,潜在空间中的状态表示又被分为基于重构方式的状态表示方法、基于互模拟等价的状态表示方法及其他状态表示方法。最后,列举了现有基于潜在空间的强化学习在游戏领域、智能控制领域、推荐领域及其他领域的成功应用,并浅谈了该领域的未来发展趋势。

基于LatLRR与NSP分解的红外与可见光图像融合

针对现有红外与可见光图像融合过程中存在的图像对比度低、红外特征不明显等问题,提出了一种基于非采样金字塔滤波(Nonsubsampled Pyramid,NSP)与潜在低秩表示(Latent Low Rank Representation,LatLRR)分解的红外与可见光图像融合算法。首先,对红外与可见光图像进行分解,采用NSP分解提取源图像的低频信息,LatLRR分解提取源图像的局部结构信息;其次,根据红外低频信息与可见光低频信息的特征及融合结果图像中低频分量占比,利用红外像素强度权重调控策略完成对低频信息的融合,同时,为使红外与可见光的局部结构信息在融合时保持均衡,使用基于像素灰度值求和的策略进行1∶1融合;最后,图像重构中引入非线性变换思想,使局部结构信息与低频信息有更加完美的契合。实验结果表明,融合结果图像在极大保留红外特征的同时又能兼顾可见光图像中的细节信息,该算法能够对红外与可见光图像进行有效融合。

潜在表示学习框架下的低冗余多视图聚类算法

基于潜在空间学习的多视图聚类研究得到了较大发展,但其通常忽略了原始数据中冗余信息的存在可能会带来不理想的聚类结果。为解决这个问题,提出一种潜在表示学习框架下的低冗余多视图聚类算法。基于k-means的方法,直接从各视图数据学习其低维表示,由于该低维表示的各个特征相互正交,学习到的低维表示通常含有较少的冗余信息。基于潜在空间的假设,各视图的低维表示可由同一个潜在表示投影得到。将两者结合,就能得到一个具有低冗余信息的统一的潜在表示。设计了一个优化算法来求解目标问题,在多个公开数据集上的实验表明了该算法的有效性。

基于显著性检测与MDLatLRR分解的红外与可见光图像融合

针对红外与可见光图像融合过程中细节信息的缺失、融合结果对比度较低等问题,提出一种基于显著性检测与多层潜在低秩表示的红外与可见光图像融合方法。首先,使用基于显著性检测的方法对红外与可见光图像进行预融合;然后,使用多层潜在低秩表示方法依次将红外图像、可见光图像和预融合图像分解为低秩层和细节层;其中细节层采用结构相似性和L2范数相结合的方法进行融合,低秩层使用基于能量属性的方法进行融合;最后,将低秩层和细节层的融合结果重构便得到最终的融合图像。文中将该方法与11种具有代表性的图像融合方法进行了评估比较,通过对比多组融合图像的主客观评价,其结果表明,相较于对比方法,本方法能够保留红外与可见光图像融合过程中源图像的有效细节,且融合结果具有较高的对比度,更符合人们的视觉理解。

潜在低秩表示下的双判别器生成对抗网络的图像融合

为了改善红外与可见光图像融合的视觉效果,通过潜在低秩表示将两种不同源的图像分别分解为各自的低秩分量和去除噪声的稀疏分量,采用KL变换确定权重对稀疏分量进行加权融合得到融合稀疏图。再对双判别器的生成对抗网络重设计,借助VGG16网络提取两种源的低秩分量特征作为该网络的输入,通过生成器和判别器的博弈来生成融合低秩图。最后,将融合稀疏图与融合低秩图进行叠加获得最终的融合结果。实验结果表明,在TNO数据集上,与所列的5种先进方法相比,本文所提出的方法在熵、标准差、互信息、差异相关性总和及多尺度结构相似度5种指标上均获得最优结果,相比于次优值,5种指标分别提高了2.43%,4.68%,2.29%,2.24%,1.74%。在RoadScene数据集上只在差异相关性总和及多尺度结构相似度两种指标上取得最优,另外3种指标仅次于GTF(gradient transfer and total variation minimization)方法,但图像视觉效果明显优于GTF方法。综合主观评价和客观评价分析,本文所提方法确实能获得高质量的融合图像,与多种方法相比具有明显的优势。

基于潜在低秩表示的多聚焦图像融合方法

低秩表示(LRR)侧重于图像主要特征表示,在实现多焦距图像融合中容易造成细节信息模糊。鉴于此,本文在LRR的基础上对细节信息进一步分解,提出了一种基于潜在LRR(LatLRR)的多聚焦图像融合方法。该方法首先通过预先训练产生一个潜在低秩字典矩阵,以用于后续细节信息的分解并减少运算量。采用滑动窗口将待融合图像分割成多个图像块并构建成一个源矩阵,采用LatLRR对源矩阵进行分解得到低秩部分和细节部分。最后,对低秩部分和细节部分分别采用加权平均与核函数的奇异值分解策略进行融合。对比实验结果表明:在主客观评价中,本文提出的LatLRR的多聚焦图像融合方法获得更好的性能,更好的多焦距图像融合性能。

潜在多步马尔可夫概率的鲁棒无监督特征选择

无监督特征选择是机器学习和数据挖掘中的一种重要的降维技术。然而当前的无监督特征选择方法侧重于从数据的邻接矩阵中学习数据的流形结构,忽视非邻接数据对之间的关联。其次这些方法都假设数据实例具有独立同一性,但现实中的数据样本其来源是不同的,这样的假设就不成立。此外,在原始数据空间中特征重要性的衡量会受到数据和特征中的噪声影响。基于以上问题,本文提出了潜在多步马尔可夫概率的鲁棒无监督特征选择方法(unsupervised feature selection via multi-step Markov probability and latent representation,MMLRL),其思想是通过最大多步马尔可夫转移概率学习数据流形结构,然后通过对称非负矩阵分解模型学习数据的潜在表示,最后在数据的潜在表示空间中选择特征。同时在6个不同类型的数据集上验证了所提出算法的有效性。

基于LatLRR多层分解与显著性检测的抗晕光方法

针对夜间行车过程中的晕光干扰问题,提出一种基于潜在低秩表示(LatLRR)多层分解与显著性检测的抗晕光方法。该方法先对可见光图像做HSV色彩空间转换,提取出H、S、V分量;采用LatLRR与复合滤波结合的方式将可见光图像的亮度分量V和红外图像多层分解,得到对应的低频层和高频层;通过平均规则和基于显著性检测与权重映射的融合策略得到低频融合层和高频融合层,线性叠加得到新的亮度分量V′;对H、S、V′分量做HSV色彩空间逆变换得到消除晕光且清晰度较高的结果图像。实验结果表明,该方法可以有效消除图像中的晕光,并显著提升图像的细节信息和清晰度,使色彩和亮度更适于人眼视觉系统。

Latent discriminative representation learning for speaker recognition

Extracting discriminative speaker-specific representations from speech signals and transforming them into fixed length vectors are key steps in speaker identification and verification systems.In this study,we propose a latent discriminative representation learning method for speaker recognition.We mean that the learned representations in this study are not only discriminative but also relevant.Specifically,we introduce an additional speaker embedded lookup table to explore the relevance between different utterances from the same speaker.Moreover,a reconstruction constraint intended to learn a linear mapping matrix is introduced to make representation discriminative.Experimental results demonstrate that the proposed method outperforms state-of-the-art methods based on the Apollo dataset used in the Fearless Steps Challenge in INTERSPEECH2019 and the TIMIT dataset.

应用稀疏非负矩阵分解聚类实现高光谱影像波段的优化选择

针对高光谱影像数据高维性、高度相关性和冗余性等特点,提出应用稀疏非负矩阵分解聚类实现高光谱影像波段的优化选择。通过稀疏非负矩阵分解方法对高光谱影像进行稀疏化表示,同时顾及其可聚类的特性,在保留所选波段物理意义的基础上,得到波段选择后的高光谱影像降维数据。通过该方法对PHI-3高光谱影像进行波段选择的试验分析,应用聚类特征有效性分析波段聚类结果,并采用波段子集的信息量、相关性和可分性3类评价指标来验证方法的效果。最终,从运行效率和分类精度两方面证明了基于无监督聚类的稀疏非负矩阵分解对高光谱影像的波段选择的实用性。

基于潜在低秩表示与复合滤波的红外与弱可见光增强图像融合方法

针对传统红外与弱可见光图像融合算法中存在的亮度与对比度低、细节轮廓信息缺失、可视性差等问题,提出一种基于潜在低秩表示与复合滤波的红外与弱可见光增强图像融合方法.该方法首先利用改进的高动态范围压缩增强方法增强可见光图像提高亮度;然后利用基于潜在低秩表示与复合滤波的分解方法分别对红外与增强后的弱可见光图像进行分解,得到相应的低频和高频层;再分别使用改进的对比度增强视觉显著图融合方法与改进的加权最小二乘优化融合方法对得到的低频和高频层进行融合;最后将得到的低频和高频融合层进行线性叠加得到最终的融合图像.与其他方法的对比实验结果表明,用该方法得到的融合图像细节信息丰富,清晰度高,具有良好的可视性.