CNN联合多尺度Transformer的高光谱与多光谱图像融合

高光谱图像具有丰富的光谱信息,多光谱图像具有精妙的几何特征,融合高分辨率的多光谱图像和低分辨率的高光谱图像可以获取更为全面的遥感数据图像。然而现有的融合网络大多数基于卷积神经网络所设计,对于结构复杂的遥感类图像而言,依赖于核大小的卷积运算,容易导致特征融合阶段缺乏一些全局上下文信息。为保证图像融合的质量,本文提出了一种CNN(Convolutional Neural Network,CNN)联合多尺度transformer网络来实现多光谱和高光谱图像融合,结合了CNN的特征提取能力与transformer的全局建模优势。网络将融合任务分为了两个阶段,特征提取阶段和融合阶段。特征提取阶段,针对图像特性,基于卷积神经网络分别设计了不同模块用于特征提取。融合阶段,通过多尺度transformer模块从局部到全局建立信息间长距离关联,最后通过多层卷积层将特征映射为高分辨率的高光谱图像。经过在CAVE和Harvard数据集的实验结果表明,本文所提算法与其他经典算法相比,能更好地提升融合图像的质量。

基于Transformer模型的时序数据预测方法综述

时序数据预测(TSF)是指通过分析历史数据的趋势性、季节性等潜在信息,预测未来时间点或时间段的数值和趋势。时序数据由传感器生成,在金融、医疗、能源、交通、气象等众多领域都发挥着重要作用。随着物联网传感器的发展,海量的时序数据难以使用传统的机器学习解决,而Transformer在自然语言处理和计算机视觉等领域的诸多任务表现优秀,学者们利用Transformer模型有效捕获长期依赖关系,使得时序数据预测任务取得了飞速发展。综述了基于Transformer模型的时序数据预测方法,按时间梳理了时序数据预测的发展进程,系统介绍了时序数据预处理过程和方法,介绍了常用的时序预测评价指标和数据集。以算法框架为研究内容系统阐述了基于Transformer的各类模型在TSF任务中的应用方法和工作原理。通过实验对比了各个模型的性能、优点和局限性,并对实验结果展开了分析与讨论。结合Transformer模型在时序数据预测任务中现有工作存在的挑战提出了该方向未来发展趋势。

基于Transformer与注意力机制的肺部肿瘤分割方法

肺部肿瘤的准确分割对于肿瘤的诊断和治疗具有重要作用,然而肺部肿瘤分割中存在病灶与周围组织的对比度低、肿瘤与正常组织易粘连和背景噪声大等问题。针对这些问题,本文提出了一种基于Transformer和注意力机制的肺部肿瘤分割方法。在Transformer编码器阶段引入全局和局部的注意力机制,使得网络可以同时关注全局和局部的上下文信息;在跳跃连接阶段,使用通道优先卷积注意力机制,可以增强复杂病灶的空间感知能力和降低通道维度冗余,从而提高肿瘤的分割精度。在私有数据集GDPH和公共数据集LUNG1上的测试结果表明,本文方法相比其他8种分割方法,Dice指标在两个数据集上表现最优,分别为90.96%和88.18%,可以为临床的诊疗提供可靠辅助。

A new insight into LPSO phase transformation and mechanical properties uniformity of large-scale Mg-Gd-Y-Zn-Zr alloy prepared by multi-pass friction stir processing

A large-scale fine-grained Mg-Gd-Y-Zn-Zr alloy plate with high strength and ductility was successfully prepared by multi-pass friction stir processing(MFSP)technology in this work.The structure of grains and long period stacking ordered(LPSO)phase were characterized,and the mechanical properties uniformity was investigated.Moreover,a quantitative relationship between the microstructure and tensile yield strength was established.The results showed that the grains in the processed zone(PZ)and interfacial zone(IZ)were refined from 50μm to 3μm and 4μm,respectively,and numerous original LPSO phases were broken.In IZ,some block-shaped 18R LPSO phases were transformed into needle-like 14H LPSO phases due to stacking faults and the short-range diffusion of solute atoms.The severe shear deformation in the form of kinetic energy caused profuse stacking fault to be generated and move rapidly,greatly increasing the transformation rate of LPSO phase.After MFSP,the ultimate tensile strength,yield strength and elongation to failure of the large-scale plate were 367 MPa,305 MPa and 18.0% respectively.Grain refinement and LPSO phase strengthening were the major strengthening mechanisms for the MFSP sample.In particularly,the strength of IZ was comparable to that of PZ because the strength contribution of the 14H LPSO phase offsets the lack of grain refinement strengthening in IZ.This result opposes the widely accepted notion that IZ is a weak region in MFSP-prepared large-scale fine-grained plate.

Advanced Lung Adenocarcinoma with EGFR 19-del Mutation Transformed into SCC after EGFR-tyrosine Kinase inhibitors Treatment:A Case report

BACKGROUND Epidermal growth factor receptor tyrosine kinase inhibitors(EGFR-TKIs)significantly improve the survival of patients with Epidermal growth factor receptor(EGFR)sensitive mutations in non-small cell lung cancer(NSCLC).CASE SUMMARY A 67-year-old female patient in advanced lung adenocarcinoma suffered from drug resistance after EGFR-TKIs treatment.Secondary pathological tissue biopsy confirmed squamous cell carcinoma(SCC)transformation.Patients inevitably encountered drug resistance issues after receiving EGFR-TKIs treatment for a certain period of time,while EGFR-TKIs can significantly improve the survival of patients with EGFR-sensitive mutations in NSCLC.Notably,EGFR-TKIs resistance includes primary and acquired.Pathological transformation is one of the mechanisms of acquired resistance in EGFR-TKIs,with SCC transformation being relatively rare.Our results provide more detailed results of the patient’s diagnosis and treatment process on SCC transformation after EGFR-TKIs treatment for lung adenocarcinoma.CONCLUSION Squamous cell carcinoma transformation is one of the acquired resistance mechanisms of EGFR-TKIs in advanced lung adenocarcinoma with EGFR mutations.

基于Transformer的胃癌显微高光谱图像分割方法

胃癌是全球第五常见的恶性肿瘤并死亡率较高,严重威胁人类的生命健康。因此,早期识别胃癌病变对早期胃癌诊断至关重要。显微高光谱成像技术作为一种新兴技术,可以在微观层面同时获取生物组织丰富的光谱信息和空间信息,为早期病理切片诊断提供了一种新的思路。利用显微高光谱成像系统,采集了在400~1000 nm波段范围的胃癌显微高光谱病理图像,通过光谱校正等预处理构建了包含230张图像的胃癌显微高光谱数据集。尽管基于空间注意力的方法在图像分类、分割等领域已取得了显著成果,但在处理高光谱图像时仍面临计算复杂度高以及光谱信息利用不充分的问题。为此,提出了基于卷积和注意力机制的混合双分支Transformer(MDBT)的特征提取主干网络模型。该模型通过交替应用空间混合模块和通道混合模块,实现块间和块内的空间和通道特征聚合。具体而言,设计了窗口注意力和卷积双分支以及空间和通道交互结构。这种设计不仅降低了计算复杂度,还通过卷积交互实现了窗口间信息交互和特征融合,从而克服了窗口注意力感受野受限的问题,进一步提高了Transformer的全局建模能力。在进行图像分割实验中,采用UperNet模型作为解码头网络对主干网络提取得到的特征进行还原,以得到最终的分割结果。在采集得到的胃癌高光谱数据集上进行了五折交叉验证实验,结果表明本模型的平均mDice和mIoU分别达到85.39、74.66,性能优于目前UNet、Swin、PVT、VIT等主流图像分割网络模型。同时设计一系列消融实验,验证本文提出空间和通道双混合模块、卷积与窗口注意力双分支等结构对实验结果的优化效果。实验结果表明本文提出的MDBT模型能够有效利用高光谱图像丰富的空间和光谱信息,提高胃癌图像分割准确率,证明显微高光谱成像技术在胃癌诊断方面具有一定的研究意义和应用价值。

图像处理中CNN与视觉Transformer混合模型研究综述

卷积神经网络(CNN)与视觉Transformer是目前图像处理领域中两大重要的深度学习模型,两者经过多年来不断的研究与进步,已在该领域取得了非凡的成就。近些年来,CNN与视觉Transformer的混合模型正在逐步兴起,广泛的研究不断克服两种模型存在的弱项,高效地发挥出各自的亮点,在图像处理任务中表现出优异的效果。基于CNN与视觉Transformer混合模型进行深入阐述。总体概述了CNN与Vision Transformer模型的架构和优缺点,并总结混合模型的概念及优势。围绕串行结构融合方式、并行结构融合方式、层级交叉结构融合方式以及其他融合方式等四个方面全面回顾梳理了混合模型的研究现状和实际进展,并针对各种融合方式的主要代表模型进行总结与剖析,从多方面对典型混合模型进行评价对比。多角度叙述了混合模型在图像识别、图像分类、目标检测和图像分割等实际图像处理特定领域中应用研究,展现出混合模型在具体实践中的适用性和高效性。深入分析混合模型未来研究方向,并为后续该模型在图像处理中的研究与应用提出展望。

Molecular investigation into the transformation of recalcitrant dissolved organic sulfur in refinery sour water during stripping process

Refinery sour water primarily originates from the tops of towers in various units and coker condensate,and cannot be discharged directly to a wastewater treatment plant due to high levels of chemical oxygen demand(COD)and organic sulfur contents.Even after the recovery of H_(2)S from the sour water by the stripping process,the effluent still contains a high concentration of dissolved organic sulfur(DOS),which can have a huge bad influence.While chemical composition of dissolved organic matter(DOM)in refinery wastewater has been extensively studied,the investigation of recalcitrant DOS from sour waters remains unclear.In the present study,chemical composition of sour water DOMs(especially DOS)was investigated using fluorescence spectroscopy(excitation-emission matrix,EEM)and mass spectrometry,including gas chromatography-mass spectrometry(GC-MS)and high-resolution Orbitrap MS.The GC-MS and EEM results showed that volatile and low-aromaticity compounds were effectively removed during the stripping process,while compounds with high hydrophilicity and humification degree were found to be more recalcitrant.The Orbitrap MS results showed that weak-polar oxygenated sulfur compounds were easier to be removed than oxygenated compounds.However,the effluent still contained significant amounts of sulfur-containing compounds with multiple sulfur atoms,particularly in the form of highly unsaturated and aromatic compounds.The Orbitrap MS/MS results of CHOS-containing compounds from the effluent indicate that the sulfur atoms may exist as sulfonates,disulfide bonds,thioethers.Understanding the composition and structure of sour water DOS is crucial for the development of effective treatment processes that can target polysulfide compounds and minimize their impact on the environment.

基于改进Transformer模型的多元时间序列预测

在无线数据传输中,环境干扰和网络拥塞导致的数据丢包和缺失问题显著影响了时间序列预测的稳定性。为了解决这个问题,提出了一种名为TFKNet的时间序列预测模型。该模型基于Transformer的多维时间序列数据预测方法,在传统Transformer模型的基础上,结合时间卷积网络(TCN)和傅立叶频率特征提取技术增强了模型对局部特征的捕捉能力和多频率特征的提取能力,引入Kernel Attention Networks(KAN)提高了模型的预测性能。实验结果表明,与Transformer、Informer、Reformer、Autoformer传统方法相比,TFKNet模型在时间序列长预测任务中预测误差MAE分别平均降低0.0522、0.1117、0.1209、0.1922。

ViTAU:基于Vision transformer和面部动作单元的面瘫识别与分析

面部神经麻痹(Facial nerve paralysis,FNP),通常称为贝尔氏麻痹或面瘫,对患者的日常生活和心理健康产生显著影响,面瘫的及时识别和诊断对于患者的早期治疗和康复至关重要.随着深度学习和计算机视觉技术的快速发展,面瘫的自动识别变得可行,为诊断提供了一种更准确和客观的方式.目前的研究主要集中关注面部的整体变化,而忽略了面部细节的重要性.面部不同部位对识别结果的影响力并不相同,这些研究尚未对面部各个区域进行细致区分和分析.本项研究引入结合Vision transformer(ViT)模型和动作单元(Action unit,AU)区域检测网络的创新性方法用于面瘫的自动识别及区域分析.ViT模型通过自注意力机制精准识别是否面瘫,同时,基于AU的策略从StyleGAN2模型提取的特征图中,利用金字塔卷积神经网络分析受影响区域.这一综合方法在YouTube Facial Palsy(YFP)和经过扩展的Cohn Kanade(CK+)数据集上的实验中分别达到99.4%的面瘫识别准确率和81.36%的面瘫区域识别准确率.通过与最新方法的对比,实验结果展示了所提的自动面瘫识别方法的有效性.

基于贝叶斯图注意力Transformer的航空发动机剩余使用寿命概率预测

航空发动机作为飞机的心脏,其健康状态对飞机的安全飞行至关重要.深度学习强大的数据挖掘能力,为通过海量历史数据预测航空发动机的剩余使用寿命提供了新方法.然而,传统基于深度学习的方法大都关注于挖掘数据在时间上的关联,而忽略了多个传感器监测数据之间复杂的非欧氏空间关系.此外,少有研究考虑数据或者预测过程本身具有的不确定性,缺乏对预测结果可靠性的评估.为解决上述问题,本文提出了一种基于贝叶斯网络和图注意力Transformer的航空发动机剩余使用寿命概率预测方法.将图注意力机制融入Transformer的时间多头注意力模块,结合图注意力网络在空间特征提取上的优势和Transformer模型在时间特征提取的优势,实现数据特征时空关系的联合提取.同时,利用改进的贝叶斯网络度量预测不确定性,在得到剩余使用寿命预测点值的同时给出相应的置信区间.最后,通过在公开航空发动机数据集上的实验,证明了所提模型的有效性和先进性.

基于Point Transformer方法的鱼类三维点云模型分类

为实现对不同鱼类的精准分类,研究共采集110尾真实鱼类的三维模型,对获取的3D模型进行基于预处理、旋转增强和下采样等操作后,获取了1650尾实验样本。然后基于Point Transformer网络和2个三维分类的对比网络进行数据集的分类训练和验证。结果表明,利用本实验的目标方法Point Transformer获得了比2个对比网络更好的分类结果,整体的分类准确率能够达到91.9%。同时对所使用的三维分类网络进行有效性评估,3个模型对于5种真实鱼类模型的分类是有意义的,其中Point Transformer的模型ROC曲线准确率最高,AUC面积最大,对于三维鱼类数据集的分类最为有效。研究提供了一种可以实现对鱼类三维模型进行精准分类的方法,为以后的智能化渔业资源监测提供一种新的技术手段。

基于Transformer的状态−动作−奖赏预测表征学习

为了提升具有高维动作空间的复杂连续控制任务的性能和样本效率,提出一种基于Transformer的状态−动作−奖赏预测表征学习框架(Transformer-based state-action-reward prediction representation learning framework,TSAR).具体来说,TSAR提出一种基于Transformer的融合状态−动作−奖赏信息的序列预测任务.该预测任务采用随机掩码技术对序列数据进行预处理,通过最大化掩码序列的预测状态特征与实际目标状态特征间的互信息,同时学习状态与动作表征.为进一步强化状态和动作表征与强化学习(Reinforcement learning,RL)策略的相关性,TSAR引入动作预测学习和奖赏预测学习作为附加的学习约束以指导状态和动作表征学习.TSAR同时将状态表征和动作表征显式地纳入到强化学习策略的优化中,显著提高了表征对策略学习的促进作用.实验结果表明,在DMControl的9个具有挑战性的困难环境中,TSAR的性能和样本效率超越了现有最先进的方法.

MSMVT:多尺度和多视图Transformer半监督医学图像分割框架

近年来,Transformer在众多监督式计算机视觉任务中取得了显著进展,然而由于高质量医学标注图像的缺乏,其在半监督图像分割领域的性能仍有待提高。为此,提出了一种基于多尺度和多视图Transformer的半监督医学图像分割框架:MSMVT(multi-scale and multi-view transformer)。鉴于对比学习在Transformer的预训练中取得的良好效果,设计了一个基于伪标签引导的多尺度原型对比学习模块。该模块利用图像金字塔数据增强技术,为无标签图像生成富有语义信息的多尺度原型表示;通过对比学习,强化了不同尺度原型之间的一致性,从而有效缓解了由标签稀缺性导致的Transformer训练不足的问题。此外,为了增强Transformer模型训练的稳定性,提出了多视图一致性学习策略。通过弱扰动视图,以校正多个强扰动视图。通过最小化不同视图之间的输出差异性,使得模型能够对不同扰动保持多层次的一致性。实验结果表明,当仅采用10%的标注比例时,提出的MSMVT框架在ACDC、LIDC和ISIC三个公共数据集上的DSC图像分割性能指标分别达到了88.93%、84.75%和85.38%,优于现有的半监督医学图像分割方法。

Advanced lung adenocarcinoma with EGFR 19-del mutation transforms into squamous cell carcinoma after EGFR tyrosine kinase inhibitor treatment

In this editorial we comment on the article by Ji et al.We focus specifically on the EGFR tyrosine kinase inhibitor(EGFR-TKI)treatment and the development of drug resistance to EGFR-TKIs.

三维卷积与Transformer支持下联合空谱特征的高光谱影像分类

由于CNN对局部特征提取能力强,目前仍是高光谱影像处理和分析中的主流深度模型,但是CNN感受野有限,无法建立长距离依赖关系,学习全局语义信息受限。Transformer的自注意力机制可以对输入序列中的每个位置进行注意力计算,从而能有效获取全局上下文信息。如何实现CNN和Transformer的技术耦合并充分利用空间信息和光谱信息进行高光谱遥感影像分类是一个重要的待研问题。鉴于此,提出一种新的基于三维卷积和Transformer的高光谱遥感影像分类方法,尝试联合空谱特征实现解译能力的提升。使用主成分分析方法对高光谱遥感影像沿垂直方向降维;用非负矩阵分解算法对降维后遥感影像沿水平方向进行空间特征提取,将两种工具处理后遥感影像进行拼接,以充分保留信息;再用三维卷积核对拼接后遥感影像进行空间特征和光谱特征的综合提取;用Transformer的注意力机制对提取空间信息和光谱信息的遥感影像序列建立长距离依赖关系并使用多层感知机完成分类任务。实验表明,所提方法在WHU-Hi龙口、汉川、洪湖以及雄安新区马蹄湾村数据集上均表现出比对比方法更优异的分类性能,表明该方法具有一定的泛化性和稳健性。

Transformation of lung adenocarcinoma into small cell lung cancer after treatment with epidermal growth factor receptor tyrosine kinase inhibitors

Epidermal growth factor receptor tyrosine kinase inhibitors effectively improve the prognosis of patients with epidermal growth factor receptor–mutant lung adenocarcinoma.However,acquired resistance inevitably develops with small cell lung cancer transformation emerging as a rare but increasingly frequent mechanism of tyrosine kinase inhibitor resistance.This transformation poses significant chal-lenges to the health of patients with lung cancer and complicates their clinical management.This article comprehensively reviews the di-agnostic,predictive,mechanistic,and therapeutic aspects of small cell lung cancer transformation to enhance our understanding and clin-ical awareness of this phenomenon.

一种基于Transformer的双流文档图像质量评价算法

为了解决文档图像质量评价网络对图像特征提取不充分、评价指标不恰当等问题,提出了一种基于Transformer的双流文档图像质量评价算法。首先,利用Transformer提取图像特征,计算特征通道间注意力;其次,使用权重模块预测文档图像OCR(光学字符识别)准确率作为文档图像质量得分,使用CNN(卷积神经网络)提取文档全局特征,全连接后预测图像的自然图像得分;最后,将两者得分结合作为预测图像的质量得分。实验结果表明,基于Transformer的双流文档图像质量评价算法在数据集上的皮尔逊线性相关系数(PLCC)达到0.9045,史比尔曼等级相关系数(SROCC)达到0.8775,证明该算法可以预测出更符合人类视觉标准的文档图像质量分数。

Unraveling the significance of cobalt on transformation kinetics,crystallography and impact toughness in high-strength steels

This work reveals the significant effects of cobalt(Co)on the microstructure and impact toughness of as-quenched highstrength steels by experimental characterizations and thermo-kinetic analyses.The results show that the Co-bearing steel exhibits finer blocks and a lower ductile-brittle transition temperature than the steel without Co.Moreover,the Co-bearing steel reveals higher transformation rates at the intermediate stage with bainite volume fraction ranging from around 0.1 to 0.6.The improved impact toughness of the Co-bearing steel results from the higher dense block boundaries dominated by the V1/V2 variant pair.Furthermore,the addition of Co induces a larger transformation driving force and a lower bainite start temperature(BS),thereby contributing to the refinement of blocks and the increase of the V1/V2 variant pair.These findings would be instructive for the composition,microstructure design,and property optimization of high-strength steels.

A Deep Dive into the Trajectory and Transformation of Fintech in Colombia Through the Case of PTM Colombia-A Secondary Publication

This research focuses on analyzing and understanding the evolution and impact of fintech companies in Colombia,using PTM Colombia as a case study,a company that has undergone a significant transformation from its beginnings as a provider of physical recharges to its current position.as a comprehensive platform for digital financial services.The current research uses a qualitative descriptive approach and emphasizes understanding the perceptions and experiences in the field of Fintech in Colombia,particularly in the case of PTM Colombia.The study revealed that adaptability and customer orientation have been fundamental in the evolution of PTM,allowing the company to expand its range of services by actively listening to the needs of its users and adjusting its strategies accordingly.In addition,an accelerated and diversified growth of Fintech will be observed in Colombia,particularly in sectors such as credit and payments,driven by a constantly advancing technological environment and a regulatory framework that,although it presents challenges,also offers opportunities for financial innovation.This growth has impacted the traditional banking sector,which,instead of being replaced by Fintech,is finding new forms of collaboration and coexistence that benefit both parties and,above all,the end consumer.In conclusion,the study highlights how Fintech in Colombia,represented by cases such as PTM,is redefining the financial landscape through the adoption of advanced technologies,collaboration with traditional banking,and strong customer orientation.