Description of martensitic transformation kinetics in Fe-C-X(X = Ni,Cr,Mn,Si) system by a modified model

Controlling the content of athermal martensite and retained austenite is important to improving the mechanical properties of high-strength steels,but a mechanism for the accurate description of martensitic transformation during the cooling process must be addressed.At present,frequently used semi-empirical kinetics models suffer from huge errors at the beginning of transformation,and most of them fail to exhibit the sigmoidal shape characteristic of transformation curves.To describe the martensitic transformation process accurately,based on the Magee model,we introduced the changes in the nucleation activation energy of martensite with temperature,which led to the varying nucleation rates of this model during martensitic transformation.According to the calculation results,the relative error of the modified model for the martensitic transformation kinetics curves of Fe-C-X(X = Ni,Cr,Mn,Si) alloys reached 9.5% compared with those measured via the thermal expansion method.The relative error was approximately reduced by two-thirds compared with that of the Magee model.The incorporation of nucleation activation energy into the kinetics model contributes to the improvement of its precision.

A Comprehensive Survey of Recent Transformers in Image,Video and Diffusion Models

Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.

Research on Restoration of Murals Based on Diffusion Model and Transformer

Due to the limitations of a priori knowledge and convolution operation,the existing image restoration techniques cannot be directly applied to the cultural relics mural restoration,in order to more accurately restore the original appearance of the cultural relics mural images,an image restoration based on the denoising diffusion probability model(Denoising Diffusion Probability Model(DDPM))and the Transformer method.The process involves two steps:in the first step,the damaged mural image is firstly utilized as the condition to generate the noise image,using the time,condition and noise image patch as the inputs to the noise prediction network,capturing the global dependencies in the input sequence through the multi-attentionmechanismof the input sequence and feedforward neural network processing,and designing a long skip connection between the shallow and deep layers in the Transformer blocks between the shallow and deep layers using long skip connections to fuse the feature information of global and local outputs to maintain the overall consistency of the restoration results;In the second step,taking the noisy image as a condition to direct the diffusion model to back sample to generate the restored image.Experiment results show that the PSNR and SSIM of the proposedmethod are improved by 2%to 9%and 1%to 3.3%,respectively,which are compared to the comparison methods.This study proposed synthesizes the advantages of the diffusionmodel and deep learningmodel to make themural restoration results more accurate.

Model Agnostic Meta-Learning(MAML)-Based Ensemble Model for Accurate Detection of Wheat Diseases Using Vision Transformer and Graph Neural Networks

Wheat is a critical crop,extensively consumed worldwide,and its production enhancement is essential to meet escalating demand.The presence of diseases like stem rust,leaf rust,yellow rust,and tan spot significantly diminishes wheat yield,making the early and precise identification of these diseases vital for effective disease management.With advancements in deep learning algorithms,researchers have proposed many methods for the automated detection of disease pathogens;however,accurately detectingmultiple disease pathogens simultaneously remains a challenge.This challenge arises due to the scarcity of RGB images for multiple diseases,class imbalance in existing public datasets,and the difficulty in extracting features that discriminate between multiple classes of disease pathogens.In this research,a novel method is proposed based on Transfer Generative Adversarial Networks for augmenting existing data,thereby overcoming the problems of class imbalance and data scarcity.This study proposes a customized architecture of Vision Transformers(ViT),where the feature vector is obtained by concatenating features extracted from the custom ViT and Graph Neural Networks.This paper also proposes a Model AgnosticMeta Learning(MAML)based ensemble classifier for accurate classification.The proposedmodel,validated on public datasets for wheat disease pathogen classification,achieved a test accuracy of 99.20%and an F1-score of 97.95%.Compared with existing state-of-the-art methods,this proposed model outperforms in terms of accuracy,F1-score,and the number of disease pathogens detection.In future,more diseases can be included for detection along with some other modalities like pests and weed.

Intelligent Transformation: General Intelligence Theory

This paper aims to formalize a general definition of intelligence beyond human intelligence. We accomplish this by re-imagining the concept of equality as a fundamental abstraction for relation. We discover that the concept of equality = limits the sensitivity of our mathematics to abstract relationships. We propose a new relation principle that does not rely on the concept of equality but is consistent with existing mathematical abstractions. In essence, this paper proposes a conceptual framework for general interaction and argues that this framework is also an abstraction that satisfies the definition of Intelligence. Hence, we define intelligence as a formalization of generality, represented by the abstraction ∆∞Ο, where each symbol represents the concepts infinitesimal, infinite, and finite respectively. In essence, this paper proposes a General Language Model (GLM), where the abstraction ∆∞Ο represents the foundational relationship of the model. This relation is colloquially termed “The theory of everything”.

Application of Elzaki Transform Method to Market Volatility Using the Black-Scholes Model

Black-Scholes Model (B-SM) simulates the dynamics of financial market and contains instruments such as options and puts which are major indices requiring solution. B-SM is known to estimate the correct prices of European Stock options and establish the theoretical foundation for Option pricing. Therefore, this paper evaluates the Black-Schole model in simulating the European call in a cash flow in the dependent drift and focuses on obtaining analytic and then approximate solution for the model. The work also examines Fokker Planck Equation (FPE) and extracts the link between FPE and B-SM for non equilibrium systems. The B-SM is then solved via the Elzaki transform method (ETM). The computational procedures were obtained using MAPLE 18 with the solution provided in the form of convergent series.

Extension of analytical model of solid-state phase transformation

Departing from an analytical phase transformation model, a new analytical approach to deduce transformed fraction for non-isothermal phase transformation was developed. In the new approach, the effect of the initial transformation temperature and the accurate ＂temperature integral＂ approximations are incorporated to obtain an extended analytical model. Numerical approach demonstrated that the extended analytical model prediction for transformed fraction and transformation rate is in good agreement with the exact numerical calculation. The new model can describe more precisely the kinetic behavior than the original analytical model, especially for transformation with relatively high initial transformation temperature. The kinetic parameters obtained from the new model are more accurate and reasonable than those from the original analytical model.

Error modeling of 3-RSR parallel robot based on D-H transformation matrix

By selecting any one limb of 3-RSR parallel robot as a research object, the paper establishes a position and orienta- tion relationship matrix between the moving platform and the base by means of Denavit-Hartenberg （D-H） transformation matrix. The error mapping model is derived from original error to the error of the platform by using matrix differential method. This model contains all geometric original errors of the robot. The nonlinear implicit function relation between po- sition and orientation error of the platform and the original geometric errors is simplified as a linear explicit function rela- tion. The results provide a basis for further studying error analysis and error compensation.

基于RoBERTa和图增强Transformer的序列推荐方法

自推荐系统出现以来,有限的数据信息就一直制约着推荐算法的进一步发展。为降低数据稀疏性的影响,增强非评分数据的利用率,基于神经网络的文本推荐模型相继被提出,但主流的卷积或循环神经网络在文本语义理解和长距离关系捕捉方面存在明显劣势。为了更好地挖掘用户与商品之间的深层潜在特征,进一步提高推荐质量,提出一种基于Ro BERTa和图增强Transformer的序列推荐(RGT)模型。引入评论文本数据,首先利用预训练的Ro BERTa模型捕获评论文本中的字词语义特征,初步建模用户的个性化兴趣,然后根据用户与商品的历史交互信息,构建具有时序特性的商品关联图注意力机制网络模型,通过图增强Transformer的方法将图模型学习到的各个商品的特征表示以序列的形式输入Transformer编码层,最后将得到的输出向量与之前捕获的语义表征以及计算得到的商品关联图的全图表征输入全连接层,以捕获用户全局的兴趣偏好,实现用户对商品的预测评分。在3组真实亚马逊公开数据集上的实验结果表明,与Deep FM、Conv MF等经典文本推荐模型相比,RGT模型在均方根误差(RMSE)和平均绝对误差(MAE)2种指标上有显著提升,相较于最优对比模型最高分别提升4.7%和5.3%。

考虑特征重组与改进Transformer的风电功率短期日前预测方法

短期日前风电功率预测对电力系统调度计划制定有重要意义,该文为提高风电功率预测的准确性,提出了一种基于Transformer的预测模型Powerformer。模型通过因果注意力机制挖掘序列的时序依赖;通过去平稳化模块优化因果注意力以提高数据本身的可预测性;通过设计趋势增强和周期增强模块提高模型的预测能力;通过改进解码器的多头注意力层,使模型提取周期特征和趋势特征。该文首先对风电数据进行预处理,采用完全自适应噪声集合经验模态分解(complete ensemble empirical mode decomposition with adaptive noise,CEEMDAN)将风电数据序列分解为不同频率的本征模态函数并计算其样本熵,使得风电功率序列重组为周期序列和趋势序列,然后将序列输入到Powerformer模型,实现对风电功率短期日前准确预测。结果表明,虽然训练时间长于已有预测模型,但Poweformer模型预测精度得到提升;同时,消融实验结果验证了模型各模块的必要性和有效性,具有一定的应用价值。

基于遥感多参数和CNN-Transformer的冬小麦单产估测

为了提高冬小麦单产估测精度,改善估产模型存在的高产低估和低产高估等现象,以陕西省关中平原为研究区域,选取旬尺度条件植被温度指数(VTCI)、叶面积指数(LAI)和光合有效辐射吸收比率(FPAR)为遥感特征参数,结合卷积神经网络(CNN)局部特征提取能力和基于自注意力机制的Transformer网络的全局信息提取能力,构建CNN-Transformer深度学习模型,用于估测关中平原冬小麦产量。与Transformer模型(R^(2)为0.64,RMSE为465.40 kg/hm^(2),MAPE为8.04%)相比,CNN-Transformer模型具有更高的冬小麦单产估测精度(R^(2)为0.70,RMSE为420.39 kg/hm^(2),MAPE为7.65%),能够从遥感多参数中提取更多与产量相关的信息,且对于Transformer模型存在的高产低估和低产高估现象均有所改善。基于5折交叉验证法和留一法进一步验证了CNN-Transformer模型的鲁棒性和泛化能力。此外,基于CNN-Transformer模型捕获冬小麦生长过程的累积效应,分析逐步累积旬尺度输入参数对产量估测的影响,评估模型对于冬小麦不同生长阶段的累积过程的表征能力。结果表明,模型能有效捕捉冬小麦生长的关键时期,3月下旬至5月上旬是冬小麦生长的关键时期。

基于LSTM与Transformer的地面沉降智能预测方法研究——以上海市为例

受地面沉降严重威胁到生命财产安全的人口已达19%,开展地面沉降模拟预测对防灾减灾具有非常重要的现实意义。针对现有地面沉降预测在模型参数难以获取、单一深度学习方法在预测精度低等方面的局限性,本文提出了集成大模型核心技术的地面沉降预测方法。首先,从地面沉降模拟预测的顶层设计,提出了基于深度学习的地面沉降预测包括算力层、数据层、模型层、评估层与应用层的总体架构;其次,基于LSTM与Transformer提出了地面沉降预测的实用方法;最后,利用上海的地面沉降数据进行了实验研究。结果表明:深度学习技术可以在地面沉降模拟预测中取得较好的结果,多模型法对地面沉降变化不大、回弹、变化较大均可进行预测,iTransformer模型对地面沉降变化较小的情况预测效果较好;在微量地面沉降时代,利用大模型的核心技术Transformer可以取得较高的精度。

基于Swin Transformer和CNN的汉字书法教学系统

针对日益增长的汉字书法学习需求,将滑动窗口自注意力(Swin Transformer,ST)模型和卷积神经网络(Convolutional Neural Network,CNN)模型相结合,提出手写体汉字识别ST-CNN模型,进而开发了汉字书法教学系统。实测结果表明,ST-CNN模型识别准确率约为91.6%,较传统的ST模型提升了约0.5个百分点,较传统的CNN模型与ST模型,在收敛速度上分别提升了约10和30个百分点,开发的汉字书法教学系统性能良好。

基于多模态掩码Transformer网络的社会事件分类

多模态社会事件分类的关键是充分且准确地利用图像和文字2种模态的特征。然而,现有的大多数方法存在以下局限性:简单地将事件的图像特征和文本特征连接起来,不同模态之间存在不相关的上下文信息导致相互干扰。因此,仅仅考虑多模态数据模态间的关系是不够的,还要考虑模态之间不相关的上下文信息(即区域或单词)。为克服这些局限性,提出一种新颖的基于多模态掩码Transformer网络(MMTN)模型的社会事件分类方法。通过图-文编码网络来学习文本和图像的更好的表示。将获得的图像和文本表示输入多模态掩码Transformer网络来融合多模态信息,并通过计算多模态信息之间的相似性,对多模态信息的模态间的关系进行建模,掩盖模态之间的不相关上下文。在2个基准数据集上的大量实验表明:所提模型达到了最先进的性能。

基于Transformer的司法文书命名实体识别方法

命名实体识别是自然语言处理领域的关键任务之一,是实现下游任务的基础。目前针对司法领域的相关研究相对较少,司法系统的信息化和智能化转型仍有许多问题亟需解决。相比其他领域的文本,司法文书存在专业性强、语料资源少等局限,导致现有的司法文书识别结果较低。因此,从以下3方面开展研究:首先,提出了一种多标签层级迭代的文本标注方式,可以对原始司法文书文本进行自动化标注,同时有效地提升司法文书命名实体识别任务的实体识别效果;其次,提出了一种交融式的Transformer神经网络模型,对汉字固有属性的深层特征进行了充分利用,用于对司法文书进行命名实体识别;最后,对所提出的标注方法和模型与其他神经网络模型进行了对比实验。所提出的文本标注方式可以较为准确地实现司法文书的标注任务;同时,所提出的模型在通用数据集中相对于对照模型有较大的提高,并在司法领域数据集中取得了良好的效果。

基于Transformer视觉特征融合的图像描述方法

现有图像描述方法只利用区域型视觉特征生成描述语句,忽略了网格型视觉特征的重要性,并且均为两阶段方法,从而影响了图像描述的质量。针对该问题,提出一种基于Transformer视觉特征融合的端到端图像描述方法。首先,在特征提取阶段,利用视觉特征提取器提取出区域型视觉特征和网格型视觉特征;其次,在特征融合阶段,通过视觉特征融合模块对区域型视觉特征和网格型视觉特征进行拼接;最后,将所有的视觉特征送入语言生成器中以生成图像描述。该方法各部分均基于Transformer模型实现,实现了一阶段方法。在MS-COCO数据集上的实验结果表明,所提方法能够充分利用区域型视觉特征与网格型视觉特征的优势,BLEU-1、BLEU-4、METEOR、ROUGE-L、CIDEr、SPICE指标分别达到83.1%、41.5%、30.2%、60.1%、140.3%、23.9%,优于目前主流的图像描述方法,能够生成更加准确和丰富的描述语句。

基于Swin-Transformer的黑色素瘤图像病灶分割研究

黑色素瘤图像病灶分割的主流模型大多基于卷积神经网络(CNN)或Vision Transformer(ViT)网络,但是CNN模型受限于感受野大小,无法获取全局上下文信息,而ViT模型只能提取固定分辨率的特征,无法提取不同粒度的特征。为解决该问题,建立一种基于Swin-Transformer的融合双分支的混合模型SwinTransFuse。在编码阶段,首先利用Noise Reduction图像降噪模块去除图像中的毛发等噪声,然后采用CNN和Swin-Transformer构成的双分支特征提取模块来提取图像的局部细粒度信息和全局上下文信息,并对来自Swin-Transformer分支的全局上下文信息使用SE模块进行通道注意力操作以增强全局特征的提取,对来自CNN分支的局部细粒度信息使用卷积块注意力机制模块(CBAM)进行空间注意力操作以增强局部细粒度特征的提取,接下来利用Hadamard积运算对两个分支输出的特征进行特征交互以实现特征的融合,最后将SE模块输出的特征、CBAM模块输出的特征和特征融合后的特征进行拼接以实现多层次特征融合,并通过一个残差块输出交互后的特征。在解码阶段,将特征输入到上采样模块得到图像最终的分割结果。实验结果表明,该模型在ISIC2017和ISIC2018皮肤病数据集上的平均交并比分别为78.72%和78.56%,优于同类型的其他医学分割模型,具有更高的实用价值。

基于卷积神经网络与Transformer的电能质量扰动分类方法

复杂电能质量扰动(power quality disturbances, PQD)的智能分类对于智能电网发展具有重要意义。扰动特征的提取与定位、模式识别与分类是电能质量扰动分类方法研究的难点。采用深度学习算法,将具有关注全局信息的Transformer与善于提取局部特征的卷积神经网络相融合,提出一种基于卷积神经网络(convolutional neural network, CNN)与Transformer的电能质量扰动分类方法,即CTranCBA。这种双深度学习模型分类方法主要是通过一维卷积神经网络提取电能质量扰动信号特征,利用Transformer自注意力机制引导模型关注序列中不同位置间的依赖关系,实现对扰动信号局部特征与全局特征的互补,克服了因感受野的限制而带来的识别不清、分类不准等问题。使用23种不同电能质量扰动信号,将CTranCBA与Deep-CNN、CNN-LSTM、CNN-CBAM方法进行比较。结果表明:该方法在分类准确率和抗噪性方面表现优异,可为电能质量扰动智能分类提供一种新的方法。

规则压缩模型和灵活架构的Transformer加速器设计

基于注意力机制的Transformer模型具有优越的性能,设计专用的Transformer加速器能大幅提高推理性能以及降低推理功耗。Transformer模型复杂性包括数量上和结构上的复杂性,其中结构上的复杂性导致不规则模型和规则硬件之间的失配,降低了模型映射到硬件的效率。目前的加速器研究主要聚焦在解决模型数量上的复杂性,但对如何解决模型结构上的复杂性研究得不多。该文首先提出规则压缩模型,降低模型的结构复杂度,提高模型和硬件的匹配度,提高模型映射到硬件的效率。接着提出一种硬件友好的模型压缩方法,采用规则的偏移对角权重剪枝方案和简化硬件量化推理逻辑。此外,提出一个高效灵活的硬件架构,包括一种以块为单元的权重固定脉动运算阵列,同时包括一种准分布的存储架构。该架构可以高效实现算法到运算阵列的映射,同时实现高效的数据存储效率和降低数据移动。实验结果表明,该文工作在性能损失极小的情况下实现93.75%的压缩率,在FPGA上实现的加速器可以高效处理压缩后的Transformer模型,相比于中央处理器(CPU)和图形处理器(GPU)能效分别提高了12.45倍和4.17倍。

基于语言和视觉融合Transformer的指代图像分割

针对指代图像分割任务中存在语言表达歧义、多模态特征对齐不充分、对图像整体理解不全面等问题,提出一种基于Transformer特征融合与对齐的多模态深度学习模型。该模型使用优化的Darknet53图像特征提取骨干网络,加强了对全局特征理解能力。使用了卷积神经网络结构、双向门控循环单元Bi-GRU结构和自注意力机制相互结合的语言特征提取结构,挖掘深层次语义特征,消除语言表达的歧义性。构建了基于Transformer的特征对齐结构,以提升模型的分割细节和分割精度。最后,采用平均的交并比mIoU和在不同阈值的识别精度作为模型评估指标,通过实验证明所提模型可以充分融合多模态的特征,理解多模态特征的深层语义信息,模型识别结果更加准确。