基于Transformer模型的时序数据预测方法综述

时序数据预测(TSF)是指通过分析历史数据的趋势性、季节性等潜在信息,预测未来时间点或时间段的数值和趋势。时序数据由传感器生成,在金融、医疗、能源、交通、气象等众多领域都发挥着重要作用。随着物联网传感器的发展,海量的时序数据难以使用传统的机器学习解决,而Transformer在自然语言处理和计算机视觉等领域的诸多任务表现优秀,学者们利用Transformer模型有效捕获长期依赖关系,使得时序数据预测任务取得了飞速发展。综述了基于Transformer模型的时序数据预测方法,按时间梳理了时序数据预测的发展进程,系统介绍了时序数据预处理过程和方法,介绍了常用的时序预测评价指标和数据集。以算法框架为研究内容系统阐述了基于Transformer的各类模型在TSF任务中的应用方法和工作原理。通过实验对比了各个模型的性能、优点和局限性,并对实验结果展开了分析与讨论。结合Transformer模型在时序数据预测任务中现有工作存在的挑战提出了该方向未来发展趋势。

小数据集上基于语义的局部注意视觉Transformer方法

在小数据集上从零开始训练时,视觉Transformer无法与同规模的卷积神经网络媲美。基于图像的局部注意力方法,可以显著提高ViT的数据效率,但是会丢失距离较远但相关的补丁之间的信息。为了解决上述问题,提出一种双向并行局部注意力视觉Transformer的方法。该方法首先在特征层面上对补丁进行分组,在组内执行局部注意力,以利用特征空间中补丁之间的关系弥补信息丢失。其次,为了有效融合补丁之间的信息,将基于语义的局部注意力和基于图像的局部注意力并行结合起来,通过双向自适应学习来增强ViT模型在小数据上的性能。实验结果表明,该方法在计算量为15.2 GFLOPs和参数量为57.2 M的情况下,分别在CIFAR-10和CIFAR-100数据集上实现了97.93%和85.80%的准确性。相比于其他方法,双向并行局部注意力视觉Transformer在增强局部引导能力的同时,保持了局部注意力所需属性的有效性。

ON THE GROWTH OF ZERO ORDER LAPLACE-STIELTJES TRANSFORM CONVERGENT IN THE RIGHT HALF-PLANE

In this article, authors study the growth of Laplace-Stieltjes transform with zero order convergent in the right half-plane, define the exponential order and the exponential low order, and find the relations between them. Some results similar to Dirichlet series are obtained.

ENTIRE FUNCTIONS REPRESENTED BY LAPLACE-STIELTJES TRANSFORMS CONCERNING THE APPROXIMATION AND GENERALIZED ORDER

The first aim of this paper is to investigate the growth of the entire function defined by the Laplace-Stieltjes transform converges on the whole complex plane.By introducing the concept of generalized order,we obtain two equivalence theorems of Laplace-Stiettjes transforms related to the generalized order,A_(n)^(*)andλ_(n).The second purpose of this paper is to study the problem on the approximation of this Laplace-Stieltjes transform.We also obtain some theorems about the generalized order,the error,and the coefficients of Laplace-Stieltjes transforms,which are generalization and improvement of the previous results.

局部特征增强的磁Transformer重构

研究了一种基于多头自注意力与卷积特征融合的磁共振图像Transformer重构方法.采用U型网络结构,通过学习图像的多尺度特征以提升重构性能.采用深度分离卷积与多头自注意力融合的Swin Transformer结构,改善网络的特征学习能力.在CC359-Brain数据集下基于多种采样模式进行仿真实验,结果证明了该方法在提升磁共振图像重构质量与降低系统复杂度方面的有效性.

On the Approximation of an Analytic Function Represented by Laplace-Stieltjes Transformation

In the present paper,we have considered the approximation of analytic functions represented by Laplace-Stieltjes transformations using sequence of definite integrals. We have characterized their order and type in terms of the rate of decrease of E;（F,β） where E;（F,β） is the error in approximating of the function F（s） by definite integral polynomials in the half plane Res≤β<α.

图像处理中CNN与视觉Transformer混合模型研究综述

卷积神经网络(CNN)与视觉Transformer是目前图像处理领域中两大重要的深度学习模型,两者经过多年来不断的研究与进步,已在该领域取得了非凡的成就。近些年来,CNN与视觉Transformer的混合模型正在逐步兴起,广泛的研究不断克服两种模型存在的弱项,高效地发挥出各自的亮点,在图像处理任务中表现出优异的效果。基于CNN与视觉Transformer混合模型进行深入阐述。总体概述了CNN与Vision Transformer模型的架构和优缺点,并总结混合模型的概念及优势。围绕串行结构融合方式、并行结构融合方式、层级交叉结构融合方式以及其他融合方式等四个方面全面回顾梳理了混合模型的研究现状和实际进展,并针对各种融合方式的主要代表模型进行总结与剖析,从多方面对典型混合模型进行评价对比。多角度叙述了混合模型在图像识别、图像分类、目标检测和图像分割等实际图像处理特定领域中应用研究,展现出混合模型在具体实践中的适用性和高效性。深入分析混合模型未来研究方向,并为后续该模型在图像处理中的研究与应用提出展望。

基于改进Transformer模型的多元时间序列预测

在无线数据传输中,环境干扰和网络拥塞导致的数据丢包和缺失问题显著影响了时间序列预测的稳定性。为了解决这个问题,提出了一种名为TFKNet的时间序列预测模型。该模型基于Transformer的多维时间序列数据预测方法,在传统Transformer模型的基础上,结合时间卷积网络(TCN)和傅立叶频率特征提取技术增强了模型对局部特征的捕捉能力和多频率特征的提取能力,引入Kernel Attention Networks(KAN)提高了模型的预测性能。实验结果表明,与Transformer、Informer、Reformer、Autoformer传统方法相比,TFKNet模型在时间序列长预测任务中预测误差MAE分别平均降低0.0522、0.1117、0.1209、0.1922。

ViTAU:基于Vision transformer和面部动作单元的面瘫识别与分析

面部神经麻痹(Facial nerve paralysis,FNP),通常称为贝尔氏麻痹或面瘫,对患者的日常生活和心理健康产生显著影响,面瘫的及时识别和诊断对于患者的早期治疗和康复至关重要.随着深度学习和计算机视觉技术的快速发展,面瘫的自动识别变得可行,为诊断提供了一种更准确和客观的方式.目前的研究主要集中关注面部的整体变化,而忽略了面部细节的重要性.面部不同部位对识别结果的影响力并不相同,这些研究尚未对面部各个区域进行细致区分和分析.本项研究引入结合Vision transformer(ViT)模型和动作单元(Action unit,AU)区域检测网络的创新性方法用于面瘫的自动识别及区域分析.ViT模型通过自注意力机制精准识别是否面瘫,同时,基于AU的策略从StyleGAN2模型提取的特征图中,利用金字塔卷积神经网络分析受影响区域.这一综合方法在YouTube Facial Palsy(YFP)和经过扩展的Cohn Kanade(CK+)数据集上的实验中分别达到99.4%的面瘫识别准确率和81.36%的面瘫区域识别准确率.通过与最新方法的对比,实验结果展示了所提的自动面瘫识别方法的有效性.

基于贝叶斯图注意力Transformer的航空发动机剩余使用寿命概率预测

航空发动机作为飞机的心脏,其健康状态对飞机的安全飞行至关重要.深度学习强大的数据挖掘能力,为通过海量历史数据预测航空发动机的剩余使用寿命提供了新方法.然而,传统基于深度学习的方法大都关注于挖掘数据在时间上的关联,而忽略了多个传感器监测数据之间复杂的非欧氏空间关系.此外,少有研究考虑数据或者预测过程本身具有的不确定性,缺乏对预测结果可靠性的评估.为解决上述问题,本文提出了一种基于贝叶斯网络和图注意力Transformer的航空发动机剩余使用寿命概率预测方法.将图注意力机制融入Transformer的时间多头注意力模块,结合图注意力网络在空间特征提取上的优势和Transformer模型在时间特征提取的优势,实现数据特征时空关系的联合提取.同时,利用改进的贝叶斯网络度量预测不确定性,在得到剩余使用寿命预测点值的同时给出相应的置信区间.最后,通过在公开航空发动机数据集上的实验,证明了所提模型的有效性和先进性.

The Grow th of the Entire Func tion Represented by the Laplace-Stieltjes Transform in the Whole Plane

In the paper, the α-order of the Laplace-Stieltjes Transform is introducedfirstly, then we get the relationship between α-order represented by the maximum modulus and α-order represented by A^*n, λn. Lastly, we obtain the relationship between type τrepresented by the maximum modulus and type τ represented by A^*n, λn.

On the Implementation of the Method of Derivation of Lorentz Transformations Used by Einstein

The Lorentz transformations are the mathematical basis of Einstein’s theory of special relativity. We conduct a thorough examination of the method of derivation of the Lorentz transformations used by Einstein and identify the cause of the incorrect implementation of the method. The cause is related to the incorrect proof of the equality φ(v)=1for the unknown function φ(v)arising in the process of derivation of the Lorentz transformations. We develop proof of the equality φ(v)=1and eliminate the cause of the incorrect implementation of the method of derivation of the Lorentz transformations used by Einstein.

How Software Engineering Transforms Organizations: An Open and Qualitative Study on the Organizational Objectives and Motivations in Agile Transformations

Agile Transformations are challenging processes for organizations that look to extend the benefits of Agile philosophy and methods beyond software engineering.Despite the impact of these transformations on orga-nizations,they have not been extensively studied in academia.We conducted a study grounded in workshops and interviews with 99 participants from 30 organizations,including organizations undergoing transformations(“final organizations”)and companies supporting these processes(“consultants”).The study aims to understand the motivations,objectives,and factors driving and challenging these transformations.Over 700 responses were collected to the question and categorized into 32 objectives.The findings show that organizations primarily aim to achieve customer centricity and adaptability,both with 8%of the mentions.Other primary important objectives,with above 4%of mentions,include alignment of goals,lean delivery,sustainable processes,and a flatter,more team-based organizational structure.We also detect discrepancies in perspectives between the objectives identified by the two kinds of organizations and the existing agile literature and models.This misalignment highlights the need for practitioners to understand with the practical realities the organizations face.

融合快速边缘注意力的Transformer跟踪算法

针对长期目标跟踪中出现模型退化和跟踪漂移的问题,提出了一种融合快速边缘注意力的Transformer跟踪算法TransFEA(fast edge attention on Transformer)。使用ResNet-50作为Siamese网络的骨干网络,并在其每个残差块后端引入注意力网络进行特征提取,增强目标的关键信息和全局信息;边缘注意力网络(edge attention network,EA)提取模板与搜索区域的特征向量,快速注意力网络(fast attention network,FA)计算注意响应值,确定两个区域的相似度,以此调整目标位置。设计多层感知器预测边界框,避免过多超参数,使跟踪器实现了准确性与轻量化的平衡。实验结果表明,TransFEA在LaSOT数据集上成功率和准确率分别为65.3%、69.1%,运行可以达到90 FPS,提高了长期跟踪的成功率和准确率。

视觉Transformer(ViT)发展综述

视觉Transformer(Vision Transformer,ViT)是基于编码器-解码器结构的Transformer改进模型,已经被成功应用于计算机视觉领域。近几年基于ViT的研究层出不穷且效果显著,基于该模型的工作已经成为计算机视觉任务的重要研究方向,因此针对近年来ViT的发展进行概述。首先,简要回顾了ViT的基本原理及迁移过程,并分析了ViT模型的结构特点和优势;然后,根据各ViT变体模型的改进特点,归纳和梳理了基于ViT的主要骨干网络变体改进方向及其代表性改进模型,包括局部性改进、结构改进、自监督、轻量化及效率改进等改进方向,并对其进行分析比较;最后,讨论了当前ViT及其改进模型仍存在的不足,对ViT未来的研究方向进行了展望。可以作为研究人员进行基于ViT骨干网络的研究时选择深度学习相关方法的一个权衡和参考。

Transformations in medical education:A prudential perspective

1 | BACKGROUND.A decade ago we asked the question, “Is there something inmedicine that is eternal or enduring?” Our aim was to writea manuscript entitled, “That which does not change inmedicine.” The publication begins as follows: “The practiceof medicine involves continual change, driven by a constantstream of developments in the understanding of biologicalstructure and function relevant to human diseases, and theparallel improvements in pharmacologic and other technologicalinterventions. This change is also driven by evolvingsocial philosophies, ethical trends, and lifestyles.” [1] Thatpreamble reverberates as strongly now as then, perhapseven more so, given the velocity of technological change.When we deliberated on which aspects of medical practiceshould remain stable, we had few premonitions thatimplantable chips, robotic surgery, virtual reality, andartificial intelligence (AI) would soon become ubiquitous.The needle has clearly moved, propelled by extraordinaryadvances in bioengineering, computer and data sciences;major shifts in the governance and organization of clinicalpractice;and powerful sociocultural trends.

A Cross Attention Transformer-Mixed Feedback Video Recommendation Algorithm Based on DIEN

The rapid development of short video platforms poses new challenges for traditional recommendation systems.Recommender systems typically depend on two types of user behavior feedback to construct user interest profiles:explicit feedback(interactive behavior),which significantly influences users’short-term interests,and implicit feedback(viewing time),which substantially affects their long-term interests.However,the previous model fails to distinguish between these two feedback methods,leading it to predict only the overall preferences of users based on extensive historical behavior sequences.Consequently,it cannot differentiate between users’long-term and shortterm interests,resulting in low accuracy in describing users’interest states and predicting the evolution of their interests.This paper introduces a video recommendationmodel calledCAT-MFRec(CrossAttention Transformer-Mixed Feedback Recommendation)designed to differentiate between explicit and implicit user feedback within the DIEN(Deep Interest Evolution Network)framework.This study emphasizes the separate learning of the two types of behavioral feedback,effectively integrating them through the cross-attention mechanism.Additionally,it leverages the long sequence dependence capabilities of Transformer technology to accurately construct user interest profiles and predict the evolution of user interests.Experimental results indicate that CAT-MF Rec significantly outperforms existing recommendation methods across various performance indicators.This advancement offers new theoretical and practical insights for the development of video recommendations,particularly in addressing complex and dynamic user behavior patterns.

Unraveling the significance of cobalt on transformation kinetics,crystallography and impact toughness in high-strength steels

This work reveals the significant effects of cobalt(Co)on the microstructure and impact toughness of as-quenched highstrength steels by experimental characterizations and thermo-kinetic analyses.The results show that the Co-bearing steel exhibits finer blocks and a lower ductile-brittle transition temperature than the steel without Co.Moreover,the Co-bearing steel reveals higher transformation rates at the intermediate stage with bainite volume fraction ranging from around 0.1 to 0.6.The improved impact toughness of the Co-bearing steel results from the higher dense block boundaries dominated by the V1/V2 variant pair.Furthermore,the addition of Co induces a larger transformation driving force and a lower bainite start temperature(BS),thereby contributing to the refinement of blocks and the increase of the V1/V2 variant pair.These findings would be instructive for the composition,microstructure design,and property optimization of high-strength steels.

Historical Changes,Contemporary Characteristics,and Five Transformations of Environmental Design in China

Starting from the characteristics of modernization in three different historical stages since the founding of the People’s Republic of China,this paper analyses the internal changes in the professional name,subordinate disciplines and categories,and professional mission of the environmental design profession,and summarizes the changes in the characteristics of the environmental design profession in 10 aspects such as subordinate disciplines,design objects,design types,cultural characteristics,and aesthetic values on the basis of the results of the analysis,and explores five transformations of the environmental design profession in the process of China’s development in the future.The five transformations in the future development of China are to serve the harmony and beauty of the three major urban and rural spaces with“small and micro-renewal and reconstruction”;to serve the transmission of Chinese culture and the spread of Chinese civilization with“spatial heritage and innovation”;to serve the balance and sufficiency of the four functions of urban and rural settlements with“friendliness and equilibrium”;to serve the intelligence,wisdom,and enjoyment of indoor and outdoor living environment space with“scientific and technological achievements”;and to serve the co-construction,co-management,and sharing of indoor and outdoor public space in urban and rural areas with“public participation.”

Soliton Solutions of a Coupled KdV System via Backlund Transformation

In this paper we use Böcklund transformation to construct soliton solutions for a coupled KdV system.This system was first proposed by Wang in 2010.First we generalize the well-known Bäcklund transformation for the KdV equation to such coupled KdV system.Then from a trivial seed solution,we construct soliton solutions.We also give a nonlinear superposition formula,which allows us to generate multi-soliton solutions.