基于双向稀疏Transformer的多变量时序分类模型

针对多变量时序(Multivariate Time Series,MTS)分类中长序列数据难以捕捉时序特征的问题,提出一种基于双向稀疏Transformer的时序分类模型BST(Bidirectional Sparse Transformer),提高了MTS分类任务的准确度.BST模型使用Transformer框架,构建了一种基于活跃度得分的双向稀疏注意力机制.基于KL散度构建活跃度评价函数,并将评价函数的非对称问题转变为对称权重问题.据此,对原有查询矩阵、键值矩阵进行双向稀疏化,从而降低原Transformer模型中自注意力机制运算的时间复杂度.实验结果显示,BST模型在9个长序列数据集上取得最高平均排名,在临界差异图中领先第2名35.7%,对于具有强时序性的乙醇浓度数据集(Ethanol Concentration,EC),分类准确率提高30.9%.

结合坐标Transformer的轻量级人体姿态估计算法

针对现有的大多数自底向上人体姿态估计算法存在模型规模大、计算成本高及对边缘设备不友好等问题,提出了一种基于YOLOv5s6-Pose的轻量级多人姿态估计网络模型YOLOv5s6-Pose-CT。该模型在颈部网络中引入空间和通道重建卷积,以减少空间和通道维度上的特征冗余。同时,提出了一种坐标Transformer嵌入于主干网络中,使模型专注于长距离依赖和拥有高效的局部特征提取能力。其次,通过使用无偏特征位置对齐来解决多尺度融合过程中出现的特征错位问题。最后,使用损失函数MPDIoU对边界框的回归损失重新定义。在COCO 2017数据集上的实验结果表明,本文优化的网络模型与主流的轻量级网络EfficientHRNet-H1模型相比,在保持相同精度的同时,参数量和计算量分别减少16.2%和66.1%。相比于基准模型YOLOv5s6-Pose,参数量减少11.2%,计算量降低5.8%,平均检测精度和平均召回率分别提升2.5%和2.6%。

QUATERNIONIC SLICE REGULAR FUNCTIONS AND QUATERNIONIC LAPLACE TRANSFORMS

The functions studied in the paper are the quaternion-valued functions of a quaternionic variable.It is shown that the left slice regular functions and right slice regular functions are related by a particular involution,and that the intrinsic slice regular functions play a central role in the theory of slice regular functions.The relation between left slice regular functions,right slice regular functions and intrinsic slice regular functions is revealed.As an application,the classical Laplace transform is generalized naturally to quaternions in two different ways,which transform a quaternion-valued function of a real variable to a left or right slice regular function.The usual properties of the classical Laplace transforms are generalized to quaternionic Laplace transforms.

Brain Functional Network Generation Using Distribution-Regularized Adversarial Graph Autoencoder with Transformer for Dementia Diagnosis

The topological connectivity information derived from the brain functional network can bring new insights for diagnosing and analyzing dementia disorders.The brain functional network is suitable to bridge the correlation between abnormal connectivities and dementia disorders.However,it is challenging to access considerable amounts of brain functional network data,which hinders the widespread application of data-driven models in dementia diagnosis.In this study,a novel distribution-regularized adversarial graph auto-Encoder(DAGAE)with transformer is proposed to generate new fake brain functional networks to augment the brain functional network dataset,improving the dementia diagnosis accuracy of data-driven models.Specifically,the label distribution is estimated to regularize the latent space learned by the graph encoder,which canmake the learning process stable and the learned representation robust.Also,the transformer generator is devised to map the node representations into node-to-node connections by exploring the long-term dependence of highly-correlated distant brain regions.The typical topological properties and discriminative features can be preserved entirely.Furthermore,the generated brain functional networks improve the prediction performance using different classifiers,which can be applied to analyze other cognitive diseases.Attempts on the Alzheimer’s Disease Neuroimaging Initiative(ADNI)dataset demonstrate that the proposed model can generate good brain functional networks.The classification results show adding generated data can achieve the best accuracy value of 85.33%,sensitivity value of 84.00%,specificity value of 86.67%.The proposed model also achieves superior performance compared with other related augmentedmodels.Overall,the proposedmodel effectively improves cognitive disease diagnosis by generating diverse brain functional networks.

Development of a rapid and efficient system for CR genes identification based on hairy root transformation in Brassicaceae

Many economically important crops and vegetables belonging to the cruciferous family are heavily endangered by clubroot disease caused by Plasmodiophora brassicae infection.Breeding of clubroot resistant cultivars based on mapping and cloning of resistant genes is commonly regarded as the most cost-effective and efficient way to fight against this disease.The traditional way of R gene functional validation requires stable transformation that is both time-and labor-consuming.In this study,a rapid and efficient hairy-root transgenic protocol mediated by Agrobacterium rhizogenes was developed.The transformation positive rate was over 80%in Brassica napus showed by GUS reporter gene and this transformation only took 1/6 of the time compared with stable transformation.The system was applicable to different B.napus varieties and other cruciferous crops including Brassica rapa and Brassica oleracea.In particular,two known CR genes,CRA3.7.1 and CRA8.2.4 were used respectively,as example to show that the system works well for CR gene study combined with subsequent P.brassicae infection in B.napus.Most importantly,it works both in over-expression that led to disease resistance,as well as in RNAi which led to disease susceptible phenotype.Therefore,this system can be used in batch-wise identification of CR genes,and also offered the possibility of manipulating key genes within the P.brassicae genome that could improve our knowledge on host-pathogen interaction.

多尺度Transformer的在线更新无锚框工件跟踪方法研究

针对工业场景目标工件跟踪任务精度低、失败率高的问题,提出了多尺度Transformer在线更新的工件跟踪算法。首先,采用Transformer特征金字塔结构,融合多层次特征信息,以实现鲁棒的对目标表观建模;其次,使用Transformer模块对高级语义信息进行特征融合,使得网络模型专注于目标工件本身;然后,提出了基于排序的交并化(IoU)损失函数优化策略,有效地抑制干扰物对跟踪器影响;最后,设计一种在线更新策略更新目标模板,增强网络的鲁棒性。实验结果表明,在VOT-2018上准确率和失败率分别比基准跟踪器提高3.8%和4.1%,且能保持53 fps的实时跟踪速度;在LaSOT数据集上精度与成功率别为0.578和0.573,均优于基准跟踪器。通过CCD工业相机采集视频序列验证算法可以准确且鲁棒的跟踪目标工件。

基于Swin-Transformer与生成对抗网络的地震随机噪声压制方法

目前深度学习类地震数据去噪方法大多基于卷积神经网络,而此类方法受限于卷积核的局部操作,缺少对地震数据全局特征的分析,因而降低了去噪效果。另外,以L 1,L 2损失函数为指标的网络模型容易出现过度平滑效应,产生虚假同相轴以及虚高的峰值信噪比(PSNR)与结构相似性(SSIM)值。为此,提出一种基于Swin-Transformer(Swin-T)和生成对抗网络的去噪方法(ST-GAN)。该方法以Swin-Transformer作为生成对抗网络中的生成网络对地震数据去噪,判别网络基于卷积神经网络。Transformer的自注意力机制是全局操作,可以有效提取地震数据的全局特征,并能与卷积神经网络的局部操作互补,提升网络模型的特征提取能力。而对抗损失则提升了网络模型的细节恢复能力,有效避免因过度平滑效应产生的同相轴假象。将该方法应用于地震数据去噪,并与现有去噪方法进行对比分析,实验结果表明,该方法具有更加优异的特征提取能力,能在有效压制随机噪声的同时,恢复和保留更多的细节信息,提高了地震信号的信噪比。

改进视觉Transformer的人脸识别方法

目前大多数人脸识别方法依赖于卷积神经网络,通过级联的形式构建多层处理单元,利用卷积操作融合局部特征,忽略了人脸全局语义信息,缺乏对人脸重点区域的关注度。针对上述问题,提出一种基于改进视觉Transformer的人脸识别方法,引入Shuffle Transformer作为特征提取骨干网络,通过自注意力机制以及Shuffle操作捕捉特征图全局信息,建立特征点之间的长距离依赖关系,提高模型的特征感知能力;同时,结合ArcFace损失函数和中心损失函数的特点,设计融合损失作为目标函数,利用类内约束扩大角度间隔,提高特征空间的辨别性。该方法在LFW、CALFW、CPLFW、AgeDB-30和CFP五个具有挑战性的基准测试人脸数据集上分别取得了99.83%、95.87%、90.05%、98.05%、97.23%的平均准确率,能够有效提升人脸特征提取能力,识别效果优于同等规模卷积神经网络。

融合Swin Transformer与UNet的云检测架构

目前在遥感图像云检测的领域中,对于薄云和碎云以及不同下垫面的云检测精度还有待进一步提高,且存在较多误判漏判和数据不均衡等问题。针对这些问题,提出了一个结合Transformer和卷积思想的云检测网络架构Cloud TransUnet。首先,使用Transformer模块替换掉UNet编码器阶段的卷积模块,利用Transformer的全局注意力机制提取出更多的细节特征信息,减少误判和漏判的发生;同时,添加边缘检测模块,加强对边缘信息的提取,提高检测的精度;最后,使用中值频率平衡对损失函数进行改进以处理数据不平衡的问题。实验结果表明,Cloud TransUnet对薄云和碎云的检测、不同下垫面的云检测都具有良好的表现,与卷积语义分割网络UNet、SegNet、ResUnet和Swin_Unet相比,Cloud TransUnet在检测速度略微提升的同时,其MIoU、总体精度、精确率等指标都得到了一定的提升。

Transforming growth factor beta 1, a cytokine with regenerative functions

We review the biology and role of transforming growth factor beta 1 （TGF-β1） in peripheral nerve injury and regeneration, as it relates to injuries to large nerve trunks （i.e., sciatic nerve, brachial plexus）, which often leads to suboptimal functional recovery. Experimental studies have suggested that the reason for the lack of functional recovery resides in the lack of sufficient mature axons reaching their targets, which is a result of the loss of the growth-supportive environment provided by the Schwann cells in the distal stump of injured nerves. Using an established chronic nerve injury and delayed repair animal model that accu- rately mimics chronic nerve injuries in humans, we summarize our key findings as well as others to better understand the pathophysiology of poor functional recovery. We demonstrated that 6 month TGF-β1 treat- ment for chronic nerve injury significantly improved Schwann cell capacity to support axonal regeneration. When combined with forskolin, the effect was additive, as evidenced by a near doubling of regenerated axons proximal to the repair site. We showed that in vivo application of TGF-β1 and forskolin directly onto chronically injured nerves reactivated chronically denervated Schwann cells, induced their proliferation, and upregulated the expression of regeneration-associated proteins. The effect of TGF-β1 and forskolin on old nerve injuries is quite impressive and the treatment regiment appears to mediate a growth-supportive milieu in the injured peripheral nerves. In summary, TGF-β1 and forskolin treatment reactivates chronical- ly denervated Schwann cells and could potentially be used to extend and prolong the regenerative responses to promote axonal regeneration.

REFINEMENTS OF TRANSFORMATION INEQUALITIES FOR ZERO-BALANCED HYPERGEOMETRIC FUNCTIONS

In the article, we present some refinements of three classes of transformation inequalities for zero-balanced hypergeometric functions by use of the updated monotonicity criterion for the quotient of power series.

Changes in Transformation of Soil Organic C and Functional Diversity of Soil Microbial Community Under Different Land Uses

Changes in soil biological and biochemical properties under different land uses in the subtropical region of China were investigated in order to develop rational cultivation and fertilization management. A small watershed of subtropical region of China was selected for this study. Land uses covered paddy fields, vegetable farming, fruit trees, upland crops, bamboo stands, and forestry. Soil biological and biochemical properties included soil organic C and nutrient contents, mineralization of soil organic C, and soil microbial biomass and community functional diversity. Soil organic C and total N contents, microbial biomass C and N, and respiration intensity under different land uses were changed in the following order： paddy fields （and vegetable farming） 〉 bamboo stands 〉 fruit trccs （and upland）. The top surface （0-15 cm） paddy fields （and vegetable farming） were 76.4 and 80.8% higher in soil organic C and total N contents than fruit trees （and upland） soils, respectively. Subsurface paddy soils （15-30 cm） were 59.8 and 67.3% higher in organic C and total N than upland soils, respectively. Soil microbial C, N and respiration intensity in paddy soils （0-15 cm） were 6.36, 3.63 and 3.20 times those in fruit tree （and upland） soils respectively. Soil microbial metabolic quotient was in the order： fruit trees （and upland） 〉 forestry 〉 paddy fields. Metabolic quotient in paddy soils was only 47.7% of that in fruit tree （and upland） soils. Rates of soil organic C mineralization during incubation changed in the order： paddy fields 〉 bamboo stands 〉 fruit trees （and upland） and soil bacteria population： paddy fields 〉 fruit trees （and upland） 〉 forestry. No significant difference was found for fungi and actinomycetes populations. BIOLOG analysis indicated a changing order of paddy fields 〉 fruit trees （and upland） 〉 forestry in values of the average well cell development （AWCD） and functional diversity indexes of microbial community. Results also showed that the conversion from paddy fields to vegetable farming for 5 years resulted in a dramatic increase in soil available phosphorus content while insignificant changes in soil organic C and total N content due to a large inputs of phosphate fertilizers. This conversion caused 53, 41.5, and 41.3% decreases in soil microbial biomass C, N, and respiration intensity, respectively, while 23.6% increase in metabolic quotient and a decrease in soil organic C mineralization rate. Moreover, soil bacteria and actinomycetes populations were increased slightly, while fungi population increased dramatically. Functional diversity indexes of soil microbial community decreased significantly. It was concluded that land uses in the subtropical region of China strongly affected soil biological and biochemical properties. Soil organic C and nutrient contents, mineralization of organic C and functional diversity of microbial community in paddy fields were higher than those in upland and forestry. Overuse of chemical fertilizers in paddy fields with high fertility might degrade soil biological properties and biochemical function, resulting in deterioration of soil biological quality.

Local Extrema of Periodic Function’s Wavelet Transform

The theory of detecling ridges in the modulus of the continuous wavelet transform is presented as well as reconstructing signal by using information on ridges,To periodic signal we suppose Morlet wavelet as basic wavelet, and research the local extreme point and extrema of the wavelet transform on periodic function for the collection of signal＇ s instantaneous amplitude and period.

Thermodynamic Functions and Phase Transformation of Metal Nanocrystals

A model to calculate the thermodynamic functions of the pure metal nanocrystals has been developed, with the consideration of the effects of both the interfaces and the crystal in the nano-grain interior. As an example, the enthalpy, entropy and Gibbs free energy, as functions of the excess free volume at interfaces, temperature and grain size, are calculated for the Co nanocrystals. Furthermore, the characteristics of β-Co→α-Co phase transformation are studied, and the transformation temperatures at different levels of grain size, as well as the critical grain sizes at different temperatures, are predicted. The calculation results show that, the nano-grained β-Co （fcc） is thermodynamically stable at temperatures much lower than that for the conventional coarse-grained materials, and may also stably exist at room temperature when the grain size is reduced to be small enough. The present model is verified by comparisons between the experimental findings and the theoretical predictions.

An extended functional transformation method and its application in some evolution equations

In this paper, an extended functional transformation is given to solve some nonlinear evolution equations. This function, in fact, is a solution of the famous KdV equation, so this transformation gives a transformation between KdV equation and other soliton equations. Then many new exact solutions can be given by virtue of the solutions of KdV equation.

A whole-space transform formula of cylindrical wave functions for scattering problems

The theory of elastic wave scattering is a fundamental concept in the study of elastic dynamics and wave motion,and the wave function expansion technique has been widely used in many subjects.To supply the essential tools for solving wave scattering problems induced by an eccentric source or multi-sources as well as multi-scatters,a whole-space transform formula of cylindrical wave functions is presented and its applicability to some simple cases is demonstrated in this study.The transforms of wave functions in cylindrical coordinates can be classifi ed into two basic types: interior transform and exterior transform,and the existing Graf’s addition theorem is only suitable for the former.By performing a new replacement between the two coordinates,the exterior transform formula is fi rst deduced.It is then combined with Graf’s addition theorem to establish a whole-space transform formula.By using the whole-space transform formula,the scattering solutions by the sources outside and inside a cylindrical cavity are constructed as examples of its application.The effectiveness and advantages of the whole-space transform formula is illustrated by comparison with the approximate model based on a large cycle method.The whole-space transform formula presented herein can be used to perform the transform between two different cylindrical coordinates in the whole space.In addition,its concept and principle are universal and can be further extended to establish the coordinate transform formula of wave functions in other coordinate systems.

Steganography based on wavelet transform and modulus function

In order to provide larger capacity of the hidden secret data while maintaining a good visual quality of stego-image, in accordance with the visual property that human eyes are less sensitive to strong texture, a novel steganographic method based on wavelet and modulus function is presented. First, an image is divided into blocks of prescribed size, and every block is decomposed into one-level wavelet. Then, the capacity of the hidden secret data is decided with the number of wavelet coefficients of larger magnitude. Finally, secret information is embedded by steganography based on modulus function. From the experimental results, the proposed method hides much more information and maintains a good visual quality of stego-image. Besides, the embedded data can be extracted from the stego-image without referencing the original image.

Mechanisms of Transforming CH_x to CO on Ni(111) Surface by Density Functional Theory

To elucidate feasible routes of producing CO from CH 3 and unravel the eff ect of adsorbed O on CH x transformation, the reactivity of CH x ( x = 1-3) with and without the assistance of adsorbed atomic O on Ni(111) was explored using density functional theory calculations. The adsorption energies of CH x ( x = 0-3) were found to be signifi cantly reduced on an O-preadsorbed Ni(111) surface compared to a pure surface. Furthermore, O-assisted one-step dehydrogenation of CH x ( x = 1-3) features energy barriers and thus is diffi cult to proceed. In terms of energy, the direct dissociation of CH 3 is favorable, except for the last CH dehydrogenation, which is energy intensive. Interestingly, in O-assisted two-step CH transformation to CO via CHO intermediate, the barrier is dramatically lowered. The successive dehydrogenations of CH x O ( x = 1-3) were also found to be a route for CO formation. Finally, two possible pathways from CH 3 to CO are proposed:(a) CH 3 → CH 2 → CH → CHO → CO;(b) CH 3 → CH 3 O → CH 2 O → CHO → CO.

Comprehensive Reliability Allocation Method for CNC Lathes Based on Cubic Transformed Functions of Failure Mode and Effects Analysis

Reliability allocation of computerized numerical controlled（CNC）lathes is very important in industry.Traditional allocation methods only focus on high-failure rate components rather than moderate failure rate components,which is not applicable in some conditions.Aiming at solving the problem of CNC lathes reliability allocating,a comprehensive reliability allocation method based on cubic transformed functions of failure modes and effects analysis（FMEA）is presented.Firstly,conventional reliability allocation methods are introduced.Then the limitations of direct combination of comprehensive allocation method with the exponential transformed FMEA method are investigated.Subsequently,a cubic transformed function is established in order to overcome these limitations.Properties of the new transformed functions are discussed by considering the failure severity and the failure occurrence.Designers can choose appropriate transform amplitudes according to their requirements.Finally,a CNC lathe and a spindle system are used as an example to verify the new allocation method.Seven criteria are considered to compare the results of the new method with traditional methods.The allocation results indicate that the new method is more flexible than traditional methods.By employing the new cubic transformed function,the method covers a wider range of problems in CNC reliability allocation without losing the advantages of traditional methods.

Relation between Fresnel transform of input light field and the two-parameter Radon transform of Wigner function of the field

This paper proves a new theorem on the relationship between optical field Wigner function＇s two-parameter Radon transform and optical Fresnel transform of the field, i.e., when an input field ψ （x＇） propagates through an optical [D （-B） （-C） A] system, the energy density of the output field is equal to the Radon transform of the Wigner function of the input field, where the Radon transform parameters are D, B. It prove this theorem in both spatial-domain and frequency-domain, in the latter case the Radon transform parameters are A, C.