基于双向稀疏Transformer的多变量时序分类模型

针对多变量时序(Multivariate Time Series,MTS)分类中长序列数据难以捕捉时序特征的问题,提出一种基于双向稀疏Transformer的时序分类模型BST(Bidirectional Sparse Transformer),提高了MTS分类任务的准确度.BST模型使用Transformer框架,构建了一种基于活跃度得分的双向稀疏注意力机制.基于KL散度构建活跃度评价函数,并将评价函数的非对称问题转变为对称权重问题.据此,对原有查询矩阵、键值矩阵进行双向稀疏化,从而降低原Transformer模型中自注意力机制运算的时间复杂度.实验结果显示,BST模型在9个长序列数据集上取得最高平均排名,在临界差异图中领先第2名35.7%,对于具有强时序性的乙醇浓度数据集(Ethanol Concentration,EC),分类准确率提高30.9%.

结合Transformer与生成对抗网络的水下图像增强算法

由于水下环境的多样性和光在水中受到的散射及选择性吸收作用,采集到的水下图像通常会产生严重的质量退化问题,如颜色偏差、清晰度低和亮度低等,为解决以上问题,本文提出了一种基于Transformer和生成对抗网络的水下图像增强算法。以生成对抗网络为基础架构,结合编码解码结构、基于空间自注意力机制的全局特征建模Transformer模块和通道级多尺度特征融合Transformer模块构建了TGAN(generative adversarial network with transformer)网络增强模型,重点关注水下图像衰减更严重的颜色通道和空间区域,有效增强了图像细节并解决了颜色偏差问题。此外,设计了一种结合RGB和LAB颜色空间的多项损失函数,约束网络增强模型的对抗训练。实验结果表明,与CLAHE(contrast limited adaptive histogram equalization)、UDCP(underwater dark channel prior)、UWCNN(underwater based on convolutional neural network)、FUnIE-GAN(fast underwater image enhancement for improved visual perception)等典型水下图像增强算法相比,所提算法增强后的水下图像在清晰度、细节纹理和色彩表现等方面都有所提升,客观评价指标如峰值信噪比、结构相似性和水下图像质量度量的平均值分别提升了5.8%、1.8%和3.6%,有效地提升了水下图像的视觉感知效果。

结合坐标Transformer的轻量级人体姿态估计算法

针对现有的大多数自底向上人体姿态估计算法存在模型规模大、计算成本高及对边缘设备不友好等问题,提出了一种基于YOLOv5s6-Pose的轻量级多人姿态估计网络模型YOLOv5s6-Pose-CT。该模型在颈部网络中引入空间和通道重建卷积,以减少空间和通道维度上的特征冗余。同时,提出了一种坐标Transformer嵌入于主干网络中,使模型专注于长距离依赖和拥有高效的局部特征提取能力。其次,通过使用无偏特征位置对齐来解决多尺度融合过程中出现的特征错位问题。最后,使用损失函数MPDIoU对边界框的回归损失重新定义。在COCO 2017数据集上的实验结果表明,本文优化的网络模型与主流的轻量级网络EfficientHRNet-H1模型相比,在保持相同精度的同时,参数量和计算量分别减少16.2%和66.1%。相比于基准模型YOLOv5s6-Pose,参数量减少11.2%,计算量降低5.8%,平均检测精度和平均召回率分别提升2.5%和2.6%。

Brain Functional Network Generation Using Distribution-Regularized Adversarial Graph Autoencoder with Transformer for Dementia Diagnosis

The topological connectivity information derived from the brain functional network can bring new insights for diagnosing and analyzing dementia disorders.The brain functional network is suitable to bridge the correlation between abnormal connectivities and dementia disorders.However,it is challenging to access considerable amounts of brain functional network data,which hinders the widespread application of data-driven models in dementia diagnosis.In this study,a novel distribution-regularized adversarial graph auto-Encoder(DAGAE)with transformer is proposed to generate new fake brain functional networks to augment the brain functional network dataset,improving the dementia diagnosis accuracy of data-driven models.Specifically,the label distribution is estimated to regularize the latent space learned by the graph encoder,which canmake the learning process stable and the learned representation robust.Also,the transformer generator is devised to map the node representations into node-to-node connections by exploring the long-term dependence of highly-correlated distant brain regions.The typical topological properties and discriminative features can be preserved entirely.Furthermore,the generated brain functional networks improve the prediction performance using different classifiers,which can be applied to analyze other cognitive diseases.Attempts on the Alzheimer’s Disease Neuroimaging Initiative(ADNI)dataset demonstrate that the proposed model can generate good brain functional networks.The classification results show adding generated data can achieve the best accuracy value of 85.33%,sensitivity value of 84.00%,specificity value of 86.67%.The proposed model also achieves superior performance compared with other related augmentedmodels.Overall,the proposedmodel effectively improves cognitive disease diagnosis by generating diverse brain functional networks.

QUATERNIONIC SLICE REGULAR FUNCTIONS AND QUATERNIONIC LAPLACE TRANSFORMS

The functions studied in the paper are the quaternion-valued functions of a quaternionic variable.It is shown that the left slice regular functions and right slice regular functions are related by a particular involution,and that the intrinsic slice regular functions play a central role in the theory of slice regular functions.The relation between left slice regular functions,right slice regular functions and intrinsic slice regular functions is revealed.As an application,the classical Laplace transform is generalized naturally to quaternions in two different ways,which transform a quaternion-valued function of a real variable to a left or right slice regular function.The usual properties of the classical Laplace transforms are generalized to quaternionic Laplace transforms.

Development of a rapid and efficient system for CR genes identification based on hairy root transformation in Brassicaceae

Many economically important crops and vegetables belonging to the cruciferous family are heavily endangered by clubroot disease caused by Plasmodiophora brassicae infection.Breeding of clubroot resistant cultivars based on mapping and cloning of resistant genes is commonly regarded as the most cost-effective and efficient way to fight against this disease.The traditional way of R gene functional validation requires stable transformation that is both time-and labor-consuming.In this study,a rapid and efficient hairy-root transgenic protocol mediated by Agrobacterium rhizogenes was developed.The transformation positive rate was over 80%in Brassica napus showed by GUS reporter gene and this transformation only took 1/6 of the time compared with stable transformation.The system was applicable to different B.napus varieties and other cruciferous crops including Brassica rapa and Brassica oleracea.In particular,two known CR genes,CRA3.7.1 and CRA8.2.4 were used respectively,as example to show that the system works well for CR gene study combined with subsequent P.brassicae infection in B.napus.Most importantly,it works both in over-expression that led to disease resistance,as well as in RNAi which led to disease susceptible phenotype.Therefore,this system can be used in batch-wise identification of CR genes,and also offered the possibility of manipulating key genes within the P.brassicae genome that could improve our knowledge on host-pathogen interaction.

多尺度Transformer的在线更新无锚框工件跟踪方法研究

针对工业场景目标工件跟踪任务精度低、失败率高的问题,提出了多尺度Transformer在线更新的工件跟踪算法。首先,采用Transformer特征金字塔结构,融合多层次特征信息,以实现鲁棒的对目标表观建模;其次,使用Transformer模块对高级语义信息进行特征融合,使得网络模型专注于目标工件本身;然后,提出了基于排序的交并化(IoU)损失函数优化策略,有效地抑制干扰物对跟踪器影响;最后,设计一种在线更新策略更新目标模板,增强网络的鲁棒性。实验结果表明,在VOT-2018上准确率和失败率分别比基准跟踪器提高3.8%和4.1%,且能保持53 fps的实时跟踪速度;在LaSOT数据集上精度与成功率别为0.578和0.573,均优于基准跟踪器。通过CCD工业相机采集视频序列验证算法可以准确且鲁棒的跟踪目标工件。

基于Swin-Transformer与生成对抗网络的地震随机噪声压制方法

目前深度学习类地震数据去噪方法大多基于卷积神经网络,而此类方法受限于卷积核的局部操作,缺少对地震数据全局特征的分析,因而降低了去噪效果。另外,以L 1,L 2损失函数为指标的网络模型容易出现过度平滑效应,产生虚假同相轴以及虚高的峰值信噪比(PSNR)与结构相似性(SSIM)值。为此,提出一种基于Swin-Transformer(Swin-T)和生成对抗网络的去噪方法(ST-GAN)。该方法以Swin-Transformer作为生成对抗网络中的生成网络对地震数据去噪,判别网络基于卷积神经网络。Transformer的自注意力机制是全局操作,可以有效提取地震数据的全局特征,并能与卷积神经网络的局部操作互补,提升网络模型的特征提取能力。而对抗损失则提升了网络模型的细节恢复能力,有效避免因过度平滑效应产生的同相轴假象。将该方法应用于地震数据去噪,并与现有去噪方法进行对比分析,实验结果表明,该方法具有更加优异的特征提取能力,能在有效压制随机噪声的同时,恢复和保留更多的细节信息,提高了地震信号的信噪比。

基于轻量化Transformer模型的多变量风电功率预测

风电功率预测对电力调度和新能源管理极为重要。为准确高效预测多变量风电功率,提出一种基于轻量化Transformer(Light Transformer)的风电功率预测方法。首先,采用滚动序列建模方法,确定输入数据,然后采用Transformer预测模型,改进和精简原始结构,在前馈网络模块中使用GeLU激活函数来代替传统的ReLU激活函数,提升模型的质量,轻量化网络结构,利用多头注意力机制,加快模型训练速度,提升预测模型精度。

The Similarity Transformation of Regularizing Functionals and Simularity Property of Their Fourier Transformations

Let and denote respectively the functionswhere λ≥1, The author discusses the similarity transformation of the regularizing functionals of these functions and the similar property of their Fourier transformation.

The Fourier Transform of a Sort of Generalized Function

The Fourier transforms of functions X ln e are given. where λ is a complex constant (λ≠-1,-2. …), a is a complex number. m=0. 1. 2,…,and

改进视觉Transformer的人脸识别方法

目前大多数人脸识别方法依赖于卷积神经网络,通过级联的形式构建多层处理单元,利用卷积操作融合局部特征,忽略了人脸全局语义信息,缺乏对人脸重点区域的关注度。针对上述问题,提出一种基于改进视觉Transformer的人脸识别方法,引入Shuffle Transformer作为特征提取骨干网络,通过自注意力机制以及Shuffle操作捕捉特征图全局信息,建立特征点之间的长距离依赖关系,提高模型的特征感知能力;同时,结合ArcFace损失函数和中心损失函数的特点,设计融合损失作为目标函数,利用类内约束扩大角度间隔,提高特征空间的辨别性。该方法在LFW、CALFW、CPLFW、AgeDB-30和CFP五个具有挑战性的基准测试人脸数据集上分别取得了99.83%、95.87%、90.05%、98.05%、97.23%的平均准确率,能够有效提升人脸特征提取能力,识别效果优于同等规模卷积神经网络。

融合Swin Transformer与UNet的云检测架构

目前在遥感图像云检测的领域中,对于薄云和碎云以及不同下垫面的云检测精度还有待进一步提高,且存在较多误判漏判和数据不均衡等问题。针对这些问题,提出了一个结合Transformer和卷积思想的云检测网络架构Cloud TransUnet。首先,使用Transformer模块替换掉UNet编码器阶段的卷积模块,利用Transformer的全局注意力机制提取出更多的细节特征信息,减少误判和漏判的发生;同时,添加边缘检测模块,加强对边缘信息的提取,提高检测的精度;最后,使用中值频率平衡对损失函数进行改进以处理数据不平衡的问题。实验结果表明,Cloud TransUnet对薄云和碎云的检测、不同下垫面的云检测都具有良好的表现,与卷积语义分割网络UNet、SegNet、ResUnet和Swin_Unet相比,Cloud TransUnet在检测速度略微提升的同时,其MIoU、总体精度、精确率等指标都得到了一定的提升。

Transforming growth factor beta 1, a cytokine with regenerative functions

We review the biology and role of transforming growth factor beta 1 （TGF-β1） in peripheral nerve injury and regeneration, as it relates to injuries to large nerve trunks （i.e., sciatic nerve, brachial plexus）, which often leads to suboptimal functional recovery. Experimental studies have suggested that the reason for the lack of functional recovery resides in the lack of sufficient mature axons reaching their targets, which is a result of the loss of the growth-supportive environment provided by the Schwann cells in the distal stump of injured nerves. Using an established chronic nerve injury and delayed repair animal model that accu- rately mimics chronic nerve injuries in humans, we summarize our key findings as well as others to better understand the pathophysiology of poor functional recovery. We demonstrated that 6 month TGF-β1 treat- ment for chronic nerve injury significantly improved Schwann cell capacity to support axonal regeneration. When combined with forskolin, the effect was additive, as evidenced by a near doubling of regenerated axons proximal to the repair site. We showed that in vivo application of TGF-β1 and forskolin directly onto chronically injured nerves reactivated chronically denervated Schwann cells, induced their proliferation, and upregulated the expression of regeneration-associated proteins. The effect of TGF-β1 and forskolin on old nerve injuries is quite impressive and the treatment regiment appears to mediate a growth-supportive milieu in the injured peripheral nerves. In summary, TGF-β1 and forskolin treatment reactivates chronical- ly denervated Schwann cells and could potentially be used to extend and prolong the regenerative responses to promote axonal regeneration.

REFINEMENTS OF TRANSFORMATION INEQUALITIES FOR ZERO-BALANCED HYPERGEOMETRIC FUNCTIONS

In the article, we present some refinements of three classes of transformation inequalities for zero-balanced hypergeometric functions by use of the updated monotonicity criterion for the quotient of power series.

Changes in Transformation of Soil Organic C and Functional Diversity of Soil Microbial Community Under Different Land Uses

Changes in soil biological and biochemical properties under different land uses in the subtropical region of China were investigated in order to develop rational cultivation and fertilization management. A small watershed of subtropical region of China was selected for this study. Land uses covered paddy fields, vegetable farming, fruit trees, upland crops, bamboo stands, and forestry. Soil biological and biochemical properties included soil organic C and nutrient contents, mineralization of soil organic C, and soil microbial biomass and community functional diversity. Soil organic C and total N contents, microbial biomass C and N, and respiration intensity under different land uses were changed in the following order： paddy fields （and vegetable farming） 〉 bamboo stands 〉 fruit trccs （and upland）. The top surface （0-15 cm） paddy fields （and vegetable farming） were 76.4 and 80.8% higher in soil organic C and total N contents than fruit trees （and upland） soils, respectively. Subsurface paddy soils （15-30 cm） were 59.8 and 67.3% higher in organic C and total N than upland soils, respectively. Soil microbial C, N and respiration intensity in paddy soils （0-15 cm） were 6.36, 3.63 and 3.20 times those in fruit tree （and upland） soils respectively. Soil microbial metabolic quotient was in the order： fruit trees （and upland） 〉 forestry 〉 paddy fields. Metabolic quotient in paddy soils was only 47.7% of that in fruit tree （and upland） soils. Rates of soil organic C mineralization during incubation changed in the order： paddy fields 〉 bamboo stands 〉 fruit trees （and upland） and soil bacteria population： paddy fields 〉 fruit trees （and upland） 〉 forestry. No significant difference was found for fungi and actinomycetes populations. BIOLOG analysis indicated a changing order of paddy fields 〉 fruit trees （and upland） 〉 forestry in values of the average well cell development （AWCD） and functional diversity indexes of microbial community. Results also showed that the conversion from paddy fields to vegetable farming for 5 years resulted in a dramatic increase in soil available phosphorus content while insignificant changes in soil organic C and total N content due to a large inputs of phosphate fertilizers. This conversion caused 53, 41.5, and 41.3% decreases in soil microbial biomass C, N, and respiration intensity, respectively, while 23.6% increase in metabolic quotient and a decrease in soil organic C mineralization rate. Moreover, soil bacteria and actinomycetes populations were increased slightly, while fungi population increased dramatically. Functional diversity indexes of soil microbial community decreased significantly. It was concluded that land uses in the subtropical region of China strongly affected soil biological and biochemical properties. Soil organic C and nutrient contents, mineralization of organic C and functional diversity of microbial community in paddy fields were higher than those in upland and forestry. Overuse of chemical fertilizers in paddy fields with high fertility might degrade soil biological properties and biochemical function, resulting in deterioration of soil biological quality.

Analysis of Ocean Wave Characteristic Distributions Modeled by Two Different Transformed Functions

The probability distributions of wave characteristics from three groups of sampled ocean data with different significant wave heights have been analyzed using two transformation functions estimated by non-parametric and parametric methods. The marginal wave characteristic distribution and the joint density of wave properties have been calculated using the two transformations, with the results and accuracy of both transformations presented here. The two transformations deviate slightly between each other for the calculation of the crest and trough height marginal wave distributions, as well as the joint densities of wave amplitude with other wave properties. The transformation methods for the calculation of the wave crest and trough height distributions are shown to provide good agreement with real ocean data. Our work will help in the determination of the most appropriate transformation procedure for the prediction of extreme values.

Local Extrema of Periodic Function’s Wavelet Transform

The theory of detecling ridges in the modulus of the continuous wavelet transform is presented as well as reconstructing signal by using information on ridges,To periodic signal we suppose Morlet wavelet as basic wavelet, and research the local extreme point and extrema of the wavelet transform on periodic function for the collection of signal＇ s instantaneous amplitude and period.

Thermodynamic Functions and Phase Transformation of Metal Nanocrystals

A model to calculate the thermodynamic functions of the pure metal nanocrystals has been developed, with the consideration of the effects of both the interfaces and the crystal in the nano-grain interior. As an example, the enthalpy, entropy and Gibbs free energy, as functions of the excess free volume at interfaces, temperature and grain size, are calculated for the Co nanocrystals. Furthermore, the characteristics of β-Co→α-Co phase transformation are studied, and the transformation temperatures at different levels of grain size, as well as the critical grain sizes at different temperatures, are predicted. The calculation results show that, the nano-grained β-Co （fcc） is thermodynamically stable at temperatures much lower than that for the conventional coarse-grained materials, and may also stably exist at room temperature when the grain size is reduced to be small enough. The present model is verified by comparisons between the experimental findings and the theoretical predictions.

ANALYSIS OF REMOTE SENSING ARCHAEOLOGY ON TRAFFIC FUNCTION TRANSFORMATION OF TONGJI GRAND CANAL IN SUI AND TANG DYNASTIES

In China, most directions of river flowing are from west to east. During historic period, since the water traffic played an important role, it was very important to form a cross-horizontal net of water carriage route. Canals should be dug so as to make up the lack of north-south river. Tongji Grand Canal, dug in the first year of Daye (605 A.D.) in the Sui Dynasty, was the important component of north-south system of Grand Canals in China. It promoted economic and social development of the Sui, Tang and Song dynasties (605 A.D.-1279 A.D.). As Tongji Canal (i.e. Tongji Grand Canal) flowed across the Huaibei Plain, which is aggraded by abundant mud and sand deposit resulted from the Huanghe (Yellow) River flooding, many traces (such as old channel) and human culture heritages were buried under mud-sand. Tongji Canal was silted up, and disappeared in the Jin Dynasty (1115 A.D.-1234 A.D.). From then on, there were many different stories about the flowing route of the canal in historical literature. Based on space-bone and air-bone remote sensing imagery, we attempt to search the old channel of Tongji Canal, and supplement historical record. The paper discusses transformation process of Tongji Canal’s traffic function, and resumptively summarizes the reasons of the transformation, which results from synthetic function of physical geographical, political, economic, and social conditions.