基于Transformer-LSTM的闽南语唇语识别

针对端到端句子级闽南语唇语识别的问题,提出一种基于Transformer和长短时记忆网络(LSTM)的编解码模型.编码器采用时空卷积神经网络及Transformer编码器用于提取唇读序列时空特征,解码器采用长短时记忆网络并结合交叉注意力机制用于文本序列预测.最后,在自建闽南语唇语数据集上进行实验.实验结果表明:模型能有效地提高唇语识别的准确率.

融合Transformer和卷积LSTM的轨迹分类网络

为了减少原始轨迹数据的噪声,充分提取轨迹的时空特征,提高基于轨迹数据的交通模式分类精度,提出一种融合堆叠降噪自编码器、Transformer和卷积长短期记忆网络的轨迹分类网络(networks fusing stacked denoising auto-encoder, Transformer and ConvLSTM,SDAETC)。通过堆叠降噪自编码器减少原始轨迹数据中的噪声;利用结合了Transformer的递归图自编码器,提取到更为丰富的时间特征,同时利用特征图自编码器提取空间特征;改进卷积长短期记忆网络,充分提取轨迹中的时空特征,并与提取到的时间特征和空间特征相融合,从而实现交通模式分类。实验结果表明,提出的SDAETC与基线模型相比,在GeoLife和SHL数据集上的准确率分别提升了1.8%和2%。此外,消融实验结果和模型训练时间分析表明,引入堆叠降噪自编码器、Transfomer和ConvLSTM虽然增加了时间消耗,但是对分类精度有积极贡献。

基于LSTM与Transformer的地面沉降智能预测方法研究——以上海市为例

受地面沉降严重威胁到生命财产安全的人口已达19%,开展地面沉降模拟预测对防灾减灾具有非常重要的现实意义。针对现有地面沉降预测在模型参数难以获取、单一深度学习方法在预测精度低等方面的局限性,本文提出了集成大模型核心技术的地面沉降预测方法。首先,从地面沉降模拟预测的顶层设计,提出了基于深度学习的地面沉降预测包括算力层、数据层、模型层、评估层与应用层的总体架构;其次,基于LSTM与Transformer提出了地面沉降预测的实用方法;最后,利用上海的地面沉降数据进行了实验研究。结果表明:深度学习技术可以在地面沉降模拟预测中取得较好的结果,多模型法对地面沉降变化不大、回弹、变化较大均可进行预测,iTransformer模型对地面沉降变化较小的情况预测效果较好;在微量地面沉降时代,利用大模型的核心技术Transformer可以取得较高的精度。

一种基于Transformer编码器与LSTM的飞机轨迹预测方法

为了解决飞机目标机动数据集缺失的问题,文章利用运动学建模生成了丰富的轨迹数据集,为网络训练提供了必要的数据支持。针对现阶段轨迹预测运动学模型建立困难及时序预测方法难以提取时空特征的问题,提出了一种结合Transformer编码器和长短期记忆网络(Long Short Term Memory,LSTM)的飞机目标轨迹预测方法,即Transformer-Encoder-LSTM模型。新模型可同时提供LSTM和Transformer编码器模块的补充历史信息和基于注意力的信息表示,提高了模型能力。通过与一些经典神经网络模型进行对比分析,发现在数据集上,新方法的平均位移误差减小到0.22,显著优于CNN-LSTMAttention模型的0.35。相比其他网络,该算法能够提取复杂轨迹中的隐藏特征,在面对飞机连续转弯、大机动转弯的复杂轨迹时,能够保证模型的鲁棒性,提升了对于复杂轨迹预测的准确性。

基于Transformer-LSTM网络的轴承寿命预测

轴承是旋转机械设备中的重要部件,由于工况、材质、加工方式等原因,轴承寿命时长相差许多。传统的并行或串行神经网络预测方式,对数据集有一定要求。因此,需要一种能够适用于不同数据长短的轴承剩余使用寿命预测网络。为此提出了一种能够预测不同寿命时长的Transformer-LSTM串并行神经网络预测模型。通过将Transformer解码层进行重构,并与长短期记忆时序神经网络(long short-term memory,LSTM)网络结构融合,实现轴承寿命数据的串并行预测处理。试验结果表明Transformer-LSTM神经网络能够精准预测长、中、短不同寿命时长的轴承失效时间,具有较强的模型泛化能力,提升轴承寿命预测精度与模型的泛化能力。

基于MsTCN-Transformer模型的轴承剩余使用寿命预测研究

剩余使用寿命(remaining useful life, RUL)预测是PHM的核心问题之一,复杂的运行工况往往导致设备部件经历不同的故障退化过程,给RUL准确预测带来了巨大挑战。为此,提出了一种多尺度时间卷积网络(multi-scale temporal convolutional network, MsTCN)与Transformer(MsTCN-Transformer)融合模型用于变工况下滚动轴承RUL预测。该方法设计了一种新的多尺度膨胀因果卷积单元(multi-scale dilated causal convolution unit, MsDCCU),能够自适应地挖掘滚动轴承全寿命信号中固有的时序特征信息;然后构建了基于自注意力机制的Transformer网络模型,在克服预测序列记忆力退化的基础上,准确学习时序特征与轴承RUL之间的映射关系。此外,通过对轴承不同故障退化阶段所提取的时序特征可视化分析,验证了所提方法在变工况下所提取的时序特征泛化性较好。多种工况条件下滚动轴承RUL预测试验表明,所提方法能够较为准确地实现变工况下轴承的RUL预测,相比当前多种方法RUL预测结果准确性更高。

基于LSTM与Transformer的大坝变形预测研究

变形是大坝运行过程中受内外荷载作用下其状态的直观表现,构建高精度的变形预测模型对大坝安全预警与运行状态评估有着重要的意义。针对已有的大坝位移模型训练时间长,预测模型预测精度和泛化能力一般,无法满足大坝位移中短期准确预测的问题,耦合长短时记忆网络(LSTM)与Transformer框架,引入改进的粒子群优化算法(IPSO)进行优化,建立了IPSO-LSTM-Transformer(ILT)大坝变形预测模型。以紧水滩拱坝正垂线11-1测点为例,选取6150组变形时间序列数据进行分析与预测。研究结果表明,模型预测精度会随着预测期的增大而出现一定程度的下降,但在预测步长10以内均具有良好的预测能力;与传统粒子群优化算法相比,ILT模型显著提升了模型的寻优精度和收敛速度;与RNN、LSTM、IPSO-Transformer神经网络模型单步与多步预测结果对比,ILT模型具有更高的精度和更好的稳定性,即使在训练数据较少时也能保证较好的预测效果。研究成果为实现运行期大坝位移的中短期精确预测提供了新的技术手段。

基于Transformer改进的Faster RCNN在复杂环境下的车辆检测

在监控视角中目标车辆较小、遮挡较为严重,导致检测精度低。通过探讨卷积神经网络和Transformer模型的互相借鉴和联系,并结合损失函数等常规改进,提出了新的Faster RCNN模型。通过借鉴Transformer模型的思想,对原有的特征提取网络进行了改进,将原block比例3∶4∶6∶3改为3∶3∶27∶3、卷积核由3×3改为7×7,增大其感受野,能够更好捕捉图像中的全局特征,使用DW卷积来减少参数量并略微提高性能,使用Channel shuffle解决通道间信息不交流的问题。将原先交并比IoU改为CIoU,与改进后的特征提取网络结合,进一步提高小目标和遮挡目标的检测效果。在UA-DETRAC数据集上,改进后的模型在mAP@0.5:0.95方面比原算法提高了20.20%,并在大、中、小目标下分别提高了15.8%、23%和45.8%,相较于其他模型,如YO⁃LOv7、YOLOv5和Cascade RCNN,mAP@0.5:0.95分别提高了3.3%、5%和6.69%。

基于ResNeSt和改进Transformer的多标签图像分类算法

目前,基于深度学习的多标签分类算法还存在一些问题,如标签之间的相关性有待提高,如何解决小目标分类等。为此提出了一种多标签图像分类算法,该算法使用分裂注意力网络ResNeSt进行特征提取,并使用BatchFormerV2与Transformer形成双分支网络对特征进行编码,解码阶段使用Transformer Decoder的交叉注意模块来自适应地处理特征以达到更好的分类效果。实验结果表明:该模型在COCO数据集上的mAP为88.4%,在VOC2007数据集上的平均精度为96.0%,一定程度上提高了多标签图像分类的准确率。

基于Transformer_LSTM编解码器模型的船舶轨迹异常检测方法

[目的]为提升船舶轨迹异常检测的精度和效率,解决传统异常检测方法存在的特征表征能力有限、补偿精度不足、容易出现梯度消失、过拟合等问题,提出一种基于Transformer_LSTM编解码器模型的无监督船舶轨迹异常检测方法。[方法]该方法基于编码器解码器架构,由Transformer_LSTM模块替代传统神经网络实现轨迹特征提取和轨迹重构;将Transformer嵌入LSTM的递归机制,结合循环单元和注意力机制,利用自注意力和交叉注意力实现对循环单元状态向量的计算,实现对长序列模型的有效构建;通过最小化重构输出和原始输入之间的差异,使模型学习一般轨迹的特征和运动模式,将重构误差大于异常阈值的轨迹判定为异常轨迹。[结果]采用2021年1月的船舶AIS数据进行实验,结果表明,模型在准确率、精确率以及召回率上相较于LOF,DBSCAN,VAE,LSTM等经典模型有着明显提升;F1分数相较于VAE_LSTM模型提升约8.11%。[结论]该方法的异常检测性能在各项指标上显著优于传统算法,可有效、可靠地运用于海上船舶轨迹异常检测。

TransTM:A device-free method based on time-streaming multiscale transformer for human activity recognition

RFID-based human activity recognition(HAR)attracts attention due to its convenience,noninvasiveness,and privacy protection.Existing RFID-based HAR methods use modeling,CNN,or LSTM to extract features effectively.Still,they have shortcomings:1)requiring complex hand-crafted data cleaning processes and 2)only addressing single-person activity recognition based on specific RF signals.To solve these problems,this paper proposes a novel device-free method based on Time-streaming Multiscale Transformer called TransTM.This model leverages the Transformer's powerful data fitting capabilities to take raw RFID RSSI data as input without pre-processing.Concretely,we propose a multiscale convolutional hybrid Transformer to capture behavioral features that recognizes singlehuman activities and human-to-human interactions.Compared with existing CNN-and LSTM-based methods,the Transformer-based method has more data fitting power,generalization,and scalability.Furthermore,using RF signals,our method achieves an excellent classification effect on human behaviorbased classification tasks.Experimental results on the actual RFID datasets show that this model achieves a high average recognition accuracy(99.1%).The dataset we collected for detecting RFID-based indoor human activities will be published.

Hemorrhagic transformation in patients with large-artery atherosclerotic stroke is associated with the gut microbiota and lipopolysaccharide

Hemorrhagic transformation is a major complication of large-artery atheroscle rotic stroke(a major ischemic stro ke subtype)that wo rsens outcomes and increases mortality.Disruption of the gut microbiota is an important feature of stroke,and some specific bacteria and bacterial metabolites may contribute to hemorrhagic transformation pathogenesis.We aimed to investigate the relationship between the gut microbiota and hemorrhagic transformation in largearte ry atheroscle rotic stro ke.An observational retrospective study was conducted.From May 2020 to September 2021,blood and fecal samples were obtained upon admission from 32 patients with first-ever acute ischemic stroke and not undergoing intravenous thrombolysis or endovascular thrombectomy,as well as 16 healthy controls.Patients with stro ke who developed hemorrhagic transfo rmation(n=15)were compared to those who did not develop hemorrhagic transformation(n=17)and with healthy controls.The gut microbiota was assessed through 16S ribosomal ribonucleic acid sequencing.We also examined key components of the lipopolysaccharide pathway:lipopolysaccharide,lipopolysaccharide-binding protein,and soluble CD14.We observed that bacterial diversity was decreased in both the hemorrhagic transformation and non-hemorrhagic transfo rmation group compared with the healthy controls.The patients with ischemic stro ke who developed hemorrhagic transfo rmation exhibited altered gut micro biota composition,in particular an increase in the relative abundance and dive rsity of members belonging to the Enterobacteriaceae family.Plasma lipopolysaccharide and lipopolysaccharide-binding protein levels were higher in the hemorrhagic transformation group compared with the non-hemorrhagic transfo rmation group.lipopolysaccharide,lipopolysaccharide-binding protein,and soluble CD14 concentrations were associated with increased abundance of Enterobacte riaceae.Next,the role of the gut microbiota in hemorrhagic transformation was evaluated using an experimental stroke rat model.In this model,transplantation of the gut microbiota from hemorrhagic transformation rats into the recipient rats triggered higher plasma levels of lipopolysaccharide,lipopolysaccharide-binding protein,and soluble CD14.Ta ken togethe r,our findings demonstrate a noticeable change in the gut microbiota and lipopolysaccharide-related inflammatory response in stroke patients with hemorrhagic transformation.This suggests that maintaining a balanced gut microbiota may be an important factor in preventing hemorrhagic transfo rmation after stro ke.

Critical approaches in the catalytic transformation of sugar isomerization and epimerization after Fischer-History,challenges,and prospects

The transformation of aldose to ketose or common sugars into rare saccharides,including rare ketoses and aldoses,is of great value and interest to the food industry and for saccharidic biomass utilization,medicine,and the synthesis of drugs.Nowadays,high-fructose corn syrup(HFCS)is industrially produced in more than 10 million tons annually using immobilized glucose isomerase.Some low-calorie saccharides such as tagatose and psicose,which are becoming popular sweeteners,have also been produced on a pilot scale in order to replace sucrose and HFCS.However,current catalysts and catalytic processes are still difficult to utilize in biomass conversion and also have strong substrate dependence in producing high-value,rare sugars.Considering the specific reaction properties of saccharides and catalysts,since the pioneering discovery by Fischer,various catalysts and catalytic systems have been discovered or developed in attempts to extend the reaction pathways,improve the reaction efficiency,and to potentially produce commercial products.In this review,we trace the history of sugar isomerization/epimerization reactions and summarize the important breakthroughs for each reaction as well as the difficulties that remain unresolved to date.

Identification of earthquake induced structural damage based on synchroextracting transform

Several popular time-frequency techniques,including the Wigner-Ville distribution,smoothed pseudo-Wigner-Ville distribution,wavelet transform,synchrosqueezing transform,Hilbert-Huang transform,and Gabor-Wigner transform,are investigated to determine how well they can identify damage to structures.In this work,a synchroextracting transform(SET)based on the short-time Fourier transform is proposed for estimating post-earthquake structural damage.The performance of SET for artificially generated signals and actual earthquake signals is examined with existing methods.Amongst other tested techniques,SET improves frequency resolution to a great extent by lowering the influence of smearing along the time-frequency plane.Hence,interpretation and readability with the proposed method are improved,and small changes in the time-varying frequency characteristics of the damaged buildings are easily detected through the SET method.

Formation and transformation of metastable LPSO building blocks clusters in Mg-Gd-Y-Zn-Zr alloys by spinodal decomposition and heterogeneous nucleation

To study the formation and transformation mechanism of long-period stacked ordered(LPSO)structures,a systematic atomic scale analysis was conducted for the structural evolution of long-period stacked ordered(LPSO)structures in the Mg-Gd-Y-Zn-Zr alloy annealed at 300℃~500℃.Various types of metastable LPSO building block clusters were found to exist in alloy structures at different temperatures,which precipitate during the solidification and homogenization process.The stability of Zn/Y clusters is explained by the first principles of density functional theory.The LPSO structure is distinguished by the arrangement of its different Zn/Y enriched LPSO structural units,which comprises local fcc stacking sequences upon a tightly packed plane.The presence of solute atoms causes local lattice distortion,thereby enabling the rearrangement of Mg atoms in the different configurations in the local lattice,and local HCP-FCC transitions occur between Mg and Zn atoms occupying the nearest neighbor positions.This finding indicates that LPSO structures can generate necessary Schockley partial dislocations on specific slip surfaces,providing direct evidence of the transition from 18R to 14H.Growth of the LPSO,devoid of any defects and non-coherent interfaces,was observed separately from other precipitated phases.As a result,the precipitation sequence of LPSO in the solidification stage was as follows:Zn/Ycluster+Mg layers→various metastable LPSO building block clusters→18R/24R LPSO;whereas the precipitation sequence of LPSO during homogenization treatment was observed to be as follows:18R LPSO→various metastable LPSO building block clusters→14H LPSO.Of these,14H LPSO was found to be the most thermodynamically stable structure.

Pre-existing orthorhombic embryos-induced hexagonal-orthorhombic martensitic transformation in MnNiSi_(1-x)(CoNiGe)_x alloy

The thermal-elastic martensitic transformation from high-temperature Ni_(2)In-type hexagonal structure to low-temperature TiNiSi-type orthorhombic structure has been widely studied in MnMX(M=Ni or Co,and X=Ge or Si)alloys.However,the answer to how the orthorhombic martensite nucleates and grows within the hexagonal parent is still unclear.In this work,the hexagonal-orthorhombic martensitic transformation in a Co and Ge co-substituted MnNiSi is investigated.One can find some orthorhombic laths embedded in the hexagonal parent at a temperature above the martensitic transformation start temperature(M_(s)).With the the sample cooing to M_(s),the laths turn broader,indicating that the martensitic transformation starts from these pre-existing orthorhombic laths.Microstructure observation suggests that these pre-existing orthorhombic laths do not originate from the hexagonal-orthorhombic martensitic transformation because of the difference between atomic occupations of doping elements in the hexagonal parent and those in the preexisting orthorhombic laths.The phenomenological crystallographic theory and experimental investigations prove that the pre-existing orthorhombic lath and generated orthorhombic martensite have the same crystallography relationship to the hexagonal parent.Therefore,the orthorhombic martensite can take these pre-existing laths as embryos and grow up.This work implies that the martensitic transformation in MnNiSi_(1-x)(CoNiGe)_(x) alloy is initiated by orthorhombic embryos.

Precise regulation of the phase transformation for pyrolusite during the reduction roasting process

The mechanism involved in the phase transformation process of pyrolusite (MnO_(2)) during roasting in a reducing atmosphere was systematically elucidated in this study,with the aim of effectively using low-grade complex manganese ore resources.According to single-factor experiment results,the roasted product with a divalent manganese (Mn^(2+)) distribution rate of 95.30% was obtained at a roasting time of 25 min,a roasting temperature of 700℃,a CO concentration of 20at%,and a total gas volume of 500 mL·min^(-1),in which the manganese was mainly in the form of manganosite (MnO).Scanning electron microscopy and Brunauer–Emmett–Teller theory demonstrated the microstructural evolution of the roasted product and the gradual reduction in the pyrolusite ore from the surface to the core Thermodynamic calculations,X-ray photoelectron spectroscopy,and X-ray diffractometry analyses determined that the phase transformation of pyrolusite followed the order of MnO_(2)→Mn_(2)O_(3)→Mn_(3)O_(4)→MnO phase by phase,and the reduction of manganese oxides in each valence state proceeded simultaneously.

Description of martensitic transformation kinetics in Fe-C-X(X = Ni,Cr,Mn,Si) system by a modified model

Controlling the content of athermal martensite and retained austenite is important to improving the mechanical properties of high-strength steels,but a mechanism for the accurate description of martensitic transformation during the cooling process must be addressed.At present,frequently used semi-empirical kinetics models suffer from huge errors at the beginning of transformation,and most of them fail to exhibit the sigmoidal shape characteristic of transformation curves.To describe the martensitic transformation process accurately,based on the Magee model,we introduced the changes in the nucleation activation energy of martensite with temperature,which led to the varying nucleation rates of this model during martensitic transformation.According to the calculation results,the relative error of the modified model for the martensitic transformation kinetics curves of Fe-C-X(X = Ni,Cr,Mn,Si) alloys reached 9.5% compared with those measured via the thermal expansion method.The relative error was approximately reduced by two-thirds compared with that of the Magee model.The incorporation of nucleation activation energy into the kinetics model contributes to the improvement of its precision.

Silencing transformer and transformer-2 in Zeugodacus cucurbitae causes defective sex determination with inviability of most pseudomales

transformer is a switch gene for sex determination in many insects, which cooperates with transformer-2 that is expressed in both sexes to regulate female differentiation, particularly in dipterans. Zeugodacus cucurbitae(Coquillett) is a very destructive pest worldwide, however, its sex determination pathway remains largely uncharacterized. Here, we show that the female sex ratio is sharply reduced with knockdown of either transformer or transformer-2 by RNA interference in early embryos of Z. cucurbitae. Most of the males grown from the embryos with transient transformer and transformer-2 suppression mated with wild-type females and produced mixed sex progeny, with one exception that produced only female progeny, and all of the few remaining males failed to mate with wild-type females and produced no progeny. The exceptional male and those males with mating failure were XX pseudomales as determined by the detection of Y chromosome-linked Maleness-on-the-Y, indicating that most XX pseudomales are not viable. The phenotypes of transformer and transformer-2 suggest that they play a key role in regulating sex determination and are required for female sexual development of Z. cucurbitae. Our results will be beneficial to the understanding of sex determination in Z. cucurbitae and can facilitate the development of genetic sexing strains for its biological control.

Point Cloud Classification Using Content-Based Transformer via Clustering in Feature Space

Recently, there have been some attempts of Transformer in 3D point cloud classification. In order to reduce computations, most existing methods focus on local spatial attention,but ignore their content and fail to establish relationships between distant but relevant points. To overcome the limitation of local spatial attention, we propose a point content-based Transformer architecture, called PointConT for short. It exploits the locality of points in the feature space(content-based), which clusters the sampled points with similar features into the same class and computes the self-attention within each class, thus enabling an effective trade-off between capturing long-range dependencies and computational complexity. We further introduce an inception feature aggregator for point cloud classification, which uses parallel structures to aggregate high-frequency and low-frequency information in each branch separately. Extensive experiments show that our PointConT model achieves a remarkable performance on point cloud shape classification. Especially, our method exhibits 90.3% Top-1 accuracy on the hardest setting of ScanObjectN N. Source code of this paper is available at https://github.com/yahuiliu99/PointC onT.