Graph Transformers研究进展综述

随着图结构数据在各种实际场景中的广泛应用,对其进行有效建模和处理的需求日益增加。Graph Transformers(GTs)作为一类使用Transformers处理图数据的模型,能够有效缓解传统图神经网络(GNN)中存在的过平滑和过挤压等问题,因此可以学习到更好的特征表示。根据对近年来GTs相关文献的研究,将现有的模型架构分为两类:第一类通过绝对编码和相对编码向Transformers中加入图的位置和结构信息,以增强Transformers对图结构数据的理解和处理能力;第二类根据不同的方式(串行、交替、并行)将GNN与Transformers进行结合,以充分利用两者的优势。介绍了GTs在信息安全、药物发现和知识图谱等领域的应用,对比总结了不同用途的模型及其优缺点。最后,从可扩展性、复杂图、更好的结合方式等方面分析了GTs未来研究面临的挑战。

Efficient Vision Transformers for Autonomous Off-Road Perception Systems

The development of autonomous vehicles has become one of the greatest research endeavors in recent years. These vehicles rely on many complex systems working in tandem to make decisions. For practical use and safety reasons, these systems must not only be accurate, but also quickly detect changes in the surrounding environment. In autonomous vehicle research, the environment perception system is one of the key components of development. Environment perception systems allow the vehicle to understand its surroundings. This is done by using cameras, light detection and ranging (LiDAR), with other sensor systems and modalities. Deep learning computer vision algorithms have been shown to be the strongest tool for translating camera data into accurate and safe traversability decisions regarding the environment surrounding a vehicle. In order for a vehicle to safely traverse an area in real time, these computer vision algorithms must be accurate and have low latency. While much research has studied autonomous driving for traversing well-structured urban environments, limited research exists evaluating perception system improvements in off-road settings. This research aims to investigate the adaptability of several existing deep-learning architectures for semantic segmentation in off-road environments. Previous studies of two Convolutional Neural Network (CNN) architectures are included for comparison with new evaluation of Vision Transformer (ViT) architectures for semantic segmentation. Our results demonstrate viability of ViT architectures for off-road perception systems, having a strong segmentation accuracy, lower inference speed and memory footprint compared to previous results with CNN architectures.

A Comprehensive Survey of Recent Transformers in Image,Video and Diffusion Models

Transformer models have emerged as dominant networks for various tasks in computer vision compared to Convolutional Neural Networks(CNNs).The transformers demonstrate the ability to model long-range dependencies by utilizing a self-attention mechanism.This study aims to provide a comprehensive survey of recent transformerbased approaches in image and video applications,as well as diffusion models.We begin by discussing existing surveys of vision transformers and comparing them to this work.Then,we review the main components of a vanilla transformer network,including the self-attention mechanism,feed-forward network,position encoding,etc.In the main part of this survey,we review recent transformer-based models in three categories:Transformer for downstream tasks,Vision Transformer for Generation,and Vision Transformer for Segmentation.We also provide a comprehensive overview of recent transformer models for video tasks and diffusion models.We compare the performance of various hierarchical transformer networks for multiple tasks on popular benchmark datasets.Finally,we explore some future research directions to further improve the field.

TC-Fuse: A Transformers Fusing CNNs Network for Medical Image Segmentation

In medical image segmentation task,convolutional neural networks(CNNs)are difficult to capture long-range dependencies,but transformers can model the long-range dependencies effectively.However,transformers have a flexible structure and seldom assume the structural bias of input data,so it is difficult for transformers to learn positional encoding of the medical images when using fewer images for training.To solve these problems,a dual branch structure is proposed.In one branch,Mix-Feed-Forward Network(Mix-FFN)and axial attention are adopted to capture long-range dependencies and keep the translation invariance of the model.Mix-FFN whose depth-wise convolutions can provide position information is better than ordinary positional encoding.In the other branch,traditional convolutional neural networks(CNNs)are used to extract different features of fewer medical images.In addition,the attention fusion module BiFusion is used to effectively integrate the information from the CNN branch and Transformer branch,and the fused features can effectively capture the global and local context of the current spatial resolution.On the public standard datasets Gland Segmentation(GlaS),Colorectal adenocarcinoma gland(CRAG)and COVID-19 CT Images Segmentation,the F1-score,Intersection over Union(IoU)and parameters of the proposed TC-Fuse are superior to those by Axial Attention U-Net,U-Net,Medical Transformer and other methods.And F1-score increased respectively by 2.99%,3.42%and 3.95%compared with Medical Transformer.

基于孪生Transformers的遥感目标多元变化检测方法

多元变化检测是目前遥感领域重要的研究主题之一,在军事侦察方面有着广泛应用。考虑到目前常用的卷积神经网络中感受野对于提取变化特征的局限性,提出了一种基于孪生Transformers的多元变化检测模型。设计了基于多级Transformers融合结构的编码器进行远距离上下文建模,引入了残差连接的轴向注意力机制对变化特征信息进行解码,从而生成准确完整的多元变化图。并在针对飞机和舰船军事目标变化检测的遥感数据集上进行了训练与测试以及消融实验。结果表明:该方法的平均IoU和F 1分数分别达到68.69%和80.43%,其性能优于其他流行的变化检测方法。

Text Augmentation-Based Model for Emotion Recognition Using Transformers

Emotion Recognition in Conversations(ERC)is fundamental in creating emotionally intelligentmachines.Graph-BasedNetwork(GBN)models have gained popularity in detecting conversational contexts for ERC tasks.However,their limited ability to collect and acquire contextual information hinders their effectiveness.We propose a Text Augmentation-based computational model for recognizing emotions using transformers(TA-MERT)to address this.The proposed model uses the Multimodal Emotion Lines Dataset(MELD),which ensures a balanced representation for recognizing human emotions.Themodel used text augmentation techniques to producemore training data,improving the proposed model’s accuracy.Transformer encoders train the deep neural network(DNN)model,especially Bidirectional Encoder(BE)representations that capture both forward and backward contextual information.This integration improves the accuracy and robustness of the proposed model.Furthermore,we present a method for balancing the training dataset by creating enhanced samples from the original dataset.By balancing the dataset across all emotion categories,we can lessen the adverse effects of data imbalance on the accuracy of the proposed model.Experimental results on the MELD dataset show that TA-MERT outperforms earlier methods,achieving a weighted F1 score of 62.60%and an accuracy of 64.36%.Overall,the proposed TA-MERT model solves the GBN models’weaknesses in obtaining contextual data for ERC.TA-MERT model recognizes human emotions more accurately by employing text augmentation and transformer-based encoding.The balanced dataset and the additional training samples also enhance its resilience.These findings highlight the significance of transformer-based approaches for special emotion recognition in conversations.

Simulation and Modeling of Common Mode EMI Noise in Planar Transformers

The transformer is the key circuit component of the common-mode noise current when an isolated converter is working.The highfrequency characteristics of the transformer have an important influence on the common-mode noise of the converter.Traditionally,the measurement method is used for transformer modeling,and a single lumped device is used to establish the transformer model,which cannot be predicted in the transformer design stage.Based on the transformer common-mode noise transmission mechanism,this paper derives the transformer common-mode equivalent capacitance under ideal conditions.According to the principle of experimental measurement of the network analyzer,the electromagnetic field finite element simulation software three-dimensional(3D)modeling and simulation method is used to obtain the two-port parameters of the transformer,extract the high-frequency parameters of the transformer,and establish its electromagnetic compatibility equivalent circuit model.Finally,an experimental prototype is used to verify the correctness of the model by comparing the experimental measurement results with the simulation prediction results.

基于时域融合Transformers的可解释预测模型及其应用研究

为提高时间序列模型预测的准确性及可解释能力,提出了变分模态分解(variational mode decomposition,VMD)和时域融合变换器(temporal fusion transformers,TFT)相结合的高效可解释预测模型,通过VMD将原始数据分解为多个模态,充分挖掘原始数据特征,将分解结果输入到TFT预测模型中,得出可解释性的预测结果。TFT是一种新的基于注意力的深度学习模型,将高性能的多水平预测和对时间动态的可解释见解结合在一起。以白卡纸价格为研究对象,证明了所设计模型的有效性。TFT的可解释输出包括分解的白卡纸价格子序列的重要性排序,所提出的白卡纸的可解释预测方法可为从业者的相关决策提供有力的支撑。

Experimental and Numerical Study of the Key Non-Dimensional Geometrical Parameters on the Noise Level of Dry-Type Cast Resin Transformers

Dry-Type Cast Resin Distribution Transformers(CRT)is the secondgeneration of air-cooled distribution transformers where oil is replaced by resin for electrical insulation.CRT transformers may installed indoor adjacent to or near residential areas since they are clean and safe comparing to the conventional transformers.But,as it is obvious,noise discrepancy is intrinsically accompanied with all types of transformers and is inevitable for CRT transformers too.Minimization of noise level caused by such these transformers has biological and ergonomic importance.As it is known the core of transformers is the main source of the noise generation.In this paper,experimental and numerical investigation is implemented for a large number of fabricated CRT transformers in IT Co(Iran Transfo Company)to evaluate the effective geometrical parameters of the core on the overall sound level of transformers.Noise Level of each sample is measured according to criteria of IEC60651 and is reported in units of Decibel(dB).Numerical simulation is done using noncommercial version of ANSYS Workbench software to extract first six natural frequencies and mode shapes of CRT cores which is reported in units of Hz.Three novel non-dimensional variables for geometry of the transformer core are introduced.Both experimental and numerical results show approximately similar response to these variables.Correlation between natural frequencies and noise level is evaluated statistically.Pearson factor shows that there is a robust conjunction between first two natural frequencies and noise level of CRTs.Results show that noise level decreases as the two first natural frequencies increases and vice versa,noise level increases as the two natural frequencies of the core decreases.Finally the noise level decomposed to two parts.

Structured Microgrids (SμGs) and Flexible Electronic Large Power Transformers (FeLPTs)

Structured microgrids(SμGs)and Flexible electronic large power transformers(FeLPTs)are emerging as two essential technologies for renewable energy integration,flexible power transmission,and active control.SμGs provide the integration of renewable energy and storage to balance the energy demand and supply as needed for a given system design.FeLPT’s flexibility for processing,control,and re-configurability offers the capability for flexible transmission for effective flow control and enable SμGs connectivity while still keeping multiscale system level control.Early adaptors for combined heat and power have demonstrated significant economic benefits while reducing environmental foot prints.They bring tremendous benefits to utility companies also.With storage and active control capabilities,a 300-percent increase in bulk transmission and distribution lines are possible without having to increase capacity.SμGs and FeLPTs will also enable the utility industry to be better prepared for the emerging large increase in base load demand from electric transportation and data centers.This is a win-win-win situation for the consumer,the utilities(grid operators),and the environment.SμGs and FeLPTs provide value in power substation,energy surety,reliability,resiliency,and security.It is also shown that the initial cost associated with SμG and FeLPTs deployment can be easily offset with reduced operating cost,which in turn reduces the total life-cycle cost by 33%to 67%.

Analysis of Thermal Behavior of High Frequency Transformers Using Finite Element Method

High frequency transformer is used in many applications among the Switch Mode Power Supply (SMPS), high voltage pulse power and etc can be mentioned. Regarding that the core of these transformers is often the ferrite core;their functions partly depend on this core characteristic. One of the characteristics of the ferrite core is thermal behavior that should be paid attention to because it affects the transformer function and causes heat generation. In this paper, a typical high frequency transformer with ferrite core is designed and simulated in ANSYS software. Temperature rise due to winding current (Joule-heat) is considered as heat generation source for thermal behavior analysis of the transformer. In this simulation, the temperature rise and heat distribution are studied and the effects of parameters such as flux density, winding loss value, using a fan to cool the winding and core and thermal conductivity are investigated.

A Frequency-Independent Equivalent Circuit for High-k Stacked Monolithic Transformers

A new 2-Π lumped element equivalent circuit model for high-k stacked on-chip transformers is proposed. The model parameters are extracted with high precision, mainly based on analytical methods. The developed model enables fast and accurate time domain transient analysis and noise analysis in RFIC simulation since all elements in the model are fre- quency independent. The validity of the proposed model has been demonstrated by a fabricated monolithic stacked trans- former in TSMC＇s 0.13μm mixed-signal （MS）/RF CMOS＇ process.

SIMULATING METHOD OF MAGNETIZING INRUSH CURRENT OF POWER TRANSFORMERS USING CONCEPT OF INSTANTANEOUS POWER

In this paper,a new simulating method is presented,using only the normal magnetizing curve (B-H) of the transformer core material,its geometric dimensions,the no-load power loss data and the concept of instantaneous power. At the end of this paper the simulating calculation using EMTP has been also performed for the same transformer. The comparison shows that the two sets of results are very close to each other,and proves the correctness of the new method. The new method presented in this paper is helpful to verify the correctness of the power transformer design,analyze the behavior of the transformer protection under switching and study the new transformer protection principles.

Optimal Economic Life Interval Evaluation Method of Transformers

This paper proposed a method of optimal economic life interval evaluation of transformer, thus providing scientific support for power system planning and reconstruction. First, obtaining the transformer failure rate by using Levenberg-Marquardt based least squares algorithm on the consumption that transformer failure mode meets three-parameters Weibull distribution. Second, calculating operation cost and failure cost of transformer during its life cycle on the basis of transformer failure rate. Then, acquiring the optimal economic life interval of transformer through life cycle minimal average annual cost based on the theory of interval analysis. Finally, the proposed method is proved to be effective by applying to some real transformers.

Early Failure Detection in Power Transformers

This paper presents equipment for early detection of failures in the insulation of power transformers, checking existing partial discharges inside. The equipment involves hardware, control and signal acquisition software, and signal analysis software. This equipment has a set of algorithms that were made with intelligent extraction techniques and interpretation of data. The degradation diagnosis of the equipment insulation is based on digital signal processing algorithms for extraction of features and also in artificial intelligence algorithms that allies a mining involving all the data linked to the equipment throughout its operating life can make an assessment of the operating conditions of the equipment and suggest interventions and provide an estimated time so that they have to be made.

Aging Effect on Characteristics of Oil Impregnated Insulation Paper for Power Transformers

This paper deals with experimental investigations of the electrical and physical properties of oil impregnated insulation paper for power transformers at different temperatures. The ac breakdown voltage, tensile strength and water content of insulation papers impregnated in mineral oil for different time periods were investigated. The effect of insulation paper thickness on the electrical and mechanical properties has also been studied. The results showed that the breakdown voltage and the tensile strength decreased with increasing the time of immersion of insulation paper in oil at room temperature, at 5℃ and at -12℃. Also, the thermal aging effect on the characteristic of insulation paper has been studied. It was found that high temperatures affect the breakdown voltage and the tensile strength to a great extent.

Optimal PV Allocation&Minimal tap-Changing Transformers Achieving Best Distribution Voltage Profile&Minimum Losses in active distribution networks

In distribution systems,voltage levels of the various buses should be maintained within the permissible limits for satisfactory operation of all electrical installations and equipment.The task of voltage control is closely associated with fluctuating load conditions and corresponding requirements of reactive power compensation.The problem of load bus voltage optimization in distribution systems that have distributed generation(DG)has recently become an issue.In Oman,the distribution code limits the load bus voltage variations within±6%of the nominal value.Several voltage control methods are employed in active distribution systems with a high share of photovoltaic systems(PV)to keep the voltage levels within the desirable limits.In addition to the constraint of targeting the best voltage profile,another constraint has to be achieved which is the minimum loss in the distribution network.An optimised solution for voltage of load busses with on-load tap-changing(OLTC)tarnsformers and PV sources is presented in this paper.This study addresses the problem of optimizing the injected power from PV systems associated with the facilities of tap-changing transformers,as it is an important means of controlling voltage throughout the system.To avoid violating tap-changing constraints,a method is depicted for determining the minimal changes in transformer taps to control voltage levels with distributed PV sources.The taps of a range+5 to-15%,can be achieved by tap-changing transformers.The OLTC operation was designed to keep the secondary bus within the voltage standard for MV networks.

750-kV class transformers exported to India

According to the People’s Daily report, in early March Baoding Tianwei Baobian Electric Co., Ltd, in the competitive bidding for the transmission &