Lightweight and polarized self-attention mechanism for abnormal morphology classification algorithm during traditional Chinese medicine inspection

Objective To propose a Light-Atten-Pose-based algorithm for classifying abnormal morphol-ogy in traditional Chinese medicine(TCM)inspection to solve the problem of relying on manual labor or expensive equipment with personal subjectivity or high cost.Methods First,this paper establishes a dataset of abnormal morphology for Chinese medi-cine diagnosis,with images from public resources and labeled with category labels by several Chinese medicine experts,including three categories:normal,shoulder abnormality,and leg abnormality.Second,the key points of human body are extracted by Light-Atten-Pose algo-rithm.Light-Atten-Pose algorithm uses lightweight EfficientNet network and polarized self-attention(PSA)mechanism on the basis of AlphaPose,which reduces the computation amount by using EfficientNet network,and the data is finely processed by using PSA mecha-nism in spatial and channel dimensions.Finally,according to the theory of TCM inspection,the abnormal morphology standard based on the joint angle difference is defined,and the classification of abnormal morphology of Chinese medical diagnosis is realized by calculat-ing the angle between key points.Accuracy,frames per second(FPS),model size,parameter set(Params),and giga floating-point operations per second(GFLOPs)are chosen as the eval-uation indexes for lightweighting.Results Validation of the Light-Atten-Pose algorithm on the dataset showed a classification accuracy of 96.23%,which is close to the original AlphaPose model.However,the FPS of the improved model reaches 41.6 fps from 16.5 fps,the model size is reduced from 155.11 MB to 33.67 MB,the Params decreases from 40.5 M to 8.6 M,and the GFLOPs reduces from 11.93 to 2.10.Conclusion The Light-Atten-Pose algorithm achieves lightweight while maintaining high ro-bustness,resulting in lower complexity and resource consumption and higher classification accuracy,and the experiments prove that the Light-Atten-Pose algorithm has a better overall performance and has practical application in the pose estimation task.

Sentiment classification model for bullet screen based on self-attention mechanism

With the development of short video industry,video and bullet screen have become important ways to spread public opinions.Public attitudes can be timely obtained through emotional analysis on bullet screen,which can also reduce difficulties in management of online public opinions.A convolutional neural network model based on multi-head attention is proposed to solve the problem of how to effectively model relations among words and identify key words in emotion classification tasks with short text contents and lack of complete context information.Firstly,encode word positions so that order information of input sequences can be used by the model.Secondly,use a multi-head attention mechanism to obtain semantic expressions in different subspaces,effectively capture internal relevance and enhance dependent relationships among words,as well as highlight emotional weights of key emotional words.Then a dilated convolution is used to increase the receptive field and extract more features.On this basis,the above multi-attention mechanism is combined with a convolutional neural network to model and analyze the seven emotional categories of bullet screens.Testing from perspectives of model and dataset,experimental results can validate effectiveness of our approach.Finally,emotions of bullet screens are visualized to provide data supports for hot event controls and other fields.

Keyphrase Generation Based on Self-Attention Mechanism

Keyphrase greatly provides summarized and valuable information.This information can help us not only understand text semantics,but also organize and retrieve text content effectively.The task of automatically generating it has received considerable attention in recent decades.From the previous studies,we can see many workable solutions for obtaining keyphrases.One method is to divide the content to be summarized into multiple blocks of text,then we rank and select the most important content.The disadvantage of this method is that it cannot identify keyphrase that does not include in the text,let alone get the real semantic meaning hidden in the text.Another approach uses recurrent neural networks to generate keyphrases from the semantic aspects of the text,but the inherently sequential nature precludes parallelization within training examples,and distances have limitations on context dependencies.Previous works have demonstrated the benefits of the self-attention mechanism,which can learn global text dependency features and can be parallelized.Inspired by the above observation,we propose a keyphrase generation model,which is based entirely on the self-attention mechanism.It is an encoder-decoder model that can make up the above disadvantage effectively.In addition,we also consider the semantic similarity between keyphrases,and add semantic similarity processing module into the model.This proposed model,which is demonstrated by empirical analysis on five datasets,can achieve competitive performance compared to baseline methods.

Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

Hierarchical multihead self-attention for time-series-based fault diagnosis

Fault diagnosis is important for maintaining the safety and effectiveness of chemical process.Considering the multivariate,nonlinear,and dynamic characteristic of chemical process,many time-series-based data-driven fault diagnosis methods have been developed in recent years.However,the existing methods have the problem of long-term dependency and are difficult to train due to the sequential way of training.To overcome these problems,a novel fault diagnosis method based on time-series and the hierarchical multihead self-attention(HMSAN)is proposed for chemical process.First,a sliding window strategy is adopted to construct the normalized time-series dataset.Second,the HMSAN is developed to extract the time-relevant features from the time-series process data.It improves the basic self-attention model in both width and depth.With the multihead structure,the HMSAN can pay attention to different aspects of the complicated chemical process and obtain the global dynamic features.However,the multiple heads in parallel lead to redundant information,which cannot improve the diagnosis performance.With the hierarchical structure,the redundant information is reduced and the deep local time-related features are further extracted.Besides,a novel many-to-one training strategy is introduced for HMSAN to simplify the training procedure and capture the long-term dependency.Finally,the effectiveness of the proposed method is demonstrated by two chemical cases.The experimental results show that the proposed method achieves a great performance on time-series industrial data and outperforms the state-of-the-art approaches.

Self-Attention Spatio-Temporal Deep Collaborative Network for Robust FDIA Detection in Smart Grids

False data injection attack(FDIA)can affect the state estimation of the power grid by tampering with the measured value of the power grid data,and then destroying the stable operation of the smart grid.Existing work usually trains a detection model by fusing the data-driven features from diverse power data streams.Data-driven features,however,cannot effectively capture the differences between noisy data and attack samples.As a result,slight noise disturbances in the power grid may cause a large number of false detections for FDIA attacks.To address this problem,this paper designs a deep collaborative self-attention network to achieve robust FDIA detection,in which the spatio-temporal features of cascaded FDIA attacks are fully integrated.Firstly,a high-order Chebyshev polynomials-based graph convolution module is designed to effectively aggregate the spatio information between grid nodes,and the spatial self-attention mechanism is involved to dynamically assign attention weights to each node,which guides the network to pay more attention to the node information that is conducive to FDIA detection.Furthermore,the bi-directional Long Short-Term Memory(LSTM)network is introduced to conduct time series modeling and long-term dependence analysis for power grid data and utilizes the temporal self-attention mechanism to describe the time correlation of data and assign different weights to different time steps.Our designed deep collaborative network can effectively mine subtle perturbations from spatiotemporal feature information,efficiently distinguish power grid noise from FDIA attacks,and adapt to diverse attack intensities.Extensive experiments demonstrate that our method can obtain an efficient detection performance over actual load data from New York Independent System Operator(NYISO)in IEEE 14,IEEE 39,and IEEE 118 bus systems,and outperforms state-of-the-art FDIA detection schemes in terms of detection accuracy and robustness.

基于PSA引导双分支神经网络特征融合的同步电机故障诊断

针对单一传感器信号在同步电机故障诊断中精度不高的问题,提出了1种基于金字塔切分注意力机制(PSA)的神经网络模型。首先,将三相电流信号和振动信号作为双分支输入到卷积神经网络进行特征提取,之后通过特征融合层将提取的信号特征进行融合。其次,添加PSA注意力机制捕获不同尺度的空间信息来丰富特征空间。最后,通过输出层输出诊断结果。实验表明所提模型能够显著提升同步电机故障诊断的准确率。

PSA和PPS耐高温纤维的结构与性能研究

通过红外光谱仪和X-射线衍射仪测定芳砜纶(PSA)纤维和聚苯硫醚(PPS)纤维的微观结构,用热分析仪测定两者的热分解温度,通过Instron电子强伸度仪测定两者受热前后的力学性能,并对两者结构与性能的差异进行比较。结果表明:PPS纤维的结构较PSA纤维稳定,结晶度高于PSA纤维;PSA纤维和PPS纤维的热分解温度分别为435.6℃和480℃,两种纤维均具有优异的耐热性能;PPS纤维的断裂强度和伸长均显著高于PSA纤维,两种纤维经高温处理后,强力损失均较小,经300℃处理200h后,强力仍保持90%左右。

改进Mask R-CNN的车辆检测算法

为提升在不同复杂场景下的车辆检测性能,提出一种基于改进Mask R-CNN的车辆检测算法.在算法的主干网络ResNet50中引入PSA极自注意力机制提升主干网络特征提取能力;在特征金字塔顶层网络中添加一个带有ECA注意力机制的分支与原分支进行特征融合,缓解顶层特征由于通道降维造成的信息损失.重新设计卷积检测头使得边框回归更为准确,并使用余弦退火算法和Soft-NMS算法来优化训练过程和后处理结果.实验结果表明,改进的Mask R-CNN车辆检测算法相比原Mask R-CNN算法在复杂场景下具有更高的检测精度,在CNRPark-EXT测试集中平均精确度提高3.8%,在更具挑战性的MiniPark测试集中平均精确度提高7.9%.

NFHP-RN:AMethod of Few-Shot Network Attack Detection Based on the Network Flow Holographic Picture-ResNet

Due to the rapid evolution of Advanced Persistent Threats(APTs)attacks,the emergence of new and rare attack samples,and even those never seen before,make it challenging for traditional rule-based detection methods to extract universal rules for effective detection.With the progress in techniques such as transfer learning and meta-learning,few-shot network attack detection has progressed.However,challenges in few-shot network attack detection arise from the inability of time sequence flow features to adapt to the fixed length input requirement of deep learning,difficulties in capturing rich information from original flow in the case of insufficient samples,and the challenge of high-level abstract representation.To address these challenges,a few-shot network attack detection based on NFHP(Network Flow Holographic Picture)-RN(ResNet)is proposed.Specifically,leveraging inherent properties of images such as translation invariance,rotation invariance,scale invariance,and illumination invariance,network attack traffic features and contextual relationships are intuitively represented in NFHP.In addition,an improved RN network model is employed for high-level abstract feature extraction,ensuring that the extracted high-level abstract features maintain the detailed characteristics of the original traffic behavior,regardless of changes in background traffic.Finally,a meta-learning model based on the self-attention mechanism is constructed,achieving the detection of novel APT few-shot network attacks through the empirical generalization of high-level abstract feature representations of known-class network attack behaviors.Experimental results demonstrate that the proposed method can learn high-level abstract features of network attacks across different traffic detail granularities.Comparedwith state-of-the-artmethods,it achieves favorable accuracy,precision,recall,and F1 scores for the identification of unknown-class network attacks through cross-validation onmultiple datasets.

A New Industrial Intrusion Detection Method Based on CNN-BiLSTM

Nowadays,with the rapid development of industrial Internet technology,on the one hand,advanced industrial control systems(ICS)have improved industrial production efficiency.However,there are more and more cyber-attacks targeting industrial control systems.To ensure the security of industrial networks,intrusion detection systems have been widely used in industrial control systems,and deep neural networks have always been an effective method for identifying cyber attacks.Current intrusion detection methods still suffer from low accuracy and a high false alarm rate.Therefore,it is important to build a more efficient intrusion detection model.This paper proposes a hybrid deep learning intrusion detection method based on convolutional neural networks and bidirectional long short-term memory neural networks(CNN-BiLSTM).To address the issue of imbalanced data within the dataset and improve the model’s detection capabilities,the Synthetic Minority Over-sampling Technique-Edited Nearest Neighbors(SMOTE-ENN)algorithm is applied in the preprocessing phase.This algorithm is employed to generate synthetic instances for the minority class,simultaneously mitigating the impact of noise in the majority class.This approach aims to create a more equitable distribution of classes,thereby enhancing the model’s ability to effectively identify patterns in both minority and majority classes.In the experimental phase,the detection performance of the method is verified using two data sets.Experimental results show that the accuracy rate on the CICIDS-2017 data set reaches 97.7%.On the natural gas pipeline dataset collected by Lan Turnipseed from Mississippi State University in the United States,the accuracy rate also reaches 85.5%.

Intelligent Fault Diagnosis Method of Rolling Bearings Based on Transfer Residual Swin Transformer with Shifted Windows

Due to their robust learning and expression ability for complex features,the deep learning(DL)model plays a vital role in bearing fault diagnosis.However,since there are fewer labeled samples in fault diagnosis,the depth of DL models in fault diagnosis is generally shallower than that of DL models in other fields,which limits the diagnostic performance.To solve this problem,a novel transfer residual Swin Transformer(RST)is proposed for rolling bearings in this paper.RST has 24 residual self-attention layers,which use the hierarchical design and the shifted window-based residual self-attention.Combined with transfer learning techniques,the transfer RST model uses pre-trained parameters from ImageNet.A new end-to-end method for fault diagnosis based on deep transfer RST is proposed.Firstly,wavelet transform transforms the vibration signal into a wavelet time-frequency diagram.The signal’s time-frequency domain representation can be represented simultaneously.Secondly,the wavelet time-frequency diagram is the input of the RST model to obtain the fault type.Finally,our method is verified on public and self-built datasets.Experimental results show the superior performance of our method by comparing it with a shallow neural network.

An Affective EEG Analysis Method Without Feature Engineering

Emotional electroencephalography(EEG)signals are a primary means of recording emotional brain activity.Currently,the most effective methods for analyzing emotional EEG signals involve feature engineering and neural networks.However,neural networks possess a strong ability for automatic feature extraction.Is it possible to discard feature engineering and directly employ neural networks for end-to-end recognition?Based on the characteristics of EEG signals,this paper proposes an end-to-end feature extraction and classification method for a dynamic self-attention network(DySAT).The study reveals significant differences in brain activity patterns associated with different emotions across various experimenters and time periods.The results of this experiment can provide insights into the reasons behind these differences.

低温等离子活化水对猕猴桃溃疡病菌抗菌活性及机制

为了探究低温等离子体活化水(cold plasma activated water,PAW)对丁香假单胞杆菌猕猴桃致病变种(Pseudomonas syringae pv.actinidiae,PSA)的抗菌活性和潜在机制。该研究通过介质阻挡放电(dielectric barrier discharge,DBD)低温等离子体发生装置制备不同激活时间的PAW,考察放电时间与杀菌效果之间的关系。此外,通过评估PSA形态特征,细胞粒径、DNA、细胞膜损伤情况和胞内活性氧积累(reactive oxygen species,ROS)情况探究PAW对PSA的杀菌机制。结果表明:PAW对PSA的杀灭效果与PAW激活时间呈依赖性,与对照相比PAW处理120 s后,PSA显著(P<0.05)减少了4.38 lg CFU/mL。扫描电子显微镜(field emission scanning electron microscope,FESEM)结果清楚地表明,由于PAW处理,PSA细胞发生了明显的质壁分离。荧光染色结果显示,PSA细胞DNA、膜渗透屏障被破坏程度、内容物泄漏量和ROS积累量与PAW激活时间表现为正相关关系。因此,推测PAW由于自身酸性、较高的氧化还原电位(oxidation-reduction potential,ORP),以及水性活性物质导致细胞的氧化损伤,而且这可能是杀灭PSA的主要原因。研究结果可为PAW控制猕猴桃细菌性溃疡病提供参考。

煤矸石灰渣研制聚硅酸铝混凝剂及应用研究

利用煤矸石灰渣制取聚硅酸铝（ＰＳＡ）混凝剂以及利用ＰＳＡ混凝剂处理印染和钢厂除尘废水。实验表明，该混凝剂具有优良的除浊、除色、除ＳＳ性能。与常规混凝剂相比。ＰＳＡ混凝剂的脱色率和ＳＳ去除率均提高２０％～４０％。综合考虑ＳＳ去除率、脱色率两方面因素，在处理废水时，选择ｐＨ值为６．０，沉降时间１５～２０ｍｉｎ为最佳。本文还对ＰＳＡ混凝剂的混凝脱色机理进行了初步探讨。

等离子体处理对芳砜纶纤维力学性能的影响

为改善芳砜纶纤维的可纺性,对其进行了低温常态等离子体处理,经数理统计分析处理前后纤维力学性能指标的测试数据表明:等离子体处理不会对芳砜纶纤维的力学性能形成损伤;处理工艺中时间、功率指数对芳砜纶纤维的断裂伸长率有显著影响.

改性丙烯酸酯乳液压敏胶的表面张力研究

以丙烯酸酯类单体为主要原料、丙烯酸(AA)为交联剂、十二烷基硫醇(CTA)为链转移剂、氢化松香树脂(HR)为增黏树脂和阴/非离子型乳化剂为复合乳化剂,采用半连续乳液聚合法制备了一系列丙烯酸酯PSA(压敏胶)乳液;研究了复合乳化剂含量、CTA含量和HR含量等对该PSA乳液表面张力的影响,并探讨了不同表面能的被粘薄膜基材对PSA乳液180°剥离强度的影响。结果表明:乳液的表面张力随复合乳化剂含量增加呈先降后升态势,随CTA或AA含量增加呈先升后降态势,随HR含量增加而不断下降;PSA乳液对BOPP薄膜的接触角随HR含量增加而减小;调节PSA乳液的表面张力,能有效改善PSA的粘接性能。

芳砜纶纤维在酸性条件下的失效特性

为揭示芳砜纶(PSA)纤维在高温、高湿、高酸性腐蚀条件下的失效特性,对PSA纤维在85℃、不同质量分数的硫酸腐蚀条件下的力学性能进行了研究,并采用扫描电镜(SEM)、红外光谱(IR)、热重(TG)分析等方法,对其微观结构及热稳定性能进行了分析.研究结果表明:PSA纤维在强酸加热条件下会发生水解反应,随着酸性腐蚀强度的增加,纤维失效加剧,纤维的力学性能、热稳定性能逐步下降;纤维的表面形态及分子结构发生改变,表面出现明显的侵蚀痕迹、甚至开裂,酰胺键水解形成羧基.这说明,芳砜纶在工业炉窑高温、高湿、高酸性腐蚀条件下应用时必须要做好防腐处理.

芳砜纶针刺非织造布耐酸性能研究

芳砜纶针刺非织造布经过硫酸溶液处理后,纤维断裂强度略有下降,纤维线密度减小;非织造布的尺寸稳定性很好,而其断裂强力却有明显增加。通过对针刺加固形式的纤维缠结模型的力学分析表明:芳砜纶经硫酸溶液处理后变细,在非织造布受拉伸时有助于增强纤维间的缠结并可以提高针刺缠结加固方式对纤维强度的利用率。

芳砜纶纤维在碱腐蚀下失效特性研究

为了揭示PSA纤维在工业炉窑高温、高湿、高腐蚀烟气环境下的失效特性,对PSA纤维在85℃、不同质量分数氢氧化钠腐蚀条件下的力学性能变化进行了试验研究,并采用扫描电镜、热重分析等材料分析方法,对其表面形态及热稳定性能变化进行分析。结果表明,PSA纤维的耐低碱性能较好,耐强碱性能较差,经强碱腐蚀之后,PSA纤维的表面结构遭到了破坏,热稳定性能下降;PSA纤维的耐低碱性能明显好于PI,略逊色于PMIA,耐强碱性能明显不如PPS和PMIA。这说明,PSA可以在工业炉窑高温、低碱性烟气环境下应用,力学稳定性明显好于PI,但必须要做好防腐蚀处理。