A Self-Attention Based Dynamic Resource Management for Satellite-Terrestrial Networks

The satellite-terrestrial networks possess the ability to transcend geographical constraints inherent in traditional communication networks,enabling global coverage and offering users ubiquitous computing power support,which is an important development direction of future communications.In this paper,we take into account a multi-scenario network model under the coverage of low earth orbit(LEO)satellite,which can provide computing resources to users in faraway areas to improve task processing efficiency.However,LEO satellites experience limitations in computing and communication resources and the channels are time-varying and complex,which makes the extraction of state information a daunting task.Therefore,we explore the dynamic resource management issue pertaining to joint computing,communication resource allocation and power control for multi-access edge computing(MEC).In order to tackle this formidable issue,we undertake the task of transforming the issue into a Markov decision process(MDP)problem and propose the self-attention based dynamic resource management(SABDRM)algorithm,which effectively extracts state information features to enhance the training process.Simulation results show that the proposed algorithm is capable of effectively reducing the long-term average delay and energy consumption of the tasks.

Stroke Electroencephalogram Data Synthesizing through Progressive Efficient Self-Attention Generative Adversarial Network

Early and timely diagnosis of stroke is critical for effective treatment,and the electroencephalogram(EEG)offers a low-cost,non-invasive solution.However,the shortage of high-quality patient EEG data often hampers the accuracy of diagnostic classification methods based on deep learning.To address this issue,our study designed a deep data amplification model named Progressive Conditional Generative Adversarial Network with Efficient Approximating Self Attention(PCGAN-EASA),which incrementally improves the quality of generated EEG features.This network can yield full-scale,fine-grained EEG features from the low-scale,coarse ones.Specially,to overcome the limitations of traditional generative models that fail to generate features tailored to individual patient characteristics,we developed an encoder with an effective approximating self-attention mechanism.This encoder not only automatically extracts relevant features across different patients but also reduces the computational resource consumption.Furthermore,the adversarial loss and reconstruction loss functions were redesigned to better align with the training characteristics of the network and the spatial correlations among electrodes.Extensive experimental results demonstrate that PCGAN-EASA provides the highest generation quality and the lowest computational resource usage compared to several existing approaches.Additionally,it significantly improves the accuracy of subsequent stroke classification tasks.

An Aerial Target Recognition Algorithm Based on Self-Attention and LSTM

In the application of aerial target recognition,on the one hand,the recognition error produced by the single measurement of the sensor is relatively large due to the impact of noise.On the other hand,it is difficult to apply machine learning methods to improve the intelligence and recognition effect due to few or no actual measurement samples.Aiming at these problems,an aerial target recognition algorithm based on self-attention and Long Short-Term Memory Network(LSTM)is proposed.LSTM can effectively extract temporal dependencies.The attention mechanism calculates the weight of each input element and applies the weight to the hidden state of the LSTM,thereby adjusting the LSTM’s attention to the input.This combination retains the learning ability of LSTM and introduces the advantages of the attention mechanism,making the model have stronger feature extraction ability and adaptability when processing sequence data.In addition,based on the prior information of the multidimensional characteristics of the target,the three-point estimation method is adopted to simulate an aerial target recognition dataset to train the recognition model.The experimental results show that the proposed algorithm achieves more than 91%recognition accuracy,lower false alarm rate and higher robustness compared with the multi-attribute decision-making(MADM)based on fuzzy numbers.

Hashtag Recommendation Using LSTM Networks with Self-Attention

On Twitter,people often use hashtags to mark the subject of a tweet.Tweets have specific themes or content that are easy for people to manage.With the increase in the number of tweets,how to automatically recommend hashtags for tweets has received wide attention.The previous hashtag recommendation methods were to convert the task into a multi-class classification problem.However,these methods can only recommend hashtags that appeared in historical information,and cannot recommend the new ones.In this work,we extend the self-attention mechanism to turn the hashtag recommendation task into a sequence labeling task.To train and evaluate the proposed method,we used the real tweet data which is collected from Twitter.Experimental results show that the proposed method can be significantly better than the most advanced method.Compared with the state-of-the-art methods,the accuracy of our method has been increased 4%.

Clothing Parsing Based on Multi-Scale Fusion and Improved Self-Attention Mechanism

Due to the lack of long-range association and spatial location information,fine details and accurate boundaries of complex clothing images cannot always be obtained by using the existing deep learning-based methods.This paper presents a convolutional structure with multi-scale fusion to optimize the step of clothing feature extraction and a self-attention module to capture long-range association information.The structure enables the self-attention mechanism to directly participate in the process of information exchange through the down-scaling projection operation of the multi-scale framework.In addition,the improved self-attention module introduces the extraction of 2-dimensional relative position information to make up for its lack of ability to extract spatial position features from clothing images.The experimental results based on the colorful fashion parsing dataset(CFPD)show that the proposed network structure achieves 53.68%mean intersection over union(mIoU)and has better performance on the clothing parsing task.

Joint Self-Attention Based Neural Networks for Semantic Relation Extraction

Relation extraction is an important task in NLP community.However,some models often fail in capturing Long-distance dependence on semantics,and the interaction between semantics of two entities is ignored.In this paper,we propose a novel neural network model for semantic relation classification called joint self-attention bi-LSTM(SA-Bi-LSTM)to model the internal structure of the sentence to obtain the importance of each word of the sentence without relying on additional information,and capture Long-distance dependence on semantics.We conduct experiments using the SemEval-2010 Task 8 dataset.Extensive experiments and the results demonstrated that the proposed method is effective against relation classification,which can obtain state-ofthe-art classification accuracy just with minimal feature engineering.

Self-attention transfer networks for speech emotion recognition

Background A crucial element of human-machine interaction,the automatic detection of emotional states from human speech has long been regarded as a challenging task for machine learning models.One vital challenge in speech emotion recognition(SER)is learning robust and discriminative representations from speech.Although machine learning methods have been widely applied in SER research,the inadequate amount of available annotated data has become a bottleneck impeding the extended application of such techniques(e.g.,deep neural networks).To address this issue,we present a deep learning method that combines knowledge transfer and self-attention for SER tasks.Herein,we apply the log-Mel spectrogram with deltas and delta-deltas as inputs.Moreover,given that emotions are time dependent,we apply temporal convolutional neural networks to model the variations in emotions.We further introduce an attention transfer mechanism,which is based on a self-attention algorithm to learn long-term dependencies.The self-attention transfer network(SATN)in our proposed approach takes advantage of attention transfer to learn attention from speech recognition,followed by transferring this knowledge into SER.An evaluation built on Interactive Emotional Dyadic Motion Capture(IEMOCAP)dataset demonstrates the effectiveness of the proposed model.

NFHP-RN:AMethod of Few-Shot Network Attack Detection Based on the Network Flow Holographic Picture-ResNet

Due to the rapid evolution of Advanced Persistent Threats(APTs)attacks,the emergence of new and rare attack samples,and even those never seen before,make it challenging for traditional rule-based detection methods to extract universal rules for effective detection.With the progress in techniques such as transfer learning and meta-learning,few-shot network attack detection has progressed.However,challenges in few-shot network attack detection arise from the inability of time sequence flow features to adapt to the fixed length input requirement of deep learning,difficulties in capturing rich information from original flow in the case of insufficient samples,and the challenge of high-level abstract representation.To address these challenges,a few-shot network attack detection based on NFHP(Network Flow Holographic Picture)-RN(ResNet)is proposed.Specifically,leveraging inherent properties of images such as translation invariance,rotation invariance,scale invariance,and illumination invariance,network attack traffic features and contextual relationships are intuitively represented in NFHP.In addition,an improved RN network model is employed for high-level abstract feature extraction,ensuring that the extracted high-level abstract features maintain the detailed characteristics of the original traffic behavior,regardless of changes in background traffic.Finally,a meta-learning model based on the self-attention mechanism is constructed,achieving the detection of novel APT few-shot network attacks through the empirical generalization of high-level abstract feature representations of known-class network attack behaviors.Experimental results demonstrate that the proposed method can learn high-level abstract features of network attacks across different traffic detail granularities.Comparedwith state-of-the-artmethods,it achieves favorable accuracy,precision,recall,and F1 scores for the identification of unknown-class network attacks through cross-validation onmultiple datasets.

Automatic infrared image recognition method for substation equipment based on a deep self-attention network and multi-factor similarity calculation

Infrared image recognition plays an important role in the inspection of power equipment.Existing technologies dedicated to this purpose often require manually selected features,which are not transferable and interpretable,and have limited training data.To address these limitations,this paper proposes an automatic infrared image recognition framework,which includes an object recognition module based on a deep self-attention network and a temperature distribution identification module based on a multi-factor similarity calculation.First,the features of an input image are extracted and embedded using a multi-head attention encoding-decoding mechanism.Thereafter,the embedded features are used to predict the equipment component category and location.In the located area,preliminary segmentation is performed.Finally,similar areas are gradually merged,and the temperature distribution of the equipment is obtained to identify a fault.Our experiments indicate that the proposed method demonstrates significantly improved accuracy compared with other related methods and,hence,provides a good reference for the automation of power equipment inspection.

An Intrusion Detection Scheme Based on Federated Learning and Self-Attention Fusion Convolutional Neural Network for IoT

Traditional based deep learning intrusion detection methods face problems such as insufficient cloud storage,data privacy leaks,high com-munication costs,unsatisfactory detection rates,and false positive rate.To address existing issues in intrusion detection,this paper presents a novel approach called CS-FL,which combines Federated Learning and a Self-Attention Fusion Convolutional Neural Network.Federated Learning is a new distributed computing model that enables individual training of client data without uploading local data to a central server.at the same time,local training results are uploaded and integrated across all participating clients to produce a global model.The sharing model reduces communication costs,protects data privacy,and solves problems such as insufficient cloud storage and“data islands”for each client.In the proposed method,a hybrid model is formed by integrating the self-Attention and similar parts of the Convolutional Neural Network in the local data processing.This approach not only enhances the performance of the hybrid model but also reduces computational overhead compared to pure hybrid neural networks.Results from experiments on the NSL-KDD dataset show that the proposed method outperforms other intrusion detection techniques,resulting in a significant improvement in performance.This demonstrates the effectiveness of the proposed approach in improving intrusion detection accuracy.

windows Vista Network Cente简化的管理操作带来的便捷体验

当用户认为网络出现故障时。他首先想到的就是打电话给身为网络管理员的你。经过艰苦的努力。你终于悲哀地发现用户永远都学不会使用IPconfig和Ping。他只希望你能尽快出现在身边解决问题。Windows Vista网络中心（Network Center）的出现。让管理员又燃起了新的希望。

QAR Data Imputation Using Generative Adversarial Network with Self-Attention Mechanism

Quick Access Recorder(QAR),an important device for storing data from various flight parameters,contains a large amount of valuable data and comprehensively records the real state of the airline flight.However,the recorded data have certain missing values due to factors,such as weather and equipment anomalies.These missing values seriously affect the analysis of QAR data by aeronautical engineers,such as airline flight scenario reproduction and airline flight safety status assessment.Therefore,imputing missing values in the QAR data,which can further guarantee the flight safety of airlines,is crucial.QAR data also have multivariate,multiprocess,and temporal features.Therefore,we innovatively propose the imputation models A-AEGAN("A"denotes attention mechanism,"AE"denotes autoencoder,and"GAN"denotes generative adversarial network)and SA-AEGAN("SA"denotes self-attentive mechanism)for missing values of QAR data,which can be effectively applied to QAR data.Specifically,we apply an innovative generative adversarial network to impute missing values from QAR data.The improved gated recurrent unit is then introduced as the neural unit of GAN,which can successfully capture the temporal relationships in QAR data.In addition,we modify the basic structure of GAN by using an autoencoder as the generator and a recurrent neural network as the discriminator.The missing values in the QAR data are imputed by using the adversarial relationship between generator and discriminator.We introduce an attention mechanism in the autoencoder to further improve the capability of the proposed model to capture the features of QAR data.Attention mechanisms can maintain the correlation among QAR data and improve the capability of the model to impute missing data.Furthermore,we improve the proposed model by integrating a self-attention mechanism to further capture the relationship between different parameters within the QAR data.Experimental results on real datasets demonstrate that the model can reasonably impute the missing values in QAR data with excellent results.

The Dynamic Relationship of Brain Networks Across Time Windows During Product-Based Creative Thinking

Consensus of creativity research suggests that the measurement of both originality and valuableness is necessary when designing creativity tasks.However,few studies have emphasized valuableness when exploring underlying neural substrates of creative thinking.The present study employs product-based creativity tasks that measure both originality and valuableness in an exploration of the dynamic relationship between the default mode(DMN),executive control(ECN),and salience(SN)networks through time windows.This methodology highlights relevance,or valuableness,in creativity evaluation as opposed to divergent thinking tasks solely measuring originality.The researchers identified seven brain regions belonging to the ECN,DMN,and SN as regions of interest(ROIs),as well as four representative seeds to analyze functional connectivity in 25 college student participants.Results showed that all of the identified ROIs were involved during the creative task.The insula,precuneus,and ventrolateral prefrontal cortex(vlPFC)remained active across all stages of product-based creative thinking.Moreover,the connectivity analyses revealed varied interaction patterns of DMN,ECN,and SN at different thinking stages.The integrated findings of the whole brain,ROI,and connectivity analyses suggest a trend that the DMN and SN(which relate to bottom-up thinking)attenuate as time proceeds,whereas the vlPFC(which relates to top-down thinking)gets stronger at later stages;these findings reflect the nature of our creativity tasks and decision-making of valuableness in later stages.Based on brain region activation throughout execution of the task,we propose that product-based creative process may include three stages:exploration and association,incubation and insight,and finally,evaluation and decision making.This model provides a thinking frame for further research and classroom instruction.

基于Windows服务器的网络安全问题研究

针对Windows服务器网络安全框架,分析了Windows服务器网络安全功能及应用,指出了Windows服务器网络安全的常见问题,包括操作系统漏洞、病毒攻击、恶意软件、黑客攻击、暴力破解及系统攻击等。提出了Windows服务器网络安全常用算法,包括Windows进程自动加载技术、端口隐藏、密码算法。分析了经典Windows服务器网络安全框架,需将网络监控系统放在同一个网络中,采用加密数据传输,以保证客户通信安全。

Prediction of the Wastewater’s pH Based on Deep Learning Incorporating Sliding Windows

To protect the environment,the discharged sewage’s quality must meet the state’s discharge standards.There are many water quality indicators,and the pH(Potential of Hydrogen)value is one of them.The natural water’s pH value is 6.0–8.5.The sewage treatment plant uses some data in the sewage treatment process to monitor and predict whether wastewater’s pH value will exceed the standard.This paper aims to study the deep learning prediction model of wastewater’s pH.Firstly,the research uses the random forest method to select the data features and then,based on the sliding window,convert the data set into a time series which is the input of the deep learning training model.Secondly,by analyzing and comparing relevant references,this paper believes that the CNN(Convolutional Neural Network)model is better at nonlinear data modeling and constructs a CNN model including the convolution and pooling layers.After alternating the combination of the convolutional layer and pooling layer,all features are integrated into a full-connected neural network.Thirdly,the number of input samples of the CNN model directly affects the prediction effect of the model.Therefore,this paper adopts the sliding window method to study the optimal size.Many experimental results show that the optimal prediction model can be obtained when alternating six convolutional layers and three pooling layers.The last full-connection layer contains two layers and 64 neurons per layer.The sliding window size selects as 12.Finally,the research has carried out data prediction based on the optimal CNN deep learning model.The predicted pH of the sewage is between 7.2 and 8.6 in this paper.The result is applied in the monitoring system platform of the“Intelligent operation and maintenance platform of the reclaimed water plant.”

基于CNN-LSTM的水泥熟料f-CaO预测模型

水泥熟料中游离氧化钙(f-CaO)含量的传统人工离线检测缺乏时效性,不利于生产指导。针对离线检测的滞后问题和软测量模型中f-CaO含量与辅助变量的时序匹配问题,提出了一种基于卷积神经网络(convolutional neural network,CNN)和长短时记忆(long short-term memory,LSTM)神经网络的f-CaO含量预测模型。首先,利用滑动窗口截取辅助变量的区间数据;然后,采用CNN提取区间数据的时序特征;之后,构建LSTM神经网络模型;最后,控制截取辅助变量的延迟时间和间隔时间,根据模型预测拟合度提取辅助变量的最优时序特征。仿真结果表明,所提模型提高了水泥熟料中f-CaO含量的预测精度。

基于深度学习的桥梁表观裂缝检测算法研究

针对在复杂背景条件下难以直接对桥梁表观裂缝进行检测的问题,文章提出一种基于深度学习的桥梁表观裂缝检测算法。首先利用滑动窗口算法将采集到的桥梁表观裂缝图像切分为小尺寸的桥梁裂缝面元图像和桥梁背景面元图像,并根据对面元图像的分析,提出一种基于Inception网络和残差网络(ResNet)的桥梁裂缝分类模型,用于桥梁裂缝面元和桥梁背景面元的识别;然后结合桥梁裂缝分类模型与滑动窗口算法对桥梁表观裂缝图像进行检测;最后利用数字图像处理技术测量裂缝宽度。结果表明:该文算法对桥梁表观裂缝有超过99%的分类精度,可满足实际工程需要;实现了裂缝的提取并能准确地定位出裂缝在图像中的位置;根据成像原理能测量出裂缝宽度。与传统的深度学习模型相比,该模型拥有更高的执行效率,可用于大规模检测,更易于应用在桥梁健康检测中。

资源约束情境下政策企业家的朴素式创新:网络、窗口、塑造——基于C市团委的个案分析

政策创新一直是公共部门关注的核心议题。通过构建“网络—窗口—塑造”框架,以C市团委“第二课堂成绩单”制度的朴素式创新案例为例,探讨资源约束情境下政策企业家如何推动朴素式创新,展示政策企业家在朴素式创新过程中的行为细节和动力机制。就创新路径而言,政策企业家在资源约束下与同级公共部门、专家智库等多主体进行正式或非正式互动中增强政治势能,以政治合力推进政策创新;就创新时机而言,问题流、政治流、政策流三流合一形成“政策之窗”的机会转瞬即逝,把握“前置窗口”则是完成政策创新既定目标的另一优化选择;就创新动因而言,政策创新既需要对创新主体的动因进行分析,也需要清晰地理解政策创新背后的影响机制。据此,提出针对性政策建议:政策企业家在资源约束情境下,应注重朴素式创新,通过创新的政策设计、实施方式以及资源获取方式推动政策发展和治理进程;充分发挥政策网络的作用,进行有效的资源整合;积极引导和培育公众参与,构建多方协作的政策环境。

考虑客户满意度的实时取送货路径优化问题

针对即时配送订单出现的动态性、服务客户的满意度、末端配送路网的非对称性,在配送订单的时间窗限制下,建立实时取送货路径优化模型。通过定义并调整非对称网络系数,构建非对称配送网络,提出滚动时域忽略策略(Ignore策略)和滚动时域实时判断策略(Real-time策略)。Ignore策略要求配送员一旦出发,返回配送起点前忽略所有新出现的订单,Real-time策略要求配送途中实时判断当新订单出现时,是否返回起点取货,重新规划配送路线。使用数值仿真软件,在不同非对称系数、客户时间窗、订单数量等的情形下分析策略适用性。数值算例分析表明,Real-time策略适用于网络较大的情形,而网络较小且订单数量较少时,Ignore策略更适用。研究结果可为城市末端合单配送和路径优化策略提供新的思路和参考。

集合空间关键字内聚组查询方法

给定一个道路网络和社交网络,集合空间关键字查询的目的是找到一组兴趣点,该组兴趣点的文本信息包含所有查询关键字,与查询的位置较近且彼此之间的距离较小。内聚组查询的目的是找到在地理位置和社交关系上紧密联系的一组用户;而集合空间关键字内聚组查询的目的是找到满足查询要求的一对最佳匹配的兴趣点集合和用户集合。针对这一问题,提出一种新的集合空间关键字内聚组查询处理模式。首先通过快速贪心查询过程获得候选兴趣点集合,然后使用core-tree结构存储(k,c)-core核心分解的结果,从而提高内聚组查询效率,并且保证查询结果能够同时满足用户之间的社会关系约束和兴趣点之间的空间位置约束。通过在真实数据集上开展实验,结果表明提出的方法比枚举方法的查询效率快1~2个数量级,并且具有较高查询准确性。