CNN Channel Attention Intrusion Detection SystemUsing NSL-KDD Dataset

Intrusion detection systems(IDS)are essential in the field of cybersecurity because they protect networks from a wide range of online threats.The goal of this research is to meet the urgent need for small-footprint,highly-adaptable Network Intrusion Detection Systems(NIDS)that can identify anomalies.The NSL-KDD dataset is used in the study;it is a sizable collection comprising 43 variables with the label’s“attack”and“level.”It proposes a novel approach to intrusion detection based on the combination of channel attention and convolutional neural networks(CNN).Furthermore,this dataset makes it easier to conduct a thorough assessment of the suggested intrusion detection strategy.Furthermore,maintaining operating efficiency while improving detection accuracy is the primary goal of this work.Moreover,typical NIDS examines both risky and typical behavior using a variety of techniques.On the NSL-KDD dataset,our CNN-based approach achieves an astounding 99.728%accuracy rate when paired with channel attention.Compared to previous approaches such as ensemble learning,CNN,RBM(Boltzmann machine),ANN,hybrid auto-encoders with CNN,MCNN,and ANN,and adaptive algorithms,our solution significantly improves intrusion detection performance.Moreover,the results highlight the effectiveness of our suggested method in improving intrusion detection precision,signifying a noteworthy advancement in this field.Subsequent efforts will focus on strengthening and expanding our approach in order to counteract growing cyberthreats and adjust to changing network circumstances.

New Fusion Approach of Spatial and Channel Attention for Semantic Segmentation of Very High Spatial Resolution Remote Sensing Images

The semantic segmentation of very high spatial resolution remote sensing images is difficult due to the complexity of interpreting the interactions between the objects in the scene. Indeed, effective segmentation requires considering spatial local context and long-term dependencies. To address this problem, the proposed approach is inspired by the MAC-UNet network which is an extension of U-Net, densely connected combined with channel attention. The advantages of this solution are as follows: 4) The new model introduces a new attention called propagate attention to build an attention-based encoder. 2) The fusion of multi-scale information is achieved by a weighted linear combination of the attentions whose coefficients are learned during the training phase. 3) Introducing in the decoder, the Spatial-Channel-Global-Local block which is an attention layer that uniquely combines channel attention and spatial attention locally and globally. The performances of the model are evaluated on 2 datasets WHDLD and DLRSD and show results of mean intersection over union (mIoU) index in progress between 1.54% and 10.47% for DLRSD and between 1.04% and 4.37% for WHDLD compared with the most efficient algorithms with attention mechanisms like MAU-Net and transformers like TMNet.

MCBAN: A Small Object Detection Multi-Convolutional Block Attention Network

Object detection has made a significant leap forward in recent years.However,the detection of small objects continues to be a great difficulty for various reasons,such as they have a very small size and they are susceptible to missed detection due to background noise.Additionally,small object information is affected due to the downsampling operations.Deep learning-based detection methods have been utilized to address the challenge posed by small objects.In this work,we propose a novel method,the Multi-Convolutional Block Attention Network(MCBAN),to increase the detection accuracy of minute objects aiming to overcome the challenge of information loss during the downsampling process.The multi-convolutional attention block(MCAB);channel attention and spatial attention module(SAM)that make up MCAB,have been crafted to accomplish small object detection with higher precision.We have carried out the experiments on the Karlsruhe Institute of Technology and Toyota Technological Institute(KITTI)and Pattern Analysis,Statical Modeling and Computational Learning(PASCAL)Visual Object Classes(VOC)datasets and have followed a step-wise process to analyze the results.These experiment results demonstrate that significant gains in performance are achieved,such as 97.75%for KITTI and 88.97%for PASCAL VOC.The findings of this study assert quite unequivocally the fact that MCBAN is much more efficient in the small object detection domain as compared to other existing approaches.

Deep neural network based on multi-level wavelet and attention for structured illumination microscopy

Structured illumination microscopy(SIM)is a popular and powerful super-resolution(SR)technique in biomedical research.However,the conventional reconstruction algorithm for SIM heavily relies on the accurate prior knowledge of illumination patterns and signal-to-noise ratio(SNR)of raw images.To obtain high-quality SR images,several raw images need to be captured under high fluorescence level,which further restricts SIM’s temporal resolution and its applications.Deep learning(DL)is a data-driven technology that has been used to expand the limits of optical microscopy.In this study,we propose a deep neural network based on multi-level wavelet and attention mechanism(MWAM)for SIM.Our results show that the MWAM network can extract high-frequency information contained in SIM raw images and accurately integrate it into the output image,resulting in superior SR images compared to those generated using wide-field images as input data.We also demonstrate that the number of SIM raw images can be reduced to three,with one image in each illumination orientation,to achieve the optimal tradeoff between temporal and spatial resolution.Furthermore,our MWAM network exhibits superior reconstruction ability on low-SNR images compared to conventional SIM algorithms.We have also analyzed the adaptability of this network on other biological samples and successfully applied the pretrained model to other SIM systems.

Facial Expression Recognition Based on Multi-Channel Attention Residual Network

For the problems of complex model structure and too many training parameters in facial expression recognition algorithms,we proposed a residual network structure with a multi-headed channel attention(MCA)module.The migration learning algorithm is used to pre-train the convolutional layer parameters and mitigate the overfitting caused by the insufficient number of training samples.The designed MCA module is integrated into the ResNet18 backbone network.The attention mechanism highlights important information and suppresses irrelevant information by assigning different coefficients or weights,and the multi-head structure focuses more on the local features of the pictures,which improves the efficiency of facial expression recognition.Experimental results demonstrate that the model proposed in this paper achieves excellent recognition results in Fer2013,CK+and Jaffe datasets,with accuracy rates of 72.7%,98.8%and 93.33%,respectively.

Attention Mechanism-Based Method for Intrusion Target Recognition in Railway

The detection of foreign object intrusion is crucial for ensuring the safety of railway operations.To address challenges such as low efficiency,suboptimal detection accuracy,and slow detection speed inherent in conventional comprehensive video monitoring systems for railways,a railway foreign object intrusion recognition and detection system is conceived and implemented using edge computing and deep learning technologies.In a bid to raise detection accuracy,the convolutional block attention module(CBAM),including spatial and channel attention modules,is seamlessly integrated into the YOLOv5 model,giving rise to the CBAM-YOLOv5 model.Furthermore,the distance intersection-over-union_non-maximum suppression(DIo U_NMS)algorithm is employed in lieu of the weighted nonmaximum suppression algorithm,resulting in improved detection performance for intrusive targets.To accelerate detection speed,the model undergoes pruning based on the batch normalization(BN)layer,and Tensor RT inference acceleration techniques are employed,culminating in the successful deployment of the algorithm on edge devices.The CBAM-YOLOv5 model exhibits a notable 2.1%enhancement in detection accuracy when evaluated on a selfconstructed railway dataset,achieving 95.0%for mean average precision(m AP).Furthermore,the inference speed on edge devices attains a commendable 15 frame/s.

An Assisted Diagnosis of Alzheimer’s Disease Incorporating Attention Mechanisms Med-3D Transfer Modeling

Alzheimer’s disease(AD)is a complex,progressive neurodegenerative disorder.The subtle and insidious onset of its pathogenesis makes early detection of a formidable challenge in both contemporary neuroscience and clinical practice.In this study,we introduce an advanced diagnostic methodology rooted in theMed-3D transfermodel and enhanced with an attention mechanism.We aim to improve the precision of AD diagnosis and facilitate its early identification.Initially,we employ a spatial normalization technique to address challenges like clarity degradation and unsaturation,which are commonly observed in imaging datasets.Subsequently,an attention mechanism is incorporated to selectively focus on the salient features within the imaging data.Building upon this foundation,we present the novelMed-3D transfermodel,designed to further elucidate and amplify the intricate features associated withADpathogenesis.Our proposedmodel has demonstrated promising results,achieving a classification accuracy of 92%.To emphasize the robustness and practicality of our approach,we introduce an adaptive‘hot-updating’auxiliary diagnostic system.This system not only enables continuous model training and optimization but also provides a dynamic platform to meet the real-time diagnostic and therapeutic demands of AD.

基于CSI与Attention-BiLSTM的动作识别算法

与传统动作识别技术相比,基于信道状态信息的动作识别具有成本低、安全便利等特点,应用前景广阔。利用乐鑫ESP32采集信道子载波幅值信息,结合预处理算法,并基于结合注意力机制的双向长短期记忆网络的动作识别算法,实现对走路、拖地、捡起、坐下、蹲下和站起六种动作的特征提取与分类识别。测试结果表明:算法在测试集上的平均识别准确率高达95.8%,相较于常规的长短期记忆算法,识别准确率更高、收敛速度更快;与传统基于统计特征与机器学习的分类算法相比,算法直接利用神经网络自动提取时序特征,特征提取更精确,准确率提升超过10%。试验结果验证了该算法在基于信道状态信息的动作识别上的有效性,说明该算法具有较高的实用价值。

A hybrid attention deep learning network for refined segmentation of cracks from shield tunnel lining images

This research developed a hybrid position-channel network (named PCNet) through incorporating newly designed channel and position attention modules into U-Net to alleviate the crack discontinuity problem in channel and spatial dimensions. In PCNet, the U-Net is used as a baseline to extract informative spatial and channel-wise features from shield tunnel lining crack images. A channel and a position attention module are designed and embedded after each convolution layer of U-Net to model the feature interdependencies in channel and spatial dimensions. These attention modules can make the U-Net adaptively integrate local crack features with their global dependencies. Experiments were conducted utilizing the dataset based on the images from Shanghai metro shield tunnels. The results validate the effectiveness of the designed channel and position attention modules, since they can individually increase balanced accuracy (BA) by 11.25% and 12.95%, intersection over union (IoU) by 10.79% and 11.83%, and F1 score by 9.96% and 10.63%, respectively. In comparison with the state-of-the-art models (i.e. LinkNet, PSPNet, U-Net, PANet, and Mask R–CNN) on the testing dataset, the proposed PCNet outperforms others with an improvement of BA, IoU, and F1 score owing to the implementation of the channel and position attention modules. These evaluation metrics indicate that the proposed PCNet presents refined crack segmentation with improved performance and is a practicable approach to segment shield tunnel lining cracks in field practice.

Improved Blending Attention Mechanism in Visual Question Answering

Visual question answering(VQA)has attracted more and more attention in computer vision and natural language processing.Scholars are committed to studying how to better integrate image features and text features to achieve better results in VQA tasks.Analysis of all features may cause information redundancy and heavy computational burden.Attention mechanism is a wise way to solve this problem.However,using single attention mechanism may cause incomplete concern of features.This paper improves the attention mechanism method and proposes a hybrid attention mechanism that combines the spatial attention mechanism method and the channel attention mechanism method.In the case that the attention mechanism will cause the loss of the original features,a small portion of image features were added as compensation.For the attention mechanism of text features,a selfattention mechanism was introduced,and the internal structural features of sentences were strengthened to improve the overall model.The results show that attention mechanism and feature compensation add 6.1%accuracy to multimodal low-rank bilinear pooling network.

Attention-based neural network for end-to-end music separation

The end-to-end separation algorithm with superior performance in the field of speech separation has not been effectively used in music separation.Moreover,since music signals are often dual channel data with a high sampling rate,how to model longsequence data and make rational use of the relevant information between channels is also an urgent problem to be solved.In order to solve the above problems,the performance of the end-to-end music separation algorithm is enhanced by improving the network structure.Our main contributions include the following:(1)A more reasonable densely connected U-Net is designed to capture the long-term characteristics of music,such as main melody,tone and so on.(2)On this basis,the multi-head attention and dualpath transformer are introduced in the separation module.Channel attention units are applied recursively on the feature map of each layer of the network,enabling the network to perform long-sequence separation.Experimental results show that after the introduction of the channel attention,the performance of the proposed algorithm has a stable improvement compared with the baseline system.On the MUSDB18 dataset,the average score of the separated audio exceeds that of the current best-performing music separation algorithm based on the time-frequency domain(T-F domain).

A Multi-View Gait Recognition Method Using Deep Convolutional Neural Network and Channel Attention Mechanism

In many existing multi-view gait recognition methods based on images or video sequences,gait sequences are usually used to superimpose and synthesize images and construct energy-like template.However,information may be lost during the process of compositing image and capture EMG signals.Errors and the recognition accuracy may be introduced and affected respectively by some factors such as period detection.To better solve the problems,a multi-view gait recognition method using deep convolutional neural network and channel attention mechanism is proposed.Firstly,the sliding time window method is used to capture EMG signals.Then,the back-propagation learning algorithm is used to train each layer of convolution,which improves the learning ability of the convolutional neural network.Finally,the channel attention mechanism is integrated into the neural network,which will improve the ability of expressing gait features.And a classifier is used to classify gait.As can be shown from experimental results on two public datasets,OULP and CASIA-B,the recognition rate of the proposed method can be achieved at 88.44%and 97.25%respectively.As can be shown from the comparative experimental results,the proposed method has better recognition effect than several other newer convolutional neural network methods.Therefore,the combination of convolutional neural network and channel attention mechanism is of great value for gait recognition.

Channel attention based wavelet cascaded network for image super-resolution

Convolutional neural networks(CNNs) have shown great potential for image super-resolution(SR).However,most existing CNNs only reconstruct images in the spatial domain,resulting in insufficient high-frequency details of reconstructed images.To address this issue,a channel attention based wavelet cascaded network for image super-resolution(CWSR) is proposed.Specifically,a second-order channel attention(SOCA) mechanism is incorporated into the network,and the covariance matrix normalization is utilized to explore interdependencies between channel-wise features.Then,to boost the quality of residual features,the non-local module is adopted to further improve the global information integration ability of the network.Finally,taking the image loss in the spatial and wavelet domains into account,a dual-constrained loss function is proposed to optimize the network.Experimental results illustrate that CWSR outperforms several state-of-the-art methods in terms of both visual quality and quantitative metrics.

Attention-YOLO:引入注意力机制的YOLO检测算法

实时目标检测算法YOLOv3的检测速度较快且精度良好,但存在边界框定位不够精确、难以区分重叠物体等不足。提出了Attention-YOLO算法,该算法借鉴了基于项的注意力机制,将通道注意力及空间注意力机制加入特征提取网络之中,使用经过筛选加权的特征向量来替换原有的特征向量进行残差融合,同时添加二阶项来减少融合过程中的信息损失并加速模型收敛。通过在COCO和PASCAL VOC数据集上的实验表明,该算法有效降低了边界框的定位误差并提升了检测精度。相比YOLOv3算法在COCO测试集上的mAP_(@IoU[0.5:0.95])提升了最高2.5 mAP,在PASCAL VOC 2007测试集上达到了最高81.9 mAP。

An attention-based prototypical network for forest fire smoke few-shot detection

Existing almost deep learning methods rely on a large amount of annotated data, so they are inappropriate for forest fire smoke detection with limited data. In this paper, a novel hybrid attention-based few-shot learning method, named Attention-Based Prototypical Network, is proposed for forest fire smoke detection. Specifically, feature extraction network, which consists of convolutional block attention module, could extract high-level and discriminative features and further decrease the false alarm rate resulting from suspected smoke areas. Moreover, we design a metalearning module to alleviate the overfitting issue caused by limited smoke images, and the meta-learning network enables achieving effective detection via comparing the distance between the class prototype of support images and the features of query images. A series of experiments on forest fire smoke datasets and miniImageNet dataset testify that the proposed method is superior to state-of-the-art few-shot learning approaches.

Mixed Attention Densely Residual Network for Single Image Super-Resolution

Recent applications of convolutional neural networks(CNNs)in single image super-resolution(SISR)have achieved unprecedented performance.However,existing CNN-based SISR network structure design consider mostly only channel or spatial information,and cannot make full use of both channel and spatial information to improve SISR performance further.The present work addresses this problem by proposing a mixed attention densely residual network architecture that can make full and simultaneous use of both channel and spatial information.Specifically,we propose a residual in dense network structure composed of dense connections between multiple dense residual groups to form a very deep network.This structure allows each dense residual group to apply a local residual skip connection and enables the cascading of multiple residual blocks to reuse previous features.A mixed attention module is inserted into each dense residual group,to enable the algorithm to fuse channel attention with laplacian spatial attention effectively,and thereby more adaptively focus on valuable feature learning.The qualitative and quantitative results of extensive experiments have demonstrate that the proposed method has a comparable performance with other stateof-the-art methods.

Image Inpainting Detection Based on High-Pass Filter Attention Network

Image inpainting based on deep learning has been greatly improved.The original purpose of image inpainting was to repair some broken photos, suchas inpainting artifacts. However, it may also be used for malicious operations,such as destroying evidence. Therefore, detection and localization of imageinpainting operations are essential. Recent research shows that high-pass filteringfull convolutional network (HPFCN) is applied to image inpainting detection andachieves good results. However, those methods did not consider the spatial location and channel information of the feature map. To solve these shortcomings, weintroduce the squeezed excitation blocks (SE) and propose a high-pass filter attention full convolutional network (HPACN). In feature extraction, we apply concurrent spatial and channel attention (scSE) to enhance feature extraction and obtainmore information. Channel attention (cSE) is introduced in upsampling toenhance detection and localization. The experimental results show that the proposed method can achieve improvement on ImageNet.

复杂战场环境下改进YOLOv5军事目标识别算法研究

复杂战场环境下军事目标识别技术是提升战场情报获取能力的基础和关键。针对当前军事目标识别技术在复杂战场环境下漏检误检率高、实时性差等问题,提出一种基于改进YOLOv5模型的PB-YOLO军事目标识别算法。将改进的目标识别算法对于陆战场军事单元的识别锚框进行重新聚类,以提升模型对于目标大小适应度,加速模型收敛;采用通道-空间并行注意力机制,增加模型对复杂战场环境下目标特征信息与位置信息关注度;在特征融合网络部分使用BiFPN以提升模型对于特征的融合能力与速度;采用Alpha_IoU损失函数加速模型收敛,解决当真实框与预测框重合时IoU计算退化问题。实验结果表明,在自建军事目标数据集下,改进算法与主流目标识别算法相比,在保证模型空间复杂度的同时,mAP值达到了90.17%。消融实验对比结果表明,改进后网络较原模型精度提升11.57%,具有较好的识别性能,能够为战场情报获取提供有效的技术支撑。

基于改进ResNet18的干香菇等级识别

为解决干香菇等级识别技术复杂及识别精度不高的问题,提出了一种基于残差神经网络ResNet18的干香菇等级识别方法。首先将传统的ResNet18中Stem的7×7卷积层替换为3个3×3卷积层串联,保证在感受野保持不变的情况下进一步减小计算量;其次针对残差块中线性变换和非线性变换不足的问题,引入融合非对称卷积和h-swish激活函数,增加了模型的复杂性,使其能够进行更深层次的特征学习;最后在ResNet18骨干网络中引入高效通道注意力机制,加强模型提取特征的能力。实验结果表明,改进后的ResNet18网络模型准确度达97.04%,相比ResNet18网络模型方法提升了4.81%,且性能优于VGG16、MobileNetV2、DenseNet121、ResNet34等网络模型方法,可提高干香菇等级的识别精度,单幅图像的检测时间为5.91 ms,对干香菇智能分拣过程中的等级识别具有借鉴意义。

基于改进YOLO v7轻量化模型的自然果园环境下苹果识别方法

针对自然果园环境下苹果果实识别中,传统的目标检测算法往往很难在检测模型的检测精度、速度和轻量化方面实现平衡,提出了一种基于改进YOLO v7的轻量化苹果检测模型。首先,引入部分卷积(Partial convolution, PConv)替换多分支堆叠模块中的部分常规卷积进行轻量化改进,以降低模型的参数量和计算量;其次,添加轻量化的高效通道注意力(Efficient channel attention, ECA)模块以提高网络的特征提取能力,改善复杂环境下遮挡目标的错检漏检问题;在模型训练过程中采用基于麻雀搜索算法(Sparrow search algorithm, SSA)的学习率优化策略来进一步提高模型的检测精度。试验结果显示:相比于YOLO v7原始模型,改进后模型的精确率、召回率和平均精度分别提高4.15、0.38、1.39个百分点,其参数量和计算量分别降低22.93%和27.41%,在GPU和CPU上检测单幅图像的平均用时分别减少0.003 s和0.014 s。结果表明,改进后的模型可以实时准确地识别复杂果园环境中的苹果,模型参数量和计算量较小,适合部署于苹果采摘机器人的嵌入式设备上,为实现苹果的无人化智能采摘奠定了基础。