Abnormal Traffic Detection for Internet of Things Based on an Improved Residual Network

Along with the progression of Internet of Things(IoT)technology,network terminals are becoming continuously more intelligent.IoT has been widely applied in various scenarios,including urban infrastructure,transportation,industry,personal life,and other socio-economic fields.The introduction of deep learning has brought new security challenges,like an increment in abnormal traffic,which threatens network security.Insufficient feature extraction leads to less accurate classification results.In abnormal traffic detection,the data of network traffic is high-dimensional and complex.This data not only increases the computational burden of model training but also makes information extraction more difficult.To address these issues,this paper proposes an MD-MRD-ResNeXt model for abnormal network traffic detection.To fully utilize the multi-scale information in network traffic,a Multi-scale Dilated feature extraction(MD)block is introduced.This module can effectively understand and process information at various scales and uses dilated convolution technology to significantly broaden the model’s receptive field.The proposed Max-feature-map Residual with Dual-channel pooling(MRD)block integrates the maximum feature map with the residual block.This module ensures the model focuses on key information,thereby optimizing computational efficiency and reducing unnecessary information redundancy.Experimental results show that compared to the latest methods,the proposed abnormal traffic detection model improves accuracy by about 2%.

Workout Action Recognition in Video Streams Using an Attention Driven Residual DC-GRU Network

Regular exercise is a crucial aspect of daily life, as it enables individuals to stay physically active, lowers thelikelihood of developing illnesses, and enhances life expectancy. The recognition of workout actions in videostreams holds significant importance in computer vision research, as it aims to enhance exercise adherence, enableinstant recognition, advance fitness tracking technologies, and optimize fitness routines. However, existing actiondatasets often lack diversity and specificity for workout actions, hindering the development of accurate recognitionmodels. To address this gap, the Workout Action Video dataset (WAVd) has been introduced as a significantcontribution. WAVd comprises a diverse collection of labeled workout action videos, meticulously curated toencompass various exercises performed by numerous individuals in different settings. This research proposes aninnovative framework based on the Attention driven Residual Deep Convolutional-Gated Recurrent Unit (ResDCGRU)network for workout action recognition in video streams. Unlike image-based action recognition, videoscontain spatio-temporal information, making the task more complex and challenging. While substantial progresshas been made in this area, challenges persist in detecting subtle and complex actions, handling occlusions,and managing the computational demands of deep learning approaches. The proposed ResDC-GRU Attentionmodel demonstrated exceptional classification performance with 95.81% accuracy in classifying workout actionvideos and also outperformed various state-of-the-art models. The method also yielded 81.6%, 97.2%, 95.6%, and93.2% accuracy on established benchmark datasets, namely HMDB51, Youtube Actions, UCF50, and UCF101,respectively, showcasing its superiority and robustness in action recognition. The findings suggest practicalimplications in real-world scenarios where precise video action recognition is paramount, addressing the persistingchallenges in the field. TheWAVd dataset serves as a catalyst for the development ofmore robust and effective fitnesstracking systems and ultimately promotes healthier lifestyles through improved exercise monitoring and analysis.

Attention-Based Residual Dense Shrinkage Network for ECG Denoising

Electrocardiogram(ECG)signal is one of the noninvasive physiological measurement techniques commonly usedin cardiac diagnosis.However,in real scenarios,the ECGsignal is susceptible to various noise erosion,which affectsthe subsequent pathological analysis.Therefore,the effective removal of the noise from ECG signals has becomea top priority in cardiac diagnostic research.Aiming at the problem of incomplete signal shape retention andlow signal-to-noise ratio(SNR)after denoising,a novel ECG denoising network,named attention-based residualdense shrinkage network(ARDSN),is proposed in this paper.Firstly,the shallow ECG characteristics are extractedby a shallow feature extraction network(SFEN).Then,the residual dense shrinkage attention block(RDSAB)isused for adaptive noise suppression.Finally,feature fusion representation(FFR)is performed on the hierarchicalfeatures extracted by a series of RDSABs to reconstruct the de-noised ECG signal.Experiments on the MIT-BIHarrhythmia database and MIT-BIH noise stress test database indicate that the proposed scheme can effectively resistthe interference of different sources of noise on the ECG signal.

Radar Signal Intra-Pulse Modulation Recognition Based on Deep Residual Network

In view of low recognition rate of complex radar intra-pulse modulation signal type by traditional methods under low signal-to-noise ratio(SNR),the paper proposes an automatic recog-nition method of complex radar intra-pulse modulation signal type based on deep residual network.The basic principle of the recognition method is to obtain the transformation relationship between the time and frequency of complex radar intra-pulse modulation signal through short-time Fourier transform(STFT),and then design an appropriate deep residual network to extract the features of the time-frequency map and complete a variety of complex intra-pulse modulation signal type recognition.In addition,in order to improve the generalization ability of the proposed method,label smoothing and L2 regularization are introduced.The simulation results show that the proposed method has a recognition accuracy of more than 95%for complex radar intra-pulse modulation sig-nal types under low SNR(2 dB).

Automatic modulation recognition of radiation source signals based on two-dimensional data matrix and improved residual neural network

Automatic modulation recognition(AMR)of radiation source signals is a research focus in the field of cognitive radio.However,the AMR of radiation source signals at low SNRs still faces a great challenge.Therefore,the AMR method of radiation source signals based on two-dimensional data matrix and improved residual neural network is proposed in this paper.First,the time series of the radiation source signals are reconstructed into two-dimensional data matrix,which greatly simplifies the signal preprocessing process.Second,the depthwise convolution and large-size convolutional kernels based residual neural network(DLRNet)is proposed to improve the feature extraction capability of the AMR model.Finally,the model performs feature extraction and classification on the two-dimensional data matrix to obtain the recognition vector that represents the signal modulation type.Theoretical analysis and simulation results show that the AMR method based on two-dimensional data matrix and improved residual network can significantly improve the accuracy of the AMR method.The recognition accuracy of the proposed method maintains a high level greater than 90% even at -14 dB SNR.

A Normalizing Flow-Based Bidirectional Mapping Residual Network for Unsupervised Defect Detection

Unsupervised methods based on density representation have shown their abilities in anomaly detection,but detection performance still needs to be improved.Specifically,approaches using normalizing flows can accurately evaluate sample distributions,mapping normal features to the normal distribution and anomalous features outside it.Consequently,this paper proposes a Normalizing Flow-based Bidirectional Mapping Residual Network(NF-BMR).It utilizes pre-trained Convolutional Neural Networks(CNN)and normalizing flows to construct discriminative source and target domain feature spaces.Additionally,to better learn feature information in both domain spaces,we propose the Bidirectional Mapping Residual Network(BMR),which maps sample features to these two spaces for anomaly detection.The two detection spaces effectively complement each other’s deficiencies and provide a comprehensive feature evaluation from two perspectives,which leads to the improvement of detection performance.Comparative experimental results on the MVTec AD and DAGM datasets against the Bidirectional Pre-trained Feature Mapping Network(B-PFM)and other state-of-the-art methods demonstrate that the proposed approach achieves superior performance.On the MVTec AD dataset,NF-BMR achieves an average AUROC of 98.7%for all 15 categories.Especially,it achieves 100%optimal detection performance in five categories.On the DAGM dataset,the average AUROC across ten categories is 98.7%,which is very close to supervised methods.

Quality assessment of the Topo-Iberia CGPS stations and data quality's effects on postfit ionosphere-free phase residuals

This paper analyzed GPS data from the Topo-Iberia network spanning almost 12 years(2008-2020).The data quality information for all 26 Topo-Iberia stations is provided for the first time,complementing the Spanish Geological Survey’s storage work.Data analyses based on quality indicators obtained using TEQC have been carried out.The guidelines and data quality information from the IGS stations have been considered as the quality references,with the stations ALJI,EPCU,and TIOU standing out as the worst stations,while on the contrary,FUEN,PALM,PILA,and TRIA meet the quality requirements to become an IGS station.The relationship between the GPS data quality and their GAMIT-and Gipsy X-derived postfit ionosphere-free phase residuals has also been investigated,and the results reveal an inversely proportional relationship.It has been found that the stations showing an increase in elevation of the horizon line,also show an increase in cycle slips and multipath,are among the poorest quality stations,and among those with the highest postfit RMS of phase residuals.Moreover,the evolution of the vegetation around the antenna should be considered as it could cause a progressive loss of quality,which is not complying with the IGS standards.The quality assessment shows that the Topo-Iberia stations are appropriate for geodetic purposes,but permanent monitoring would be necessary to avoid the least possible loss of data and quality.In addition,a method to characterize the GNSS data quality is proposed.

Review of Artificial Intelligence for Oil and Gas Exploration: Convolutional Neural Network Approaches and the U-Net 3D Model

Deep learning, especially through convolutional neural networks (CNN) such as the U-Net 3D model, has revolutionized fault identification from seismic data, representing a significant leap over traditional methods. Our review traces the evolution of CNN, emphasizing the adaptation and capabilities of the U-Net 3D model in automating seismic fault delineation with unprecedented accuracy. We find: 1) The transition from basic neural networks to sophisticated CNN has enabled remarkable advancements in image recognition, which are directly applicable to analyzing seismic data. The U-Net 3D model, with its innovative architecture, exemplifies this progress by providing a method for detailed and accurate fault detection with reduced manual interpretation bias. 2) The U-Net 3D model has demonstrated its superiority over traditional fault identification methods in several key areas: it has enhanced interpretation accuracy, increased operational efficiency, and reduced the subjectivity of manual methods. 3) Despite these achievements, challenges such as the need for effective data preprocessing, acquisition of high-quality annotated datasets, and achieving model generalization across different geological conditions remain. Future research should therefore focus on developing more complex network architectures and innovative training strategies to refine fault identification performance further. Our findings confirm the transformative potential of deep learning, particularly CNN like the U-Net 3D model, in geosciences, advocating for its broader integration to revolutionize geological exploration and seismic analysis.

融合Residual Network-50残差块与卷积注意力模块的地震断层自动识别

传统的断层识别是由地质解释人员以人工标记的方式进行检测,不仅耗时长、效率低,且识别结果存在一定的人为误差。为解决以上问题,提高断层识别的精度,提出了一种基于深度学习的断层识别方法,利用注意力机制聚焦目标特征的能力,在U-Net网络的解码层引入了卷积注意力模块(Convolutional Block Attention Module,CBAM),在编码层引入了ResNet-50残差块,建立基于卷积神经网络(Convolutional Neural Networks,CNN)的断层识别方法(Res-CBAM-UNet)。将合成地震数据与相应的断层标签进行数据增强操作,新生成的训练数据集作为输入对网络模型进行训练,以提高模型的泛化能力。随后将该模型与CBAM-UNet、ResNet34-UNet和ResNet50-UNet网络进行对比分析,利用实际工区地震数据进行测试。结果表明,设计的Res-CBAM-UNet网络对断层具有较好的识别效果,且识别出的断层连续性好,计算效率高。

基于双通道Residual-LSTM的SINS/GNSS组合导航算法

针对全球导航卫星系统信号中断情况下SINS/GNSS组合导航系统无法持续进行误差校正的问题,提出一种基于双通道Residual-LSTM的SINS/GNSS组合导航算法。首先,考虑到SINS经度、纬度误差传播特性不同所导致的模型输入、输出信息之间的非线性相关性差异化,构建具有不同权重系数的双通道长短期记忆神经网络模型结构,并引入遗忘信息共享机制自适应地利用历史导航数据对经度、纬度信息进行拟合预测。其次,针对深层神经网络存在的模型退化和梯度消失问题,在多层双通道LSTM网络之间建立残差高速通道形成Residual-LSTM模型结构,以增加不同网络层次之间的信息传播路径。最后,通过实船数据验证本文所提算法的有效性。实验结果表明,与基于常规智能方法的SINS/GNSS组合导航算法相比,所提组合导航算法在GNSS信号中断期间经度误差降低了51.97%,纬度误差降低了31.45%。

Defending Adversarial Examples by a Clipped Residual U-Net Model

Deep learning-based systems have succeeded in many computer vision tasks.However,it is found that the latest study indicates that these systems are in danger in the presence of adversarial attacks.These attacks can quickly spoil deep learning models,e.g.,different convolutional neural networks(CNNs),used in various computer vision tasks from image classification to object detection.The adversarial examples are carefully designed by injecting a slight perturbation into the clean images.The proposed CRU-Net defense model is inspired by state-of-the-art defense mechanisms such as MagNet defense,Generative Adversarial Net-work Defense,Deep Regret Analytic Generative Adversarial Networks Defense,Deep Denoising Sparse Autoencoder Defense,and Condtional Generattive Adversarial Network Defense.We have experimentally proved that our approach is better than previous defensive techniques.Our proposed CRU-Net model maps the adversarial image examples into clean images by eliminating the adversarial perturbation.The proposed defensive approach is based on residual and U-Net learning.Many experiments are done on the datasets MNIST and CIFAR10 to prove that our proposed CRU-Net defense model prevents adversarial example attacks in WhiteBox and BlackBox settings and improves the robustness of the deep learning algorithms especially in the computer visionfield.We have also reported similarity(SSIM and PSNR)between the original and restored clean image examples by the proposed CRU-Net defense model.

Deep Learned Singular Residual Network for Super Resolution Reconstruction

Single image super resolution(SISR)techniques produce images of high resolution(HR)as output from input images of low resolution(LR).Motivated by the effectiveness of deep learning methods,we provide a framework based on deep learning to achieve super resolution(SR)by utilizing deep singular-residual neural network(DSRNN)in training phase.Residuals are obtained from the difference between HR and LR images to generate LR-residual example pairs.Singular value decomposition(SVD)is applied to each LR-residual image pair to decompose into subbands of low and high frequency components.Later,DSRNN is trained on these subbands through input and output channels by optimizing the weights and biases of the network.With fewer layers in DSRNN,the influence of exploding gradients is reduced.This speeds up the learning process and also improves accuracy by using skip connections.The trained DSRNN parameters yield residuals to recover the HR subbands in the testing phase.Experimental analysis shows that the proposed method results in superior performance to existingmethods in terms of subjective quality.Extensive testing results on popular benchmark datasets such as set5,set14,and urban100 for a scaling factor of 4 show the effectiveness of the proposed method across different qualitative evaluation metrics.

Prediction of Load-Displacement Curve of Flexible Pipe Carcass Under Radial Compression Based on Residual Neural Network

The carcass layer of flexible pipe comprises a large-angle spiral structure with a complex interlocked stainless steel cross-section profile, which is mainly used to resist radial load. With the complex structure of the carcass layer, an equivalent simplified model is used to study the mechanical properties of the carcass layer. However, the current equivalent carcass model only considers the elastic deformation, and this simplification leads to huge errors in the calculation results. In this study, radial compression experiments were carried out to make the carcasses to undergo plastic deformation. Subsequently, a residual neural network based on the experimental data was established to predict the load-displacement curves of carcasses with different inner diameter in plastic states under radial compression.The established neural network model’s high precision was verified by experimental data, and the influence of the number of input variables on the accuracy of the neural network was discussed. The conclusion shows that the residual neural network model established based on the experimental data of the small-diameter carcass layer can predict the load-displacement curve of the large-diameter carcass layer in the plastic stage. With the decrease of input data, the prediction accuracy of residual network model in plasticity stage will decrease.

A COVID-19 Detection Model Based on Convolutional Neural Network and Residual Learning

Amodel that can obtain rapid and accurate detection of coronavirus disease 2019(COVID-19)plays a significant role in treating and preventing the spread of disease transmission.However,designing such amodel that can balance the detection accuracy andweight parameters ofmemorywell to deploy a mobile device is challenging.Taking this point into account,this paper fuses the convolutional neural network and residual learning operations to build a multi-class classification model,which improves COVID-19 pneumonia detection performance and keeps a trade-off between the weight parameters and accuracy.The convolutional neural network can extract the COVID-19 feature information by repeated convolutional operations.The residual learning operations alleviate the gradient problems caused by stacking convolutional layers and enhance the ability of feature extraction.The ability further enables the proposed model to acquire effective feature information at a lowcost,which canmake ourmodel keep smallweight parameters.Extensive validation and comparison with other models of COVID-19 pneumonia detection on the well-known COVIDx dataset show that(1)the sensitivity of COVID-19 pneumonia detection is improved from 88.2%(non-COVID-19)and 77.5%(COVID-19)to 95.3%(non-COVID-19)and 96.5%(COVID-19),respectively.The positive predictive value is also respectively increased from72.8%(non-COVID-19)and 89.0%(COVID-19)to 88.8%(non-COVID-19)and 95.1%(COVID-19).(2)Compared with the weight parameters of the COVIDNet-small network,the value of the proposed model is 13 M,which is slightly higher than that(11.37 M)of the COVIDNet-small network.But,the corresponding accuracy is improved from 85.2%to 93.0%.The above results illustrate the proposed model can gain an efficient balance between accuracy and weight parameters.

Deep Capsule Residual Networks for Better Diagnosis Rate in Medical Noisy Images

With the advent of Machine and Deep Learning algorithms,medical image diagnosis has a new perception of diagnosis and clinical treatment.Regret-tably,medical images are more susceptible to capturing noises despite the peak in intelligent imaging techniques.However,the presence of noise images degrades both the diagnosis and clinical treatment processes.The existing intelligent meth-ods suffer from the deficiency in handling the diverse range of noise in the ver-satile medical images.This paper proposes a novel deep learning network which learns from the substantial extent of noise in medical data samples to alle-viate this challenge.The proposed deep learning architecture exploits the advan-tages of the capsule network,which is used to extract correlation features and combine them with redefined residual features.Additionally,thefinal stage of dense learning is replaced with powerful extreme learning machines to achieve a better diagnosis rate,even for noisy and complex images.Extensive experimen-tation has been conducted using different medical images.Various performances such as Peak-Signal-To-Noise Ratio(PSNR)and Structural-Similarity-Index-Metrics(SSIM)are compared with the existing deep learning architectures.Addi-tionally,a comprehensive analysis of individual algorithms is analyzed.The experimental results prove that the proposed model has outperformed the other existing algorithms by a substantial margin and proved its supremacy over the other learning models.

Deep Capsule Residual Networks for Better Diagnosis Rate in Medical Noisy Images

With the advent of Machine and Deep Learning algorithms,medical image diagnosis has a new perception of diagnosis and clinical treatment.Regret-tably,medical images are more susceptible to capturing noises despite the peak in intelligent imaging techniques.However,the presence of noise images degrades both the diagnosis and clinical treatment processes.The existing intelligent meth-ods suffer from the deficiency in handling the diverse range of noise in the ver-satile medical images.This paper proposes a novel deep learning network which learns from the substantial extent of noise in medical data samples to alle-viate this challenge.The proposed deep learning architecture exploits the advan-tages of the capsule network,which is used to extract correlation features and combine them with redefined residual features.Additionally,the final stage of dense learning is replaced with powerful extreme learning machines to achieve a better diagnosis rate,even for noisy and complex images.Extensive experimen-tation has been conducted using different medical images.Various performances such as Peak-Signal-To-Noise Ratio(PSNR)and Structural-Similarity-Index-Metrics(SSIM)are compared with the existing deep learning architectures.Addi-tionally,a comprehensive analysis of individual algorithms is analyzed.The experimental results prove that the proposed model has outperformed the other existing algorithms by a substantial margin and proved its supremacy over the other learning models.

Rockburst Intensity Grade Prediction Model Based on Batch Gradient Descent and Multi-Scale Residual Deep Neural Network

Rockburst is a phenomenon in which free surfaces are formed during excavation,which subsequently causes the sudden release of energy in the construction of mines and tunnels.Light rockburst only peels off rock slices without ejection,while severe rockburst causes casualties and property loss.The frequency and degree of rockburst damage increases with the excavation depth.Moreover,rockburst is the leading engineering geological hazard in the excavation process,and thus the prediction of its intensity grade is of great significance to the development of geotechnical engineering.Therefore,the prediction of rockburst intensity grade is one problem that needs to be solved urgently.By comprehensively considering the occurrence mechanism of rockburst,this paper selects the stress index(σθ/σc),brittleness index(σ_(c)/σ_(t)),and rock elastic energy index(Wet)as the rockburst evaluation indexes through the Spearman coefficient method.This overcomes the low accuracy problem of a single evaluation index prediction method.Following this,the BGD-MSR-DNN rockburst intensity grade prediction model based on batch gradient descent and a multi-scale residual deep neural network is proposed.The batch gradient descent(BGD)module is used to replace the gradient descent algorithm,which effectively improves the efficiency of the network and reduces the model training time.Moreover,the multi-scale residual(MSR)module solves the problem of network degradation when there are too many hidden layers of the deep neural network(DNN),thus improving the model prediction accuracy.The experimental results reveal the BGDMSR-DNN model accuracy to reach 97.1%,outperforming other comparable models.Finally,actual projects such as Qinling Tunnel and Daxiangling Tunnel,reached an accuracy of 100%.The model can be applied in mines and tunnel engineering to realize the accurate and rapid prediction of rockburst intensity grade.

Pre-Impact and Impact Fall Detection Based on a Multimodal Sensor Using a Deep Residual Network

Falls are the contributing factor to both fatal and nonfatal injuries in the elderly.Therefore,pre-impact fall detection,which identifies a fall before the body collides with the floor,would be essential.Recently,researchers have turned their attention from post-impact fall detection to pre-impact fall detection.Pre-impact fall detection solutions typically use either a threshold-based or machine learning-based approach,although the threshold value would be difficult to accu-rately determine in threshold-based methods.Moreover,while additional features could sometimes assist in categorizing falls and non-falls more precisely,the esti-mated determination of the significant features would be too time-intensive,thus using a significant portion of the algorithm’s operating time.In this work,we developed a deep residual network with aggregation transformation called FDSNeXt for a pre-impact fall detection approach employing wearable inertial sensors.The proposed network was introduced to address the limitations of fea-ture extraction,threshold definition,and algorithm complexity.After training on a large-scale motion dataset,the KFall dataset,and straightforward evaluation with standard metrics,the proposed approach identified pre-impact and impact falls with high accuracy of 91.87 and 92.52%,respectively.In addition,we have inves-tigated fall detection’s performances of three state-of-the-art deep learning models such as a convolutional neural network(CNN),a long short-term memory neural network(LSTM),and a hybrid model(CNN-LSTM).The experimental results showed that the proposed FDSNeXt model outperformed these deep learning models(CNN,LSTM,and CNN-LSTM)with significant improvements.

Deep Pyramidal Residual Network for Indoor-Outdoor Activity Recognition Based on Wearable Sensor

Recognition of human activity is one of the most exciting aspects of time-series classification,with substantial practical and theoretical impli-cations.Recent evidence indicates that activity recognition from wearable sensors is an effective technique for tracking elderly adults and children in indoor and outdoor environments.Consequently,researchers have demon-strated considerable passion for developing cutting-edge deep learning sys-tems capable of exploiting unprocessed sensor data from wearable devices and generating practical decision assistance in many contexts.This study provides a deep learning-based approach for recognizing indoor and outdoor movement utilizing an enhanced deep pyramidal residual model called Sen-PyramidNet and motion information from wearable sensors(accelerometer and gyroscope).The suggested technique develops a residual unit based on a deep pyramidal residual network and introduces the concept of a pyramidal residual unit to increase detection capability.The proposed deep learning-based model was assessed using the publicly available 19Nonsens dataset,which gathered motion signals from various indoor and outdoor activities,including practicing various body parts.The experimental findings demon-strate that the proposed approach can efficiently reuse characteristics and has achieved an identification accuracy of 96.37%for indoor and 97.25%for outdoor activity.Moreover,comparison experiments demonstrate that the SenPyramidNet surpasses other cutting-edge deep learning models in terms of accuracy and F1-score.Furthermore,this study explores the influence of several wearable sensors on indoor and outdoor action recognition ability.

End-to-End Auto-Encoder System for Deep Residual Shrinkage Network for AWGN Channels

With the rapid development of deep learning methods, the data-driven approach has shown powerful advantages over the model-driven one. In this paper, we propose an end-to-end autoencoder communication system based on Deep Residual Shrinkage Networks (DRSNs), where neural networks (DNNs) are used to implement the coding, decoding, modulation and demodulation functions of the communication system. Our proposed autoencoder communication system can better reduce the signal noise by adding an “attention mechanism” and “soft thresholding” modules and has better performance at various signal-to-noise ratios (SNR). Also, we have shown through comparative experiments that the system can operate at moderate block lengths and support different throughputs. It has been shown to work efficiently in the AWGN channel. Simulation results show that our model has a higher Bit-Error-Rate (BER) gain and greatly improved decoding performance compared to conventional modulation and classical autoencoder systems at various signal-to-noise ratios.