End-to-End Auto-Encoder System for Deep Residual Shrinkage Network for AWGN Channels

With the rapid development of deep learning methods, the data-driven approach has shown powerful advantages over the model-driven one. In this paper, we propose an end-to-end autoencoder communication system based on Deep Residual Shrinkage Networks (DRSNs), where neural networks (DNNs) are used to implement the coding, decoding, modulation and demodulation functions of the communication system. Our proposed autoencoder communication system can better reduce the signal noise by adding an “attention mechanism” and “soft thresholding” modules and has better performance at various signal-to-noise ratios (SNR). Also, we have shown through comparative experiments that the system can operate at moderate block lengths and support different throughputs. It has been shown to work efficiently in the AWGN channel. Simulation results show that our model has a higher Bit-Error-Rate (BER) gain and greatly improved decoding performance compared to conventional modulation and classical autoencoder systems at various signal-to-noise ratios.

Radar Signal Intra-Pulse Modulation Recognition Based on Deep Residual Network

In view of low recognition rate of complex radar intra-pulse modulation signal type by traditional methods under low signal-to-noise ratio(SNR),the paper proposes an automatic recog-nition method of complex radar intra-pulse modulation signal type based on deep residual network.The basic principle of the recognition method is to obtain the transformation relationship between the time and frequency of complex radar intra-pulse modulation signal through short-time Fourier transform(STFT),and then design an appropriate deep residual network to extract the features of the time-frequency map and complete a variety of complex intra-pulse modulation signal type recognition.In addition,in order to improve the generalization ability of the proposed method,label smoothing and L2 regularization are introduced.The simulation results show that the proposed method has a recognition accuracy of more than 95%for complex radar intra-pulse modulation sig-nal types under low SNR(2 dB).

A Deep Residual PLS for Data-Driven Quality Prediction Modeling in Industrial Process

Partial least squares(PLS)model is the most typical data-driven method for quality-related industrial tasks like soft sensor.However,only linear relations are captured between the input and output data in the PLS.It is difficult to obtain the remaining nonlinear information in the residual subspaces,which may deteriorate the prediction performance in complex industrial processes.To fully utilize data information in PLS residual subspaces,a deep residual PLS(DRPLS)framework is proposed for quality prediction in this paper.Inspired by deep learning,DRPLS is designed by stacking a number of PLSs successively,in which the input residuals of the previous PLS are used as the layer connection.To enhance representation,nonlinear function is applied to the input residuals before using them for stacking highlevel PLS.For each PLS,the output parts are just the output residuals from its previous PLS.Finally,the output prediction is obtained by adding the results of each PLS.The effectiveness of the proposed DRPLS is validated on an industrial hydrocracking process.

Abnormal Traffic Detection for Internet of Things Based on an Improved Residual Network

Along with the progression of Internet of Things(IoT)technology,network terminals are becoming continuously more intelligent.IoT has been widely applied in various scenarios,including urban infrastructure,transportation,industry,personal life,and other socio-economic fields.The introduction of deep learning has brought new security challenges,like an increment in abnormal traffic,which threatens network security.Insufficient feature extraction leads to less accurate classification results.In abnormal traffic detection,the data of network traffic is high-dimensional and complex.This data not only increases the computational burden of model training but also makes information extraction more difficult.To address these issues,this paper proposes an MD-MRD-ResNeXt model for abnormal network traffic detection.To fully utilize the multi-scale information in network traffic,a Multi-scale Dilated feature extraction(MD)block is introduced.This module can effectively understand and process information at various scales and uses dilated convolution technology to significantly broaden the model’s receptive field.The proposed Max-feature-map Residual with Dual-channel pooling(MRD)block integrates the maximum feature map with the residual block.This module ensures the model focuses on key information,thereby optimizing computational efficiency and reducing unnecessary information redundancy.Experimental results show that compared to the latest methods,the proposed abnormal traffic detection model improves accuracy by about 2%.

Residual Attention Deep SVDD for COVID-19 Diagnosis Using CT Scans

COVID-19 is the common name of the disease caused by the novel coronavirus(2019-nCoV)that appeared in Wuhan,China in 2019.Discovering the infected people is the most important factor in the fight against the disease.The gold-standard test to diagnose COVID-19 is polymerase chain reaction(PCR),but it takes 5–6 h and,in the early stages of infection,may produce false-negative results.Examining Computed Tomography(CT)images to diagnose patients infected with COVID-19 has become an urgent necessity.In this study,we propose a residual attention deep support vector data description SVDD(RADSVDD)approach to diagnose COVID-19.It is a novel approach combining residual attention with deep support vector data description(DSVDD)to classify the CT images.To the best of our knowledge,we are the first to combine residual attention with DSVDD in general,and specifically in the diagnosis of COVID-19.Combining attention with DSVDD naively may cause model collapse.Attention in the proposed RADSVDD guides the network during training and enables quick learning,residual connectivity prevents vanishing gradients.Our approach consists of three models,each model is devoted to recognizing one certain disease and classifying other diseases as anomalies.These models learn in an end-to-end fashion.The proposed approach attained high performance in classifying CT images into intact,COVID-19,and non-COVID-19 pneumonia.To evaluate the proposed approach,we created a dataset from published datasets and had it assessed by an experienced radiologist.The proposed approach achieved high performance,with the normal model attained sensitivity(0.96–0.98),specificity(0.97–0.99),F1-score(0.97–0.98),and area under the receiver operator curve(AUC)0.99;the COVID-19 model attained sensitivity(0.97–0.98),specificity(0.97–0.99),F1-score(0.97–0.99),and AUC 0.99;and the non-COVID pneumoniamodel attained sensitivity(0.97–1),specificity(0.98–0.99),F1-score(0.97–0.99),and AUC 0.99.

Deep Learned Singular Residual Network for Super Resolution Reconstruction

Single image super resolution(SISR)techniques produce images of high resolution(HR)as output from input images of low resolution(LR).Motivated by the effectiveness of deep learning methods,we provide a framework based on deep learning to achieve super resolution(SR)by utilizing deep singular-residual neural network(DSRNN)in training phase.Residuals are obtained from the difference between HR and LR images to generate LR-residual example pairs.Singular value decomposition(SVD)is applied to each LR-residual image pair to decompose into subbands of low and high frequency components.Later,DSRNN is trained on these subbands through input and output channels by optimizing the weights and biases of the network.With fewer layers in DSRNN,the influence of exploding gradients is reduced.This speeds up the learning process and also improves accuracy by using skip connections.The trained DSRNN parameters yield residuals to recover the HR subbands in the testing phase.Experimental analysis shows that the proposed method results in superior performance to existingmethods in terms of subjective quality.Extensive testing results on popular benchmark datasets such as set5,set14,and urban100 for a scaling factor of 4 show the effectiveness of the proposed method across different qualitative evaluation metrics.

基于双通道Residual-LSTM的SINS/GNSS组合导航算法

针对全球导航卫星系统信号中断情况下SINS/GNSS组合导航系统无法持续进行误差校正的问题,提出一种基于双通道Residual-LSTM的SINS/GNSS组合导航算法。首先,考虑到SINS经度、纬度误差传播特性不同所导致的模型输入、输出信息之间的非线性相关性差异化,构建具有不同权重系数的双通道长短期记忆神经网络模型结构,并引入遗忘信息共享机制自适应地利用历史导航数据对经度、纬度信息进行拟合预测。其次,针对深层神经网络存在的模型退化和梯度消失问题,在多层双通道LSTM网络之间建立残差高速通道形成Residual-LSTM模型结构,以增加不同网络层次之间的信息传播路径。最后,通过实船数据验证本文所提算法的有效性。实验结果表明,与基于常规智能方法的SINS/GNSS组合导航算法相比,所提组合导航算法在GNSS信号中断期间经度误差降低了51.97%,纬度误差降低了31.45%。

Pre-Impact and Impact Fall Detection Based on a Multimodal Sensor Using a Deep Residual Network

Falls are the contributing factor to both fatal and nonfatal injuries in the elderly.Therefore,pre-impact fall detection,which identifies a fall before the body collides with the floor,would be essential.Recently,researchers have turned their attention from post-impact fall detection to pre-impact fall detection.Pre-impact fall detection solutions typically use either a threshold-based or machine learning-based approach,although the threshold value would be difficult to accu-rately determine in threshold-based methods.Moreover,while additional features could sometimes assist in categorizing falls and non-falls more precisely,the esti-mated determination of the significant features would be too time-intensive,thus using a significant portion of the algorithm’s operating time.In this work,we developed a deep residual network with aggregation transformation called FDSNeXt for a pre-impact fall detection approach employing wearable inertial sensors.The proposed network was introduced to address the limitations of fea-ture extraction,threshold definition,and algorithm complexity.After training on a large-scale motion dataset,the KFall dataset,and straightforward evaluation with standard metrics,the proposed approach identified pre-impact and impact falls with high accuracy of 91.87 and 92.52%,respectively.In addition,we have inves-tigated fall detection’s performances of three state-of-the-art deep learning models such as a convolutional neural network(CNN),a long short-term memory neural network(LSTM),and a hybrid model(CNN-LSTM).The experimental results showed that the proposed FDSNeXt model outperformed these deep learning models(CNN,LSTM,and CNN-LSTM)with significant improvements.

Deep Pyramidal Residual Network for Indoor-Outdoor Activity Recognition Based on Wearable Sensor

Recognition of human activity is one of the most exciting aspects of time-series classification,with substantial practical and theoretical impli-cations.Recent evidence indicates that activity recognition from wearable sensors is an effective technique for tracking elderly adults and children in indoor and outdoor environments.Consequently,researchers have demon-strated considerable passion for developing cutting-edge deep learning sys-tems capable of exploiting unprocessed sensor data from wearable devices and generating practical decision assistance in many contexts.This study provides a deep learning-based approach for recognizing indoor and outdoor movement utilizing an enhanced deep pyramidal residual model called Sen-PyramidNet and motion information from wearable sensors(accelerometer and gyroscope).The suggested technique develops a residual unit based on a deep pyramidal residual network and introduces the concept of a pyramidal residual unit to increase detection capability.The proposed deep learning-based model was assessed using the publicly available 19Nonsens dataset,which gathered motion signals from various indoor and outdoor activities,including practicing various body parts.The experimental findings demon-strate that the proposed approach can efficiently reuse characteristics and has achieved an identification accuracy of 96.37%for indoor and 97.25%for outdoor activity.Moreover,comparison experiments demonstrate that the SenPyramidNet surpasses other cutting-edge deep learning models in terms of accuracy and F1-score.Furthermore,this study explores the influence of several wearable sensors on indoor and outdoor action recognition ability.

Rockburst Intensity Grade Prediction Model Based on Batch Gradient Descent and Multi-Scale Residual Deep Neural Network

Rockburst is a phenomenon in which free surfaces are formed during excavation,which subsequently causes the sudden release of energy in the construction of mines and tunnels.Light rockburst only peels off rock slices without ejection,while severe rockburst causes casualties and property loss.The frequency and degree of rockburst damage increases with the excavation depth.Moreover,rockburst is the leading engineering geological hazard in the excavation process,and thus the prediction of its intensity grade is of great significance to the development of geotechnical engineering.Therefore,the prediction of rockburst intensity grade is one problem that needs to be solved urgently.By comprehensively considering the occurrence mechanism of rockburst,this paper selects the stress index(σθ/σc),brittleness index(σ_(c)/σ_(t)),and rock elastic energy index(Wet)as the rockburst evaluation indexes through the Spearman coefficient method.This overcomes the low accuracy problem of a single evaluation index prediction method.Following this,the BGD-MSR-DNN rockburst intensity grade prediction model based on batch gradient descent and a multi-scale residual deep neural network is proposed.The batch gradient descent(BGD)module is used to replace the gradient descent algorithm,which effectively improves the efficiency of the network and reduces the model training time.Moreover,the multi-scale residual(MSR)module solves the problem of network degradation when there are too many hidden layers of the deep neural network(DNN),thus improving the model prediction accuracy.The experimental results reveal the BGDMSR-DNN model accuracy to reach 97.1%,outperforming other comparable models.Finally,actual projects such as Qinling Tunnel and Daxiangling Tunnel,reached an accuracy of 100%.The model can be applied in mines and tunnel engineering to realize the accurate and rapid prediction of rockburst intensity grade.

Deep Capsule Residual Networks for Better Diagnosis Rate in Medical Noisy Images

With the advent of Machine and Deep Learning algorithms,medical image diagnosis has a new perception of diagnosis and clinical treatment.Regret-tably,medical images are more susceptible to capturing noises despite the peak in intelligent imaging techniques.However,the presence of noise images degrades both the diagnosis and clinical treatment processes.The existing intelligent meth-ods suffer from the deficiency in handling the diverse range of noise in the ver-satile medical images.This paper proposes a novel deep learning network which learns from the substantial extent of noise in medical data samples to alle-viate this challenge.The proposed deep learning architecture exploits the advan-tages of the capsule network,which is used to extract correlation features and combine them with redefined residual features.Additionally,thefinal stage of dense learning is replaced with powerful extreme learning machines to achieve a better diagnosis rate,even for noisy and complex images.Extensive experimen-tation has been conducted using different medical images.Various performances such as Peak-Signal-To-Noise Ratio(PSNR)and Structural-Similarity-Index-Metrics(SSIM)are compared with the existing deep learning architectures.Addi-tionally,a comprehensive analysis of individual algorithms is analyzed.The experimental results prove that the proposed model has outperformed the other existing algorithms by a substantial margin and proved its supremacy over the other learning models.

Deep Capsule Residual Networks for Better Diagnosis Rate in Medical Noisy Images

With the advent of Machine and Deep Learning algorithms,medical image diagnosis has a new perception of diagnosis and clinical treatment.Regret-tably,medical images are more susceptible to capturing noises despite the peak in intelligent imaging techniques.However,the presence of noise images degrades both the diagnosis and clinical treatment processes.The existing intelligent meth-ods suffer from the deficiency in handling the diverse range of noise in the ver-satile medical images.This paper proposes a novel deep learning network which learns from the substantial extent of noise in medical data samples to alle-viate this challenge.The proposed deep learning architecture exploits the advan-tages of the capsule network,which is used to extract correlation features and combine them with redefined residual features.Additionally,the final stage of dense learning is replaced with powerful extreme learning machines to achieve a better diagnosis rate,even for noisy and complex images.Extensive experimen-tation has been conducted using different medical images.Various performances such as Peak-Signal-To-Noise Ratio(PSNR)and Structural-Similarity-Index-Metrics(SSIM)are compared with the existing deep learning architectures.Addi-tionally,a comprehensive analysis of individual algorithms is analyzed.The experimental results prove that the proposed model has outperformed the other existing algorithms by a substantial margin and proved its supremacy over the other learning models.

Method of Multi-Mode Sensor Data Fusion with an Adaptive Deep Coupling Convolutional Auto-Encoder

To address the difficulties in fusing multi-mode sensor data for complex industrial machinery, an adaptive deep coupling convolutional auto-encoder (ADCCAE) fusion method was proposed. First, the multi-mode features extracted synchronously by the CCAE were stacked and fed to the multi-channel convolution layers for fusion. Then, the fused data was passed to all connection layers for compression and fed to the Softmax module for classification. Finally, the coupling loss function coefficients and the network parameters were optimized through an adaptive approach using the gray wolf optimization (GWO) algorithm. Experimental comparisons showed that the proposed ADCCAE fusion model was superior to existing models for multi-mode data fusion.

Research on the Application of Residual Apparent Polarization to Surveying Deep Jiaojia-type Gold Mines

The theory and equations of the residual apparent polarization method are proposed and described in this article. Based on studies of existing mines, the residual apparent polarization ηα^sy, calculated from the induced-current middle-gradient apparent polarizations ηα^sy at large and small electrode spaces over the known deep Jiaojia-type gold mines, have been shown to separate the effects of mines from the anomalous polarizations generated from the strongly altered rocks in fracture zones.

Distribution of soil aggregates and organic carbon in deep soil under long-term conservation tillage with residual retention in dryland

To ascertain the effects of long-term conservation tillage and residue retention on soil organic carbon(SOC) content and aggregate distribution in a deep soil(>20-cm depth) in a dryland environment,this paper analyzed the SOC and aggregate distribution in soil, and the aggregate-associated organic carbon(OC) and SOC physical fractions. Conservation tillage(reduced tillage with residue incorporated(RT) and no-tillage with residue mulch(NT)) significantly increased SOC sequestration and soil aggregation in deep soil compared with conventional tillage with residue removal(CT). Compared with CT, RT significantly increased the proportion of small macroaggregates by 23%–81% in the 10–80 cm layer, and the OC content in small macroaggregates by 1%–58% in the 0–80 cm layer. RT significantly increased(by 24%–90%) the OC content in mineral-SOC within small macroaggregates in the 0–60 cm layer, while there was a 23%–80% increase in the 0–40 cm layer with NT. These results indicated that:(1) conservation tillage treatments are beneficial for soil aggregation and SOC sequestration in a deep soil in a dryland environment; and(2)the SOC in mineral-associated OC plays important roles in soil aggregation and SOC sequestration. In conclusion, RT with NT is recommended as an agricultural management tool in dryland soils because of its role in improving soil aggregation and SOC sequestration.

融合Residual Network-50残差块与卷积注意力模块的地震断层自动识别

传统的断层识别是由地质解释人员以人工标记的方式进行检测,不仅耗时长、效率低,且识别结果存在一定的人为误差。为解决以上问题,提高断层识别的精度,提出了一种基于深度学习的断层识别方法,利用注意力机制聚焦目标特征的能力,在U-Net网络的解码层引入了卷积注意力模块(Convolutional Block Attention Module,CBAM),在编码层引入了ResNet-50残差块,建立基于卷积神经网络(Convolutional Neural Networks,CNN)的断层识别方法(Res-CBAM-UNet)。将合成地震数据与相应的断层标签进行数据增强操作,新生成的训练数据集作为输入对网络模型进行训练,以提高模型的泛化能力。随后将该模型与CBAM-UNet、ResNet34-UNet和ResNet50-UNet网络进行对比分析,利用实际工区地震数据进行测试。结果表明,设计的Res-CBAM-UNet网络对断层具有较好的识别效果,且识别出的断层连续性好,计算效率高。

Specific Emitter Identification for IoT Devices Based on Deep Residual Shrinkage Networks

Specific emitter identification can distin-guish individual transmitters by analyzing received signals and extracting inherent features of hard-ware circuits.Feature extraction is a key part of traditional machine learning-based methods,but manual extrac-tion is generally limited by prior professional knowl-edge.At the same time,it has been noted that the per-formance of most specific emitter identification meth-ods degrades in the low signal-to-noise ratio(SNR)environments.The deep residual shrinkage network(DRSN)is proposed for specific emitter identification,particularly in the low SNRs.The soft threshold can preserve more key features for the improvement of performance,and an identity shortcut can speed up the training process.We collect signals via the receiver to create a dataset in the actual environments.The DRSN is trained to automatically extract features and imple-ment the classification of transmitters.Experimental results show that DRSN obtains the best accuracy un-der different SNRs and has less running time,which demonstrates the effectiveness of DRSN in identify-ing specific emitters.

Deep Spectrum Prediction in High Frequency Communication Based on Temporal-Spectral Residual Network

High frequency(HF) communication is widely spread due to some merits like easy deployment and wide communication coverage. Spectrum prediction is a promising technique to facilitate the working frequency selection and enhance the function of automatic link establishment. Most of the existing spectrum prediction algorithms focus on predicting spectrum values in a slot-by-slot manner and therefore are lack of timeliness. Deep learning based spectrum prediction is developed in this paper by simultaneously predicting multi-slot ahead states of multiple spectrum points within a period of time. Specifically, we first employ supervised learning and construct samples depending on longterm and short-term HF spectrum data. Then, advanced residual units are introduced to build multiple residual network modules to respectively capture characteristics in these data with diverse time scales. Further, convolution neural network fuses the outputs of residual network modules above for temporal-spectral prediction, which is combined with residual network modules to construct the deep temporal-spectral residual network. Experiments have demonstrated that the approach proposed in this paper has a significant advantage over the benchmark schemes.

The deep mantle upwelling beneath the northwestern South China Sea:Insights from the time-varying residual subsidence in the Qiongdongnan Basin

Deep hot mantle upwelling is widely revealed around the Qiongdongnan Basin on the northwestern South China Sea margin. However, when and how it influenced the hyper-extended basin is unclear.To resolve these issues, a detailed analysis of the Cenozoic time-varying residual subsidence derived by subtracting the predicted subsidence from the backstripped subsidence was performed along a new seismic reflection line in the western Qiongdongnan Basin. For the first time, a method is proposed to calculate the time-varying strain rates constrained by the faults growth rates, on basis of which, the predicted basement subsidence is obtained with a basin-and lithosphere-scale coupled finite extension model, and the backstripped subsidence is accurately recovered with a modified technique of backstripping to eliminate the effects of later episodes of rifting on earlier sediment thickness. Results show no residual subsidence in 45–28.4 Ma. But after 28.4 Ma, negative residual subsidence occurred, reached and remained ca. -1000 m during 23–11.6 Ma, and reduced dramatically after 11.6 Ma. In the syn-rift period(45–23 Ma), the residual subsidence is ca. -1000 m, however in the post-rift period(23–0 Ma),it is positive of ca. 300 to 1300 m increasing southeastwards. These results suggest that the syn-rift subsidence deficit commenced at 28.4 Ma, while the post-rift excess subsidence occurred after 11.6 Ma.Combined with previous studies, it is inferred that the opposite residual subsidence in the syn-and post-rift periods with similar large wavelengths(>10^(2) km) and km-scale amplitudes are the results of transient dynamic topography induced by deep mantle upwelling beneath the central QDNB, which started to influence the basin at ca. 28.4 Ma, continued into the Middle Miocene, and decayed at ca.11.6 Ma. The initial mantle upwelling with significant dynamic uplift had precipitated considerable continental extension and faulting in the Late Oligocene(28.4–23 Ma). After ca. 11.6 Ma, strong mantle upwelling probably occurred beneath the Leizhou–Hainan area to form vast basaltic lava flow.

Underdetermined DOA estimation via multiple time-delay covariance matrices and deep residual network

Higher-order statistics based approaches and signal sparseness based approaches have emerged in recent decades to resolve the underdetermined direction-of-arrival(DOA)estimation problem.These model-based methods face great challenges in practical applications due to high computational complexity and dependence on ideal assumptions.This paper presents an effective DOA estimation approach based on a deep residual network(DRN)for the underdetermined case.We first extract an input feature from a new matrix calculated by stacking several covariance matrices corresponding to different time delays.We then provide the input feature to the trained DRN to construct the super resolution spectrum.The DRN learns the mapping relationship between the input feature and the spatial spectrum by training.The proposed approach is superior to existing model-based estimation methods in terms of calculation efficiency,independence of source sparseness and adaptive capacity to non-ideal conditions(e.g.,low signal to noise ratio,short bit sequence).Simulations demonstrate the validity and strong performance of the proposed algorithm on both overdetermined and underdetermined cases.