A Deep Residual PLS for Data-Driven Quality Prediction Modeling in Industrial Process

Partial least squares(PLS)model is the most typical data-driven method for quality-related industrial tasks like soft sensor.However,only linear relations are captured between the input and output data in the PLS.It is difficult to obtain the remaining nonlinear information in the residual subspaces,which may deteriorate the prediction performance in complex industrial processes.To fully utilize data information in PLS residual subspaces,a deep residual PLS(DRPLS)framework is proposed for quality prediction in this paper.Inspired by deep learning,DRPLS is designed by stacking a number of PLSs successively,in which the input residuals of the previous PLS are used as the layer connection.To enhance representation,nonlinear function is applied to the input residuals before using them for stacking highlevel PLS.For each PLS,the output parts are just the output residuals from its previous PLS.Finally,the output prediction is obtained by adding the results of each PLS.The effectiveness of the proposed DRPLS is validated on an industrial hydrocracking process.

Radar Signal Intra-Pulse Modulation Recognition Based on Deep Residual Network

In view of low recognition rate of complex radar intra-pulse modulation signal type by traditional methods under low signal-to-noise ratio(SNR),the paper proposes an automatic recog-nition method of complex radar intra-pulse modulation signal type based on deep residual network.The basic principle of the recognition method is to obtain the transformation relationship between the time and frequency of complex radar intra-pulse modulation signal through short-time Fourier transform(STFT),and then design an appropriate deep residual network to extract the features of the time-frequency map and complete a variety of complex intra-pulse modulation signal type recognition.In addition,in order to improve the generalization ability of the proposed method,label smoothing and L2 regularization are introduced.The simulation results show that the proposed method has a recognition accuracy of more than 95%for complex radar intra-pulse modulation sig-nal types under low SNR(2 dB).

Pre-Impact and Impact Fall Detection Based on a Multimodal Sensor Using a Deep Residual Network

Falls are the contributing factor to both fatal and nonfatal injuries in the elderly.Therefore,pre-impact fall detection,which identifies a fall before the body collides with the floor,would be essential.Recently,researchers have turned their attention from post-impact fall detection to pre-impact fall detection.Pre-impact fall detection solutions typically use either a threshold-based or machine learning-based approach,although the threshold value would be difficult to accu-rately determine in threshold-based methods.Moreover,while additional features could sometimes assist in categorizing falls and non-falls more precisely,the esti-mated determination of the significant features would be too time-intensive,thus using a significant portion of the algorithm’s operating time.In this work,we developed a deep residual network with aggregation transformation called FDSNeXt for a pre-impact fall detection approach employing wearable inertial sensors.The proposed network was introduced to address the limitations of fea-ture extraction,threshold definition,and algorithm complexity.After training on a large-scale motion dataset,the KFall dataset,and straightforward evaluation with standard metrics,the proposed approach identified pre-impact and impact falls with high accuracy of 91.87 and 92.52%,respectively.In addition,we have inves-tigated fall detection’s performances of three state-of-the-art deep learning models such as a convolutional neural network(CNN),a long short-term memory neural network(LSTM),and a hybrid model(CNN-LSTM).The experimental results showed that the proposed FDSNeXt model outperformed these deep learning models(CNN,LSTM,and CNN-LSTM)with significant improvements.

End-to-End Auto-Encoder System for Deep Residual Shrinkage Network for AWGN Channels

With the rapid development of deep learning methods, the data-driven approach has shown powerful advantages over the model-driven one. In this paper, we propose an end-to-end autoencoder communication system based on Deep Residual Shrinkage Networks (DRSNs), where neural networks (DNNs) are used to implement the coding, decoding, modulation and demodulation functions of the communication system. Our proposed autoencoder communication system can better reduce the signal noise by adding an “attention mechanism” and “soft thresholding” modules and has better performance at various signal-to-noise ratios (SNR). Also, we have shown through comparative experiments that the system can operate at moderate block lengths and support different throughputs. It has been shown to work efficiently in the AWGN channel. Simulation results show that our model has a higher Bit-Error-Rate (BER) gain and greatly improved decoding performance compared to conventional modulation and classical autoencoder systems at various signal-to-noise ratios.

Cross-Modal Hashing Retrieval Based on Deep Residual Network

In the era of big data rich inWe Media,the single mode retrieval system has been unable to meet people’s demand for information retrieval.This paper proposes a new solution to the problem of feature extraction and unified mapping of different modes:A Cross-Modal Hashing retrieval algorithm based on Deep Residual Network(CMHR-DRN).The model construction is divided into two stages:The first stage is the feature extraction of different modal data,including the use of Deep Residual Network(DRN)to extract the image features,using the method of combining TF-IDF with the full connection network to extract the text features,and the obtained image and text features used as the input of the second stage.In the second stage,the image and text features are mapped into Hash functions by supervised learning,and the image and text features are mapped to the common binary Hamming space.In the process of mapping,the distance measurement of the original distance measurement and the common feature space are kept unchanged as far as possible to improve the accuracy of Cross-Modal Retrieval.In training the model,adaptive moment estimation(Adam)is used to calculate the adaptive learning rate of each parameter,and the stochastic gradient descent(SGD)is calculated to obtain the minimum loss function.The whole training process is completed on Caffe deep learning framework.Experiments show that the proposed algorithm CMHR-DRN based on Deep Residual Network has better retrieval performance and stronger advantages than other Cross-Modal algorithms CMFH,CMDN and CMSSH.

Specific Emitter Identification for IoT Devices Based on Deep Residual Shrinkage Networks

Specific emitter identification can distin-guish individual transmitters by analyzing received signals and extracting inherent features of hard-ware circuits.Feature extraction is a key part of traditional machine learning-based methods,but manual extrac-tion is generally limited by prior professional knowl-edge.At the same time,it has been noted that the per-formance of most specific emitter identification meth-ods degrades in the low signal-to-noise ratio(SNR)environments.The deep residual shrinkage network(DRSN)is proposed for specific emitter identification,particularly in the low SNRs.The soft threshold can preserve more key features for the improvement of performance,and an identity shortcut can speed up the training process.We collect signals via the receiver to create a dataset in the actual environments.The DRSN is trained to automatically extract features and imple-ment the classification of transmitters.Experimental results show that DRSN obtains the best accuracy un-der different SNRs and has less running time,which demonstrates the effectiveness of DRSN in identify-ing specific emitters.

Underdetermined DOA estimation via multiple time-delay covariance matrices and deep residual network

Higher-order statistics based approaches and signal sparseness based approaches have emerged in recent decades to resolve the underdetermined direction-of-arrival(DOA)estimation problem.These model-based methods face great challenges in practical applications due to high computational complexity and dependence on ideal assumptions.This paper presents an effective DOA estimation approach based on a deep residual network(DRN)for the underdetermined case.We first extract an input feature from a new matrix calculated by stacking several covariance matrices corresponding to different time delays.We then provide the input feature to the trained DRN to construct the super resolution spectrum.The DRN learns the mapping relationship between the input feature and the spatial spectrum by training.The proposed approach is superior to existing model-based estimation methods in terms of calculation efficiency,independence of source sparseness and adaptive capacity to non-ideal conditions(e.g.,low signal to noise ratio,short bit sequence).Simulations demonstrate the validity and strong performance of the proposed algorithm on both overdetermined and underdetermined cases.

Sand-bed defect recognition for 3D sand printing based on deep residual network

The 3D sand printing(3DSP),by binder jetting technology for rapid casting,has a pivotal role in promoting the development of the traditional casting industry as a result of producing high-quality and economical sand molds.This work presents an approach for monitoring and analyzing powder sand-bed images to serve as a real-time control system in a 3DSP machine.A deep residual network(ResNet)is used to classify the defects occurring during the powder spreading stage of the process.Firstly,a pre-trained network was applied as the initial parameter;then it was fine-tuned on the labelled defective sample dataset to accomplish the task,which defines the sand-bed defects induced in the 3DSP processing.Furthermore,the recognition and positioning of sand-bed defects were readily achieved by dividing the sand-bed images into blocks.Experiments show that the fine-tuned network has a 98.7%classification accuracy on the validation dataset of sand-bed defects and 95.4%recognition accuracy for the sand-bed images.

Action Recognition Based on CSI Signal Using Improved Deep Residual Network Model

In this paper,we propose an improved deep residual network model to recognize human actions.Action data is composed of channel state information signals,which are continuous fine-grained signals.We replaced the traditional identity connection with the shrinking thresholdmodule.Themodule automatically adjusts the threshold of the action data signal,and filters out signals that are not related to the principal components.We use the attention mechanism to improve the memory of the network model to the action signal,so as to better recognize the action.To verify the validity of the experiment more accurately,we collected action data in two different environments.The experimental results show that the improved network model is much better than the traditional network in recognition.The accuracy of recognition in complex places can reach 92.85%,among which the recognition rate of raising hands is up to 96%.We combine the improved residual deep network model with channel state information action data,and prove the effectiveness of our model for classification through experimental data.

Deep Pyramidal Residual Network for Indoor-Outdoor Activity Recognition Based on Wearable Sensor

Recognition of human activity is one of the most exciting aspects of time-series classification,with substantial practical and theoretical impli-cations.Recent evidence indicates that activity recognition from wearable sensors is an effective technique for tracking elderly adults and children in indoor and outdoor environments.Consequently,researchers have demon-strated considerable passion for developing cutting-edge deep learning sys-tems capable of exploiting unprocessed sensor data from wearable devices and generating practical decision assistance in many contexts.This study provides a deep learning-based approach for recognizing indoor and outdoor movement utilizing an enhanced deep pyramidal residual model called Sen-PyramidNet and motion information from wearable sensors(accelerometer and gyroscope).The suggested technique develops a residual unit based on a deep pyramidal residual network and introduces the concept of a pyramidal residual unit to increase detection capability.The proposed deep learning-based model was assessed using the publicly available 19Nonsens dataset,which gathered motion signals from various indoor and outdoor activities,including practicing various body parts.The experimental findings demon-strate that the proposed approach can efficiently reuse characteristics and has achieved an identification accuracy of 96.37%for indoor and 97.25%for outdoor activity.Moreover,comparison experiments demonstrate that the SenPyramidNet surpasses other cutting-edge deep learning models in terms of accuracy and F1-score.Furthermore,this study explores the influence of several wearable sensors on indoor and outdoor action recognition ability.

A physics-constrained deep residual network for solving the sine-Gordon equation

Despite some empirical successes for solving nonlinear evolution equations using deep learning,there are several unresolved issues.First,it could not uncover the dynamical behaviors of some equations where highly nonlinear source terms are included very well.Second,the gradient exploding and vanishing problems often occur for the traditional feedforward neural networks.In this paper,we propose a new architecture that combines the deep residual neural network with some underlying physical laws.Using the sine-Gordon equation as an example,we show that the numerical result is in good agreement with the exact soliton solution.In addition,a lot of numerical experiments show that the model is robust under small perturbations to a certain extent.

Data-driven fault diagnosis of control valve with missing data based on modeling and deep residual shrinkage network

A control valve is one of the most widely used machines in hydraulic systems.However,it often works in harsh environments and failure occurs from time to time.An intelligent and robust control valve fault diagnosis is therefore important for operation of the system.In this study,a fault diagnosis based on the mathematical model(MM)imputation and the modified deep residual shrinkage network(MDRSN)is proposed to solve the problem that data-driven models for control valves are susceptible to changing operating conditions and missing data.The multiple fault time-series samples of the control valve at different openings are collected for fault diagnosis to verify the effectiveness of the proposed method.The effects of the proposed method in missing data imputation and fault diagnosis are analyzed.Compared with random and k-nearest neighbor(KNN)imputation,the accuracies of MM-based imputation are improved by 17.87%and 21.18%,in the circumstances of a20.00%data missing rate at valve opening from 10%to 28%.Furthermore,the results show that the proposed MDRSN can maintain high fault diagnosis accuracy with missing data.

A novel face recognition in uncontrolled environment based on block 2D-CS-LBP features and deep residual network

Purpose-In order to solve the problem that the performance of the existing local feature descriptors in uncontrolled environment is greatly affected by illumination,background,occlusion and other factors,we propose a novel face recognition algorithm in uncontrolled environment which combines the block central symmetry local binary pattern(CS-LBP)and deep residual network(DRN)model.Design/methodology/approach-The algorithm first extracts the block CSP-LBP features of the face image,then incorporates the extracted features into the DRN model,and gives the face recognition results by using a well-trained DRN model.The features obtained by the proposed algorithm have the characteristics of both local texture features and deep features that robust to illumination.Findings-Compared with the direct usage of the original image,the usage of local texture features of the image as the input of DRN model significantly improves the computation efficiency.Experimental results on the face datasets of FERET,YALE-B and CMU-PIE have shown that the recognition rate of the proposed algorithm is significantly higher than that of other compared algorithms.Originality/value-The proposed algorithm fundamentally solves the problem of face identity recognition in uncontrolled environment,and it is particularly robust to the change of illumination,which proves its superiority.

Underwater sea cucumber identification via deep residual networks

Sea cucumber culture and fishing are primarily dependent on manual work.For fast and accurate automatic identification of sea cucumbers,deep residual networks with different configures were conducted in this experiment to identify underwater sea cucumber.Sea cucumber images were captured by a C-Watch remotely operated underwater vehicle(ROV)in a sea cucumber fishery at Haiyang Qiandao Lake in Shandong Province,China and sliced to positive samples and negative samples.Two training algorithms,namely,the stochastic gradient descent algorithm(SGD)and Adam,activation functions ReLU and leaky ReLU,as well as learning rates of 0.001,0.005,0.01,0.05,and 0.1 were combined to form different models,which were trained with epochs 200 times and mini-batch of 100.The results showed that the accuracy of each model was higher than 82%,and the highest accuracy reached 89.53%under the SGD algorithm with ReLU and a learning rate of 0.05 or 0.1,which showed better generalization ability than that of other models.The performance of the proposed method indicates a great potential for automatic sea cucumber identification.

Self-adaptive scale pedestrian detection algorithm based on deep residual network

Purpose–The conventional pedestrian detection algorithms lack in scale sensitivity.The purpose of this paper is to propose a novel algorithm of self-adaptive scale pedestrian detection,based on deep residual network(DRN),to address such lacks.Design/methodology/approach–First,the“Edge boxes”algorithm is introduced to extract region of interestsfrompedestrian images.Then,the extracted boundingboxesare incorporatedto differentDRNs,one is a large-scale DRN and the other one is the small-scale DRN.The height of the bounding boxes is used to classify the results of pedestrians and to regress the bounding boxes to the entity of the pedestrian.At last,a weighted self-adaptive scale function,which combines the large-scale results and small-scale results,is designed for the final pedestrian detection.Findings–Tovalidatetheeffectivenessandfeasibilityoftheproposedalgorithm,somecomparisonexperiments have been done on the common pedestrian detection data sets:Caltech,INRIA,ETH and KITTI.Experimental resultsshowthattheproposedalgorithmisadaptedforthevariousscalesofthepedestrians.Fortheharddetected small-scale pedestrians,the proposed algorithm has improved the accuracy and robustness of detections.Originality/value–By applying different models to deal with different scales of pedestrians,the proposed algorithm with the weighted calculation function has improved the accuracy and robustness for different scales of pedestrians.

Feedback deer hunting optimization algorithm for intrusion detection in cloud based deep residual network

Cloud computing is the distributed computing paradigm continually exposed to different attacks and threats of various origins.The data stored in the cloud framework is easier for external and internal intruders,as access to the cloud framework is done through internet services.Various intrusion detection(ID)methods are developed to detect network intruders in the cloud,but these methods are not primarily effective in generating accurate detection results.Hence,an effective intrusion detection system(IDS)is designed to solve the security issues that unfavorably influence the sustainable development of the cloud and enhance the protection of the cloud from malicious attacks.The IDS is modeled using the proposed Feedback Deer Hunting Optimization(FDHO)-based Deep Residual network to detect network intrusions.However,the proposed FDHO algorithm is designed by integrating Feedback Artificial Tree(FAT)with Deer Hunting Optimization(DHOA),respectively.Moreover,the detection of malicious attacks is carried out using a Deep Residual network that significantly increases the training speed,reduces the computational complexity,and generates effective detection results.The performance of the proposed method is comparatively analyzed with the existing techniques,such as Stacked Contractive Auto-Encoder and Support Vector Machine(SCAE+SVM),Artificial Neural Network with ant bee colony optimization algorithm+fuzzy clustering(ANN+ABC+fuzzy clustering),Improved dynamic immune algorithm(IDIA),and Normalized K-means(NK)clustering algorithm with RNN named,(NK-RNN),FAT-based Deep Residual network,and DHOA-based Deep Residual network using the BoT-IoT dataset and KDD cup-99 dataset.The proposed method achieved outstanding performance by considering the metrics,like specificity,accuracy,and sensitivity,with the values of 0.9526,0.9498,and 0.9214 using the BoT-IoT dataset.

Facial expression recognition based on bidirectional gated recurrent units within deep residual network

Purpose-recent years,facial expression recognition has been widely used in human machine interaction,clinical medicine and safe driving.However,there is a limitation that conventional recurrent neural networks can only learn the time-series characteristics of expressions based on one-way propagation information.Design/methodology/approach-To solve such limitation,this paper proposes a novel model based on bidirectional gated recurrent unit networks(Bi-GRUs)with two-way propagations,and the theory of identity mapping residuals is adopted to effectively prevent the problem of gradient disappearance caused by the depth of the introduced network.Since the Inception-V3 network model for spatial feature extraction has too many parameters,it is prone to overfitting during training.This paper proposes a novel facial expression recognition model to add two reduction modules to reduce parameters,so as to obtain an Inception-W network with better generalization.Findings-Finally,the proposed model is pretrained to determine the best settings and selections.Then,the pretrained model is experimented on two facial expression data sets of CKþand Oulu-CASIA,and the recognition performance and efficiency are compared with the existing methods.The highest recognition rate is 99.6%,which shows that the method has good recognition accuracy in a certain range.Originality/value-By using the proposed model for the applications of facial expression,the high recognition accuracy and robust recognition results with lower time consumption will help to build more sophisticated applications in real world.

基于GRU-DRSN的双通道人体活动识别

人体活动识别(human activity recognizition, HAR)在医疗、军工、智能家居等领域有很大的应用空间。传统机器学习方法特征提取难度较大且精度不高。针对上述问题并结合传感器时序特性,提出了一种融合CBAM(convolutional block attention module)注意力机制的GRU-DRSN双通道并行模型,有效避免了传统串行模型因网络深度加深引起梯度爆炸和消失问题。同时并行结构使得两条支路具有相同的优先级,使用深度残差收缩网络(deep residual shrinkage network, DRSN)提取数据的深层空间特征,同时使用门控循环结构(gated recurrent unit, GRU)学习活动样本在时间序列上的特征,同时进行提取样本不同维度的特征,并通过CBAM模块进行特征的权重分配,最后通过Softmax层进行识别,实现了端对端的人体活动识别。使用公开数据集(wireless sensor data mining, WISDM)进行验证,模型平均精度达到了97.6%,与传统机器学习模型和前人所提神经网络模型相比,有更好的识别效果。

A Comprehensive Evaluation of State-of-the-Art Deep Learning Models for Road Surface Type Classification

In recent years,as intelligent transportation systems(ITS)such as autonomous driving and advanced driver-assistance systems have become more popular,there has been a rise in the need for different sources of traffic situation data.The classification of the road surface type,also known as the RST,is among the most essential of these situational data and can be utilized across the entirety of the ITS domain.Recently,the benefits of deep learning(DL)approaches for sensor-based RST classification have been demonstrated by automatic feature extraction without manual methods.The ability to extract important features is vital in making RST classification more accurate.This work investigates the most recent advances in DL algorithms for sensor-based RST classification and explores appropriate feature extraction models.We used different convolutional neural networks to understand the functional architecture better;we constructed an enhanced DL model called SE-ResNet,which uses residual connections and squeeze-and-excitation mod-ules to improve the classification performance.Comparative experiments with a publicly available benchmark dataset,the passive vehicular sensors dataset,have shown that SE-ResNet outperforms other state-of-the-art models.The proposed model achieved the highest accuracy of 98.41%and the highest F1-score of 98.19%when classifying surfaces into segments of dirt,cobblestone,or asphalt roads.Moreover,the proposed model significantly outperforms DL networks(CNN,LSTM,and CNN-LSTM).The proposed RE-ResNet achieved the classification accuracies of asphalt roads at 98.98,cobblestone roads at 97.02,and dirt roads at 99.56%,respectively.

A New Encrypted Traffic Identification Model Based on VAE-LSTM-DRN

Encrypted traffic identification pertains to the precise acquisition and categorization of data from traffic datasets containing imbalanced and obscured content.The extraction of encrypted traffic attributes and their subsequent identification presents a formidable challenge.The existing models have predominantly relied on direct extraction of encrypted traffic data from imbalanced datasets,with the dataset’s imbalance significantly affecting the model’s performance.In the present study,a new model,referred to as UD-VLD(Unbalanced Dataset-VAE-LSTM-DRN),was proposed to address above problem.The proposed model is an encrypted traffic identification model for handling unbalanced datasets.The encoder of the variational autoencoder(VAE)is combined with the decoder and Long-short term Memory(LSTM)in UD-VLD model to realize the data enhancement processing of the original unbalanced datasets.The enhanced data is processed by transforming the deep residual network(DRN)to address neural network gradient-related issues.Subsequently,the data is classified and recognized.The UD-VLD model integrates the related techniques of deep learning into the encrypted traffic recognition technique,thereby solving the processing problem for unbalanced datasets.The UD-VLD model was tested using the publicly available Tor dataset and VPN dataset.The UD-VLD model is evaluated against other comparative models in terms of accuracy,loss rate,precision,recall,F1-score,total time,and ROC curve.The results reveal that the UD-VLD model exhibits better performance in both binary and multi classification,being higher than other encrypted traffic recognition models that exist for unbalanced datasets.Furthermore,the evaluation performance indicates that the UD-VLD model effectivelymitigates the impact of unbalanced data on traffic classification.and can serve as a novel solution for encrypted traffic identification.