Underwater sea cucumber identification via deep residual networks

Sea cucumber culture and fishing are primarily dependent on manual work.For fast and accurate automatic identification of sea cucumbers,deep residual networks with different configures were conducted in this experiment to identify underwater sea cucumber.Sea cucumber images were captured by a C-Watch remotely operated underwater vehicle(ROV)in a sea cucumber fishery at Haiyang Qiandao Lake in Shandong Province,China and sliced to positive samples and negative samples.Two training algorithms,namely,the stochastic gradient descent algorithm(SGD)and Adam,activation functions ReLU and leaky ReLU,as well as learning rates of 0.001,0.005,0.01,0.05,and 0.1 were combined to form different models,which were trained with epochs 200 times and mini-batch of 100.The results showed that the accuracy of each model was higher than 82%,and the highest accuracy reached 89.53%under the SGD algorithm with ReLU and a learning rate of 0.05 or 0.1,which showed better generalization ability than that of other models.The performance of the proposed method indicates a great potential for automatic sea cucumber identification.

Specific Emitter Identification for IoT Devices Based on Deep Residual Shrinkage Networks

Specific emitter identification can distin-guish individual transmitters by analyzing received signals and extracting inherent features of hard-ware circuits.Feature extraction is a key part of traditional machine learning-based methods,but manual extrac-tion is generally limited by prior professional knowl-edge.At the same time,it has been noted that the per-formance of most specific emitter identification meth-ods degrades in the low signal-to-noise ratio(SNR)environments.The deep residual shrinkage network(DRSN)is proposed for specific emitter identification,particularly in the low SNRs.The soft threshold can preserve more key features for the improvement of performance,and an identity shortcut can speed up the training process.We collect signals via the receiver to create a dataset in the actual environments.The DRSN is trained to automatically extract features and imple-ment the classification of transmitters.Experimental results show that DRSN obtains the best accuracy un-der different SNRs and has less running time,which demonstrates the effectiveness of DRSN in identify-ing specific emitters.

Radar Signal Intra-Pulse Modulation Recognition Based on Deep Residual Network

In view of low recognition rate of complex radar intra-pulse modulation signal type by traditional methods under low signal-to-noise ratio(SNR),the paper proposes an automatic recog-nition method of complex radar intra-pulse modulation signal type based on deep residual network.The basic principle of the recognition method is to obtain the transformation relationship between the time and frequency of complex radar intra-pulse modulation signal through short-time Fourier transform(STFT),and then design an appropriate deep residual network to extract the features of the time-frequency map and complete a variety of complex intra-pulse modulation signal type recognition.In addition,in order to improve the generalization ability of the proposed method,label smoothing and L2 regularization are introduced.The simulation results show that the proposed method has a recognition accuracy of more than 95%for complex radar intra-pulse modulation sig-nal types under low SNR(2 dB).

Underdetermined DOA estimation via multiple time-delay covariance matrices and deep residual network

Higher-order statistics based approaches and signal sparseness based approaches have emerged in recent decades to resolve the underdetermined direction-of-arrival(DOA)estimation problem.These model-based methods face great challenges in practical applications due to high computational complexity and dependence on ideal assumptions.This paper presents an effective DOA estimation approach based on a deep residual network(DRN)for the underdetermined case.We first extract an input feature from a new matrix calculated by stacking several covariance matrices corresponding to different time delays.We then provide the input feature to the trained DRN to construct the super resolution spectrum.The DRN learns the mapping relationship between the input feature and the spatial spectrum by training.The proposed approach is superior to existing model-based estimation methods in terms of calculation efficiency,independence of source sparseness and adaptive capacity to non-ideal conditions(e.g.,low signal to noise ratio,short bit sequence).Simulations demonstrate the validity and strong performance of the proposed algorithm on both overdetermined and underdetermined cases.

Research on classification diagnosis model of psoriasis based on deep residual

Objective A classification and diagnosis model for psoriasis based on deep residual network is proposed in this paper.Which using deep learning technology to classify and diagnose psoriasis can help reduce the burden of doctors,simplify the diagnosis and treatment process,and improve the quality of diagnosis.Methods Firstly,data enhancement,image resizings,and TFRecord coding are used to preprocess the input of the model,and then a 34-layer deep residual network(ResNet-34)is constructed to extract the characteristics of psoriasis.Finally,we used the Adam algorithm as the optimizer to train ResNet-34,used cross-entropy as the loss function of ResNet-34 in this study to measure the accuracy of the model,and obtained an optimized ResNet-34 model for psoriasis diagnosis.Results The experimental results based on k-fold cross validation show that the proposed model is superior to other diagnostic methods in terms of recall rate,F1-score and ROC curve.Conclusion The ResNet-34 model can achieve accurate diagnosis of psoriasis,and provide technical support for data analysis and intelligent diagnosis and treatment of psoriasis.

Sand-bed defect recognition for 3D sand printing based on deep residual network

The 3D sand printing(3DSP),by binder jetting technology for rapid casting,has a pivotal role in promoting the development of the traditional casting industry as a result of producing high-quality and economical sand molds.This work presents an approach for monitoring and analyzing powder sand-bed images to serve as a real-time control system in a 3DSP machine.A deep residual network(ResNet)is used to classify the defects occurring during the powder spreading stage of the process.Firstly,a pre-trained network was applied as the initial parameter;then it was fine-tuned on the labelled defective sample dataset to accomplish the task,which defines the sand-bed defects induced in the 3DSP processing.Furthermore,the recognition and positioning of sand-bed defects were readily achieved by dividing the sand-bed images into blocks.Experiments show that the fine-tuned network has a 98.7%classification accuracy on the validation dataset of sand-bed defects and 95.4%recognition accuracy for the sand-bed images.

Cross-Modal Hashing Retrieval Based on Deep Residual Network

In the era of big data rich inWe Media,the single mode retrieval system has been unable to meet people’s demand for information retrieval.This paper proposes a new solution to the problem of feature extraction and unified mapping of different modes:A Cross-Modal Hashing retrieval algorithm based on Deep Residual Network(CMHR-DRN).The model construction is divided into two stages:The first stage is the feature extraction of different modal data,including the use of Deep Residual Network(DRN)to extract the image features,using the method of combining TF-IDF with the full connection network to extract the text features,and the obtained image and text features used as the input of the second stage.In the second stage,the image and text features are mapped into Hash functions by supervised learning,and the image and text features are mapped to the common binary Hamming space.In the process of mapping,the distance measurement of the original distance measurement and the common feature space are kept unchanged as far as possible to improve the accuracy of Cross-Modal Retrieval.In training the model,adaptive moment estimation(Adam)is used to calculate the adaptive learning rate of each parameter,and the stochastic gradient descent(SGD)is calculated to obtain the minimum loss function.The whole training process is completed on Caffe deep learning framework.Experiments show that the proposed algorithm CMHR-DRN based on Deep Residual Network has better retrieval performance and stronger advantages than other Cross-Modal algorithms CMFH,CMDN and CMSSH.

Action Recognition Based on CSI Signal Using Improved Deep Residual Network Model

In this paper,we propose an improved deep residual network model to recognize human actions.Action data is composed of channel state information signals,which are continuous fine-grained signals.We replaced the traditional identity connection with the shrinking thresholdmodule.Themodule automatically adjusts the threshold of the action data signal,and filters out signals that are not related to the principal components.We use the attention mechanism to improve the memory of the network model to the action signal,so as to better recognize the action.To verify the validity of the experiment more accurately,we collected action data in two different environments.The experimental results show that the improved network model is much better than the traditional network in recognition.The accuracy of recognition in complex places can reach 92.85%,among which the recognition rate of raising hands is up to 96%.We combine the improved residual deep network model with channel state information action data,and prove the effectiveness of our model for classification through experimental data.

Pre-Impact and Impact Fall Detection Based on a Multimodal Sensor Using a Deep Residual Network

Falls are the contributing factor to both fatal and nonfatal injuries in the elderly.Therefore,pre-impact fall detection,which identifies a fall before the body collides with the floor,would be essential.Recently,researchers have turned their attention from post-impact fall detection to pre-impact fall detection.Pre-impact fall detection solutions typically use either a threshold-based or machine learning-based approach,although the threshold value would be difficult to accu-rately determine in threshold-based methods.Moreover,while additional features could sometimes assist in categorizing falls and non-falls more precisely,the esti-mated determination of the significant features would be too time-intensive,thus using a significant portion of the algorithm’s operating time.In this work,we developed a deep residual network with aggregation transformation called FDSNeXt for a pre-impact fall detection approach employing wearable inertial sensors.The proposed network was introduced to address the limitations of fea-ture extraction,threshold definition,and algorithm complexity.After training on a large-scale motion dataset,the KFall dataset,and straightforward evaluation with standard metrics,the proposed approach identified pre-impact and impact falls with high accuracy of 91.87 and 92.52%,respectively.In addition,we have inves-tigated fall detection’s performances of three state-of-the-art deep learning models such as a convolutional neural network(CNN),a long short-term memory neural network(LSTM),and a hybrid model(CNN-LSTM).The experimental results showed that the proposed FDSNeXt model outperformed these deep learning models(CNN,LSTM,and CNN-LSTM)with significant improvements.

Deep Pyramidal Residual Network for Indoor-Outdoor Activity Recognition Based on Wearable Sensor

Recognition of human activity is one of the most exciting aspects of time-series classification,with substantial practical and theoretical impli-cations.Recent evidence indicates that activity recognition from wearable sensors is an effective technique for tracking elderly adults and children in indoor and outdoor environments.Consequently,researchers have demon-strated considerable passion for developing cutting-edge deep learning sys-tems capable of exploiting unprocessed sensor data from wearable devices and generating practical decision assistance in many contexts.This study provides a deep learning-based approach for recognizing indoor and outdoor movement utilizing an enhanced deep pyramidal residual model called Sen-PyramidNet and motion information from wearable sensors(accelerometer and gyroscope).The suggested technique develops a residual unit based on a deep pyramidal residual network and introduces the concept of a pyramidal residual unit to increase detection capability.The proposed deep learning-based model was assessed using the publicly available 19Nonsens dataset,which gathered motion signals from various indoor and outdoor activities,including practicing various body parts.The experimental findings demon-strate that the proposed approach can efficiently reuse characteristics and has achieved an identification accuracy of 96.37%for indoor and 97.25%for outdoor activity.Moreover,comparison experiments demonstrate that the SenPyramidNet surpasses other cutting-edge deep learning models in terms of accuracy and F1-score.Furthermore,this study explores the influence of several wearable sensors on indoor and outdoor action recognition ability.

End-to-End Auto-Encoder System for Deep Residual Shrinkage Network for AWGN Channels

With the rapid development of deep learning methods, the data-driven approach has shown powerful advantages over the model-driven one. In this paper, we propose an end-to-end autoencoder communication system based on Deep Residual Shrinkage Networks (DRSNs), where neural networks (DNNs) are used to implement the coding, decoding, modulation and demodulation functions of the communication system. Our proposed autoencoder communication system can better reduce the signal noise by adding an “attention mechanism” and “soft thresholding” modules and has better performance at various signal-to-noise ratios (SNR). Also, we have shown through comparative experiments that the system can operate at moderate block lengths and support different throughputs. It has been shown to work efficiently in the AWGN channel. Simulation results show that our model has a higher Bit-Error-Rate (BER) gain and greatly improved decoding performance compared to conventional modulation and classical autoencoder systems at various signal-to-noise ratios.

A Comprehensive Evaluation of State-of-the-Art Deep Learning Models for Road Surface Type Classification

In recent years,as intelligent transportation systems(ITS)such as autonomous driving and advanced driver-assistance systems have become more popular,there has been a rise in the need for different sources of traffic situation data.The classification of the road surface type,also known as the RST,is among the most essential of these situational data and can be utilized across the entirety of the ITS domain.Recently,the benefits of deep learning(DL)approaches for sensor-based RST classification have been demonstrated by automatic feature extraction without manual methods.The ability to extract important features is vital in making RST classification more accurate.This work investigates the most recent advances in DL algorithms for sensor-based RST classification and explores appropriate feature extraction models.We used different convolutional neural networks to understand the functional architecture better;we constructed an enhanced DL model called SE-ResNet,which uses residual connections and squeeze-and-excitation mod-ules to improve the classification performance.Comparative experiments with a publicly available benchmark dataset,the passive vehicular sensors dataset,have shown that SE-ResNet outperforms other state-of-the-art models.The proposed model achieved the highest accuracy of 98.41%and the highest F1-score of 98.19%when classifying surfaces into segments of dirt,cobblestone,or asphalt roads.Moreover,the proposed model significantly outperforms DL networks(CNN,LSTM,and CNN-LSTM).The proposed RE-ResNet achieved the classification accuracies of asphalt roads at 98.98,cobblestone roads at 97.02,and dirt roads at 99.56%,respectively.

Non-identical residual learning for image enhancement via dynamic multi-level perceptual loss

Residual learning based deep generative networks have achieved promising performance in image enhancement.However,due to the large color gap between a low-quality image and its highquality version,the identical mapping in conventional residual learning cannot explore the elaborate detail differences,resulting in color deviations and texture losses in enhanced images.To address this issue,an innovative non-identical residual learning architecture is proposed,which views image enhancement as two complementary branches,namely a holistic color adjustment branch and a finegrained residual generation branch.In the holistic color adjustment,an adjusting map is calculated for each input low-quality image,in order to regulate the low-quality image to the high-quality representation in an overall way.In the fine-grained residual generation branch,a novel attention-aware recursive network is designed to generate residual images.This design can alleviate the overfitting problem by reusing parameters and promoting the network’s adaptability for different input conditions.In addition,a novel dynamic multi-level perceptual loss based on the error feedback ideology is proposed.Consequently,the proposed network can be dynamically optimized by the hybrid perceptual loss provided by a well-trained VGG,so as to improve the perceptual quality of enhanced images in a guided way.Extensive experiments conducted on publicly available datasets demonstrate the state-of-the-art performance of the proposed method.

基于GRU-DRSN的双通道人体活动识别

人体活动识别(human activity recognizition, HAR)在医疗、军工、智能家居等领域有很大的应用空间。传统机器学习方法特征提取难度较大且精度不高。针对上述问题并结合传感器时序特性,提出了一种融合CBAM(convolutional block attention module)注意力机制的GRU-DRSN双通道并行模型,有效避免了传统串行模型因网络深度加深引起梯度爆炸和消失问题。同时并行结构使得两条支路具有相同的优先级,使用深度残差收缩网络(deep residual shrinkage network, DRSN)提取数据的深层空间特征,同时使用门控循环结构(gated recurrent unit, GRU)学习活动样本在时间序列上的特征,同时进行提取样本不同维度的特征,并通过CBAM模块进行特征的权重分配,最后通过Softmax层进行识别,实现了端对端的人体活动识别。使用公开数据集(wireless sensor data mining, WISDM)进行验证,模型平均精度达到了97.6%,与传统机器学习模型和前人所提神经网络模型相比,有更好的识别效果。

A New Encrypted Traffic Identification Model Based on VAE-LSTM-DRN

Encrypted traffic identification pertains to the precise acquisition and categorization of data from traffic datasets containing imbalanced and obscured content.The extraction of encrypted traffic attributes and their subsequent identification presents a formidable challenge.The existing models have predominantly relied on direct extraction of encrypted traffic data from imbalanced datasets,with the dataset’s imbalance significantly affecting the model’s performance.In the present study,a new model,referred to as UD-VLD(Unbalanced Dataset-VAE-LSTM-DRN),was proposed to address above problem.The proposed model is an encrypted traffic identification model for handling unbalanced datasets.The encoder of the variational autoencoder(VAE)is combined with the decoder and Long-short term Memory(LSTM)in UD-VLD model to realize the data enhancement processing of the original unbalanced datasets.The enhanced data is processed by transforming the deep residual network(DRN)to address neural network gradient-related issues.Subsequently,the data is classified and recognized.The UD-VLD model integrates the related techniques of deep learning into the encrypted traffic recognition technique,thereby solving the processing problem for unbalanced datasets.The UD-VLD model was tested using the publicly available Tor dataset and VPN dataset.The UD-VLD model is evaluated against other comparative models in terms of accuracy,loss rate,precision,recall,F1-score,total time,and ROC curve.The results reveal that the UD-VLD model exhibits better performance in both binary and multi classification,being higher than other encrypted traffic recognition models that exist for unbalanced datasets.Furthermore,the evaluation performance indicates that the UD-VLD model effectivelymitigates the impact of unbalanced data on traffic classification.and can serve as a novel solution for encrypted traffic identification.

Improved deep residual shrinkage network for a multi-cylinder heavy-duty engine fault detection with single channel surface vibration

The health monitoring and fault diagnosis of heavy-duty engines are increasingly important for energy storage ecosystem. During operation, vibration characters corresponding to the specific fault need to be extracted from the overall system vibration. Faulty characteristics emanating from one single cylinder are also mixed with those from other cylinders. Besides, the change of working condition brings strong nonlinearities in surface vibration. To solve these problems, an improved deep residual shrinkage network (IDRSN) is developed for detecting diverse engine faults at various degrees using single channel surface vibration signal. Within IDRSN, a wide convolution kernel is utilized in first convolution layer to capture the long-term fault-related impacts and eliminate the short-time random impact. The residual network module is adopted to enhance the focus the relevant components of vibration signals. Mini-batch training strategy is used to improve the model stability. Meanwhile, Gradient-weighted class activation map is adopted to assess the consistency between the learned knowledge and the fault-related information. The IDRSN is implemented to diagnosing a diesel engine under various faults, faulty degrees and operating speeds. Comparisons with existing models are analyzed in terms of hyper-parameters, training samples, noise resistance, and visualization. Results demonstrate the proposed IDRSN's superior performance on fault diagnosis accuracy, stability, anti-noise performance, and anti-interference performance. An average accuracy rate of 98.38 % was achieved by the proposed IDRSN, in comparison to 96.64 % and 93.56 % achieved by the DRSN and the wide-kernel deep convolutional neural network respectively. These results highlight the proposed IDRSN's superiority in diagnosing multiple faults under various working conditions, offering a low-cost, highly effective, and applicable approach for complex fault diagnosis tasks.

A novel face recognition in uncontrolled environment based on block 2D-CS-LBP features and deep residual network

Purpose-In order to solve the problem that the performance of the existing local feature descriptors in uncontrolled environment is greatly affected by illumination,background,occlusion and other factors,we propose a novel face recognition algorithm in uncontrolled environment which combines the block central symmetry local binary pattern(CS-LBP)and deep residual network(DRN)model.Design/methodology/approach-The algorithm first extracts the block CSP-LBP features of the face image,then incorporates the extracted features into the DRN model,and gives the face recognition results by using a well-trained DRN model.The features obtained by the proposed algorithm have the characteristics of both local texture features and deep features that robust to illumination.Findings-Compared with the direct usage of the original image,the usage of local texture features of the image as the input of DRN model significantly improves the computation efficiency.Experimental results on the face datasets of FERET,YALE-B and CMU-PIE have shown that the recognition rate of the proposed algorithm is significantly higher than that of other compared algorithms.Originality/value-The proposed algorithm fundamentally solves the problem of face identity recognition in uncontrolled environment,and it is particularly robust to the change of illumination,which proves its superiority.

Facial expression recognition based on bidirectional gated recurrent units within deep residual network

Purpose-recent years,facial expression recognition has been widely used in human machine interaction,clinical medicine and safe driving.However,there is a limitation that conventional recurrent neural networks can only learn the time-series characteristics of expressions based on one-way propagation information.Design/methodology/approach-To solve such limitation,this paper proposes a novel model based on bidirectional gated recurrent unit networks(Bi-GRUs)with two-way propagations,and the theory of identity mapping residuals is adopted to effectively prevent the problem of gradient disappearance caused by the depth of the introduced network.Since the Inception-V3 network model for spatial feature extraction has too many parameters,it is prone to overfitting during training.This paper proposes a novel facial expression recognition model to add two reduction modules to reduce parameters,so as to obtain an Inception-W network with better generalization.Findings-Finally,the proposed model is pretrained to determine the best settings and selections.Then,the pretrained model is experimented on two facial expression data sets of CKþand Oulu-CASIA,and the recognition performance and efficiency are compared with the existing methods.The highest recognition rate is 99.6%,which shows that the method has good recognition accuracy in a certain range.Originality/value-By using the proposed model for the applications of facial expression,the high recognition accuracy and robust recognition results with lower time consumption will help to build more sophisticated applications in real world.

Disease Recognition of Apple Leaf Using Lightweight Multi-Scale Network with ECANet

To solve the problem of difficulty in identifying apple diseases in the natural environment and the low application rate of deep learning recognition networks,a lightweight ResNet(LW-ResNet)model for apple disease recognition is proposed.Based on the deep residual network(ResNet18),the multi-scale feature extraction layer is constructed by group convolution to realize the compression model and improve the extraction ability of different sizes of lesion features.By improving the identity mapping structure to reduce information loss.By introducing the efficient channel attention module(ECANet)to suppress noise from a complex background.The experimental results show that the average precision,recall and F1-score of the LW-ResNet on the test set are 97.80%,97.92%and 97.85%,respectively.The parameter memory is 2.32 MB,which is 94%less than that of ResNet18.Compared with the classic lightweight networks SqueezeNet and MobileNetV2,LW-ResNet has obvious advantages in recognition performance,speed,parameter memory requirement and time complexity.The proposed model has the advantages of low computational cost,low storage cost,strong real-time performance,high identification accuracy,and strong practicability,which can meet the needs of real-time identification task of apple leaf disease on resource-constrained devices.

A physics-constrained deep residual network for solving the sine-Gordon equation

Despite some empirical successes for solving nonlinear evolution equations using deep learning,there are several unresolved issues.First,it could not uncover the dynamical behaviors of some equations where highly nonlinear source terms are included very well.Second,the gradient exploding and vanishing problems often occur for the traditional feedforward neural networks.In this paper,we propose a new architecture that combines the deep residual neural network with some underlying physical laws.Using the sine-Gordon equation as an example,we show that the numerical result is in good agreement with the exact soliton solution.In addition,a lot of numerical experiments show that the model is robust under small perturbations to a certain extent.