Sparse Autoencoder-based Multi-head Deep Neural Networks for Machinery Fault Diagnostics with Detection of Novelties

Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.

Pseudo Zernike Moment and Deep Stacked Sparse Autoencoder for COVID-19 Diagnosis

(Aim)COVID-19 is an ongoing infectious disease.It has caused more than 107.45 m confirmed cases and 2.35 m deaths till 11/Feb/2021.Traditional computer vision methods have achieved promising results on the automatic smart diagnosis.(Method)This study aims to propose a novel deep learning method that can obtain better performance.We use the pseudo-Zernike moment(PZM),derived from Zernike moment,as the extracted features.Two settings are introducing:(i)image plane over unit circle;and(ii)image plane inside the unit circle.Afterward,we use a deep-stacked sparse autoencoder(DSSAE)as the classifier.Besides,multiple-way data augmentation is chosen to overcome overfitting.The multiple-way data augmentation is based on Gaussian noise,salt-and-pepper noise,speckle noise,horizontal and vertical shear,rotation,Gamma correction,random translation and scaling.(Results)10 runs of 10-fold cross validation shows that our PZM-DSSAE method achieves a sensitivity of 92.06%±1.54%,a specificity of 92.56%±1.06%,a precision of 92.53%±1.03%,and an accuracy of 92.31%±1.08%.Its F1 score,MCC,and FMI arrive at 92.29%±1.10%,84.64%±2.15%,and 92.29%±1.10%,respectively.The AUC of our model is 0.9576.(Conclusion)We demonstrate“image plane over unit circle”can get better results than“image plane inside a unit circle.”Besides,this proposed PZM-DSSAE model is better than eight state-of-the-art approaches.

Battle damage assessment based on an improved Kullback-Leibler divergence sparse autoencoder

The nodes number of the hidden layer in a deep learning network is quite difficult to determine with traditional methods. To solve this problem, an improved Kullback-Leibler divergence sparse autoencoder （KL-SAE） is proposed in this paper, which can be applied to battle damage assessment （BDA）. This method can select automatically the hidden layer feature which contributes most to data reconstruction, and abandon the hidden layer feature which contributes least. Therefore, the structure of the network can be modified. In addition, the method can select automatically hidden layer feature without loss of the network prediction accuracy and increase the computation speed. Experiments on University ofCalifomia-Irvine （UCI） data sets and BDA for battle damage data demonstrate that the method outperforms other reference data-driven methods. The following results can be found from this paper. First, the improved KL-SAE regression network can guarantee the prediction accuracy and increase the speed of training networks and prediction. Second, the proposed network can select automatically hidden layer effective feature and modify the structure of the network by optimizing the nodes number of the hidden layer.

Alzheimer’s Disease Stage Classification Using a Deep Transfer Learning and Sparse Auto Encoder Method

Alzheimer’s Disease(AD)is a progressive neurological disease.Early diagnosis of this illness using conventional methods is very challenging.Deep Learning(DL)is one of the finest solutions for improving diagnostic procedures’performance and forecast accuracy.The disease’s widespread distribution and elevated mortality rate demonstrate its significance in the older-onset and younger-onset age groups.In light of research investigations,it is vital to consider age as one of the key criteria when choosing the subjects.The younger subjects are more susceptible to the perishable side than the older onset.The proposed investigation concentrated on the younger onset.The research used deep learning models and neuroimages to diagnose and categorize the disease at its early stages automatically.The proposed work is executed in three steps.The 3D input images must first undergo image pre-processing using Weiner filtering and Contrast Limited Adaptive Histogram Equalization(CLAHE)methods.The Transfer Learning(TL)models extract features,which are subsequently compressed using cascaded Auto Encoders(AE).The final phase entails using a Deep Neural Network(DNN)to classify the phases of AD.The model was trained and tested to classify the five stages of AD.The ensemble ResNet-18 and sparse autoencoder with DNN model achieved an accuracy of 98.54%.The method is compared to state-of-the-art approaches to validate its efficacy and performance.

A Double-Weighted Deterministic Extreme Learning Machine Based on Sparse Denoising Autoencoder and Its Applications

Extreme learning machine (ELM) is a feedforward neural network-based machine learning method that has the benefits of short training times, strong generalization capabilities, and will not fall into local minima. However, due to the traditional ELM shallow architecture, it requires a large number of hidden nodes when dealing with high-dimensional data sets to ensure its classification performance. The other aspect, it is easy to degrade the classification performance in the face of noise interference from noisy data. To improve the above problem, this paper proposes a double pseudo-inverse extreme learning machine (DPELM) based on Sparse Denoising AutoEncoder (SDAE) namely, SDAE-DPELM. The algorithm can directly determine the input weight and output weight of the network by using the pseudo-inverse method. As a result, the algorithm only requires a few hidden layer nodes to produce superior classification results when classifying data. And its combination with SDAE can effectively improve the classification performance and noise resistance. Extensive numerical experiments show that the algorithm has high classification accuracy and good robustness when dealing with high-dimensional noisy data and high-dimensional noiseless data. Furthermore, applying such an algorithm to Miao character recognition substantiates its excellent performance, which further illustrates the practicability of the algorithm.

Anomaly-Based Intrusion DetectionModel Using Deep Learning for IoT Networks

The rapid growth of Internet of Things(IoT)devices has brought numerous benefits to the interconnected world.However,the ubiquitous nature of IoT networks exposes them to various security threats,including anomaly intrusion attacks.In addition,IoT devices generate a high volume of unstructured data.Traditional intrusion detection systems often struggle to cope with the unique characteristics of IoT networks,such as resource constraints and heterogeneous data sources.Given the unpredictable nature of network technologies and diverse intrusion methods,conventional machine-learning approaches seem to lack efficiency.Across numerous research domains,deep learning techniques have demonstrated their capability to precisely detect anomalies.This study designs and enhances a novel anomaly-based intrusion detection system(AIDS)for IoT networks.Firstly,a Sparse Autoencoder(SAE)is applied to reduce the high dimension and get a significant data representation by calculating the reconstructed error.Secondly,the Convolutional Neural Network(CNN)technique is employed to create a binary classification approach.The proposed SAE-CNN approach is validated using the Bot-IoT dataset.The proposed models exceed the performance of the existing deep learning approach in the literature with an accuracy of 99.9%,precision of 99.9%,recall of 100%,F1 of 99.9%,False Positive Rate(FPR)of 0.0003,and True Positive Rate(TPR)of 0.9992.In addition,alternative metrics,such as training and testing durations,indicated that SAE-CNN performs better.

A Time Series Intrusion Detection Method Based on SSAE,TCN and Bi-LSTM

In the fast-evolving landscape of digital networks,the incidence of network intrusions has escalated alarmingly.Simultaneously,the crucial role of time series data in intrusion detection remains largely underappreciated,with most systems failing to capture the time-bound nuances of network traffic.This leads to compromised detection accuracy and overlooked temporal patterns.Addressing this gap,we introduce a novel SSAE-TCN-BiLSTM(STL)model that integrates time series analysis,significantly enhancing detection capabilities.Our approach reduces feature dimensionalitywith a Stacked Sparse Autoencoder(SSAE)and extracts temporally relevant features through a Temporal Convolutional Network(TCN)and Bidirectional Long Short-term Memory Network(Bi-LSTM).By meticulously adjusting time steps,we underscore the significance of temporal data in bolstering detection accuracy.On the UNSW-NB15 dataset,ourmodel achieved an F1-score of 99.49%,Accuracy of 99.43%,Precision of 99.38%,Recall of 99.60%,and an inference time of 4.24 s.For the CICDS2017 dataset,we recorded an F1-score of 99.53%,Accuracy of 99.62%,Precision of 99.27%,Recall of 99.79%,and an inference time of 5.72 s.These findings not only confirm the STL model’s superior performance but also its operational efficiency,underpinning its significance in real-world cybersecurity scenarios where rapid response is paramount.Our contribution represents a significant advance in cybersecurity,proposing a model that excels in accuracy and adaptability to the dynamic nature of network traffic,setting a new benchmark for intrusion detection systems.

Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput.This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems.First,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data searching.Next,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job execution.As a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

Representation learning via an integrated autoencoder for unsupervised domain adaptation

The purpose of unsupervised domain adaptation is to use the knowledge of the source domain whose data distribution is different from that of the target domain for promoting the learning task in the target domain.The key bottleneck in unsupervised domain adaptation is how to obtain higher-level and more abstract feature representations between source and target domains which can bridge the chasm of domain discrepancy.Recently,deep learning methods based on autoencoder have achieved sound performance in representation learning,and many dual or serial autoencoderbased methods take different characteristics of data into consideration for improving the effectiveness of unsupervised domain adaptation.However,most existing methods of autoencoders just serially connect the features generated by different autoencoders,which pose challenges for the discriminative representation learning and fail to find the real cross-domain features.To address this problem,we propose a novel representation learning method based on an integrated autoencoders for unsupervised domain adaptation,called IAUDA.To capture the inter-and inner-domain features of the raw data,two different autoencoders,which are the marginalized autoencoder with maximum mean discrepancy(mAE)and convolutional autoencoder(CAE)respectively,are proposed to learn different feature representations.After higher-level features are obtained by these two different autoencoders,a sparse autoencoder is introduced to compact these inter-and inner-domain representations.In addition,a whitening layer is embedded for features processed before the mAE to reduce redundant features inside a local area.Experimental results demonstrate the effectiveness of our proposed method compared with several state-of-the-art baseline methods.

Method for denoising and reconstructing radar HRRP using modified sparse auto-encoder

A high resolution range profile(HRRP) is a summation vector of the sub-echoes of the target scattering points acquired by a wide-band radar.Generally, HRRPs obtained in a noncooperative complex electromagnetic environment are contaminated by strong noise.Effective pre-processing of the HRRP data can greatly improve the accuracy of target recognition.In this paper, a denoising and reconstruction method for HRRP is proposed based on a Modified Sparse Auto-Encoder, which is a representative non-linear model.To better reconstruct the HRRP, a sparse constraint is added to the proposed model and the sparse coefficient is calculated based on the intrinsic dimension of HRRP.The denoising of the HRRP is performed by adding random noise to the input HRRP data during the training process and fine-tuning the weight matrix through singular-value decomposition.The results of simulations showed that the proposed method can both reconstruct the signal with fidelity and suppress noise effectively, significantly outperforming other methods, especially in low Signal-to-Noise Ratio conditions.

A hybrid deep neural network based prediction of 300 MW coalfired boiler combustion operation condition

In power generation industries,boilers are required to be operated under a range of different conditions to accommodate demands for fuel randomness and energy fluctuation.Reliable prediction of the combustion operation condition is crucial for an in-depth understanding of boiler performance and maintaining high combustion efficiency.However,it is difficult to establish an accurate prediction model based on traditional data-driven methods,which requires prior expert knowledge and a large number of labeled data.To overcome these limitations,a novel prediction method for the combustion operation condition based on flame imaging and a hybrid deep neural network is proposed.The proposed hybrid model is a combination of convolutional sparse autoencoder(CSAE)and least support vector machine(LSSVM),i.e.,CSAE-LSSVM,where the convolutional sparse autoencoder with deep architectures is utilized to extract the essential features of flame image,and then essential features are input into the least support vector machine for operation condition prediction.A comprehensive investigation of optimal hyper-parameter and dropout technique is carried out to improve the performance of the CSAE-LSSVM.The effectiveness of the proposed model is evaluated by 300 MW tangential coal-fired boiler flame images.The prediction accuracy of the proposed hybrid model reaches 98.06%,and its prediction time is 3.06 ms/image.It is observed that the proposed model could present a superior performance in comparison to other existing neural network models.

Down image recognition based on deep convolutional neural network

Since of the scale and the various shapes of down in the image,it is difficult for traditional image recognition method to correctly recognize the type of down image and get the required recognition accuracy,even for the Traditional Convolutional Neural Network(TCNN).To deal with the above problems,a Deep Convolutional Neural Network(DCNN)for down image classification is constructed,and a new weight initialization method is proposed.Firstly,the salient regions of a down image were cut from the image using the visual saliency model.Then,these salient regions of the image were used to train a sparse autoencoder and get a collection of convolutional filters,which accord with the statistical characteristics of dataset.At last,a DCNN with Inception module and its variants was constructed.To improve the recognition accuracy,the depth of the network is deepened.The experiment results indicate that the constructed DCNN increases the recognition accuracy by 2.7% compared to TCNN,when recognizing the down in the images.The convergence rate of the proposed DCNN with the new weight initialization method is improved by 25.5% compared to TCNN.