Rock mass quality classification based on deep learning:A feasibility study for stacked autoencoders

Objective and accurate evaluation of rock mass quality classification is the prerequisite for reliable sta-bility assessment.To develop a tool that can deliver quick and accurate evaluation of rock mass quality,a deep learning approach is developed,which uses stacked autoencoders(SAEs)with several autoencoders and a softmax net layer.Ten rock parameters of rock mass rating(RMR)system are calibrated in this model.The model is trained using 75%of the total database for training sample data.The SAEs trained model achieves a nearly 100%prediction accuracy.For comparison,other different models are also trained with the same dataset,using artificial neural network(ANN)and radial basis function(RBF).The results show that the SAEs classify all test samples correctly while the rating accuracies of ANN and RBF are 97.5%and 98.7%,repectively,which are calculated from the confusion matrix.Moreover,this model is further employed to predict the slope risk level of an abandoned quarry.The proposed approach using SAEs,or deep learning in general,is more objective and more accurate and requires less human inter-vention.The findings presented here shall shed light for engineers/researchers interested in analyzing rock mass classification criteria or performing field investigation.

Bi-LSTM-Based Deep Stacked Sequence-to-Sequence Autoencoder for Forecasting Solar Irradiation and Wind Speed

Wind and solar energy are two popular forms of renewable energy used in microgrids and facilitating the transition towards net-zero carbon emissions by 2050.However,they are exceedingly unpredictable since they rely highly on weather and atmospheric conditions.In microgrids,smart energy management systems,such as integrated demand response programs,are permanently established on a step-ahead basis,which means that accu-rate forecasting of wind speed and solar irradiance intervals is becoming increasingly crucial to the optimal operation and planning of microgrids.With this in mind,a novel“bidirectional long short-term memory network”(Bi-LSTM)-based,deep stacked,sequence-to-sequence autoencoder(S2SAE)forecasting model for predicting short-term solar irradiation and wind speed was developed and evaluated in MATLAB.To create a deep stacked S2SAE prediction model,a deep Bi-LSTM-based encoder and decoder are stacked on top of one another to reduce the dimension of the input sequence,extract its features,and then reconstruct it to produce the forecasts.Hyperparameters of the proposed deep stacked S2SAE forecasting model were optimized using the Bayesian optimization algorithm.Moreover,the forecasting performance of the proposed Bi-LSTM-based deep stacked S2SAE model was compared to three other deep,and shallow stacked S2SAEs,i.e.,the LSTM-based deep stacked S2SAE model,gated recurrent unit-based deep stacked S2SAE model,and Bi-LSTM-based shallow stacked S2SAE model.All these models were also optimized and modeled in MATLAB.The results simulated based on actual data confirmed that the proposed model outperformed the alternatives by achieving an accuracy of up to 99.7%,which evidenced the high reliability of the proposed forecasting.

Iterative learning-based many-objective history matching using deep neural network with stacked autoencoder

This paper presents an innovative data-integration that uses an iterative-learning method,a deep neural network(DNN)coupled with a stacked autoencoder(SAE)to solve issues encountered with many-objective history matching.The proposed method consists of a DNN-based inverse model with SAE-encoded static data and iterative updates of supervised-learning data are based on distance-based clustering schemes.DNN functions as an inverse model and results in encoded flattened data,while SAE,as a pre-trained neural network,successfully reduces dimensionality and reliably reconstructs geomodels.The iterative-learning method can improve the training data for DNN by showing the error reduction achieved with each iteration step.The proposed workflow shows the small mean absolute percentage error below 4%for all objective functions,while a typical multi-objective evolutionary algorithm fails to significantly reduce the initial population uncertainty.Iterative learning-based manyobjective history matching estimates the trends in water cuts that are not reliably included in dynamicdata matching.This confirms the proposed workflow constructs more plausible geo-models.The workflow would be a reliable alternative to overcome the less-convergent Pareto-based multi-objective evolutionary algorithm in the presence of geological uncertainty and varying objective functions.

Optimized Stacked Autoencoder for IoT Enabled Financial Crisis Prediction Model

Recently,Financial Technology(FinTech)has received more attention among financial sectors and researchers to derive effective solutions for any financial institution or firm.Financial crisis prediction(FCP)is an essential topic in business sector that finds it useful to identify the financial condition of a financial institution.At the same time,the development of the internet of things(IoT)has altered the mode of human interaction with the physical world.The IoT can be combined with the FCP model to examine the financial data from the users and perform decision making process.This paper presents a novel multi-objective squirrel search optimization algorithm with stacked autoencoder(MOSSA-SAE)model for FCP in IoT environment.The MOSSA-SAE model encompasses different subprocesses namely preprocessing,class imbalance handling,parameter tuning,and classification.Primarily,the MOSSA-SAE model allows the IoT devices such as smartphones,laptops,etc.,to collect the financial details of the users which are then transmitted to the cloud for further analysis.In addition,SMOTE technique is employed to handle class imbalance problems.The goal of MOSSA in SMOTE is to determine the oversampling rate and area of nearest neighbors of SMOTE.Besides,SAE model is utilized as a classification technique to determine the class label of the financial data.At the same time,the MOSSA is applied to appropriately select the‘weights’and‘bias’values of the SAE.An extensive experimental validation process is performed on the benchmark financial dataset and the results are examined under distinct aspects.The experimental values ensured the superior performance of the MOSSA-SAE model on the applied dataset.

Software Defect Prediction Based on Stacked Contractive Autoencoder and Multi-Objective Optimization

Software defect prediction plays an important role in software quality assurance.However,the performance of the prediction model is susceptible to the irrelevant and redundant features.In addition,previous studies mostly regard software defect prediction as a single objective optimization problem,and multi-objective software defect prediction has not been thoroughly investigated.For the above two reasons,we propose the following solutions in this paper:(1)we leverage an advanced deep neural network-Stacked Contractive AutoEncoder(SCAE)to extract the robust deep semantic features from the original defect features,which has stronger discrimination capacity for different classes(defective or non-defective).(2)we propose a novel multi-objective defect prediction model named SMONGE that utilizes the Multi-Objective NSGAII algorithm to optimize the advanced neural network-Extreme learning machine(ELM)based on state-of-the-art Pareto optimal solutions according to the features extracted by SCAE.We mainly consider two objectives.One objective is to maximize the performance of ELM,which refers to the benefit of the SMONGE model.Another objective is to minimize the output weight norm of ELM,which is related to the cost of the SMONGE model.We compare the SCAE with six state-of-the-art feature extraction methods and compare the SMONGE model with multiple baseline models that contain four classic defect predictors and the MONGE model without SCAE across 20 open source software projects.The experimental results verify that the superiority of SCAE and SMONGE on seven evaluation metrics.

Novel Ensemble Modeling Method for Enhancing Subset Diversity Using Clustering Indicator Vector Based on Stacked Autoencoder

A single model cannot satisfy the high-precision prediction requirements given the high nonlinearity between variables.By contrast,ensemble models can effectively solve this problem.Three key factors for improving the accuracy of ensemble models are namely the high accuracy of a submodel,the diversity between subsample sets and the optimal ensemble method.This study presents an improved ensemble modeling method to improve the prediction precision and generalization capability of the model.Our proposed method first uses a bagging algorithm to generate multiple subsample sets.Second,an indicator vector is defined to describe these subsample sets.Third,subsample sets are selected on the basis of the results of agglomerative nesting clustering on indicator vectors to maximize the diversity between subsets.Subsequently,these subsample sets are placed in a stacked autoencoder for training.Finally,XGBoost algorithm,rather than the traditional simple average ensemble method,is imported to ensemble the model during modeling.Three machine learning public datasets and atmospheric column dry point dataset from a practical industrial process show that our proposed method demonstrates high precision and improved prediction ability.

Hybrid Image Compression-Encryption Scheme Based on Multilayer Stacked Autoencoder and Logistic Map

Secure transmission of images over a communication channel, with limited data transfer capacity, possesses compression and encryption schemes. A deep learning based hybrid image compression-encryption scheme is proposed by combining stacked auto-encoder with the logistic map. The proposed structure of stacked autoencoder has seven multiple layers, and back propagation algorithm is intended to extend vector portrayal of information into lower vector space. The randomly generated key is used to set initial conditions and control parameters of logistic map. Subsequently, compressed image is encrypted by substituting and scrambling of pixel sequences using key stream sequences generated from logistic map.The proposed algorithms are experimentally tested over five standard grayscale images. Compression and encryption efficiency of proposed algorithms are evaluated and analyzed based on peak signal to noise ratio(PSNR), mean square error(MSE), structural similarity index metrics(SSIM) and statistical,differential, entropy analysis respectively. Simulation results show that proposed algorithms provide high quality reconstructed images with excellent levels of security during transmission..

Data Cleaning Based on Stacked Denoising Autoencoders and Multi-Sensor Collaborations

Wireless sensor networks are increasingly used in sensitive event monitoring.However,various abnormal data generated by sensors greatly decrease the accuracy of the event detection.Although many methods have been proposed to deal with the abnormal data,they generally detect and/or repair all abnormal data without further differentiate.Actually,besides the abnormal data caused by events,it is well known that sensor nodes prone to generate abnormal data due to factors such as sensor hardware drawbacks and random effects of external sources.Dealing with all abnormal data without differentiate will result in false detection or missed detection of the events.In this paper,we propose a data cleaning approach based on Stacked Denoising Autoencoders(SDAE)and multi-sensor collaborations.We detect all abnormal data by SDAE,then differentiate the abnormal data by multi-sensor collaborations.The abnormal data caused by events are unchanged,while the abnormal data caused by other factors are repaired.Real data based simulations show the efficiency of the proposed approach.

Pseudo Zernike Moment and Deep Stacked Sparse Autoencoder for COVID-19 Diagnosis

(Aim)COVID-19 is an ongoing infectious disease.It has caused more than 107.45 m confirmed cases and 2.35 m deaths till 11/Feb/2021.Traditional computer vision methods have achieved promising results on the automatic smart diagnosis.(Method)This study aims to propose a novel deep learning method that can obtain better performance.We use the pseudo-Zernike moment(PZM),derived from Zernike moment,as the extracted features.Two settings are introducing:(i)image plane over unit circle;and(ii)image plane inside the unit circle.Afterward,we use a deep-stacked sparse autoencoder(DSSAE)as the classifier.Besides,multiple-way data augmentation is chosen to overcome overfitting.The multiple-way data augmentation is based on Gaussian noise,salt-and-pepper noise,speckle noise,horizontal and vertical shear,rotation,Gamma correction,random translation and scaling.(Results)10 runs of 10-fold cross validation shows that our PZM-DSSAE method achieves a sensitivity of 92.06%±1.54%,a specificity of 92.56%±1.06%,a precision of 92.53%±1.03%,and an accuracy of 92.31%±1.08%.Its F1 score,MCC,and FMI arrive at 92.29%±1.10%,84.64%±2.15%,and 92.29%±1.10%,respectively.The AUC of our model is 0.9576.(Conclusion)We demonstrate“image plane over unit circle”can get better results than“image plane inside a unit circle.”Besides,this proposed PZM-DSSAE model is better than eight state-of-the-art approaches.

Denoising Letter Images from Scanned Invoices Using Stacked Autoencoders

Invoice document digitization is crucial for efficient management in industries.The scanned invoice image is often noisy due to various reasons.This affects the OCR(optical character recognition)detection accuracy.In this paper,letter data obtained from images of invoices are denoised using a modified autoencoder based deep learning method.A stacked denoising autoencoder(SDAE)is implemented with two hidden layers each in encoder network and decoder network.In order to capture the most salient features of training samples,a undercomplete autoencoder is designed with non-linear encoder and decoder function.This autoencoder is regularized for denoising application using a combined loss function which considers both mean square error and binary cross entropy.A dataset consisting of 59,119 letter images,which contains both English alphabets(upper and lower case)and numbers(0 to 9)is prepared from many scanned invoices images and windows true type(.ttf)files,are used for training the neural network.Performance is analyzed in terms of Signal to Noise Ratio(SNR),Peak Signal to Noise Ratio(PSNR),Structural Similarity Index(SSIM)and Universal Image Quality Index(UQI)and compared with other filtering techniques like Nonlocal Means filter,Anisotropic diffusion filter,Gaussian filters and Mean filters.Denoising performance of proposed SDAE is compared with existing SDAE with single loss function in terms of SNR and PSNR values.Results show the superior performance of proposed SDAE method.

Automatic Detection of COVID-19 Using a Stacked Denoising Convolutional Autoencoder

The exponential increase in new coronavirus disease 2019(COVID-19)cases and deaths has made COVID-19 the leading cause of death in many countries.Thus,in this study,we propose an efficient technique for the automatic detection of COVID-19 and pneumonia based on X-ray images.A stacked denoising convolutional autoencoder(SDCA)model was proposed to classify X-ray images into three classes:normal,pneumonia,and COVID-19.The SDCA model was used to obtain a good representation of the input data and extract the relevant features from noisy images.The proposed model’s architecture mainly composed of eight autoencoders,which were fed to two dense layers and SoftMax classifiers.The proposed model was evaluated with 6356 images from the datasets from different sources.The experiments and evaluation of the proposed model were applied to an 80/20 training/validation split and for five cross-validation data splitting,respectively.The metrics used for the SDCA model were the classification accuracy,precision,sensitivity,and specificity for both schemes.Our results demonstrated the superiority of the proposed model in classifying X-ray images with high accuracy of 96.8%.Therefore,this model can help physicians accelerate COVID-19 diagnosis.

Fault Diagnosis of Wind Turbine Generator with Stacked Noise Reduction Autoencoder Based on Group Normalization

In order to improve the condition monitoring and fault diagnosis of wind turbines,a stacked noise reduction autoencoding network based on group normalization is proposed in this paper.The network is based on SCADA data of wind turbine operation,firstly,the group normalization(GN)algorithm is added to solve the problems of stack noise reduction autoencoding network training and slow convergence speed,and the RMSProp algorithm is used to update the weight and the bias of the autoenccoder,which further optimizes the problem that the loss function swings too much during the update process.Finally,in the last layer of the network,the softmax activation function is used to classify the results,and the output of the network is transformed into a probability distribution.The selected wind turbine SCADA data was substituted into the pre-improved and improved stacked denoising autoencoding(SDA)networks for comparative training and verification.The results show that the stacked denoising autoencoding network based on group normalization is more accurate and effective for wind turbine condition monitoring and fault diagnosis,and also provides a reference for wind turbine fault identification.

An attention graph stacked autoencoder for anomaly detection of electro-mechanical actuator using spatio-temporal multivariate signals

Health monitoring of electro-mechanical actuator(EMA)is critical to ensure the security of airplanes.It is difficult or even impossible to collect enough labeled failure or degradation data from actual EMA.The autoencoder based on reconstruction loss is a popular model that can carry out anomaly detection with only consideration of normal training data,while it fails to capture spatio-temporal information from multivariate time series signals of multiple monitoring sensors.To mine the spatio-temporal information from multivariate time series signals,this paper proposes an attention graph stacked autoencoder for EMA anomaly detection.Firstly,attention graph con-volution is introduced into autoencoder to convolve temporal information from neighbor features to current features based on different weight attentions.Secondly,stacked autoencoder is applied to mine spatial information from those new aggregated temporal features.Finally,based on the bench-mark reconstruction loss of normal training data,different health thresholds calculated by several statistic indicators can carry out anomaly detection for new testing data.In comparison with tra-ditional stacked autoencoder,the proposed model could obtain higher fault detection rate and lower false alarm rate in EMA anomaly detection experiment.

A Time Series Intrusion Detection Method Based on SSAE,TCN and Bi-LSTM

In the fast-evolving landscape of digital networks,the incidence of network intrusions has escalated alarmingly.Simultaneously,the crucial role of time series data in intrusion detection remains largely underappreciated,with most systems failing to capture the time-bound nuances of network traffic.This leads to compromised detection accuracy and overlooked temporal patterns.Addressing this gap,we introduce a novel SSAE-TCN-BiLSTM(STL)model that integrates time series analysis,significantly enhancing detection capabilities.Our approach reduces feature dimensionalitywith a Stacked Sparse Autoencoder(SSAE)and extracts temporally relevant features through a Temporal Convolutional Network(TCN)and Bidirectional Long Short-term Memory Network(Bi-LSTM).By meticulously adjusting time steps,we underscore the significance of temporal data in bolstering detection accuracy.On the UNSW-NB15 dataset,ourmodel achieved an F1-score of 99.49%,Accuracy of 99.43%,Precision of 99.38%,Recall of 99.60%,and an inference time of 4.24 s.For the CICDS2017 dataset,we recorded an F1-score of 99.53%,Accuracy of 99.62%,Precision of 99.27%,Recall of 99.79%,and an inference time of 5.72 s.These findings not only confirm the STL model’s superior performance but also its operational efficiency,underpinning its significance in real-world cybersecurity scenarios where rapid response is paramount.Our contribution represents a significant advance in cybersecurity,proposing a model that excels in accuracy and adaptability to the dynamic nature of network traffic,setting a new benchmark for intrusion detection systems.

基于SAE-BP神经网络的审计风险识别研究——以计算机、通信和其他电子设备制造业行业为例

审计风险的识别和评估是现代风险导向审计的重要内容,为准确地识别审计风险,建立了一套基于SAE-BP神经网络的审计风险识别模型。选取16个指标构成重大错报风险评估模型的输入指标体系,利用SAE算法提取特征,通过机器学习模型BP神经网络分类器进行识别,构建SAE-BP神经网络,并选取135个A股上市公司作为样本进行了实证分析。结果表明:该模型运算速度快,模型平均识别准确率较高,可以达到88.5%,能够对审计风险进行高质量识别,有效提高了审计的效率。

一种SSAE+BPNN的变工况飞灰含碳量软测量方法

火电机组变工况运行使数据呈现多模态特征,导致基于浅层网络结构的回归软测量模型的预测精度下降。研究一种改进的BP神经网络(back propagation neural network,BPNN)软测量方法:首先利用堆叠稀疏自编码器(stacked sparse autoencoder,SSAE)强大的深度学习能力提取原始数据特征,然后再利用BPNN对提取特征进行回归分析。经实验验证,SSAE+BPNN软测量方法的均方误差为0.135 8×10–3,平方相关系数为0.983 2,其预测精度和泛化能力显著优于BPNN。将其应用于某台灵活调峰的超超临界660 MW发电机组飞灰含碳量软测量中,预测结果的平均相对误差为0.91%,总体相对误差控制在±5%以内,具有良好的工程应用价值。

基于SAE-SA-1D-CNN-BGRU的涡扇发动机剩余寿命预测

为解决涡扇发动机监测数据维度高和寿命预测准确度低的问题,提出一种基于深度学习的寿命预测方法,开展了利用神经网络获取涡扇发动机剩余寿命的研究。利用堆叠自编码(SAE)网络从高维传感器数据中提取健康因子(HI),采用1维卷积神经网络-双向门控循环单元(1D-CNN-BGRU)方法捕捉HI序列中的空间和时间特征,并引入自注意(SA)机制对捕捉的特征分配权重,使用全连接层输出涡扇发动机剩余使用寿命(RUL),以此构建复合神经网络进行面向涡扇发动机高维数据的寿命预测。结果表明:利用NASA官方网站提供的涡扇发动机寿命试验公开数据集C-MAPSS对该方法进行验证,取得了均方根误差16.22和评分函数225的结果。证明了基于SAE-SA-1D-CNN-BGRU的寿命预测方法可实现涡扇发动机寿命的有效预测,能为涡扇发动机维修保障及健康管理提供有效决策支撑。

Enhanced Deep Autoencoder Based Feature Representation Learning for Intelligent Intrusion Detection System

In the era of Big data,learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system(IDS).Owing to the lack of accurately labeled network traffic data,many unsupervised feature representation learning models have been proposed with state-of-theart performance.Yet,these models fail to consider the classification error while learning the feature representation.Intuitively,the learnt feature representation may degrade the performance of the classification task.For the first time in the field of intrusion detection,this paper proposes an unsupervised IDS model leveraging the benefits of deep autoencoder(DAE)for learning the robust feature representation and one-class support vector machine(OCSVM)for finding the more compact decision hyperplane for intrusion detection.Specially,the proposed model defines a new unified objective function to minimize the reconstruction and classification error simultaneously.This unique contribution not only enables the model to support joint learning for feature representation and classifier training but also guides to learn the robust feature representation which can improve the discrimination ability of the classifier for intrusion detection.Three set of evaluation experiments are conducted to demonstrate the potential of the proposed model.First,the ablation evaluation on benchmark dataset,NSL-KDD validates the design decision of the proposed model.Next,the performance evaluation on recent intrusion dataset,UNSW-NB15 signifies the stable performance of the proposed model.Finally,the comparative evaluation verifies the efficacy of the proposed model against recently published state-of-the-art methods.

基于SVD-SAE-GPR的锂离子电池RUL预测

锂离子电池是重要的储能手段之一,对其剩余使用寿命(RUL)进行预测具有非常重要的实际意义。本工作首先针对传统特征提取方法依赖参数设置且对于不同锂离子电池数据集适应性差的缺陷,将电池数据视作矩阵,并引入奇异值分解(SVD)从测量数据和包含更多退化信息的特征提取对象中提取潜在健康因子(HIs)。其次,潜在HIs的冗余和不足会影响RUL的预测,同时考虑到主成分分析(PCA)的缺点,使用Spearman相关分析和堆叠自编码器(SAE)处理HIs得到一个融合HI。在此基础上,利用高斯过程回归(GPR)算法构建了融合HI与容量之间的模型,得到了带有不确定性表达的最终预测结果。最后,通过NASA提供的四个老化电池数据验证了所提预测模型的可行性和有效性。并额外选取MIT电池数据集验证特征提取方法的适应性。实验结果表明,所提出的RUL预测框架具有较好的预测性能,SVD特征提取方法在避免参数设置的前提下具有较好的适应性。本工作提取的HI与经过PCA融合的HI、其他HI相比,预测精度有显著提高。

基于SSAE和相似性匹配的航空发动机剩余寿命预测

航空发动机作为高度复杂的热力机械,其剩余寿命(RUL)预测往往作为提高安全性和经济性的重要保障。为了提高航空发动机剩余寿命预测精度,提出一种基于堆栈稀疏自编码器(SSAE)及相似性匹配的剩余寿命预测方法。以Spearman秩相关系数(SRCC)作为适应度函数,利用遗传算法(GA)对融合参数候选集进行寻优;采用SSAE的结构融合最优参数集,生成特征融合指标;采用相似性匹配的方法在历史数据库内全局搜索最优匹配的历史轨迹,得到寿命预测结果;采用美国国家航空航天局(NASA)公布的C-MAPSS数据集验证该融合指标和方法的有效性。