Rock mass quality classification based on deep learning:A feasibility study for stacked autoencoders

Objective and accurate evaluation of rock mass quality classification is the prerequisite for reliable sta-bility assessment.To develop a tool that can deliver quick and accurate evaluation of rock mass quality,a deep learning approach is developed,which uses stacked autoencoders(SAEs)with several autoencoders and a softmax net layer.Ten rock parameters of rock mass rating(RMR)system are calibrated in this model.The model is trained using 75%of the total database for training sample data.The SAEs trained model achieves a nearly 100%prediction accuracy.For comparison,other different models are also trained with the same dataset,using artificial neural network(ANN)and radial basis function(RBF).The results show that the SAEs classify all test samples correctly while the rating accuracies of ANN and RBF are 97.5%and 98.7%,repectively,which are calculated from the confusion matrix.Moreover,this model is further employed to predict the slope risk level of an abandoned quarry.The proposed approach using SAEs,or deep learning in general,is more objective and more accurate and requires less human inter-vention.The findings presented here shall shed light for engineers/researchers interested in analyzing rock mass classification criteria or performing field investigation.

Iterative learning-based many-objective history matching using deep neural network with stacked autoencoder

This paper presents an innovative data-integration that uses an iterative-learning method,a deep neural network(DNN)coupled with a stacked autoencoder(SAE)to solve issues encountered with many-objective history matching.The proposed method consists of a DNN-based inverse model with SAE-encoded static data and iterative updates of supervised-learning data are based on distance-based clustering schemes.DNN functions as an inverse model and results in encoded flattened data,while SAE,as a pre-trained neural network,successfully reduces dimensionality and reliably reconstructs geomodels.The iterative-learning method can improve the training data for DNN by showing the error reduction achieved with each iteration step.The proposed workflow shows the small mean absolute percentage error below 4%for all objective functions,while a typical multi-objective evolutionary algorithm fails to significantly reduce the initial population uncertainty.Iterative learning-based manyobjective history matching estimates the trends in water cuts that are not reliably included in dynamicdata matching.This confirms the proposed workflow constructs more plausible geo-models.The workflow would be a reliable alternative to overcome the less-convergent Pareto-based multi-objective evolutionary algorithm in the presence of geological uncertainty and varying objective functions.

Optimized Stacked Autoencoder for IoT Enabled Financial Crisis Prediction Model

Recently,Financial Technology(FinTech)has received more attention among financial sectors and researchers to derive effective solutions for any financial institution or firm.Financial crisis prediction(FCP)is an essential topic in business sector that finds it useful to identify the financial condition of a financial institution.At the same time,the development of the internet of things(IoT)has altered the mode of human interaction with the physical world.The IoT can be combined with the FCP model to examine the financial data from the users and perform decision making process.This paper presents a novel multi-objective squirrel search optimization algorithm with stacked autoencoder(MOSSA-SAE)model for FCP in IoT environment.The MOSSA-SAE model encompasses different subprocesses namely preprocessing,class imbalance handling,parameter tuning,and classification.Primarily,the MOSSA-SAE model allows the IoT devices such as smartphones,laptops,etc.,to collect the financial details of the users which are then transmitted to the cloud for further analysis.In addition,SMOTE technique is employed to handle class imbalance problems.The goal of MOSSA in SMOTE is to determine the oversampling rate and area of nearest neighbors of SMOTE.Besides,SAE model is utilized as a classification technique to determine the class label of the financial data.At the same time,the MOSSA is applied to appropriately select the‘weights’and‘bias’values of the SAE.An extensive experimental validation process is performed on the benchmark financial dataset and the results are examined under distinct aspects.The experimental values ensured the superior performance of the MOSSA-SAE model on the applied dataset.

Hybrid Image Compression-Encryption Scheme Based on Multilayer Stacked Autoencoder and Logistic Map

Secure transmission of images over a communication channel, with limited data transfer capacity, possesses compression and encryption schemes. A deep learning based hybrid image compression-encryption scheme is proposed by combining stacked auto-encoder with the logistic map. The proposed structure of stacked autoencoder has seven multiple layers, and back propagation algorithm is intended to extend vector portrayal of information into lower vector space. The randomly generated key is used to set initial conditions and control parameters of logistic map. Subsequently, compressed image is encrypted by substituting and scrambling of pixel sequences using key stream sequences generated from logistic map.The proposed algorithms are experimentally tested over five standard grayscale images. Compression and encryption efficiency of proposed algorithms are evaluated and analyzed based on peak signal to noise ratio(PSNR), mean square error(MSE), structural similarity index metrics(SSIM) and statistical,differential, entropy analysis respectively. Simulation results show that proposed algorithms provide high quality reconstructed images with excellent levels of security during transmission..

Denoising Letter Images from Scanned Invoices Using Stacked Autoencoders

Invoice document digitization is crucial for efficient management in industries.The scanned invoice image is often noisy due to various reasons.This affects the OCR(optical character recognition)detection accuracy.In this paper,letter data obtained from images of invoices are denoised using a modified autoencoder based deep learning method.A stacked denoising autoencoder(SDAE)is implemented with two hidden layers each in encoder network and decoder network.In order to capture the most salient features of training samples,a undercomplete autoencoder is designed with non-linear encoder and decoder function.This autoencoder is regularized for denoising application using a combined loss function which considers both mean square error and binary cross entropy.A dataset consisting of 59,119 letter images,which contains both English alphabets(upper and lower case)and numbers(0 to 9)is prepared from many scanned invoices images and windows true type(.ttf)files,are used for training the neural network.Performance is analyzed in terms of Signal to Noise Ratio(SNR),Peak Signal to Noise Ratio(PSNR),Structural Similarity Index(SSIM)and Universal Image Quality Index(UQI)and compared with other filtering techniques like Nonlocal Means filter,Anisotropic diffusion filter,Gaussian filters and Mean filters.Denoising performance of proposed SDAE is compared with existing SDAE with single loss function in terms of SNR and PSNR values.Results show the superior performance of proposed SDAE method.

Bi-LSTM-Based Deep Stacked Sequence-to-Sequence Autoencoder for Forecasting Solar Irradiation and Wind Speed

Wind and solar energy are two popular forms of renewable energy used in microgrids and facilitating the transition towards net-zero carbon emissions by 2050.However,they are exceedingly unpredictable since they rely highly on weather and atmospheric conditions.In microgrids,smart energy management systems,such as integrated demand response programs,are permanently established on a step-ahead basis,which means that accu-rate forecasting of wind speed and solar irradiance intervals is becoming increasingly crucial to the optimal operation and planning of microgrids.With this in mind,a novel“bidirectional long short-term memory network”(Bi-LSTM)-based,deep stacked,sequence-to-sequence autoencoder(S2SAE)forecasting model for predicting short-term solar irradiation and wind speed was developed and evaluated in MATLAB.To create a deep stacked S2SAE prediction model,a deep Bi-LSTM-based encoder and decoder are stacked on top of one another to reduce the dimension of the input sequence,extract its features,and then reconstruct it to produce the forecasts.Hyperparameters of the proposed deep stacked S2SAE forecasting model were optimized using the Bayesian optimization algorithm.Moreover,the forecasting performance of the proposed Bi-LSTM-based deep stacked S2SAE model was compared to three other deep,and shallow stacked S2SAEs,i.e.,the LSTM-based deep stacked S2SAE model,gated recurrent unit-based deep stacked S2SAE model,and Bi-LSTM-based shallow stacked S2SAE model.All these models were also optimized and modeled in MATLAB.The results simulated based on actual data confirmed that the proposed model outperformed the alternatives by achieving an accuracy of up to 99.7%,which evidenced the high reliability of the proposed forecasting.

Automatic Detection of COVID-19 Using a Stacked Denoising Convolutional Autoencoder

The exponential increase in new coronavirus disease 2019(COVID-19)cases and deaths has made COVID-19 the leading cause of death in many countries.Thus,in this study,we propose an efficient technique for the automatic detection of COVID-19 and pneumonia based on X-ray images.A stacked denoising convolutional autoencoder(SDCA)model was proposed to classify X-ray images into three classes:normal,pneumonia,and COVID-19.The SDCA model was used to obtain a good representation of the input data and extract the relevant features from noisy images.The proposed model’s architecture mainly composed of eight autoencoders,which were fed to two dense layers and SoftMax classifiers.The proposed model was evaluated with 6356 images from the datasets from different sources.The experiments and evaluation of the proposed model were applied to an 80/20 training/validation split and for five cross-validation data splitting,respectively.The metrics used for the SDCA model were the classification accuracy,precision,sensitivity,and specificity for both schemes.Our results demonstrated the superiority of the proposed model in classifying X-ray images with high accuracy of 96.8%.Therefore,this model can help physicians accelerate COVID-19 diagnosis.

Software Defect Prediction Based on Stacked Contractive Autoencoder and Multi-Objective Optimization

Software defect prediction plays an important role in software quality assurance.However,the performance of the prediction model is susceptible to the irrelevant and redundant features.In addition,previous studies mostly regard software defect prediction as a single objective optimization problem,and multi-objective software defect prediction has not been thoroughly investigated.For the above two reasons,we propose the following solutions in this paper:(1)we leverage an advanced deep neural network-Stacked Contractive AutoEncoder(SCAE)to extract the robust deep semantic features from the original defect features,which has stronger discrimination capacity for different classes(defective or non-defective).(2)we propose a novel multi-objective defect prediction model named SMONGE that utilizes the Multi-Objective NSGAII algorithm to optimize the advanced neural network-Extreme learning machine(ELM)based on state-of-the-art Pareto optimal solutions according to the features extracted by SCAE.We mainly consider two objectives.One objective is to maximize the performance of ELM,which refers to the benefit of the SMONGE model.Another objective is to minimize the output weight norm of ELM,which is related to the cost of the SMONGE model.We compare the SCAE with six state-of-the-art feature extraction methods and compare the SMONGE model with multiple baseline models that contain four classic defect predictors and the MONGE model without SCAE across 20 open source software projects.The experimental results verify that the superiority of SCAE and SMONGE on seven evaluation metrics.

Pseudo Zernike Moment and Deep Stacked Sparse Autoencoder for COVID-19 Diagnosis

(Aim)COVID-19 is an ongoing infectious disease.It has caused more than 107.45 m confirmed cases and 2.35 m deaths till 11/Feb/2021.Traditional computer vision methods have achieved promising results on the automatic smart diagnosis.(Method)This study aims to propose a novel deep learning method that can obtain better performance.We use the pseudo-Zernike moment(PZM),derived from Zernike moment,as the extracted features.Two settings are introducing:(i)image plane over unit circle;and(ii)image plane inside the unit circle.Afterward,we use a deep-stacked sparse autoencoder(DSSAE)as the classifier.Besides,multiple-way data augmentation is chosen to overcome overfitting.The multiple-way data augmentation is based on Gaussian noise,salt-and-pepper noise,speckle noise,horizontal and vertical shear,rotation,Gamma correction,random translation and scaling.(Results)10 runs of 10-fold cross validation shows that our PZM-DSSAE method achieves a sensitivity of 92.06%±1.54%,a specificity of 92.56%±1.06%,a precision of 92.53%±1.03%,and an accuracy of 92.31%±1.08%.Its F1 score,MCC,and FMI arrive at 92.29%±1.10%,84.64%±2.15%,and 92.29%±1.10%,respectively.The AUC of our model is 0.9576.(Conclusion)We demonstrate“image plane over unit circle”can get better results than“image plane inside a unit circle.”Besides,this proposed PZM-DSSAE model is better than eight state-of-the-art approaches.

Data Cleaning Based on Stacked Denoising Autoencoders and Multi-Sensor Collaborations

Wireless sensor networks are increasingly used in sensitive event monitoring.However,various abnormal data generated by sensors greatly decrease the accuracy of the event detection.Although many methods have been proposed to deal with the abnormal data,they generally detect and/or repair all abnormal data without further differentiate.Actually,besides the abnormal data caused by events,it is well known that sensor nodes prone to generate abnormal data due to factors such as sensor hardware drawbacks and random effects of external sources.Dealing with all abnormal data without differentiate will result in false detection or missed detection of the events.In this paper,we propose a data cleaning approach based on Stacked Denoising Autoencoders(SDAE)and multi-sensor collaborations.We detect all abnormal data by SDAE,then differentiate the abnormal data by multi-sensor collaborations.The abnormal data caused by events are unchanged,while the abnormal data caused by other factors are repaired.Real data based simulations show the efficiency of the proposed approach.

Fault Diagnosis of Wind Turbine Generator with Stacked Noise Reduction Autoencoder Based on Group Normalization

In order to improve the condition monitoring and fault diagnosis of wind turbines,a stacked noise reduction autoencoding network based on group normalization is proposed in this paper.The network is based on SCADA data of wind turbine operation,firstly,the group normalization(GN)algorithm is added to solve the problems of stack noise reduction autoencoding network training and slow convergence speed,and the RMSProp algorithm is used to update the weight and the bias of the autoenccoder,which further optimizes the problem that the loss function swings too much during the update process.Finally,in the last layer of the network,the softmax activation function is used to classify the results,and the output of the network is transformed into a probability distribution.The selected wind turbine SCADA data was substituted into the pre-improved and improved stacked denoising autoencoding(SDA)networks for comparative training and verification.The results show that the stacked denoising autoencoding network based on group normalization is more accurate and effective for wind turbine condition monitoring and fault diagnosis,and also provides a reference for wind turbine fault identification.

Enhanced Deep Autoencoder Based Feature Representation Learning for Intelligent Intrusion Detection System

In the era of Big data,learning discriminant feature representation from network traffic is identified has as an invariably essential task for improving the detection ability of an intrusion detection system(IDS).Owing to the lack of accurately labeled network traffic data,many unsupervised feature representation learning models have been proposed with state-of-theart performance.Yet,these models fail to consider the classification error while learning the feature representation.Intuitively,the learnt feature representation may degrade the performance of the classification task.For the first time in the field of intrusion detection,this paper proposes an unsupervised IDS model leveraging the benefits of deep autoencoder(DAE)for learning the robust feature representation and one-class support vector machine(OCSVM)for finding the more compact decision hyperplane for intrusion detection.Specially,the proposed model defines a new unified objective function to minimize the reconstruction and classification error simultaneously.This unique contribution not only enables the model to support joint learning for feature representation and classifier training but also guides to learn the robust feature representation which can improve the discrimination ability of the classifier for intrusion detection.Three set of evaluation experiments are conducted to demonstrate the potential of the proposed model.First,the ablation evaluation on benchmark dataset,NSL-KDD validates the design decision of the proposed model.Next,the performance evaluation on recent intrusion dataset,UNSW-NB15 signifies the stable performance of the proposed model.Finally,the comparative evaluation verifies the efficacy of the proposed model against recently published state-of-the-art methods.

A Time Series Intrusion Detection Method Based on SSAE,TCN and Bi-LSTM

In the fast-evolving landscape of digital networks,the incidence of network intrusions has escalated alarmingly.Simultaneously,the crucial role of time series data in intrusion detection remains largely underappreciated,with most systems failing to capture the time-bound nuances of network traffic.This leads to compromised detection accuracy and overlooked temporal patterns.Addressing this gap,we introduce a novel SSAE-TCN-BiLSTM(STL)model that integrates time series analysis,significantly enhancing detection capabilities.Our approach reduces feature dimensionalitywith a Stacked Sparse Autoencoder(SSAE)and extracts temporally relevant features through a Temporal Convolutional Network(TCN)and Bidirectional Long Short-term Memory Network(Bi-LSTM).By meticulously adjusting time steps,we underscore the significance of temporal data in bolstering detection accuracy.On the UNSW-NB15 dataset,ourmodel achieved an F1-score of 99.49%,Accuracy of 99.43%,Precision of 99.38%,Recall of 99.60%,and an inference time of 4.24 s.For the CICDS2017 dataset,we recorded an F1-score of 99.53%,Accuracy of 99.62%,Precision of 99.27%,Recall of 99.79%,and an inference time of 5.72 s.These findings not only confirm the STL model’s superior performance but also its operational efficiency,underpinning its significance in real-world cybersecurity scenarios where rapid response is paramount.Our contribution represents a significant advance in cybersecurity,proposing a model that excels in accuracy and adaptability to the dynamic nature of network traffic,setting a new benchmark for intrusion detection systems.

UCAV situation assessment method based on C-LSHADE-Means and SAE-LVQ

The unmanned combat aerial vehicle(UCAV)is a research hot issue in the world,and the situation assessment is an important part of it.To overcome shortcomings of the existing situation assessment methods,such as low accuracy and strong dependence on prior knowledge,a datadriven situation assessment method is proposed.The clustering and classification are combined,the former is used to mine situational knowledge,and the latter is used to realize rapid assessment.Angle evaluation factor and distance evaluation factor are proposed to transform multi-dimensional air combat information into two-dimensional features.A convolution success-history based adaptive differential evolution with linear population size reduc-tion-means(C-LSHADE-Means)algorithm is proposed.The convolutional pooling layer is used to compress the size of data and preserve the distribution characteristics.The LSHADE algorithm is used to initialize the center of the mean clustering,which over-comes the defect of initialization sensitivity.Comparing experi-ment with the seven clustering algorithms is done on the UCI data set,through four clustering indexes,and it proves that the method proposed in this paper has better clustering performance.A situation assessment model based on stacked autoen-coder and learning vector quantization(SAE-LVQ)network is constructed,and it uses SAE to reconstruct air combat data fea-tures,and uses the self-competition layer of the LVQ to achieve efficient classification.Compared with the five kinds of assess-ments models,the SAE-LVQ model has the highest accuracy.Finally,three kinds of confrontation processes from air combat maneuvering instrumentation(ACMI)are selected,and the model in this paper is used for situation assessment.The assessment results are in line with the actual situation.

Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput.This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems.First,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data searching.Next,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job execution.As a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

Deep learning technique for process fault detection and diagnosis in the presence of incomplete data

In modern industrial processes, timely detection and diagnosis of process abnormalities are critical for monitoring process operations. Various fault detection and diagnosis(FDD) methods have been proposed and implemented, the performance of which, however, could be drastically influenced by the common presence of incomplete or missing data in real industrial scenarios. This paper presents a new FDD approach based on an incomplete data imputation technique for process fault recognition. It employs the modified stacked autoencoder,a deep learning structure, in the phase of incomplete data treatment, and classifies data representations rather than the imputed complete data in the phase of fault identification. A benchmark process, the Tennessee Eastman process, is employed to illustrate the effectiveness and applicability of the proposed method.

Energy Aware Secure Cyber-Physical Systems with Clustered Wireless Sensor Networks

Recently,cyber physical system(CPS)has gained significant attention which mainly depends upon an effective collaboration with computation and physical components.The greatly interrelated and united characteristics of CPS resulting in the development of cyber physical energy systems(CPES).At the same time,the rising ubiquity of wireless sensor networks(WSN)in several application areas makes it a vital part of the design of CPES.Since security and energy efficiency are the major challenging issues in CPES,this study offers an energy aware secure cyber physical systems with clustered wireless sensor networks using metaheuristic algorithms(EASCPSMA).The presented EASCPS-MA technique intends to attain lower energy utilization via clustering and security using intrusion detection.The EASCPSMA technique encompasses two main stages namely improved fruit fly optimization algorithm(IFFOA)based clustering and optimal deep stacked autoencoder(OSAE)based intrusion detection.Besides,the optimal selection of stacked autoencoder(SAE)parameters takes place using root mean square propagation(RMSProp)model.The extensive performance validation of the EASCPS-MA technique takes place and the results are inspected under varying aspects.The simulation results reported the improved effectiveness of the EASCPS-MA technique over other recent approaches interms of several measures.

Research on Deep Knowledge Tracking Incorporating Rich Features and Forgetting Behaviors

The individualization of education and teaching through the computer⁃aided education system provides students with personalized learning,so that each student can obtain the knowledge they need.At this stage,there are a lot of intelligent tutoring systems.In these systems,students􀆳learning actions are tracked in real⁃time,and there are a lot of available data.From these data,personalized education that suits each student can be mined.To improve the quality of education,some models for predicting students􀆳next practice have been produced,such as Bayesian Knowledge Tracing(BKT),Performance Factor Analysis(PFA),and Deep Knowledge Tracing(DKT)with the development of deep learning.However,the model only considers the knowledge component and correctness of the problem,ignoring the breadth of other characteristics of the information collected by the intelligent tutoring system,the lag time of the previous interaction,the number of past attempts to a problem,and situations that students have forgotten the knowledge.Although some studies consider forgetting and rich information when modeling student knowledge,they often ignore student learning sequences.The main contribution of this paper is in two aspects.One is to transform the input into a position feature vector by introducing an auto⁃encoding network layer and to carry out multiple sets of bad political combinations.The other is to consider repeated time intervals,sequence time intervals,and the number of attempts to simulate forgetting behavior.This paper proposes an adaptive algorithm for the original DKT model.By using the stacked auto⁃encoder network,the input dimension is reduced to half of the original and the original features are retained and consider the forgetting memory behavior according to the time sequence of students􀆳learning.The model proposed in this paper has been experimented on two public data sets to improve the original accuracy.

Fault prediction of combine harvesters based on stacked denoising autoencoders

Accurate fault prediction is essential to ensure the safety and reliability of combine harvester operation.In this study,a combine harvester fault prediction method based on a combination of stacked denoising autoencoders(SDAE)and multi-classification support vector machines(SVM)is proposed to predict combine harvester faults by extracting operational features of key combine components.In general,SDAE contains autoencoders and uses a deep network architecture to learn complex non-linear input-output relationships in a hierarchical manner.Selected features are fed into the SDAE network,deep-level features of the input parameters are extracted by SDAE,and an SVM classifier is then added to its top layer to achieve combine harvester fault prediction.The experimental results show that the method can achieve accurate and efficient combine harvester fault prediction.In particular,the experiments used Gaussian noise with a distribution center of 0.05 to corrupt the test data samples obtained by random sampling of the whole population,and the results showed that the prediction accuracy of the method was 95.31%,which has better robustness and generalization ability compared to SVM(77.03%),BP(74.61%),and SAE(90.86%).

Predicting microbial extracellular electron transfer activity in paddy soils with soil physicochemical properties using machine learning

Soil extracellular electron transfer(EET)is a pivotal biological process within the realm of soil.Unfortunately,EET suffers from a lack of predictive models.Herein,an intricately crafted machine learning model has been developed for the purpose of predicting soil EET by using the physicochemical properties of soil as independent input variables and the EET capabilities in terms of current density(j_(max))and Coulombic charge(C_(out))as dependent output variables.An autoencoder ensemble stacking(AES)model was developed to address the aforementioned issue by integrating support vector machine,multilayer perceptron,extreme gradient boosting,and light gradient boosting machine algorithms as the stacking algorithms.With 10-fold crossvalidation,the AES model exhibited notable improvements in predicting j_(max)and C_(out),with average test R^(2)values of 0.83 and 0.84,respectively,surpassing those of single machine learning(ML)models and the basic ensemble model.By utilizing partial correlation plots(PDPs),Shapley Additive explanations(SHAP)values,and SHAP decision plots,we quantitatively explained the impact and contribution of the input molecules on the AES model’s predictions of j_(max)and C_(out).In the context of the SHAP method for the AES model,total carbon(TC)was identified as the most correlated descriptor for j_(max),while total organic carbon(TOC)stood out as the most relevant descriptor for C_(out).In the prediction tasks of j_(max)and C_(out)within the AES model,employing a multitask ML approach allowed the model to benefit from the shared information of input variables,thereby enhancing its overall generalizability.This study provides a feasible tool for the prediction of soil EET from soil physiochemical properties and an advanced understanding of the relationship between soil physiochemical properties and EET capability.