Hypergraph Regularized Deep Autoencoder for Unsupervised Unmixing Hyperspectral Images

Deep learning(DL)has shown its superior performance in dealing with various computer vision tasks in recent years.As a simple and effective DL model,autoencoder(AE)is popularly used to decompose hyperspectral images(HSIs)due to its powerful ability of feature extraction and data reconstruction.However,most existing AE-based unmixing algorithms usually ignore the spatial information of HSIs.To solve this problem,a hypergraph regularized deep autoencoder(HGAE)is proposed for unmixing.Firstly,the traditional AE architecture is specifically improved as an unsupervised unmixing framework.Secondly,hypergraph learning is employed to reformulate the loss function,which facilitates the expression of high-order similarity among locally neighboring pixels and promotes the consistency of their abundances.Moreover,L_(1/2)norm is further used to enhance abundances sparsity.Finally,the experiments on simulated data,real hyperspectral remote sensing images,and textile cloth images are used to verify that the proposed method can perform better than several state-of-the-art unmixing algorithms.

Bi-LSTM-Based Deep Stacked Sequence-to-Sequence Autoencoder for Forecasting Solar Irradiation and Wind Speed

Wind and solar energy are two popular forms of renewable energy used in microgrids and facilitating the transition towards net-zero carbon emissions by 2050.However,they are exceedingly unpredictable since they rely highly on weather and atmospheric conditions.In microgrids,smart energy management systems,such as integrated demand response programs,are permanently established on a step-ahead basis,which means that accu-rate forecasting of wind speed and solar irradiance intervals is becoming increasingly crucial to the optimal operation and planning of microgrids.With this in mind,a novel“bidirectional long short-term memory network”(Bi-LSTM)-based,deep stacked,sequence-to-sequence autoencoder(S2SAE)forecasting model for predicting short-term solar irradiation and wind speed was developed and evaluated in MATLAB.To create a deep stacked S2SAE prediction model,a deep Bi-LSTM-based encoder and decoder are stacked on top of one another to reduce the dimension of the input sequence,extract its features,and then reconstruct it to produce the forecasts.Hyperparameters of the proposed deep stacked S2SAE forecasting model were optimized using the Bayesian optimization algorithm.Moreover,the forecasting performance of the proposed Bi-LSTM-based deep stacked S2SAE model was compared to three other deep,and shallow stacked S2SAEs,i.e.,the LSTM-based deep stacked S2SAE model,gated recurrent unit-based deep stacked S2SAE model,and Bi-LSTM-based shallow stacked S2SAE model.All these models were also optimized and modeled in MATLAB.The results simulated based on actual data confirmed that the proposed model outperformed the alternatives by achieving an accuracy of up to 99.7%,which evidenced the high reliability of the proposed forecasting.

An Intelligent Fault Diagnosis Method of Multi-Scale Deep Feature Fusion Based on Information Entropy

For a single-structure deep learning fault diagnosis model,its disadvantages are an insufficient feature extraction and weak fault classification capability.This paper proposes a multi-scale deep feature fusion intelligent fault diagnosis method based on information entropy.First,a normal autoencoder,denoising autoencoder,sparse autoencoder,and contractive autoencoder are used in parallel to construct a multi-scale deep neural network feature extraction structure.A deep feature fusion strategy based on information entropy is proposed to obtain low-dimensional features and ensure the robustness of the model and the quality of deep features.Finally,the advantage of the deep belief network probability model is used as the fault classifier to identify the faults.The effectiveness of the proposed method was verified by a gearbox test-bed.Experimental results show that,compared with traditional and existing intelligent fault diagnosis methods,the proposed method can obtain representative information and features from the raw data with higher classification accuracy.

Multi-Layer Reconstruction Errors Autoencoding and Density Estimate for Network Anomaly Detection

Anomaly detection is an important method for intrusion detection.In recent years,unsupervised methods have been widely researched because they do not require labeling.For example,a nonlinear autoencoder can use reconstruction errors to attain the discrimination threshold.This method is not effective when the model complexity is high or the data contains noise.The method for detecting the density of compressed features in a hidden layer can be used to reduce the influence of noise on the selection of the threshold because the density of abnormal data in hidden layers is smaller than normal data.However,compressed features may lose some of the high-dimensional distribution information of the original data.In this paper,we present an efficient anomaly detection framework for unsupervised anomaly detection,which includes network data capturing,processing,feature extraction,and anomaly detection.We employ a deep autoencoder to obtain compressed features and multi-layer reconstruction errors,and feeds them the same to the Gaussian mixture model to estimate the density.The proposed approach is trained and tested on multiple current intrusion detection datasets and real network scenes,and performance indicators,namely accuracy,recall,and F1-score,are better than other autoencoder models.

VASC: Dimension Reduction and Visualization of Single-cell RNA-seq Data by Deep Variational Autoencoder

Single-cell RNA sequencing(scRNA-seq) is a powerful technique to analyze the transcriptomic heterogeneities at the single cell level. It is an important step for studying cell subpopulations and lineages, with an effective low-dimensional representation and visualization of the original scRNA-Seq data. At the single cell level, the transcriptional fluctuations are much larger than the average of a cell population, and the low amount of RNA transcripts will increase the rate of technical dropout events. Therefore, scRNA-seq data are much noisier than traditional bulk RNA-seq data. In this study, we proposed the deep variational autoencoder for scRNA-seq data(VASC), a deep multi-layer generative model, for the unsupervised dimension reduction and visualization of scRNA-seq data. VASC can explicitly model the dropout events and find the nonlinear hierarchical feature representations of the original data. Tested on over 20 datasets, VASC shows superior performances in most cases and exhibits broader dataset compatibility compared to four state-of-the-art dimension reduction and visualization methods. In addition, VASC provides better representations for very rare cell populations in the 2D visualization. As a case study, VASC successfully re-establishes the cell dynamics in pre-implantation embryos and identifies several candidate marker genes associated with early embryo development. Moreover, VASC also performs well on a 10× Genomics dataset with more cells and higher dropout rate.

A defect recognition model for cross-section profile of hot-rolled strip based on deep learning

The cross-section profile is a key signal for evaluating hot-rolled strip quality,and ignoring its defects can easily lead to a final failure.The characteristics of complex curve,significant irregular fluctuation and imperfect sample data make it a challenge of recognizing cross-section defects,and current industrial judgment methods rely excessively on human decision making.A novel stacked denoising autoencoders(SDAE)model optimized with support vector machine(SVM)theory was proposed for the recognition of cross-section defects.Firstly,interpolation filtering and principal component analysis were employed to linearly reduce the data dimensionality of the profile curve.Secondly,the deep learning algorithm SDAE was used layer by layer for greedy unsupervised feature learning,and its final layer of back-propagation neural network was replaced by SVM for supervised learning of the final features,and the final model SDAE_SVM was obtained by further optimizing the entire network parameters via error back-propagation.Finally,the curve mirroring and combination stitching methods were used as data augmentation for the training set,which dealt with the problem of sample imbalance in the original data set,and the accuracy of cross-section defect prediction was further improved.The approach was applied in a 1780-mm hot rolling line of a steel mill to achieve the automatic diagnosis and classification of defects in cross-section profile of hot-rolled strip,which helps to reduce flatness quality concerns in downstream processes.

Deep 3D reconstruction:methods,data,and challenges

Three-dimensional(3D)reconstruction of shapes is an important research topic in the fields of computer vision,computer graphics,pattern recognition,and virtual reality.Existing 3D reconstruction methods usually suffer from two bottlenecks:(1)they involve multiple manually designed states which can lead to cumulative errors,but can hardly learn semantic features of 3D shapes automatically;(2)they depend heavily on the content and quality of images,as well as precisely calibrated cameras.As a result,it is difficult to improve the reconstruction accuracy of those methods.3D reconstruction methods based on deep learning overcome both of these bottlenecks by automatically learning semantic features of 3D shapes from low-quality images using deep networks.However,while these methods have various architectures,in-depth analysis and comparisons of them are unavailable so far.We present a comprehensive survey of 3D reconstruction methods based on deep learning.First,based on different deep learning model architectures,we divide 3D reconstruction methods based on deep learning into four types,recurrent neural network,deep autoencoder,generative adversarial network,and convolutional neural network based methods,and analyze the corresponding methodologies carefully.Second,we investigate four representative databases that are commonly used by the above methods in detail.Third,we give a comprehensive comparison of 3D reconstruction methods based on deep learning,which consists of the results of different methods with respect to the same database,the results of each method with respect to different databases,and the robustness of each method with respect to the number of views.Finally,we discuss future development of 3D reconstruction methods based on deep learning.