Alzheimer’s Disease Stage Classification Using a Deep Transfer Learning and Sparse Auto Encoder Method

Alzheimer’s Disease(AD)is a progressive neurological disease.Early diagnosis of this illness using conventional methods is very challenging.Deep Learning(DL)is one of the finest solutions for improving diagnostic procedures’performance and forecast accuracy.The disease’s widespread distribution and elevated mortality rate demonstrate its significance in the older-onset and younger-onset age groups.In light of research investigations,it is vital to consider age as one of the key criteria when choosing the subjects.The younger subjects are more susceptible to the perishable side than the older onset.The proposed investigation concentrated on the younger onset.The research used deep learning models and neuroimages to diagnose and categorize the disease at its early stages automatically.The proposed work is executed in three steps.The 3D input images must first undergo image pre-processing using Weiner filtering and Contrast Limited Adaptive Histogram Equalization(CLAHE)methods.The Transfer Learning(TL)models extract features,which are subsequently compressed using cascaded Auto Encoders(AE).The final phase entails using a Deep Neural Network(DNN)to classify the phases of AD.The model was trained and tested to classify the five stages of AD.The ensemble ResNet-18 and sparse autoencoder with DNN model achieved an accuracy of 98.54%.The method is compared to state-of-the-art approaches to validate its efficacy and performance.

Learning sparse and smooth functions by deep Sigmoid nets

To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.

Multi-Layer Deep Sparse Representation for Biological Slice Image Inpainting

Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing,leads to problems such as difficulty in preparing slice images and breakage of slice images.Therefore,we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation,achieving the high-fidelity reconstruction of slice images.We further discussed the relationship between deep convolutional neural networks and sparse representation,ensuring the high-fidelity characteristic of the algorithm first.A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature.And multi-layer deep sparse representation is used to implement dictionary learning,acquiring better signal expression.Compared with methods such as NLABH,Shearlet,Partial Differential Equation(PDE),K-Singular Value Decomposition(K-SVD),Convolutional Sparse Coding,and Deep Image Prior,the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data,which realized high-fidelity inpainting,under the condition of small-scale image data.And theOn2-level time complexitymakes the proposed algorithm practical.The proposed algorithm can be effectively extended to other cross-sectional image inpainting problems,such as magnetic resonance images,and computed tomography images.

Sparse Autoencoder-based Multi-head Deep Neural Networks for Machinery Fault Diagnostics with Detection of Novelties

Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.

Sparse Seismic Data Reconstruction Based on a Convolutional Neural Network Algorithm

At present,the acquisition of seismic data is developing toward high-precision and high-density methods.However,complex natural environments and cultural factors in many exploration areas cause difficulties in achieving uniform and intensive acquisition,which makes complete seismic data collection impossible.Therefore,data reconstruction is required in the processing link to ensure imaging accuracy.Deep learning,as a new field in rapid development,presents clear advantages in feature extraction and modeling.In this study,the convolutional neural network deep learning algorithm is applied to seismic data reconstruction.Based on the convolutional neural network algorithm and combined with the characteristics of seismic data acquisition,two training strategies of supervised and unsupervised learning are designed to reconstruct sparse acquisition seismic records.First,a supervised learning strategy is proposed for labeled data,wherein the complete seismic data are segmented as the input of the training set and are randomly sampled before each training,thereby increasing the number of samples and the richness of features.Second,an unsupervised learning strategy based on large samples is proposed for unlabeled data,and the rolling segmentation method is used to update(pseudo)labels and training parameters in the training process.Through the reconstruction test of simulated and actual data,the deep learning algorithm based on a convolutional neural network shows better reconstruction quality and higher accuracy than compressed sensing based on Curvelet transform.

Pseudo Zernike Moment and Deep Stacked Sparse Autoencoder for COVID-19 Diagnosis

(Aim)COVID-19 is an ongoing infectious disease.It has caused more than 107.45 m confirmed cases and 2.35 m deaths till 11/Feb/2021.Traditional computer vision methods have achieved promising results on the automatic smart diagnosis.(Method)This study aims to propose a novel deep learning method that can obtain better performance.We use the pseudo-Zernike moment(PZM),derived from Zernike moment,as the extracted features.Two settings are introducing:(i)image plane over unit circle;and(ii)image plane inside the unit circle.Afterward,we use a deep-stacked sparse autoencoder(DSSAE)as the classifier.Besides,multiple-way data augmentation is chosen to overcome overfitting.The multiple-way data augmentation is based on Gaussian noise,salt-and-pepper noise,speckle noise,horizontal and vertical shear,rotation,Gamma correction,random translation and scaling.(Results)10 runs of 10-fold cross validation shows that our PZM-DSSAE method achieves a sensitivity of 92.06%±1.54%,a specificity of 92.56%±1.06%,a precision of 92.53%±1.03%,and an accuracy of 92.31%±1.08%.Its F1 score,MCC,and FMI arrive at 92.29%±1.10%,84.64%±2.15%,and 92.29%±1.10%,respectively.The AUC of our model is 0.9576.(Conclusion)We demonstrate“image plane over unit circle”can get better results than“image plane inside a unit circle.”Besides,this proposed PZM-DSSAE model is better than eight state-of-the-art approaches.

Anomaly-Based Intrusion DetectionModel Using Deep Learning for IoT Networks

The rapid growth of Internet of Things(IoT)devices has brought numerous benefits to the interconnected world.However,the ubiquitous nature of IoT networks exposes them to various security threats,including anomaly intrusion attacks.In addition,IoT devices generate a high volume of unstructured data.Traditional intrusion detection systems often struggle to cope with the unique characteristics of IoT networks,such as resource constraints and heterogeneous data sources.Given the unpredictable nature of network technologies and diverse intrusion methods,conventional machine-learning approaches seem to lack efficiency.Across numerous research domains,deep learning techniques have demonstrated their capability to precisely detect anomalies.This study designs and enhances a novel anomaly-based intrusion detection system(AIDS)for IoT networks.Firstly,a Sparse Autoencoder(SAE)is applied to reduce the high dimension and get a significant data representation by calculating the reconstructed error.Secondly,the Convolutional Neural Network(CNN)technique is employed to create a binary classification approach.The proposed SAE-CNN approach is validated using the Bot-IoT dataset.The proposed models exceed the performance of the existing deep learning approach in the literature with an accuracy of 99.9%,precision of 99.9%,recall of 100%,F1 of 99.9%,False Positive Rate(FPR)of 0.0003,and True Positive Rate(TPR)of 0.9992.In addition,alternative metrics,such as training and testing durations,indicated that SAE-CNN performs better.

Performance Evaluation of Super-Resolution Methods Using Deep-Learning and Sparse-Coding for Improving the Image Quality of Magnified Images in Chest Radiographs

Purpose: To detect small diagnostic signals such as lung nodules in chest radiographs, radiologists magnify a region-of-interest using linear interpolation methods. However, such methods tend to generate over-smoothed images with artifacts that can make interpretation difficult. The purpose of this study was to investigate the effectiveness of super-resolution methods for improving the image quality of magnified chest radiographs. Materials and Methods: A total of 247 chest X-rays were sampled from the JSRT database, then divided into 93 training cases with non-nodules and 154 test cases with lung nodules. We first trained two types of super-resolution methods, sparse-coding super-resolution (ScSR) and super-resolution convolutional neural network (SRCNN). With the trained super-resolution methods, the high-resolution image was then reconstructed using the super-resolution methods from a low-resolution image that was down-sampled from the original test image. We compared the image quality of the super-resolution methods and the linear interpolations (nearest neighbor and bilinear interpolations). For quantitative evaluation, we measured two image quality metrics: peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). For comparative evaluation of the super-resolution methods, we measured the computation time per image. Results: The PSNRs and SSIMs for the ScSR and the SRCNN schemes were significantly higher than those of the linear interpolation methods (p p p Conclusion: Super-resolution methods provide significantly better image quality than linear interpolation methods for magnified chest radiograph images. Of the two tested schemes, the SRCNN scheme processed the images fastest;thus, SRCNN could be clinically superior for processing radiographs in terms of both image quality and processing speed.

Sparsity-Aware Channel Estimation for mmWave Massive MIMO: A Deep CNN-Based Approach

The deep convolutional neural network(CNN)is exploited in this work to conduct the challenging channel estimation for mmWave massive multiple input multiple output(MIMO)systems.The inherent sparse features of the mmWave massive MIMO channels can be extracted and the sparse channel supports can be learnt by the multi-layer CNN-based network through training.Then accurate channel inference can be efficiently implemented using the trained network.The estimation accuracy and spectrum efficiency can be further improved by fully utilizing the spatial correlation among the sparse channel supports of different antennas.It is verified by simulation results that the proposed deep CNN-based scheme significantly outperforms the state-of-the-art benchmarks in both accuracy and spectrum efficiency.

稀疏编码(Sparse coding)在图像检索中的应用

稀疏编码(Sparse Coding)作为深度学习的一个分支,在机器学习领域取得了多个方面的突破。本文将探索如何将Sparse Coding结合到图像检索的多个模块中,利用Sparse Coding的优点来提高检索的效果。

Fault Diagnosis of Motor in Frequency Domain Signal by Stacked De-noising Auto-encoder

With the rapid development of mechanical equipment,mechanical health monitoring field has entered the era of big data.Deep learning has made a great achievement in the processing of large data of image and speech due to the powerful modeling capabilities,this also brings influence to the mechanical fault diagnosis field.Therefore,according to the characteristics of motor vibration signals(nonstationary and difficult to deal with)and mechanical‘big data’,combined with deep learning,a motor fault diagnosis method based on stacked de-noising auto-encoder is proposed.The frequency domain signals obtained by the Fourier transform are used as input to the network.This method can extract features adaptively and unsupervised,and get rid of the dependence of traditional machine learning methods on human extraction features.A supervised fine tuning of the model is then carried out by backpropagation.The Asynchronous motor in Drivetrain Dynamics Simulator system was taken as the research object,the effectiveness of the proposed method was verified by a large number of data,and research on visualization of network output,the results shown that the SDAE method is more efficient and more intelligent.

基于Sparse Coding和DBN的敏感图像检测

敏感图像检测,即检测图片是否含有危害青少年健康成长的不良色情信息,对于净化网络环境有重要意义。该文分析了现有的敏感图像检测算法的性能,结合稀疏编码和深度信赖网络,提出了一种改进的敏感图像检测算法。该算法通过稀疏编码来提取特征,将图像切分成标准大小的小图块,然后将其基于字典稀疏表示。接着用max-pooling池化来整合特征,获得最终使用的特征向量。将得到的特征向量输入到DBN网络中进行训练,得到DBN模型。最后将待测图像的特征向量输入到DBN模型中获得分类结果。在文献[10]的数据集上的实验显示,该检测算法较原有算法有较大提升,在以总样本的90%作为训练集时,可获得9.29%的平均错误率。

Deblending by modified dictionary learning using Sparse Parameter Training

Considerable attempts have been made on removing the crosstalk noise in a simultaneous source data using the popular K-means Singular Value Decomposition algorithm(KSVD).Several hybrids of this method have been designed and successfully deployed,but the complex nature of blending noise makes it difficult to manipulate easily.One of the challenges of the K-means Singular Value Decomposition approach is the challenge to obtain an exact KSVD for each data patch which is believed to result in a better output.In this work,we propose a learnable architecture capable of data training while retaining the K-means Singular Value Decomposition essence to deblend simultaneous source data.

Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput.This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems.First,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data searching.Next,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job execution.As a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

Deep reinforcement learning for UAV swarm rendezvous behavior

The unmanned aerial vehicle(UAV)swarm technology is one of the research hotspots in recent years.With the continuous improvement of autonomous intelligence of UAV,the swarm technology of UAV will become one of the main trends of UAV development in the future.This paper studies the behavior decision-making process of UAV swarm rendezvous task based on the double deep Q network(DDQN)algorithm.We design a guided reward function to effectively solve the problem of algorithm convergence caused by the sparse return problem in deep reinforcement learning(DRL)for the long period task.We also propose the concept of temporary storage area,optimizing the memory playback unit of the traditional DDQN algorithm,improving the convergence speed of the algorithm,and speeding up the training process of the algorithm.Different from traditional task environment,this paper establishes a continuous state-space task environment model to improve the authentication process of UAV task environment.Based on the DDQN algorithm,the collaborative tasks of UAV swarm in different task scenarios are trained.The experimental results validate that the DDQN algorithm is efficient in terms of training UAV swarm to complete the given collaborative tasks while meeting the requirements of UAV swarm for centralization and autonomy,and improving the intelligence of UAV swarm collaborative task execution.The simulation results show that after training,the proposed UAV swarm can carry out the rendezvous task well,and the success rate of the mission reaches 90%.

融合稀疏八叉树与卷积神经网络的汽车风阻系数预测

针对汽车风阻系数预测研究中参数化方法难以准确表征汽车外造型的难题,提出融合稀疏八叉树与卷积神经网络的汽车风阻系数预测方法。将汽车外造型按照八叉树结构离散,使用平均法向量对离散的复杂曲面进行简化,利用卷积神经网络对八叉树形式的汽车外造型进行特征提取,进而对汽车风阻系数进行快速预测。通过改变卷积层数与全连接层数,研究了不同卷积神经网络结构对风阻系数预测精度的影响。与参数化方法相比,本文提出的外造型表示方法能更好地描述模型细节,构建的卷积神经网络结构对风阻系数预测的最小相对误差为1.453%,且计算速度是CFD仿真的1620倍,具有较高的精度及计算效率。

基于DK-SVD的深度学习电阻抗块稀疏成像方法研究

针对电阻抗层析成像逆问题的病态性和非线性,提出一种基于DK-SVD的电阻抗块稀疏图像重建方法。该算法通过多层感知器为每组测量数据提供最优的模型参数,以适应数据集的多样性,进一步提高成像质量,并在稀疏编码阶段采用迭代收缩阈值算法加快收敛速度。仿真实验结果表明DK-SVD算法重建图像的结构相似性可达到0.95以上,误差可控制在0.1左右,平均重建速度为0.034 s,有效地提高了电阻抗层析成像的质量和效率,且经进一步实验证明了该算法具有良好的噪声鲁棒性和实际应用价值。

面向外辐射源雷达目标探测的非时变稀疏模型和深度展开网络实现方法

近年来,基于稀疏特征提取的目标探测方法成为了雷达领域的研究热点。基于正交频分复用调制的外辐射源雷达(简称“外源雷达”)由于发射波形不受控,一方面构建的稀疏模型会随未知发射波形时变,导致相应的目标探测方法计算量大;另一方面目标回波常常因被直达波等强杂波掩盖而面临探测困难。在此背景下,利用外源雷达的正交频分复用波形特点,使用导频位置处频域信道响应提出一种非时变稀疏模型。通过将稀疏模型求解的每一次迭代过程替代为一层神经网络,首次研究了基于深度展开网络的智能化外源雷达目标探测实现方法。仿真和实测数据结果表明:所提方法与传统杂波抑制方法在目标探测上性能相近,但有着更低的计算复杂度,且无需人工设计稀疏矩阵等稀疏模型求解参数。

基于探针稀疏注意力机制的门控Transformer模型

在强化学习中,智能体对状态序列进行编码,根据历史信息指导动作的选择,通常将其建模为递归型神经网络,但其存在梯度消失和梯度爆炸的问题,难以处理长序列。以自注意力机制为核心的Transformer是一种能够有效整合长时间范围内信息的机制,将传统Transformer直接应用于强化学习中存在训练不稳定和计算复杂度高的问题。门控Transformer-XL(GTrXL)解决了Transformer在强化学习中训练不稳定的问题,但仍具有很高的计算复杂度。针对此问题,本研究提出了一种具有探针稀疏注意力机制的门控Transformer(PS-GTr),其在GTrXL中的恒等映射重排和门控机制的基础上引入了探针稀疏注意力机制,降低了时间复杂度和空间复杂度,进一步提高了训练效率。通过实验验证,PS-GTr在强化学习任务中的性能与GTrXL相当,而且训练时间更短,内存占用更少。

基于卷积特征提取及深度降噪网络的大规模MIMO系统信号检测

传统多输入多输出(Multiple-Input Multiple-Output,MIMO)信号检测算法受到天线数量和收发天线比例的限制,一般仅适用于少量天线、收发天线比例较低的情况。本文提出一种基于深度学习(Deep Learning,DL)的稀疏连接卷积降噪网络模型,用于大规模MIMO系统上行链路信号检测。首先,通过简化经典的检测网络(Detection Network,DetNet),改进ScNet(Sparsely Connected Neural Network)检测算法,引入卷积神经网络(Convolutional NeuralNetworks,CNN)对三通道输入数据提取特征以减少训练参数,提出一种SConv(SparselyConnected Convolutional Neural Network)检测算法。与DetNet算法相比,该算法可同时降低计算复杂度和提高检测精度。在此基础上,进一步基于CNN构建信号降噪模块,并嵌入SConv网络,提出一种卷积神经降噪(Sparsely Connected Convolutional Denoising,SConv-D)网络辅助的大规模MIMO检测算法。此算法检测过程分为两级,第一级由SConv算法提供初始估计值,再将初始估计值作为降噪过程的输入,并由此构成算法第二级。实验结果表明,本文提出的SConv-D算法适用于QPSK、4QAM及16QAM等多种信号调制模式,在高阶调制模式下获得的性能增益尤为明显。此外,该算法能够适应各种比例的收发天线及数量规模的系统配置,尤其是在收发天线数量相等的情况下亦能获得更优的性能。本文算法还克服了MMNet在高阶调制情况下的性能平台效应,在16QAM调制、收发天线数量相等的情况下,SConv-D在10^(-2)误比特率上能获得接近2 dB的性能增益。