Anomaly-Based Intrusion DetectionModel Using Deep Learning for IoT Networks

The rapid growth of Internet of Things(IoT)devices has brought numerous benefits to the interconnected world.However,the ubiquitous nature of IoT networks exposes them to various security threats,including anomaly intrusion attacks.In addition,IoT devices generate a high volume of unstructured data.Traditional intrusion detection systems often struggle to cope with the unique characteristics of IoT networks,such as resource constraints and heterogeneous data sources.Given the unpredictable nature of network technologies and diverse intrusion methods,conventional machine-learning approaches seem to lack efficiency.Across numerous research domains,deep learning techniques have demonstrated their capability to precisely detect anomalies.This study designs and enhances a novel anomaly-based intrusion detection system(AIDS)for IoT networks.Firstly,a Sparse Autoencoder(SAE)is applied to reduce the high dimension and get a significant data representation by calculating the reconstructed error.Secondly,the Convolutional Neural Network(CNN)technique is employed to create a binary classification approach.The proposed SAE-CNN approach is validated using the Bot-IoT dataset.The proposed models exceed the performance of the existing deep learning approach in the literature with an accuracy of 99.9%,precision of 99.9%,recall of 100%,F1 of 99.9%,False Positive Rate(FPR)of 0.0003,and True Positive Rate(TPR)of 0.9992.In addition,alternative metrics,such as training and testing durations,indicated that SAE-CNN performs better.

Alzheimer’s Disease Stage Classification Using a Deep Transfer Learning and Sparse Auto Encoder Method

Alzheimer’s Disease(AD)is a progressive neurological disease.Early diagnosis of this illness using conventional methods is very challenging.Deep Learning(DL)is one of the finest solutions for improving diagnostic procedures’performance and forecast accuracy.The disease’s widespread distribution and elevated mortality rate demonstrate its significance in the older-onset and younger-onset age groups.In light of research investigations,it is vital to consider age as one of the key criteria when choosing the subjects.The younger subjects are more susceptible to the perishable side than the older onset.The proposed investigation concentrated on the younger onset.The research used deep learning models and neuroimages to diagnose and categorize the disease at its early stages automatically.The proposed work is executed in three steps.The 3D input images must first undergo image pre-processing using Weiner filtering and Contrast Limited Adaptive Histogram Equalization(CLAHE)methods.The Transfer Learning(TL)models extract features,which are subsequently compressed using cascaded Auto Encoders(AE).The final phase entails using a Deep Neural Network(DNN)to classify the phases of AD.The model was trained and tested to classify the five stages of AD.The ensemble ResNet-18 and sparse autoencoder with DNN model achieved an accuracy of 98.54%.The method is compared to state-of-the-art approaches to validate its efficacy and performance.

Learning sparse and smooth functions by deep Sigmoid nets

To pursue the outperformance of deep nets in learning,we construct a deep net with three hidden layers and prove that,implementing the empirical risk minimization(ERM)on this deep net,the estimator can theoretically realize the optimal learning rates without the classical saturation problem.In other words,deepening the networks with only three hidden layers can overcome the saturation and not degrade the optimal learning rates.The obtained results underlie the success of deep nets and provide a theoretical guidance for deep learning.

Multi-Layer Deep Sparse Representation for Biological Slice Image Inpainting

Biological slices are an effective tool for studying the physiological structure and evolutionmechanism of biological systems.However,due to the complexity of preparation technology and the presence of many uncontrollable factors during the preparation processing,leads to problems such as difficulty in preparing slice images and breakage of slice images.Therefore,we proposed a biological slice image small-scale corruption inpainting algorithm with interpretability based on multi-layer deep sparse representation,achieving the high-fidelity reconstruction of slice images.We further discussed the relationship between deep convolutional neural networks and sparse representation,ensuring the high-fidelity characteristic of the algorithm first.A novel deep wavelet dictionary is proposed that can better obtain image prior and possess learnable feature.And multi-layer deep sparse representation is used to implement dictionary learning,acquiring better signal expression.Compared with methods such as NLABH,Shearlet,Partial Differential Equation(PDE),K-Singular Value Decomposition(K-SVD),Convolutional Sparse Coding,and Deep Image Prior,the proposed algorithm has better subjective reconstruction and objective evaluation with small-scale image data,which realized high-fidelity inpainting,under the condition of small-scale image data.And theOn2-level time complexitymakes the proposed algorithm practical.The proposed algorithm can be effectively extended to other cross-sectional image inpainting problems,such as magnetic resonance images,and computed tomography images.

Sparse Seismic Data Reconstruction Based on a Convolutional Neural Network Algorithm

At present,the acquisition of seismic data is developing toward high-precision and high-density methods.However,complex natural environments and cultural factors in many exploration areas cause difficulties in achieving uniform and intensive acquisition,which makes complete seismic data collection impossible.Therefore,data reconstruction is required in the processing link to ensure imaging accuracy.Deep learning,as a new field in rapid development,presents clear advantages in feature extraction and modeling.In this study,the convolutional neural network deep learning algorithm is applied to seismic data reconstruction.Based on the convolutional neural network algorithm and combined with the characteristics of seismic data acquisition,two training strategies of supervised and unsupervised learning are designed to reconstruct sparse acquisition seismic records.First,a supervised learning strategy is proposed for labeled data,wherein the complete seismic data are segmented as the input of the training set and are randomly sampled before each training,thereby increasing the number of samples and the richness of features.Second,an unsupervised learning strategy based on large samples is proposed for unlabeled data,and the rolling segmentation method is used to update(pseudo)labels and training parameters in the training process.Through the reconstruction test of simulated and actual data,the deep learning algorithm based on a convolutional neural network shows better reconstruction quality and higher accuracy than compressed sensing based on Curvelet transform.

Sparse Autoencoder-based Multi-head Deep Neural Networks for Machinery Fault Diagnostics with Detection of Novelties

Supervised fault diagnosis typically assumes that all the types of machinery failures are known.However,in practice unknown types of defect,i.e.,novelties,may occur,whose detection is a challenging task.In this paper,a novel fault diagnostic method is developed for both diagnostics and detection of novelties.To this end,a sparse autoencoder-based multi-head Deep Neural Network(DNN)is presented to jointly learn a shared encoding representation for both unsupervised reconstruction and supervised classification of the monitoring data.The detection of novelties is based on the reconstruction error.Moreover,the computational burden is reduced by directly training the multi-head DNN with rectified linear unit activation function,instead of performing the pre-training and fine-tuning phases required for classical DNNs.The addressed method is applied to a benchmark bearing case study and to experimental data acquired from a delta 3D printer.The results show that its performance is satisfactory both in detection of novelties and fault diagnosis,outperforming other state-of-the-art methods.This research proposes a novel fault diagnostics method which can not only diagnose the known type of defect,but also detect unknown types of defects.

Deep Neural Network Based Detection and Segmentation of Ships for Maritime Surveillance

In recent years,computer visionfinds wide applications in maritime surveillance with its sophisticated algorithms and advanced architecture.Auto-matic ship detection with computer vision techniques provide an efficient means to monitor as well as track ships in water bodies.Waterways being an important medium of transport require continuous monitoring for protection of national security.The remote sensing satellite images of ships in harbours and water bodies are the image data that aid the neural network models to localize ships and to facilitate early identification of possible threats at sea.This paper proposes a deep learning based model capable enough to classify between ships and no-ships as well as to localize ships in the original images using bounding box tech-nique.Furthermore,classified ships are again segmented with deep learning based auto-encoder model.The proposed model,in terms of classification,provides suc-cessful results generating 99.5%and 99.2%validation and training accuracy respectively.The auto-encoder model also produces 85.1%and 84.2%validation and training accuracies.Moreover the IoU metric of the segmented images is found to be of 0.77 value.The experimental results reveal that the model is accu-rate and can be implemented for automatic ship detection in water bodies consid-ering remote sensing satellite images as input to the computer vision system.

Application of Improved Deep Auto-Encoder Network in Rolling Bearing Fault Diagnosis

Since the effectiveness of extracting fault features is not high under traditional bearing fault diagnosis method, a bearing fault diagnosis method based on Deep Auto-encoder Network (DAEN) optimized by Cloud Adaptive Particle Swarm Optimization (CAPSO) was proposed. On the basis of analyzing CAPSO and DAEN, the CAPSO-DAEN fault diagnosis model is built. The model uses the randomness and stability of CAPSO algorithm to optimize the connection weight of DAEN, to reduce the constraints on the weights and extract fault features adaptively. Finally, efficient and accurate fault diagnosis can be implemented with the Softmax classifier. The results of test show that the proposed method has higher diagnostic accuracy and more stable diagnosis results than those based on the DAEN, Support Vector Machine (SVM) and the Back Propagation algorithm (BP) under appropriate parameters.

Performance Evaluation of Super-Resolution Methods Using Deep-Learning and Sparse-Coding for Improving the Image Quality of Magnified Images in Chest Radiographs

Purpose: To detect small diagnostic signals such as lung nodules in chest radiographs, radiologists magnify a region-of-interest using linear interpolation methods. However, such methods tend to generate over-smoothed images with artifacts that can make interpretation difficult. The purpose of this study was to investigate the effectiveness of super-resolution methods for improving the image quality of magnified chest radiographs. Materials and Methods: A total of 247 chest X-rays were sampled from the JSRT database, then divided into 93 training cases with non-nodules and 154 test cases with lung nodules. We first trained two types of super-resolution methods, sparse-coding super-resolution (ScSR) and super-resolution convolutional neural network (SRCNN). With the trained super-resolution methods, the high-resolution image was then reconstructed using the super-resolution methods from a low-resolution image that was down-sampled from the original test image. We compared the image quality of the super-resolution methods and the linear interpolations (nearest neighbor and bilinear interpolations). For quantitative evaluation, we measured two image quality metrics: peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). For comparative evaluation of the super-resolution methods, we measured the computation time per image. Results: The PSNRs and SSIMs for the ScSR and the SRCNN schemes were significantly higher than those of the linear interpolation methods (p p p Conclusion: Super-resolution methods provide significantly better image quality than linear interpolation methods for magnified chest radiograph images. Of the two tested schemes, the SRCNN scheme processed the images fastest;thus, SRCNN could be clinically superior for processing radiographs in terms of both image quality and processing speed.

Prediction Model of Aircraft Icing Based on Deep Neural Network

Icing is an important factor threatening aircraft flight safety.According to the requirements of airworthiness regulations,aircraft icing safety assessment is needed to be carried out based on the ice shapes formed under different icing conditions.Due to the complexity of the icing process,the rapid assessment of ice shape remains an important challenge.In this paper,an efficient prediction model of aircraft icing is established based on the deep belief network(DBN)and the stacked auto-encoder(SAE),which are all deep neural networks.The detailed network structures are designed and then the networks are trained according to the samples obtained by the icing numerical computation.After that the model is applied on the ice shape evaluation of NACA0012 airfoil.The results show that the model can accurately capture the nonlinear behavior of aircraft icing and thus make an excellent ice shape prediction.The model provides an important tool for aircraft icing analysis.

Sparsity-Aware Channel Estimation for mmWave Massive MIMO: A Deep CNN-Based Approach

The deep convolutional neural network(CNN)is exploited in this work to conduct the challenging channel estimation for mmWave massive multiple input multiple output(MIMO)systems.The inherent sparse features of the mmWave massive MIMO channels can be extracted and the sparse channel supports can be learnt by the multi-layer CNN-based network through training.Then accurate channel inference can be efficiently implemented using the trained network.The estimation accuracy and spectrum efficiency can be further improved by fully utilizing the spatial correlation among the sparse channel supports of different antennas.It is verified by simulation results that the proposed deep CNN-based scheme significantly outperforms the state-of-the-art benchmarks in both accuracy and spectrum efficiency.

基于Sparse Coding和DBN的敏感图像检测

敏感图像检测,即检测图片是否含有危害青少年健康成长的不良色情信息,对于净化网络环境有重要意义。该文分析了现有的敏感图像检测算法的性能,结合稀疏编码和深度信赖网络,提出了一种改进的敏感图像检测算法。该算法通过稀疏编码来提取特征,将图像切分成标准大小的小图块,然后将其基于字典稀疏表示。接着用max-pooling池化来整合特征,获得最终使用的特征向量。将得到的特征向量输入到DBN网络中进行训练,得到DBN模型。最后将待测图像的特征向量输入到DBN模型中获得分类结果。在文献[10]的数据集上的实验显示,该检测算法较原有算法有较大提升,在以总样本的90%作为训练集时,可获得9.29%的平均错误率。

Deep Learning Based Face Detection and Identification of Criminal Suspects

Occurrence of crimes has been on the constant rise despite the emerging discoveries and advancements in the technological field in the past decade.One of the most tedious tasks is to track a suspect once a crime is committed.As most of the crimes are committed by individuals who have a history of felonies,it is essential for a monitoring system that does not just detect the person’s face who has committed the crime,but also their identity.Hence,a smart criminal detection and identification system that makes use of the OpenCV Deep Neural Network(DNN)model which employs a Single Shot Multibox Detector for detection of face and an auto-encoder model in which the encoder part is used for matching the captured facial images with the criminals has been proposed.After detection and extraction of the face in the image by face cropping,the captured face is then compared with the images in the CriminalDatabase.The comparison is performed by calculating the similarity value between each pair of images that are obtained by using the Cosine Similarity metric.After plotting the values in a graph to find the threshold value,we conclude that the confidence rate of the encoder model is 0.75 and above.

Optimizing Big Data Retrieval and Job Scheduling Using Deep Learning Approaches

Big data analytics in business intelligence do not provide effective data retrieval methods and job scheduling that will cause execution inefficiency and low system throughput.This paper aims to enhance the capability of data retrieval and job scheduling to speed up the operation of big data analytics to overcome inefficiency and low throughput problems.First,integrating stacked sparse autoencoder and Elasticsearch indexing explored fast data searching and distributed indexing,which reduces the search scope of the database and dramatically speeds up data searching.Next,exploiting a deep neural network to predict the approximate execution time of a job gives prioritized job scheduling based on the shortest job first,which reduces the average waiting time of job execution.As a result,the proposed data retrieval approach outperforms the previous method using a deep autoencoder and Solr indexing,significantly improving the speed of data retrieval up to 53%and increasing system throughput by 53%.On the other hand,the proposed job scheduling algorithmdefeats both first-in-first-out andmemory-sensitive heterogeneous early finish time scheduling algorithms,effectively shortening the average waiting time up to 5%and average weighted turnaround time by 19%,respectively.

Deep reinforcement learning for UAV swarm rendezvous behavior

The unmanned aerial vehicle(UAV)swarm technology is one of the research hotspots in recent years.With the continuous improvement of autonomous intelligence of UAV,the swarm technology of UAV will become one of the main trends of UAV development in the future.This paper studies the behavior decision-making process of UAV swarm rendezvous task based on the double deep Q network(DDQN)algorithm.We design a guided reward function to effectively solve the problem of algorithm convergence caused by the sparse return problem in deep reinforcement learning(DRL)for the long period task.We also propose the concept of temporary storage area,optimizing the memory playback unit of the traditional DDQN algorithm,improving the convergence speed of the algorithm,and speeding up the training process of the algorithm.Different from traditional task environment,this paper establishes a continuous state-space task environment model to improve the authentication process of UAV task environment.Based on the DDQN algorithm,the collaborative tasks of UAV swarm in different task scenarios are trained.The experimental results validate that the DDQN algorithm is efficient in terms of training UAV swarm to complete the given collaborative tasks while meeting the requirements of UAV swarm for centralization and autonomy,and improving the intelligence of UAV swarm collaborative task execution.The simulation results show that after training,the proposed UAV swarm can carry out the rendezvous task well,and the success rate of the mission reaches 90%.

融合稀疏八叉树与卷积神经网络的汽车风阻系数预测

针对汽车风阻系数预测研究中参数化方法难以准确表征汽车外造型的难题,提出融合稀疏八叉树与卷积神经网络的汽车风阻系数预测方法。将汽车外造型按照八叉树结构离散,使用平均法向量对离散的复杂曲面进行简化,利用卷积神经网络对八叉树形式的汽车外造型进行特征提取,进而对汽车风阻系数进行快速预测。通过改变卷积层数与全连接层数,研究了不同卷积神经网络结构对风阻系数预测精度的影响。与参数化方法相比,本文提出的外造型表示方法能更好地描述模型细节,构建的卷积神经网络结构对风阻系数预测的最小相对误差为1.453%,且计算速度是CFD仿真的1620倍,具有较高的精度及计算效率。

基于深度SSDAE网络的刀具磨损状态识别

针对刀具磨损状态识别过程中采集数据量大、干扰信号复杂且需人为选择特征参数的问题,为提高刀具磨损状态识别模型的鲁棒性与泛化性,提出了一种数据驱动下深度堆叠稀疏降噪自编码(stacking sparse denoising auto-encoder,简称SSDAE)网络的刀具磨损状态识别方法,实现隐藏在数据中深层次的数据特征自动挖掘。首先,将原始振动信号分解为一系列固有模态分量(intrinsic mode function,简称IMF),并采用皮尔逊相关系数法选取了最优固有模态来组合一个新的信号;其次,采用SSDAE网络自适应提取特征后对刀具磨损阶段进行了状态识别,识别精度达到98%;最后,对网络模型进行实验验证,并与最常用的刀具磨损状态识别方法进行了对比。实验结果表明,所提出的方法能够很好地处理非平稳振动信号,对不同刀具磨损阶段状态的识别效果良好,并具有较好的泛化性能和可靠性。

基于卷积特征提取及深度降噪网络的大规模MIMO系统信号检测

传统多输入多输出(Multiple-Input Multiple-Output,MIMO)信号检测算法受到天线数量和收发天线比例的限制,一般仅适用于少量天线、收发天线比例较低的情况。本文提出一种基于深度学习(Deep Learning,DL)的稀疏连接卷积降噪网络模型,用于大规模MIMO系统上行链路信号检测。首先,通过简化经典的检测网络(Detection Network,DetNet),改进ScNet(Sparsely Connected Neural Network)检测算法,引入卷积神经网络(Convolutional NeuralNetworks,CNN)对三通道输入数据提取特征以减少训练参数,提出一种SConv(SparselyConnected Convolutional Neural Network)检测算法。与DetNet算法相比,该算法可同时降低计算复杂度和提高检测精度。在此基础上,进一步基于CNN构建信号降噪模块,并嵌入SConv网络,提出一种卷积神经降噪(Sparsely Connected Convolutional Denoising,SConv-D)网络辅助的大规模MIMO检测算法。此算法检测过程分为两级,第一级由SConv算法提供初始估计值,再将初始估计值作为降噪过程的输入,并由此构成算法第二级。实验结果表明,本文提出的SConv-D算法适用于QPSK、4QAM及16QAM等多种信号调制模式,在高阶调制模式下获得的性能增益尤为明显。此外,该算法能够适应各种比例的收发天线及数量规模的系统配置,尤其是在收发天线数量相等的情况下亦能获得更优的性能。本文算法还克服了MMNet在高阶调制情况下的性能平台效应,在16QAM调制、收发天线数量相等的情况下,SConv-D在10^(-2)误比特率上能获得接近2 dB的性能增益。

NM-SpMM:面向国产异构向量处理器的半结构化稀疏矩阵乘算法

深度神经网络在自然语言处理、计算机视觉等领域取得了优异的成果,由于智能应用处理数据规模的增长和大模型的快速发展,对深度神经网络的推理性能要求越来越高,N∶M半结构化稀疏化技术成为平衡算力需求和应用效果的热点技术之一。国产异构向量处理器FT-M7032为智能模型处理中的数据并行和指令并行开发提供了较大空间。针对N∶M半结构化稀疏模型计算稀疏模式多样性,提出了一种面向FT-M7032的可灵活配置的稀疏矩阵乘算法NM-SpMM。NM-SpMM设计了一种高效的压缩偏移地址稀疏编码格式COA,避免了半结构化参数配置对稀疏数据访存计算的影响。基于COA编码,NM-SpMM对不同维度稀疏矩阵计算进行了细粒度优化。在FT-M7032单核上的实验结果表明,相较于稠密矩阵乘,NM-SpMM能获得1.73~21.00倍的加速,相较于采用CuSPARSE稀疏计算库的NVIDIA V100 GPU,能获得0.04~1.04倍的加速。

融合项目特征级信息的稀疏兴趣网络序列推荐

在以往提取多兴趣嵌入的序列推荐模型中仅能通过聚类的方法发现少量兴趣概念,忽视项目交互序列中特征级信息对最终推荐结果的影响。针对此问题,对传统的多兴趣序列推荐模型进行改进,提出一种融合项目特征级信息的稀疏兴趣网络序列推荐模型。实验结果表明,相比其它模型,该模型可以更好捕捉用户的多样化偏好并缓解冷启动问题。在给定数据集上,该模型比传统的序列推荐模型在命中率上平均提高了6.4%,归一化折损累计增益平均提高了8.7%。