Intrusion Detection System for Smart Industrial Environments with Ensemble Feature Selection and Deep Convolutional Neural Networks

Smart Industrial environments use the Industrial Internet of Things(IIoT)for their routine operations and transform their industrial operations with intelligent and driven approaches.However,IIoT devices are vulnerable to cyber threats and exploits due to their connectivity with the internet.Traditional signature-based IDS are effective in detecting known attacks,but they are unable to detect unknown emerging attacks.Therefore,there is the need for an IDS which can learn from data and detect new threats.Ensemble Machine Learning(ML)and individual Deep Learning(DL)based IDS have been developed,and these individual models achieved low accuracy;however,their performance can be improved with the ensemble stacking technique.In this paper,we have proposed a Deep Stacked Neural Network(DSNN)based IDS,which consists of two stacked Convolutional Neural Network(CNN)models as base learners and Extreme Gradient Boosting(XGB)as the meta learner.The proposed DSNN model was trained and evaluated with the next-generation dataset,TON_IoT.Several pre-processing techniques were applied to prepare a dataset for the model,including ensemble feature selection and the SMOTE technique.Accuracy,precision,recall,F1-score,and false positive rates were used to evaluate the performance of the proposed ensemble model.Our experimental results showed that the accuracy for binary classification is 99.61%,which is better than in the baseline individual DL and ML models.In addition,the model proposed for IDS has been compared with similar models.The proposed DSNN achieved better performance metrics than the other models.The proposed DSNN model will be used to develop enhanced IDS for threat mitigation in smart industrial environments.

Downscaling Seasonal Precipitation Forecasts over East Africa with Deep Convolutional Neural Networks

This study assesses the suitability of convolutional neural networks(CNNs) for downscaling precipitation over East Africa in the context of seasonal forecasting. To achieve this, we design a set of experiments that compare different CNN configurations and deployed the best-performing architecture to downscale one-month lead seasonal forecasts of June–July–August–September(JJAS) precipitation from the Nanjing University of Information Science and Technology Climate Forecast System version 1.0(NUIST-CFS1.0) for 1982–2020. We also perform hyper-parameter optimization and introduce predictors over a larger area to include information about the main large-scale circulations that drive precipitation over the East Africa region, which improves the downscaling results. Finally, we validate the raw model and downscaled forecasts in terms of both deterministic and probabilistic verification metrics, as well as their ability to reproduce the observed precipitation extreme and spell indicator indices. The results show that the CNN-based downscaling consistently improves the raw model forecasts, with lower bias and more accurate representations of the observed mean and extreme precipitation spatial patterns. Besides, CNN-based downscaling yields a much more accurate forecast of extreme and spell indicators and reduces the significant relative biases exhibited by the raw model predictions. Moreover, our results show that CNN-based downscaling yields better skill scores than the raw model forecasts over most portions of East Africa. The results demonstrate the potential usefulness of CNN in downscaling seasonal precipitation predictions over East Africa,particularly in providing improved forecast products which are essential for end users.

Prediction of Geopolymer Concrete Compressive Strength Using Convolutional Neural Networks

Geopolymer concrete emerges as a promising avenue for sustainable development and offers an effective solution to environmental problems.Its attributes as a non-toxic,low-carbon,and economical substitute for conventional cement concrete,coupled with its elevated compressive strength and reduced shrinkage properties,position it as a pivotal material for diverse applications spanning from architectural structures to transportation infrastructure.In this context,this study sets out the task of using machine learning(ML)algorithms to increase the accuracy and interpretability of predicting the compressive strength of geopolymer concrete in the civil engineering field.To achieve this goal,a new approach using convolutional neural networks(CNNs)has been adopted.This study focuses on creating a comprehensive dataset consisting of compositional and strength parameters of 162 geopolymer concrete mixes,all containing Class F fly ash.The selection of optimal input parameters is guided by two distinct criteria.The first criterion leverages insights garnered from previous research on the influence of individual features on compressive strength.The second criterion scrutinizes the impact of these features within the model’s predictive framework.Key to enhancing the CNN model’s performance is the meticulous determination of the optimal hyperparameters.Through a systematic trial-and-error process,the study ascertains the ideal number of epochs for data division and the optimal value of k for k-fold cross-validation—a technique vital to the model’s robustness.The model’s predictive prowess is rigorously assessed via a suite of performance metrics and comprehensive score analyses.Furthermore,the model’s adaptability is gauged by integrating a secondary dataset into its predictive framework,facilitating a comparative evaluation against conventional prediction methods.To unravel the intricacies of the CNN model’s learning trajectory,a loss plot is deployed to elucidate its learning rate.The study culminates in compelling findings that underscore the CNN model’s accurate prediction of geopolymer concrete compressive strength.To maximize the dataset’s potential,the application of bivariate plots unveils nuanced trends and interactions among variables,fortifying the consistency with earlier research.Evidenced by promising prediction accuracy,the study’s outcomes hold significant promise in guiding the development of innovative geopolymer concrete formulations,thereby reinforcing its role as an eco-conscious and robust construction material.The findings prove that the CNN model accurately estimated geopolymer concrete’s compressive strength.The results show that the prediction accuracy is promising and can be used for the development of new geopolymer concrete mixes.The outcomes not only underscore the significance of leveraging technology for sustainable construction practices but also pave the way for innovation and efficiency in the field of civil engineering.

Prediction of constrained modulus for granular soil using 3D discrete element method and convolutional neural networks

To efficiently predict the mechanical parameters of granular soil based on its random micro-structure,this study proposed a novel approach combining numerical simulation and machine learning algorithms.Initially,3500 simulations of one-dimensional compression tests on coarse-grained sand using the three-dimensional(3D)discrete element method(DEM)were conducted to construct a database.In this process,the positions of the particles were randomly altered,and the particle assemblages changed.Interestingly,besides confirming the influence of particle size distribution parameters,the stress-strain curves differed despite an identical gradation size statistic when the particle position varied.Subsequently,the obtained data were partitioned into training,validation,and testing datasets at a 7:2:1 ratio.To convert the DEM model into a multi-dimensional matrix that computers can recognize,the 3D DEM models were first sliced to extract multi-layer two-dimensional(2D)cross-sectional data.Redundant information was then eliminated via gray processing,and the data were stacked to form a new 3D matrix representing the granular soil’s fabric.Subsequently,utilizing the Python language and Pytorch framework,a 3D convolutional neural networks(CNNs)model was developed to establish the relationship between the constrained modulus obtained from DEM simulations and the soil’s fabric.The mean squared error(MSE)function was utilized to assess the loss value during the training process.When the learning rate(LR)fell within the range of 10-5e10-1,and the batch sizes(BSs)were 4,8,16,32,and 64,the loss value stabilized after 100 training epochs in the training and validation dataset.For BS?32 and LR?10-3,the loss reached a minimum.In the testing set,a comparative evaluation of the predicted constrained modulus from the 3D CNNs versus the simulated modulus obtained via DEM reveals a minimum mean absolute percentage error(MAPE)of 4.43%under the optimized condition,demonstrating the accuracy of this approach.Thus,by combining DEM and CNNs,the variation of soil’s mechanical characteristics related to its random fabric would be efficiently evaluated by directly tracking the particle assemblages.

Multi-Material Topology Optimization of 2D Structures Using Convolutional Neural Networks

In recent years,there has been significant research on the application of deep learning(DL)in topology optimization(TO)to accelerate structural design.However,these methods have primarily focused on solving binary TO problems,and effective solutions for multi-material topology optimization(MMTO)which requires a lot of computing resources are still lacking.Therefore,this paper proposes the framework of multiphase topology optimization using deep learning to accelerate MMTO design.The framework employs convolutional neural network(CNN)to construct a surrogate model for solving MMTO,and the obtained surrogate model can rapidly generate multi-material structure topologies in negligible time without any iterations.The performance evaluation results show that the proposed method not only outputs multi-material topologies with clear material boundary but also reduces the calculation cost with high prediction accuracy.Additionally,in order to find a more reasonable modeling method for MMTO,this paper studies the characteristics of surrogate modeling as regression task and classification task.Through the training of 297 models,our findings show that the regression task yields slightly better results than the classification task in most cases.Furthermore,The results indicate that the prediction accuracy is primarily influenced by factors such as the TO problem,material category,and data scale.Conversely,factors such as the domain size and the material property have minimal impact on the accuracy.

Prediction of Porous Media Fluid Flow with Spatial Heterogeneity Using Criss-Cross Physics-Informed Convolutional Neural Networks

Recent advances in deep neural networks have shed new light on physics,engineering,and scientific computing.Reconciling the data-centered viewpoint with physical simulation is one of the research hotspots.The physicsinformedneural network(PINN)is currently the most general framework,which is more popular due to theconvenience of constructing NNs and excellent generalization ability.The automatic differentiation(AD)-basedPINN model is suitable for the homogeneous scientific problem;however,it is unclear how AD can enforce fluxcontinuity across boundaries between cells of different properties where spatial heterogeneity is represented bygrid cells with different physical properties.In this work,we propose a criss-cross physics-informed convolutionalneural network(CC-PINN)learning architecture,aiming to learn the solution of parametric PDEs with spatialheterogeneity of physical properties.To achieve the seamless enforcement of flux continuity and integration ofphysicalmeaning into CNN,a predefined 2D convolutional layer is proposed to accurately express transmissibilitybetween adjacent cells.The efficacy of the proposedmethodwas evaluated through predictions of several petroleumreservoir problems with spatial heterogeneity and compared against state-of-the-art(PINN)through numericalanalysis as a benchmark,which demonstrated the superiority of the proposed method over the PINN.

A Pooling Method Developed for Use in Convolutional Neural Networks

In convolutional neural networks,pooling methods are used to reduce both the size of the data and the number of parameters after the convolution of the models.These methods reduce the computational amount of convolutional neural networks,making the neural network more efficient.Maximum pooling,average pooling,and minimum pooling methods are generally used in convolutional neural networks.However,these pooling methods are not suitable for all datasets used in neural network applications.In this study,a new pooling approach to the literature is proposed to increase the efficiency and success rates of convolutional neural networks.This method,which we call MAM(Maximum Average Minimum)pooling,is more interactive than other traditional maximum pooling,average pooling,and minimum pooling methods and reduces data loss by calculating the more appropriate pixel value.The proposed MAM pooling method increases the performance of the neural network by calculating the optimal value during the training of convolutional neural networks.To determine the success accuracy of the proposed MAM pooling method and compare it with other traditional pooling methods,training was carried out on the LeNet-5 model using CIFAR-10,CIFAR-100,and MNIST datasets.According to the results obtained,the proposed MAM pooling method performed better than the maximum pooling,average pooling,and minimum pooling methods in all pool sizes on three different datasets.

High-resolution reconstruction of the ablative RT instability flowfield via convolutional neural networks

High-resolution flow field data has important applications in meteorology,aerospace engineering,high-energy physics and other fields.Experiments and numerical simulations are two main ways to obtain high-resolution flow field data,while the high experiment cost and computing resources for simulation hinder the specificanalysis of flow field evolution.With the development of deep learning technology,convolutional neural networks areused to achieve high-resolution reconstruction of the flow field.In this paper,an ordinary convolutional neuralnetwork and a multi-time-path convolutional neural network are established for the ablative Rayleigh-Taylorinstability.These two methods can reconstruct the high-resolution flow field in just a few seconds,and further greatlyenrich the application of high-resolution reconstruction technology in fluid instability.Compared with the ordinaryconvolutional neural network,the multi-time-path convolutional neural network model has smaller error and canrestore more details of the flow field.The influence of low-resolution flow field data obtained by the two poolingmethods on the convolutional neural networks model is also discussed.

Resilience Augmentation in Unmanned Weapon Systems via Multi-Layer Attention Graph Convolutional Neural Networks

The collective Unmanned Weapon System-of-Systems(UWSOS)network represents a fundamental element in modern warfare,characterized by a diverse array of unmanned combat platforms interconnected through hetero-geneous network architectures.Despite its strategic importance,the UWSOS network is highly susceptible to hostile infiltrations,which significantly impede its battlefield recovery capabilities.Existing methods to enhance network resilience predominantly focus on basic graph relationships,neglecting the crucial higher-order dependencies among nodes necessary for capturing multi-hop meta-paths within the UWSOS.To address these limitations,we propose the Enhanced-Resilience Multi-Layer Attention Graph Convolutional Network(E-MAGCN),designed to augment the adaptability of UWSOS.Our approach employs BERT for extracting semantic insights from nodes and edges,thereby refining feature representations by leveraging various node and edge categories.Additionally,E-MAGCN integrates a regularization-based multi-layer attention mechanism and a semantic node fusion algo-rithm within the Graph Convolutional Network(GCN)framework.Through extensive simulation experiments,our model demonstrates an enhancement in resilience performance ranging from 1.2% to 7% over existing algorithms.

Application of Convolutional Neural Networks in Classification of GBM for Enhanced Prognosis

The lethal brain tumor “Glioblastoma” has the propensity to grow over time. To improve patient outcomes, it is essential to classify GBM accurately and promptly in order to provide a focused and individualized treatment plan. Despite this, deep learning methods, particularly Convolutional Neural Networks (CNNs), have demonstrated a high level of accuracy in a myriad of medical image analysis applications as a result of recent technical breakthroughs. The overall aim of the research is to investigate how CNNs can be used to classify GBMs using data from medical imaging, to improve prognosis precision and effectiveness. This research study will demonstrate a suggested methodology that makes use of the CNN architecture and is trained using a database of MRI pictures with this tumor. The constructed model will be assessed based on its overall performance. Extensive experiments and comparisons with conventional machine learning techniques and existing classification methods will also be made. It will be crucial to emphasize the possibility of early and accurate prediction in a clinical workflow because it can have a big impact on treatment planning and patient outcomes. The paramount objective is to not only address the classification challenge but also to outline a clear pathway towards enhancing prognosis precision and treatment effectiveness.

Customized Convolutional Neural Network for Accurate Detection of Deep Fake Images in Video Collections

The motivation for this study is that the quality of deep fakes is constantly improving,which leads to the need to develop new methods for their detection.The proposed Customized Convolutional Neural Network method involves extracting structured data from video frames using facial landmark detection,which is then used as input to the CNN.The customized Convolutional Neural Network method is the date augmented-based CNN model to generate‘fake data’or‘fake images’.This study was carried out using Python and its libraries.We used 242 films from the dataset gathered by the Deep Fake Detection Challenge,of which 199 were made up and the remaining 53 were real.Ten seconds were allotted for each video.There were 318 videos used in all,199 of which were fake and 119 of which were real.Our proposedmethod achieved a testing accuracy of 91.47%,loss of 0.342,and AUC score of 0.92,outperforming two alternative approaches,CNN and MLP-CNN.Furthermore,our method succeeded in greater accuracy than contemporary models such as XceptionNet,Meso-4,EfficientNet-BO,MesoInception-4,VGG-16,and DST-Net.The novelty of this investigation is the development of a new Convolutional Neural Network(CNN)learning model that can accurately detect deep fake face photos.

Development of a convolutional neural network based geomechanical upscaling technique for heterogeneous geological reservoir

Geomechanical assessment using coupled reservoir-geomechanical simulation is becoming increasingly important for analyzing the potential geomechanical risks in subsurface geological developments.However,a robust and efficient geomechanical upscaling technique for heterogeneous geological reservoirs is lacking to advance the applications of three-dimensional(3D)reservoir-scale geomechanical simulation considering detailed geological heterogeneities.Here,we develop convolutional neural network(CNN)proxies that reproduce the anisotropic nonlinear geomechanical response caused by lithological heterogeneity,and compute upscaled geomechanical properties from CNN proxies.The CNN proxies are trained using a large dataset of randomly generated spatially correlated sand-shale realizations as inputs and simulation results of their macroscopic geomechanical response as outputs.The trained CNN models can provide the upscaled shear strength(R^(2)>0.949),stress-strain behavior(R^(2)>0.925),and volumetric strain changes(R^(2)>0.958)that highly agree with the numerical simulation results while saving over two orders of magnitude of computational time.This is a major advantage in computing the upscaled geomechanical properties directly from geological realizations without the need to perform local numerical simulations to obtain the geomechanical response.The proposed CNN proxybased upscaling technique has the ability to(1)bridge the gap between the fine-scale geocellular models considering geological uncertainties and computationally efficient geomechanical models used to assess the geomechanical risks of large-scale subsurface development,and(2)improve the efficiency of numerical upscaling techniques that rely on local numerical simulations,leading to significantly increased computational time for uncertainty quantification using numerous geological realizations.

Pluggable multitask diffractive neural networks based on cascaded metasurfaces

Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.

Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks

The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.

A data-driven model of drop size prediction based on artificial neural networks using small-scale data sets

An artificial neural network(ANN)method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets.After training,the deviation between calculate and experimental results are 3.8%and 9.3%,respectively.Through ANN model,the influence of interfacial tension and pulsation intensity on the droplet diameter has been developed.Droplet size gradually increases with the increase of interfacial tension,and decreases with the increase of pulse intensity.It can be seen that the accuracy of ANN model in predicting droplet size outside the training set range is reach the same level as the accuracy of correlation obtained based on experiments within this range.For two kinds of columns,the drop size prediction deviations of ANN model are 9.6%and 18.5%and the deviations in correlations are 11%and 15%.

Multi-Scale-Matching neural networks for thin plate bending problem

Physics-informed neural networks are a useful machine learning method for solving differential equations,but encounter challenges in effectively learning thin boundary layers within singular perturbation problems.To resolve this issue,multi-scale-matching neural networks are proposed to solve the singular perturbation problems.Inspired by matched asymptotic expansions,the solution is decomposed into inner solutions for small scales and outer solutions for large scales,corresponding to boundary layers and outer regions,respectively.Moreover,to conform neural networks,we introduce exponential stretched variables in the boundary layers to avoid semiinfinite region problems.Numerical results for the thin plate problem validate the proposed method.

Multi-scale physics-informed neural networks for solving high Reynolds number boundary layer flows based on matched asymptotic expansions

Multi-scale system remains a classical scientific problem in fluid dynamics,biology,etc.In the present study,a scheme of multi-scale Physics-informed neural networks is proposed to solve the boundary layer flow at high Reynolds numbers without any data.The flow is divided into several regions with different scales based on Prandtl's boundary theory.Different regions are solved with governing equations in different scales.The method of matched asymptotic expansions is used to make the flow field continuously.A flow on a semi infinite flat plate at a high Reynolds number is considered a multi-scale problem because the boundary layer scale is much smaller than the outer flow scale.The results are compared with the reference numerical solutions,which show that the msPINNs can solve the multi-scale problem of the boundary layer in high Reynolds number flows.This scheme can be developed for more multi-scale problems in the future.

Intelligent geochemical interpretation of mass chromatograms:Based on convolution neural network

Gas chromatography-mass spectrometry(GC-MS)is an extremely important analytical technique that is widely used in organic geochemistry.It is the only approach to capture biomarker features of organic matter and provides the key evidence for oil-source correlation and thermal maturity determination.However,the conventional way of processing and interpreting the mass chromatogram is both timeconsuming and labor-intensive,which increases the research cost and restrains extensive applications of this method.To overcome this limitation,a correlation model is developed based on the convolution neural network(CNN)to link the mass chromatogram and biomarker features of samples from the Triassic Yanchang Formation,Ordos Basin,China.In this way,the mass chromatogram can be automatically interpreted.This research first performs dimensionality reduction for 15 biomarker parameters via the factor analysis and then quantifies the biomarker features using two indexes(i.e.MI and PMI)that represent the organic matter thermal maturity and parent material type,respectively.Subsequently,training,interpretation,and validation are performed multiple times using different CNN models to optimize the model structure and hyper-parameter setting,with the mass chromatogram used as the input and the obtained MI and PMI values for supervision(label).The optimized model presents high accuracy in automatically interpreting the mass chromatogram,with R2values typically above 0.85 and0.80 for the thermal maturity and parent material interpretation results,respectively.The significance of this research is twofold:(i)developing an efficient technique for geochemical research;(ii)more importantly,demonstrating the potential of artificial intelligence in organic geochemistry and providing vital references for future related studies.

A multiscale adaptive framework based on convolutional neural network:Application to fluid catalytic cracking product yield prediction

Since chemical processes are highly non-linear and multiscale,it is vital to deeply mine the multiscale coupling relationships embedded in the massive process data for the prediction and anomaly tracing of crucial process parameters and production indicators.While the integrated method of adaptive signal decomposition combined with time series models could effectively predict process variables,it does have limitations in capturing the high-frequency detail of the operation state when applied to complex chemical processes.In light of this,a novel Multiscale Multi-radius Multi-step Convolutional Neural Network(Msrt Net)is proposed for mining spatiotemporal multiscale information.First,the industrial data from the Fluid Catalytic Cracking(FCC)process decomposition using Complete Ensemble Empirical Mode Decomposition with Adaptive Noise(CEEMDAN)extract the multi-energy scale information of the feature subset.Then,convolution kernels with varying stride and padding structures are established to decouple the long-period operation process information encapsulated within the multi-energy scale data.Finally,a reconciliation network is trained to reconstruct the multiscale prediction results and obtain the final output.Msrt Net is initially assessed for its capability to untangle the spatiotemporal multiscale relationships among variables in the Tennessee Eastman Process(TEP).Subsequently,the performance of Msrt Net is evaluated in predicting product yield for a 2.80×10^(6) t/a FCC unit,taking diesel and gasoline yield as examples.In conclusion,Msrt Net can decouple and effectively extract spatiotemporal multiscale information from chemical process data and achieve a approximately reduction of 30%in prediction error compared to other time-series models.Furthermore,its robustness and transferability underscore its promising potential for broader applications.

Single event effects evaluation on convolution neural network in Xilinx 28 nm system on chip

Convolutional neural networks(CNNs) exhibit excellent performance in the areas of image recognition and object detection, which can enhance the intelligence level of spacecraft. However, in aerospace, energetic particles, such as heavy ions, protons, and alpha particles, can induce single event effects(SEEs) that lead CNNs to malfunction and can significantly impact the reliability of a CNN system. In this paper, the MNIST CNN system was constructed based on a 28 nm systemon-chip(SoC), and then an alpha particle irradiation experiment and fault injection were applied to evaluate the SEE of the CNN system. Various types of soft errors in the CNN system have been detected, and the SEE cross sections have been calculated. Furthermore, the mechanisms behind some soft errors have been explained. This research will provide technical support for the design of radiation-resistant artificial intelligence chips.