Pluggable multitask diffractive neural networks based on cascaded metasurfaces

Optical neural networks have significant advantages in terms of power consumption,parallelism,and high computing speed,which has intrigued extensive attention in both academic and engineering communities.It has been considered as one of the powerful tools in promoting the fields of imaging processing and object recognition.However,the existing optical system architecture cannot be reconstructed to the realization of multi-functional artificial intelligence systems simultaneously.To push the development of this issue,we propose the pluggable diffractive neural networks(P-DNN),a general paradigm resorting to the cascaded metasurfaces,which can be applied to recognize various tasks by switching internal plug-ins.As the proof-of-principle,the recognition functions of six types of handwritten digits and six types of fashions are numerical simulated and experimental demonstrated at near-infrared regimes.Encouragingly,the proposed paradigm not only improves the flexibility of the optical neural networks but paves the new route for achieving high-speed,low-power and versatile artificial intelligence systems.

A data-driven model of drop size prediction based on artificial neural networks using small-scale data sets

An artificial neural network(ANN)method is introduced to predict drop size in two kinds of pulsed columns with small-scale data sets.After training,the deviation between calculate and experimental results are 3.8%and 9.3%,respectively.Through ANN model,the influence of interfacial tension and pulsation intensity on the droplet diameter has been developed.Droplet size gradually increases with the increase of interfacial tension,and decreases with the increase of pulse intensity.It can be seen that the accuracy of ANN model in predicting droplet size outside the training set range is reach the same level as the accuracy of correlation obtained based on experiments within this range.For two kinds of columns,the drop size prediction deviations of ANN model are 9.6%and 18.5%and the deviations in correlations are 11%and 15%.

Hybrid model for BOF oxygen blowing time prediction based on oxygen balance mechanism and deep neural network

The amount of oxygen blown into the converter is one of the key parameters for the control of the converter blowing process,which directly affects the tap-to-tap time of converter. In this study, a hybrid model based on oxygen balance mechanism (OBM) and deep neural network (DNN) was established for predicting oxygen blowing time in converter. A three-step method was utilized in the hybrid model. First, the oxygen consumption volume was predicted by the OBM model and DNN model, respectively. Second, a more accurate oxygen consumption volume was obtained by integrating the OBM model and DNN model. Finally, the converter oxygen blowing time was calculated according to the oxygen consumption volume and the oxygen supply intensity of each heat. The proposed hybrid model was verified using the actual data collected from an integrated steel plant in China, and compared with multiple linear regression model, OBM model, and neural network model including extreme learning machine, back propagation neural network, and DNN. The test results indicate that the hybrid model with a network structure of 3 hidden layer layers, 32-16-8 neurons per hidden layer, and 0.1 learning rate has the best prediction accuracy and stronger generalization ability compared with other models. The predicted hit ratio of oxygen consumption volume within the error±300 m^(3)is 96.67%;determination coefficient (R^(2)) and root mean square error (RMSE) are0.6984 and 150.03 m^(3), respectively. The oxygen blow time prediction hit ratio within the error±0.6 min is 89.50%;R2and RMSE are0.9486 and 0.3592 min, respectively. As a result, the proposed model can effectively predict the oxygen consumption volume and oxygen blowing time in the converter.

Non-crossing Quantile Regression Neural Network as a Calibration Tool for Ensemble Weather Forecasts

Despite the maturity of ensemble numerical weather prediction(NWP),the resulting forecasts are still,more often than not,under-dispersed.As such,forecast calibration tools have become popular.Among those tools,quantile regression(QR)is highly competitive in terms of both flexibility and predictive performance.Nevertheless,a long-standing problem of QR is quantile crossing,which greatly limits the interpretability of QR-calibrated forecasts.On this point,this study proposes a non-crossing quantile regression neural network(NCQRNN),for calibrating ensemble NWP forecasts into a set of reliable quantile forecasts without crossing.The overarching design principle of NCQRNN is to add on top of the conventional QRNN structure another hidden layer,which imposes a non-decreasing mapping between the combined output from nodes of the last hidden layer to the nodes of the output layer,through a triangular weight matrix with positive entries.The empirical part of the work considers a solar irradiance case study,in which four years of ensemble irradiance forecasts at seven locations,issued by the European Centre for Medium-Range Weather Forecasts,are calibrated via NCQRNN,as well as via an eclectic mix of benchmarking models,ranging from the naïve climatology to the state-of-the-art deep-learning and other non-crossing models.Formal and stringent forecast verification suggests that the forecasts post-processed via NCQRNN attain the maximum sharpness subject to calibration,amongst all competitors.Furthermore,the proposed conception to resolve quantile crossing is remarkably simple yet general,and thus has broad applicability as it can be integrated with many shallow-and deep-learning-based neural networks.

Development of a convolutional neural network based geomechanical upscaling technique for heterogeneous geological reservoir

Geomechanical assessment using coupled reservoir-geomechanical simulation is becoming increasingly important for analyzing the potential geomechanical risks in subsurface geological developments.However,a robust and efficient geomechanical upscaling technique for heterogeneous geological reservoirs is lacking to advance the applications of three-dimensional(3D)reservoir-scale geomechanical simulation considering detailed geological heterogeneities.Here,we develop convolutional neural network(CNN)proxies that reproduce the anisotropic nonlinear geomechanical response caused by lithological heterogeneity,and compute upscaled geomechanical properties from CNN proxies.The CNN proxies are trained using a large dataset of randomly generated spatially correlated sand-shale realizations as inputs and simulation results of their macroscopic geomechanical response as outputs.The trained CNN models can provide the upscaled shear strength(R^(2)>0.949),stress-strain behavior(R^(2)>0.925),and volumetric strain changes(R^(2)>0.958)that highly agree with the numerical simulation results while saving over two orders of magnitude of computational time.This is a major advantage in computing the upscaled geomechanical properties directly from geological realizations without the need to perform local numerical simulations to obtain the geomechanical response.The proposed CNN proxybased upscaling technique has the ability to(1)bridge the gap between the fine-scale geocellular models considering geological uncertainties and computationally efficient geomechanical models used to assess the geomechanical risks of large-scale subsurface development,and(2)improve the efficiency of numerical upscaling techniques that rely on local numerical simulations,leading to significantly increased computational time for uncertainty quantification using numerous geological realizations.

Customized Convolutional Neural Network for Accurate Detection of Deep Fake Images in Video Collections

The motivation for this study is that the quality of deep fakes is constantly improving,which leads to the need to develop new methods for their detection.The proposed Customized Convolutional Neural Network method involves extracting structured data from video frames using facial landmark detection,which is then used as input to the CNN.The customized Convolutional Neural Network method is the date augmented-based CNN model to generate‘fake data’or‘fake images’.This study was carried out using Python and its libraries.We used 242 films from the dataset gathered by the Deep Fake Detection Challenge,of which 199 were made up and the remaining 53 were real.Ten seconds were allotted for each video.There were 318 videos used in all,199 of which were fake and 119 of which were real.Our proposedmethod achieved a testing accuracy of 91.47%,loss of 0.342,and AUC score of 0.92,outperforming two alternative approaches,CNN and MLP-CNN.Furthermore,our method succeeded in greater accuracy than contemporary models such as XceptionNet,Meso-4,EfficientNet-BO,MesoInception-4,VGG-16,and DST-Net.The novelty of this investigation is the development of a new Convolutional Neural Network(CNN)learning model that can accurately detect deep fake face photos.

Activation Redistribution Based Hybrid Asymmetric Quantization Method of Neural Networks

The demand for adopting neural networks in resource-constrained embedded devices is continuously increasing.Quantization is one of the most promising solutions to reduce computational cost and memory storage on embedded devices.In order to reduce the complexity and overhead of deploying neural networks on Integeronly hardware,most current quantization methods use a symmetric quantization mapping strategy to quantize a floating-point neural network into an integer network.However,although symmetric quantization has the advantage of easier implementation,it is sub-optimal for cases where the range could be skewed and not symmetric.This often comes at the cost of lower accuracy.This paper proposed an activation redistribution-based hybrid asymmetric quantizationmethod for neural networks.The proposedmethod takes data distribution into consideration and can resolve the contradiction between the quantization accuracy and the ease of implementation,balance the trade-off between clipping range and quantization resolution,and thus improve the accuracy of the quantized neural network.The experimental results indicate that the accuracy of the proposed method is 2.02%and 5.52%higher than the traditional symmetric quantization method for classification and detection tasks,respectively.The proposed method paves the way for computationally intensive neural network models to be deployed on devices with limited computing resources.Codes will be available on https://github.com/ycjcy/Hybrid-Asymmetric-Quantization.

Reliability analysis of slope stability by neural network,principal component analysis,and transfer learning techniques

The prediction of slope stability is considered as one of the critical concerns in geotechnical engineering.Conventional stochastic analysis with spatially variable slopes is time-consuming and highly computation-demanding.To assess the slope stability problems with a more desirable computational effort,many machine learning(ML)algorithms have been proposed.However,most ML-based techniques require that the training data must be in the same feature space and have the same distribution,and the model may need to be rebuilt when the spatial distribution changes.This paper presents a new ML-based algorithm,which combines the principal component analysis(PCA)-based neural network(NN)and transfer learning(TL)techniques(i.e.PCAeNNeTL)to conduct the stability analysis of slopes with different spatial distributions.The Monte Carlo coupled with finite element simulation is first conducted for data acquisition considering the spatial variability of cohesive strength or friction angle of soils from eight slopes with the same geometry.The PCA method is incorporated into the neural network algorithm(i.e.PCA-NN)to increase the computational efficiency by reducing the input variables.It is found that the PCA-NN algorithm performs well in improving the prediction of slope stability for a given slope in terms of the computational accuracy and computational effort when compared with the other two algorithms(i.e.NN and decision trees,DT).Furthermore,the PCAeNNeTL algorithm shows great potential in assessing the stability of slope even with fewer training data.

Multi-Scale-Matching neural networks for thin plate bending problem

Physics-informed neural networks are a useful machine learning method for solving differential equations,but encounter challenges in effectively learning thin boundary layers within singular perturbation problems.To resolve this issue,multi-scale-matching neural networks are proposed to solve the singular perturbation problems.Inspired by matched asymptotic expansions,the solution is decomposed into inner solutions for small scales and outer solutions for large scales,corresponding to boundary layers and outer regions,respectively.Moreover,to conform neural networks,we introduce exponential stretched variables in the boundary layers to avoid semiinfinite region problems.Numerical results for the thin plate problem validate the proposed method.

Multi-scale physics-informed neural networks for solving high Reynolds number boundary layer flows based on matched asymptotic expansions

Multi-scale system remains a classical scientific problem in fluid dynamics,biology,etc.In the present study,a scheme of multi-scale Physics-informed neural networks is proposed to solve the boundary layer flow at high Reynolds numbers without any data.The flow is divided into several regions with different scales based on Prandtl's boundary theory.Different regions are solved with governing equations in different scales.The method of matched asymptotic expansions is used to make the flow field continuously.A flow on a semi infinite flat plate at a high Reynolds number is considered a multi-scale problem because the boundary layer scale is much smaller than the outer flow scale.The results are compared with the reference numerical solutions,which show that the msPINNs can solve the multi-scale problem of the boundary layer in high Reynolds number flows.This scheme can be developed for more multi-scale problems in the future.

Finite-time Prescribed Performance Time-Varying Formation Control for Second-Order Multi-Agent Systems With Non-Strict Feedback Based on a Neural Network Observer

This paper studies the problem of time-varying formation control with finite-time prescribed performance for nonstrict feedback second-order multi-agent systems with unmeasured states and unknown nonlinearities.To eliminate nonlinearities,neural networks are applied to approximate the inherent dynamics of the system.In addition,due to the limitations of the actual working conditions,each follower agent can only obtain the locally measurable partial state information of the leader agent.To address this problem,a neural network state observer based on the leader state information is designed.Then,a finite-time prescribed performance adaptive output feedback control strategy is proposed by restricting the sliding mode surface to a prescribed region,which ensures that the closed-loop system has practical finite-time stability and that formation errors of the multi-agent systems converge to the prescribed performance bound in finite time.Finally,a numerical simulation is provided to demonstrate the practicality and effectiveness of the developed algorithm.

Trajectory tracking guidance of interceptor via prescribed performance integral sliding mode with neural network disturbance observer

This paper investigates interception missiles’trajectory tracking guidance problem under wind field and external disturbances in the boost phase.Indeed,the velocity control in such trajectory tracking guidance systems of missiles is challenging.As our contribution,the velocity control channel is designed to deal with the intractable velocity problem and improve tracking accuracy.The global prescribed performance function,which guarantees the tracking error within the set range and the global convergence of the tracking guidance system,is first proposed based on the traditional PPF.Then,a tracking guidance strategy is derived using the integral sliding mode control techniques to make the sliding manifold and tracking errors converge to zero and avoid singularities.Meanwhile,an improved switching control law is introduced into the designed tracking guidance algorithm to deal with the chattering problem.A back propagation neural network(BPNN)extended state observer(BPNNESO)is employed in the inner loop to identify disturbances.The obtained results indicate that the proposed tracking guidance approach achieves the trajectory tracking guidance objective without and with disturbances and outperforms the existing tracking guidance schemes with the lowest tracking errors,convergence times,and overshoots.

Predicting uniaxial compressive strength of tuff after accelerated freeze-thaw testing: Comparative analysis of regression models and artificial neural networks

Ignimbrites have been widely used as building materials in many historical and touristic structures in the Kayseri region of Türkiye. Their diverse colours and textures make them a popular choice for modern construction as well. However, ignimbrites are particularly vulnerable to atmospheric conditions, such as freeze-thaw cycles, due to their high porosity, which is a result of their formation process. When water enters the pores of the ignimbrites, it can freeze during cold weather. As the water freezes and expands, it generates internal stress within the stone, causing micro-cracks to develop. Over time, repeated freeze-thaw (F-T) cycles lead to the growth of these micro-cracks into larger cracks, compromising the structural integrity of the ignimbrites and eventually making them unsuitable for use as building materials. The determination of the long-term F-T performance of ignimbrites can be established after long F-T experimental processes. Determining the long-term F-T performance of ignimbrites typically requires extensive experimental testing over prolonged freeze-thaw cycles. To streamline this process, developing accurate predictive equations becomes crucial. In this study, such equations were formulated using classical regression analyses and artificial neural networks (ANN) based on data obtained from these experiments, allowing for the prediction of the F-T performance of ignimbrites and other similar building stones without the need for lengthy testing. In this study, uniaxial compressive strength, ultrasonic propagation velocity, apparent porosity and mass loss of ignimbrites after long-term F-T were determined. Following the F-T cycles, the disintegration rate was evaluated using decay function approaches, while uniaxial compressive strength (UCS) values were predicted with minimal input parameters through both regression and ANN analyses. The ANN and regression models created for this purpose were first started with a single input value and then developed with two and three combinations. The predictive performance of the models was assessed by comparing them to regression models using the coefficient of determination (R2) as the evaluation criterion. As a result of the study, higher R2 values (0.87) were obtained in models built with artificial neural network. The results of the study indicate that ANN usage can produce results close to experimental outcomes in predicting the long-term F-T performance of ignimbrite samples.

Assessment of Seasonal Rainfall Prediction in Ethiopia: Evaluating a Dynamic Recurrent Neural Network to Downscale ECMWF-SEAS5 Rainfall

Seasonal rainfall plays a vital role in both environmental dynamics and decision-making for rainfed agriculture in Ethiopia, a country often impacted by extreme climate events such as drought and flooding. Predicting the onset of the rainy season and providing localized rainfall forecasts for Ethiopia is challenging due to the changing spatiotemporal patterns and the country's rugged topography. The Climate Hazards Group Infra Red Precipitation with Station Data(CHIRPS), ERA5-Land total precipitation and temperature data are used from 1981–2022 to predict spatial rainfall by applying an artificial neural network(ANN). The recurrent neural network(RNN) is a nonlinear autoregressive network with exogenous input(NARX), which includes feed-forward connections and multiple network layers, employing the Levenberg Marquart algorithm. This method is applied to downscale data from the European Centre for Medium-range Weather Forecasts fifth-generation seasonal forecast system(ECMWF-SEAS5) and the Euro-Mediterranean Centre for Climate Change(CMCC) to the specific locations of rainfall stations in Ethiopia for the period 1980–2020. Across the stations, the results of NARX exhibit strong associations and reduced errors. The statistical results indicate that, except for the southwestern Ethiopian highlands, the downscaled monthly precipitation data exhibits high skill scores compared to the station records, demonstrating the effectiveness of the NARX approach for predicting local seasonal rainfall in Ethiopia's complex terrain. In addition to this spatial ANN of the summer season precipitation, temperature, as well as the combination of these two variables, show promising results.

Predicting microseismic,acoustic emission and electromagnetic radiation data using neural networks

Microseism,acoustic emission and electromagnetic radiation(M-A-E)data are usually used for predicting rockburst hazards.However,it is a great challenge to realize the prediction of M-A-E data.In this study,with the aid of a deep learning algorithm,a new method for the prediction of M-A-E data is proposed.In this method,an M-A-E data prediction model is built based on a variety of neural networks after analyzing numerous M-A-E data,and then the M-A-E data can be predicted.The predicted results are highly correlated with the real data collected in the field.Through field verification,the deep learning-based prediction method of M-A-E data provides quantitative prediction data for rockburst monitoring.

Improving the accuracy of mechanistic models for dynamic batch distillation enabled by neural network:An industrial plant case

Neural networks are often viewed as pure‘black box’models,lacking interpretability and extrapolation capabilities of pure mechanistic models.This work proposes a new approach that,with the help of neural networks,improves the conformity of the first-principal model to the actual plant.The final result is still a first-principal model rather than a hybrid model,which maintains the advantage of the high interpretability of first-principal model.This work better simulates industrial batch distillation which separates four components:water,ethylene glycol,diethylene glycol,and triethylene glycol.GRU(gated recurrent neural network)and LSTM(long short-term memory)were used to obtain empirical parameters of mechanistic model that are difficult to measure directly.These were used to improve the empirical processes in mechanistic model,thus correcting unreasonable model assumptions and achieving better predictability for batch distillation.The proposed method was verified using a case study from one industrial plant case,and the results show its advancement in improving model predictions and the potential to extend to other similar systems.

Regression Method for Rail Fastener Tightness Based on Center-Line Projection Distance Feature and Neural Network

In the railway system,fasteners have the functions of damping,maintaining the track distance,and adjusting the track level.Therefore,routine maintenance and inspection of fasteners are important to ensure the safe operation of track lines.Currently,assessment methods for fastener tightness include manual observation,acoustic wave detection,and image detection.There are limitations such as low accuracy and efficiency,easy interference and misjudgment,and a lack of accurate,stable,and fast detection methods.Aiming at the small deformation characteristics and large elastic change of fasteners from full loosening to full tightening,this study proposes high-precision surface-structured light technology for fastener detection and fastener deformation feature extraction based on the center-line projection distance and a fastener tightness regression method based on neural networks.First,the method uses a 3D camera to obtain a fastener point cloud and then segments the elastic rod area based on the iterative closest point algorithm registration.Principal component analysis is used to calculate the normal vector of the segmented elastic rod surface and extract the point on the centerline of the elastic rod.The point is projected onto the upper surface of the bolt to calculate the projection distance.Subsequently,the mapping relationship between the projection distance sequence and fastener tightness is established,and the influence of each parameter on the fastener tightness prediction is analyzed.Finally,by setting up a fastener detection scene in the track experimental base,collecting data,and completing the algorithm verification,the results showed that the deviation between the fastener tightness regression value obtained after the algorithm processing and the actual measured value RMSE was 0.2196 mm,which significantly improved the effect compared with other tightness detection methods,and realized an effective fastener tightness regression.

Dynamic interwell connectivity analysis of multi-layer waterflooding reservoirs based on an improved graph neural network

The analysis of interwell connectivity plays an important role in the formulation of oilfield development plans and the description of residual oil distribution. In fact, sandstone reservoirs in China's onshore oilfields generally have the characteristics of thin and many layers, so multi-layer joint production is usually adopted. It remains a challenge to ensure the accuracy of splitting and dynamic connectivity in each layer of the injection-production wells with limited field data. The three-dimensional well pattern of multi-layer reservoir and the relationship between injection-production wells can be equivalent to a directional heterogeneous graph. In this paper, an improved graph neural network is proposed to construct an interacting process mimics the real interwell flow regularity. In detail, this method is used to split injection and production rates by combining permeability, porosity and effective thickness, and to invert the dynamic connectivity in each layer of the injection-production wells by attention mechanism.Based on the material balance and physical information, the overall connectivity from the injection wells,through the water injection layers to the production layers and the output of final production wells is established. Meanwhile, the change of well pattern caused by perforation, plugging and switching of wells at different times is achieved by updated graph structure in spatial and temporal ways. The effectiveness of the method is verified by a combination of reservoir numerical simulation examples and field example. The method corresponds to the actual situation of the reservoir, has wide adaptability and low cost, has good practical value, and provides a reference for adjusting the injection-production relationship of the reservoir and the development of the remaining oil.

Prediction of the undrained shear strength of remolded soil with non-linear regression,fuzzy logic,and artificial neural network

This study aims to predict the undrained shear strength of remolded soil samples using non-linear regression analyses,fuzzy logic,and artificial neural network modeling.A total of 1306 undrained shear strength results from 230 different remolded soil test settings reported in 21 publications were collected,utilizing six different measurement devices.Although water content,plastic limit,and liquid limit were used as input parameters for fuzzy logic and artificial neural network modeling,liquidity index or water content ratio was considered as an input parameter for non-linear regression analyses.In non-linear regression analyses,12 different regression equations were derived for the prediction of undrained shear strength of remolded soil.Feed-Forward backpropagation and the TANSIG transfer function were used for artificial neural network modeling,while the Mamdani inference system was preferred with trapezoidal and triangular membership functions for fuzzy logic modeling.The experimental results of 914 tests were used for training of the artificial neural network models,196 for validation and 196 for testing.It was observed that the accuracy of the artificial neural network and fuzzy logic modeling was higher than that of the non-linear regression analyses.Furthermore,a simple and reliable regression equation was proposed for assessments of undrained shear strength values with higher coefficients of determination.

Network Security Enhanced with Deep Neural Network-Based Intrusion Detection System

This study describes improving network security by implementing and assessing an intrusion detection system(IDS)based on deep neural networks(DNNs).The paper investigates contemporary technical ways for enhancing intrusion detection performance,given the vital relevance of safeguarding computer networks against harmful activity.The DNN-based IDS is trained and validated by the model using the NSL-KDD dataset,a popular benchmark for IDS research.The model performs well in both the training and validation stages,with 91.30%training accuracy and 94.38%validation accuracy.Thus,the model shows good learning and generalization capabilities with minor losses of 0.22 in training and 0.1553 in validation.Furthermore,for both macro and micro averages across class 0(normal)and class 1(anomalous)data,the study evaluates the model using a variety of assessment measures,such as accuracy scores,precision,recall,and F1 scores.The macro-average recall is 0.9422,the macro-average precision is 0.9482,and the accuracy scores are 0.942.Furthermore,macro-averaged F1 scores of 0.9245 for class 1 and 0.9434 for class 0 demonstrate the model’s ability to precisely identify anomalies precisely.The research also highlights how real-time threat monitoring and enhanced resistance against new online attacks may be achieved byDNN-based intrusion detection systems,which can significantly improve network security.The study underscores the critical function ofDNN-based IDS in contemporary cybersecurity procedures by setting the foundation for further developments in this field.Upcoming research aims to enhance intrusion detection systems by examining cooperative learning techniques and integrating up-to-date threat knowledge.