Multi-head neural networks for simulating particle breakage dynamics

The breakage of brittle particulate materials into smaller particles under compressive or impact loads can be modelled as an instantiation of the population balance integro-differential equation.In this paper,the emerging computational science paradigm of physics-informed neural networks is studied for the first time for solving both linear and nonlinear variants of the governing dynamics.Unlike conventional methods,the proposed neural network provides rapid simulations of arbitrarily high resolution in particle size,predicting values on arbitrarily fine grids without the need for model retraining.The network is assigned a simple multi-head architecture tailored to uphold monotonicity of the modelled cumulative distribution function over particle sizes.The method is theoretically analyzed and validated against analytical results before being applied to real-world data of a batch grinding mill.The agreement between laboratory data and numerical simulation encourages the use of physics-informed neural nets for optimal planning and control of industrial comminution processes.

A Neural-network-based Alternative Scheme to Include Nonhydrostatic Processes in an Atmospheric Dynamical Core

Here,a nonhydrostatic alternative scheme(NAS)is proposed for the grey zone where the nonhydrostatic impact on the atmosphere is evident but not large enough to justify the necessity to include an implicit nonhydrostatic solver in an atmospheric dynamical core.The NAS is designed to replace this solver,which can be incorporated into any hydrostatic models so that existing well-developed hydrostatic models can effectively serve for a longer time.Recent advances in machine learning(ML)provide a potential tool for capturing the main complicated nonlinear-nonhydrostatic relationship.In this study,an ML approach called a neural network(NN)was adopted to select leading input features and develop the NAS.The NNs were trained and evaluated with 12-day simulation results of dry baroclinic-wave tests by the Weather Research and Forecasting(WRF)model.The forward time difference of the nonhydrostatic tendency was used as the target variable,and the five selected features were the nonhydrostatic tendency at the last time step,and four hydrostatic variables at the current step including geopotential height,pressure in two different forms,and potential temperature,respectively.Finally,a practical NAS was developed with these features and trained layer by layer at a 20-km horizontal resolution,which can accurately reproduce the temporal variation and vertical distribution of the nonhydrostatic tendency.Corrected by the NN-based NAS,the improved hydrostatic solver at different horizontal resolutions can run stably for at least one month and effectively reduce most of the nonhydrostatic errors in terms of system bias,anomaly root-mean-square error,and the error of the wave spatial pattern,which proves the feasibility and superiority of this scheme.

Dynamic interwell connectivity analysis of multi-layer waterflooding reservoirs based on an improved graph neural network

The analysis of interwell connectivity plays an important role in the formulation of oilfield development plans and the description of residual oil distribution. In fact, sandstone reservoirs in China's onshore oilfields generally have the characteristics of thin and many layers, so multi-layer joint production is usually adopted. It remains a challenge to ensure the accuracy of splitting and dynamic connectivity in each layer of the injection-production wells with limited field data. The three-dimensional well pattern of multi-layer reservoir and the relationship between injection-production wells can be equivalent to a directional heterogeneous graph. In this paper, an improved graph neural network is proposed to construct an interacting process mimics the real interwell flow regularity. In detail, this method is used to split injection and production rates by combining permeability, porosity and effective thickness, and to invert the dynamic connectivity in each layer of the injection-production wells by attention mechanism.Based on the material balance and physical information, the overall connectivity from the injection wells,through the water injection layers to the production layers and the output of final production wells is established. Meanwhile, the change of well pattern caused by perforation, plugging and switching of wells at different times is achieved by updated graph structure in spatial and temporal ways. The effectiveness of the method is verified by a combination of reservoir numerical simulation examples and field example. The method corresponds to the actual situation of the reservoir, has wide adaptability and low cost, has good practical value, and provides a reference for adjusting the injection-production relationship of the reservoir and the development of the remaining oil.

Study on Ecological Change Remote Sensing Monitoring Method Based on Elman Dynamic Recurrent Neural Network

In this paper, Hailin City of Heilongjiang Province, China is taken as the research area. As an important city in Heilongjiang Province, China, the sustainable development of its ecological environment is related to the opening up, economic prosperity and social stability of Northeast China. In this paper, the remote sensing ecological index (RSEI) of Hailin City in recent 20 years was calculated by using Landsat 5/8/9 series satellite images, and the temporal and spatial changes of the ecological environment in Hailin City were further analyzed and the influencing factors were discussed. From 2003 to 2023, the mean value of RSEI in Hailin City decreased and increased, and the ecological environment decreased slightly as a whole. RSEI declined most significantly from 2003 to 2008, and it increased from 2008 to 2013, decreased from 2013 to 2018, and increased from 2018 to 2023 again, with higher RSEI value in the south and lower RSEI value in the northwest. It is suggested to appropriately increase vegetation coverage in the northwest to improve ecological quality. As a result, the predicted value of Elman dynamic recurrent neural network model is consistent with the change trend of the mean value, and the prediction error converges quickly, which can accurately predict the ecological environment quality in the future study area.

Event-based nonfragile state estimation for memristive recurrent neural networks with stochastic cyber-attacks and sensor saturations

This paper addresses the issue of nonfragile state estimation for memristive recurrent neural networks with proportional delay and sensor saturations. In practical engineering, numerous unnecessary signals are transmitted to the estimator through the networks, which increases the burden of communication bandwidth. A dynamic event-triggered mechanism,instead of a static event-triggered mechanism, is employed to select useful data. By constructing a meaningful Lyapunov–Krasovskii functional, a delay-dependent criterion is derived in terms of linear matrix inequalities for ensuring the global asymptotic stability of the augmented system. In the end, two numerical simulations are employed to illustrate the feasibility and validity of the proposed theoretical results.

Multi-scale physics-informed neural networks for solving high Reynolds number boundary layer flows based on matched asymptotic expansions

Multi-scale system remains a classical scientific problem in fluid dynamics,biology,etc.In the present study,a scheme of multi-scale Physics-informed neural networks is proposed to solve the boundary layer flow at high Reynolds numbers without any data.The flow is divided into several regions with different scales based on Prandtl's boundary theory.Different regions are solved with governing equations in different scales.The method of matched asymptotic expansions is used to make the flow field continuously.A flow on a semi infinite flat plate at a high Reynolds number is considered a multi-scale problem because the boundary layer scale is much smaller than the outer flow scale.The results are compared with the reference numerical solutions,which show that the msPINNs can solve the multi-scale problem of the boundary layer in high Reynolds number flows.This scheme can be developed for more multi-scale problems in the future.

Large-scale self-normalizing neural networks

Self-normalizing neural networks(SNN)regulate the activation and gradient flows through activation functions with the self-normalization property.As SNNs do not rely on norms computed from minibatches,they are more friendly to data parallelism,kernel fusion,and emerging architectures such as ReRAM-based accelerators.However,existing SNNs have mainly demonstrated their effectiveness on toy datasets and fall short in accuracy when dealing with large-scale tasks like ImageNet.They lack the strong normalization,regularization,and expression power required for wider,deeper models and larger-scale tasks.To enhance the normalization strength,this paper introduces a comprehensive and practical definition of the self-normalization property in terms of the stability and attractiveness of the statistical fixed points.It is comprehensive as it jointly considers all the fixed points used by existing studies:the first and second moment of forward activation and the expected Frobenius norm of backward gradient.The practicality comes from the analytical equations provided by our paper to assess the stability and attractiveness of each fixed point,which are derived from theoretical analysis of the forward and backward signals.The proposed definition is applied to a meta activation function inspired by prior research,leading to a stronger self-normalizing activation function named‘‘bi-scaled exponential linear unit with backward standardized’’(bSELU-BSTD).We provide both theoretical and empirical evidence to show that it is superior to existing studies.To enhance the regularization and expression power,we further propose scaled-Mixup and channel-wise scale&shift.With these three techniques,our approach achieves 75.23%top-1 accuracy on the ImageNet with Conv MobileNet V1,surpassing the performance of existing self-normalizing activation functions.To the best of our knowledge,this is the first SNN that achieves comparable accuracy to batch normalization on ImageNet.

The dynamic relaxation form finding method aided with advanced recurrent neural network

How to establish a self‐equilibrium configuration is vital for further kinematics and dynamics analyses of tensegrity mechanism.In this study,for investigating tensegrity form‐finding problems,a concise and efficient dynamic relaxation‐noise tolerant zeroing neural network(DR‐NTZNN)form‐finding algorithm is established through analysing the physical properties of tensegrity structures.In addition,the non‐linear constrained opti-misation problem which transformed from the form‐finding problem is solved by a sequential quadratic programming algorithm.Moreover,the noise may produce in the form‐finding process that includes the round‐off errors which are brought by the approximate matrix and restart point calculating course,disturbance caused by external force and manufacturing error when constructing a tensegrity structure.Hence,for the purpose of suppressing the noise,a noise tolerant zeroing neural network is presented to solve the search direction,which can endow the anti‐noise capability to the form‐finding model and enhance the calculation capability.Besides,the dynamic relaxation method is contributed to seek the nodal coordinates rapidly when the search direction is acquired.The numerical results show the form‐finding model has a huge capability for high‐dimensional free form cable‐strut mechanisms with complicated topology.Eventually,comparing with other existing form‐finding methods,the contrast simulations reveal the excellent anti‐noise performance and calculation capacity of DR‐NTZNN form‐finding algorithm.

Fractional Order Nonlinear Bone Remodeling Dynamics Using the Supervised Neural Network

This study aims to solve the nonlinear fractional-order mathematical model(FOMM)by using the normal and dysregulated bone remodeling of themyeloma bone disease(MBD).For themore precise performance of the model,fractional-order derivatives have been used to solve the disease model numerically.The FOMM is preliminarily designed to focus on the critical interactions between bone resorption or osteoclasts(OC)and bone formation or osteoblasts(OB).The connections of OC and OB are represented by a nonlinear differential system based on the cellular components,which depict stable fluctuation in the usual bone case and unstable fluctuation through the MBD.Untreated myeloma causes by increasing the OC and reducing the osteoblasts,resulting in net bone waste the tumor growth.The solutions of the FOMM will be provided by using the stochastic framework based on the Levenberg-Marquardt backpropagation(LVMBP)neural networks(NN),i.e.,LVMBPNN.The mathematical performances of three variations of the fractional-order derivative based on the nonlinear disease model using the LVMPNN.The static structural performances are 82%for investigation and 9%for both learning and certification.The performances of the LVMBPNN are authenticated by using the results of the Adams-Bashforth-Moulton mechanism.To accomplish the capability,steadiness,accuracy,and ability of the LVMBPNN,the performances of the error histograms(EHs),mean square error(MSE),recurrence,and state transitions(STs)will be provided.

Investigation on Dynamics of Echo State Networks

Dynamics is a key issue about understanding recurrent neural networks(RNNs).Because of the complexity,the problem still remains unanswered in spite of many important progresses.Echo state network(ESN)is a simple approach to design RNNs.It is possible to investigate ESNs’dynamics deeply.However,most of dynamic studies have mainly concentrated on the shallow ESNs and seldom of them explain the dynamics of the deep ones.Therefore,this paper investigates the dynamics of four typical ESNs under a unified theoretical framework.These ESNs contain both the shallow versions and the deep ones.This investigation is helpful to clarify the dynamics of ESNs in a general sense.Also,the short-term memory(STM)of different ESNs is analyzed,which is closely related to the dynamics.This analysis is helpful to determine the hyper-parameters of ESNs for given problems.In addition,the problem-solving abilities of ESNs are investigated through modeling two time series tasks.It further explains the influence of the dynamics on ESN’s performance.

Systematic Evaluation of Deep Neural Network based Dynamic Modeling Method for AC Power Electronic System

Since the high penetration of renewable energy complicates the dynamic characteristics of the AC power electronic system(ACPES),it is essential to establish an accurate dynamic model to obtain its dynamic behavior for ensure the safe and stable operation of the system.However,due to the no or limited internal control details,the state-space modeling method cannot be realized.It leads to the ACPES system becoming a black-box dynamic system.The dynamic modeling method based on deep neural network can simulate the dynamic behavior using port data without obtaining internal control details.However,deep neural network modeling methods are rarely systematically evaluated.In practice,the construction of neural network faces the selection of massive data and various network structure parameters.However,different sample distributions make the trained network performance quite different.Different network structure hyperparameters also mean different convergence time.Due to the lack of systematic evaluation and targeted suggestions,neural network modeling with high precision and high training speed cannot be realized quickly and conveniently in practical engineering applications.To fill this gap,this paper systematically evaluates the deep neural network from sample distribution and structural hyperparameter selection.The influence on modeling accuracy is analyzed in detail,then some modeling suggestions are presented.Simulation results under multiple operating points verify the effectiveness of the proposed method.

Nonlinear Dynamics and Stability of Neural Networks with Delay-Time

In this paper we study the dynamic properties and stabilities of neural networks with delay-time (which includes the time-varying case) by differential inequalities and Lyapunov function approaches. The criteria of connective stability, robust stability, Lyapunov stability, asymptotic atability, exponential stability and Lagrange stability of neural networks with delay-time are established, and the results obtained are very useful for the design, implementation and application of adaptive learning neural networks.

Application of artificial neural networks for predicting the impact of rolling dynamic compaction using dynamic cone penetrometer test results

Rolling dynamic compaction(RDC),which involves the towing of a noncircular module,is now widespread and accepted among many other soil compaction methods.However,to date,there is no accurate method for reliable prediction of the densification of soil and the extent of ground improvement by means of RDC.This study presents the application of artificial neural networks(ANNs) for a priori prediction of the effectiveness of RDC.The models are trained with in situ dynamic cone penetration(DCP) test data obtained from previous civil projects associated with the 4-sided impact roller.The predictions from the ANN models are in good agreement with the measured field data,as indicated by the model correlation coefficient of approximately 0.8.It is concluded that the ANN models developed in this study can be successfully employed to provide more accurate prediction of the performance of the RDC on a range of soil types.

Predicting uniaxial compressive strength of serpentinites through physical,dynamic and mechanical properties using neural networks

The uniaxial compressive strength(UCS)of intact rock is one of the most important parameters required and determined for rock mechanics studies in engineering projects.The limitations and difficulty of conducting tests on rocks,specifically on thinly bedded,highly fractured,highly porous and weak rocks,as well as the fact that these tests are destructive,expensive and time-consuming,lead to development of soft computing-based techniques.Application of artificial neural networks(ANNs)for predicting UCS has become an attractive alternative for geotechnical engineering scientists.In this study,an ANN was designed with the aim of indirectly predicting UCS through the serpentinization percentage,and physical,dynamic and mechanical characteristics of serpentinites.For this purpose,data obtained in earlier experimental work from central Greece were used.The ANN-based results were compared with the experimental ones and those obtained from previous analysis.The proposed ANN-based formula was found to be very efficient in predicting UCS values and the samples could be classified with simple physical,dynamic and mechanical tests,thus the expensive,difficult,time-consuming and destructive mechanical tests could be avoided.

DYNAMIC PROPERTIES OF CONTINUOUS ASYMMETRIC NEURAL NETWORKS

The dynamic properties of continuous asymmetric neural networks are discussed in this paper. The condition in the existence of unique equilibrium point is obtained. It is also dealt with the conditions in not producing static bifurcation and Hopf’s bifurcation and is put forward the sufficient conditions for overall asymptotic stability and exponential stability.

STUDIES OF THE DYNAMIC BEHAVIORS OF A CLASS OF LEARNING ASSOCIATIVE NEURAL NETWORKS

This paper investigates exponential stability and trajectory bounds of motions of equilibria of a class of associative neural networks under structural variations as learning a new pattern. Some conditions for the possible maximum estimate of the domain of structural exponential stability are determined. The filtering ability of the associative neural networks contaminated by input noises is analyzed. Employing the obtained results as valuable guidelines, a systematic synthesis procedure for constructing a dynamical associative neural network that stores a given set of vectors as the stable equilibrium points as well as learns new patterns can be developed. Some new concepts defined here are expected to be the instruction for further studies of learning associative neural networks.

Research of Dynamic Competitive Learning in Neural Networks

Introduce a method of generation of new units within a cluster and aalgorithm of generating new clusters. The model automatically builds up its dynamically growinginternal representation structure during the learning process. Comparing model with other typicalclassification algorithm such as the Kohonen's self-organizing map, the model realizes a multilevelclassification of the input pattern with an optional accuracy and gives a strong support possibilityfor the parallel computational main processor. The idea is suitable for the high-level storage ofcomplex datas structures for object recognition.

L_2-L_∞ learning of dynamic neural networks

This paper proposes an y2-y∞ learning law as a new learning method for dynamic neural networks with external disturbance. Based on linear matrix inequality （LMI） formulation, the y2-y∞ learning law is presented to not only guarantee asymptotical stability of dynamic neural networks but also reduce the effect of external disturbance to an y2-y∞ induced norm constraint. It is shown that the design of the y2-y∞ learning law for such neural networks can be achieved by solving LMIs, which can be easily facilitated by using some standard numerical packages. A numerical example is presented to demonstrate the validity of the proposed learning law.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Dynamic Bandwidth Allocation Technique in ATM Networks Based on Fuzzy Neural Networks and Genetic Algorithm

ＤｙｎａｍｉｃＢａｎｄｗｉｄｔｈＡｌｏｃａｔｉｏｎＴｅｃｈｎｉｑｕｅｉｎＡＴＭＮｅｔｗｏｒｋｓＢａｓｅｄｏｎＦｕｚｙＮｅｕｒａｌＮｅｔｗｏｒｋｓａｎｄＧｅｎｅｔｉｃＡｌｇｏｒｉｔｈｍ①ＺｈａｎｇＬｉａｎｇｊｉｅＬｉＹａｎｄａＷａｎｇＰｕ（Ｄｅｐｔｏｆ...