TCAS-PINN:Physics-informed neural networks with a novel temporal causality-based adaptive sampling method

Physics-informed neural networks(PINNs)have become an attractive machine learning framework for obtaining solutions to partial differential equations(PDEs).PINNs embed initial,boundary,and PDE constraints into the loss function.The performance of PINNs is generally affected by both training and sampling.Specifically,training methods focus on how to overcome the training difficulties caused by the special PDE residual loss of PINNs,and sampling methods are concerned with the location and distribution of the sampling points upon which evaluations of PDE residual loss are accomplished.However,a common problem among these original PINNs is that they omit special temporal information utilization during the training or sampling stages when dealing with an important PDE category,namely,time-dependent PDEs,where temporal information plays a key role in the algorithms used.There is one method,called Causal PINN,that considers temporal causality at the training level but not special temporal utilization at the sampling level.Incorporating temporal knowledge into sampling remains to be studied.To fill this gap,we propose a novel temporal causality-based adaptive sampling method that dynamically determines the sampling ratio according to both PDE residual and temporal causality.By designing a sampling ratio determined by both residual loss and temporal causality to control the number and location of sampled points in each temporal sub-domain,we provide a practical solution by incorporating temporal information into sampling.Numerical experiments of several nonlinear time-dependent PDEs,including the Cahn–Hilliard,Korteweg–de Vries,Allen–Cahn and wave equations,show that our proposed sampling method can improve the performance.We demonstrate that using such a relatively simple sampling method can improve prediction performance by up to two orders of magnitude compared with the results from other methods,especially when points are limited.

A Fractional-Order Ultra-Local Model-Based Adaptive Neural Network Sliding Mode Control of n-DOF Upper-Limb Exoskeleton With Input Deadzone

This paper proposes an adaptive neural network sliding mode control based on fractional-order ultra-local model for n-DOF upper-limb exoskeleton in presence of uncertainties,external disturbances and input deadzone.Considering the model complexity and input deadzone,a fractional-order ultra-local model is proposed to formulate the original dynamic system for simple controller design.Firstly,the control gain of ultra-local model is considered as a constant.The fractional-order sliding mode technique is designed to stabilize the closed-loop system,while fractional-order time-delay estimation is combined with neural network to estimate the lumped disturbance.Correspondingly,a fractional-order ultra-local model-based neural network sliding mode controller(FO-NNSMC) is proposed.Secondly,to avoid disadvantageous effect of improper gain selection on the control performance,the control gain of ultra-local model is considered as an unknown parameter.Then,the Nussbaum technique is introduced into the FO-NNSMC to deal with the stability problem with unknown gain.Correspondingly,a fractional-order ultra-local model-based adaptive neural network sliding mode controller(FO-ANNSMC) is proposed.Moreover,the stability analysis of the closed-loop system with the proposed method is presented by using the Lyapunov theory.Finally,with the co-simulations on virtual prototype of 7-DOF iReHave upper-limb exoskeleton and experiments on 2-DOF upper-limb exoskeleton,the obtained compared results illustrate the effectiveness and superiority of the proposed method.

Noise-tolerate and adaptive coefficient zeroing neural network for solving dynamic matrix square root

The solving of dynamic matrix square root(DMSR)problems is frequently encountered in many scientific and engineering fields.Although the original zeroing neural network is powerful for solving the DMSR,it cannot vanish the influence of the noise perturbations,and its constant-coefficient design scheme cannot accelerate the convergence speed.Therefore,a noise-tolerate and adaptive coefficient zeroing neural network(NTACZNN)is raised to enhance the robust noise immunity performance and accelerate the conver-gence speed simultaneously.Then,the global convergence and robustness of the pro-posed NTACZNN are theoretically analysed under an ideal environment and noise-perturbed circumstances.Furthermore,some illustrative simulation examples are designed and performed in order to substantiate the efficacy and advantage of the NTACZNN for the DMSR problem solution.Compared with some existing ZNNs,the proposed NTACZNN possesses advanced performance in terms of noise tolerance,solution accuracy,and convergence rate.

Solar Radiation Estimation Based on a New Combined Approach of Artificial Neural Networks (ANN) and Genetic Algorithms (GA) in South Algeria

When designing solar systems and assessing the effectiveness of their many uses,estimating sun irradiance is a crucial first step.This study examined three approaches(ANN,GA-ANN,and ANFIS)for estimating daily global solar radiation(GSR)in the south of Algeria:Adrar,Ouargla,and Bechar.The proposed hybrid GA-ANN model,based on genetic algorithm-based optimization,was developed to improve the ANN model.The GA-ANN and ANFIS models performed better than the standalone ANN-based model,with GA-ANN being better suited for forecasting in all sites,and it performed the best with the best values in the testing phase of Coefficient of Determination(R=0.9005),Mean Absolute Percentage Error(MAPE=8.40%),and Relative Root Mean Square Error(rRMSE=12.56%).Nevertheless,the ANFIS model outperformed the GA-ANN model in forecasting daily GSR,with the best values of indicators when testing the model being R=0.9374,MAPE=7.78%,and rRMSE=10.54%.Generally,we may conclude that the initial ANN stand-alone model performance when forecasting solar radiation has been improved,and the results obtained after injecting the genetic algorithm into the ANN to optimize its weights were satisfactory.The model can be used to forecast daily GSR in dry climates and other climates and may also be helpful in selecting solar energy system installations and sizes.

A Spectral Convolutional Neural Network Model Based on Adaptive Fick’s Law for Hyperspectral Image Classification

Hyperspectral image classification stands as a pivotal task within the field of remote sensing,yet achieving highprecision classification remains a significant challenge.In response to this challenge,a Spectral Convolutional Neural Network model based on Adaptive Fick’s Law Algorithm(AFLA-SCNN)is proposed.The Adaptive Fick’s Law Algorithm(AFLA)constitutes a novel metaheuristic algorithm introduced herein,encompassing three new strategies:Adaptive weight factor,Gaussian mutation,and probability update policy.With adaptive weight factor,the algorithmcan adjust theweights according to the change in the number of iterations to improve the performance of the algorithm.Gaussianmutation helps the algorithm avoid falling into local optimal solutions and improves the searchability of the algorithm.The probability update strategy helps to improve the exploitability and adaptability of the algorithm.Within the AFLA-SCNN model,AFLA is employed to optimize two hyperparameters in the SCNN model,namely,“numEpochs”and“miniBatchSize”,to attain their optimal values.AFLA’s performance is initially validated across 28 functions in 10D,30D,and 50D for CEC2013 and 29 functions in 10D,30D,and 50D for CEC2017.Experimental results indicate AFLA’s marked performance superiority over nine other prominent optimization algorithms.Subsequently,the AFLA-SCNN model was compared with the Spectral Convolutional Neural Network model based on Fick’s Law Algorithm(FLA-SCNN),Spectral Convolutional Neural Network model based on Harris Hawks Optimization(HHO-SCNN),Spectral Convolutional Neural Network model based onDifferential Evolution(DE-SCNN),SpectralConvolutionalNeuralNetwork(SCNN)model,and SupportVector Machines(SVM)model using the Indian Pines dataset and PaviaUniversity dataset.The experimental results show that the AFLA-SCNN model outperforms other models in terms of Accuracy,Precision,Recall,and F1-score on Indian Pines and Pavia University.Among them,the Accuracy of the AFLA-SCNN model on Indian Pines reached 99.875%,and the Accuracy on PaviaUniversity reached 98.022%.In conclusion,our proposed AFLA-SCNN model is deemed to significantly enhance the precision of hyperspectral image classification.

Physics-informed neural networks with residual/gradient-based adaptive sampling methods for solving partial differential equations with sharp solutions

We consider solving the forward and inverse partial differential equations(PDEs)which have sharp solutions with physics-informed neural networks(PINNs)in this work.In particular,to better capture the sharpness of the solution,we propose the adaptive sampling methods(ASMs)based on the residual and the gradient of the solution.We first present a residual only-based ASM denoted by ASMⅠ.In this approach,we first train the neural network using a small number of residual points and divide the computational domain into a certain number of sub-domains,then we add new residual points in the sub-domain which has the largest mean absolute value of the residual,and those points which have the largest absolute values of the residual in this sub-domain as new residual points.We further develop a second type of ASM(denoted by ASMⅡ)based on both the residual and the gradient of the solution due to the fact that only the residual may not be able to efficiently capture the sharpness of the solution.The procedure of ASMⅡis almost the same as that of ASMⅠ,and we add new residual points which have not only large residuals but also large gradients.To demonstrate the effectiveness of the present methods,we use both ASMⅠand ASMⅡto solve a number of PDEs,including the Burger equation,the compressible Euler equation,the Poisson equation over an Lshape domain as well as the high-dimensional Poisson equation.It has been shown from the numerical results that the sharp solutions can be well approximated by using either ASMⅠor ASMⅡ,and both methods deliver much more accurate solutions than the original PINNs with the same number of residual points.Moreover,the ASMⅡalgorithm has better performance in terms of accuracy,efficiency,and stability compared with the ASMⅠalgorithm.This means that the gradient of the solution improves the stability and efficiency of the adaptive sampling procedure as well as the accuracy of the solution.Furthermore,we also employ the similar adaptive sampling technique for the data points of boundary conditions(BCs)if the sharpness of the solution is near the boundary.The result of the L-shape Poisson problem indicates that the present method can significantly improve the efficiency,stability,and accuracy.

Adaptive Fixed-Time Synchronization of Delayed Memristor-Based Neural Networks with Discontinuous Activations

Fixed-time synchronization(FTS)of delayed memristor-based neural networks(MNNs)with discontinuous activations is studied in this paper.Both continuous and discontinuous activations are considered forMNNs.And the mixed delays which are closer to reality are taken into the system.Besides,two kinds of control schemes are proposed,including feedback and adaptive control strategies.Based on some lemmas,mathematical inequalities and the designed controllers,a few synchronization criteria are acquired.Moreover,the upper bound of settling time(ST)which is independent of the initial values is given.Finally,the feasibility of our theory is attested by simulation examples.

Scale adaptive fitness evaluation‐based particle swarm optimisation for hyperparameter and architecture optimisation in neural networks and deep learning

Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to two issues:Both the hyperparameter and ar-chitecture should be optimised and the optimisation process is computationally expen-sive.To tackle these two issues,this paper focusses on solving the hyperparameter and architecture optimization problem for the NN and proposes a novel light‐weight scale‐adaptive fitness evaluation‐based particle swarm optimisation(SAFE‐PSO)approach.Firstly,the SAFE‐PSO algorithm considers the hyperparameters and architectures together in the optimisation problem and therefore can find their optimal combination for the globally best NN.Secondly,the computational cost can be reduced by using multi‐scale accuracy evaluation methods to evaluate candidates.Thirdly,a stagnation‐based switch strategy is proposed to adaptively switch different evaluation methods to better balance the search performance and computational cost.The SAFE‐PSO algorithm is tested on two widely used datasets:The 10‐category(i.e.,CIFAR10)and the 100−cate-gory(i.e.,CIFAR100).The experimental results show that SAFE‐PSO is very effective and efficient,which can not only find a promising NN automatically but also find a better NN than compared algorithms at the same computational cost.

Adaptive Stochastic Synchronization of Uncertain Delayed Neural Networks

This paper considers adaptive synchronization of uncertain neural networks with time delays and stochastic perturbation. A general adaptive controller is designed to deal with the difficulties deduced by uncertain parameters and stochastic perturbations, in which the controller is less conservative and optimal since its control gains can be automatically adjusted according to some designed update laws. Based on Lyapunov stability theory and Barbalat lemma, sufficient condition is obtained for synchronization of delayed neural networks by strict mathematical proof. Moreover, the obtained results of this paper are more general than most existing results of certainly neural networks with or without stochastic disturbances. Finally, numerical simulations are presented to substantiate our theoretical results.

A Novel Approach to Energy Optimization:Efficient Path Selection in Wireless Sensor Networks with Hybrid ANN

In pursuit of enhancing the Wireless Sensor Networks(WSNs)energy efficiency and operational lifespan,this paper delves into the domain of energy-efficient routing protocols.InWSNs,the limited energy resources of Sensor Nodes(SNs)are a big challenge for ensuring their efficient and reliable operation.WSN data gathering involves the utilization of a mobile sink(MS)to mitigate the energy consumption problem through periodic network traversal.The mobile sink(MS)strategy minimizes energy consumption and latency by visiting the fewest nodes or predetermined locations called rendezvous points(RPs)instead of all cluster heads(CHs).CHs subsequently transmit packets to neighboring RPs.The unique determination of this study is the shortest path to reach RPs.As the mobile sink(MS)concept has emerged as a promising solution to the energy consumption problem in WSNs,caused by multi-hop data collection with static sinks.In this study,we proposed two novel hybrid algorithms,namely“ Reduced k-means based on Artificial Neural Network”(RkM-ANN)and“Delay Bound Reduced kmeans with ANN”(DBRkM-ANN)for designing a fast,efficient,and most proficient MS path depending upon rendezvous points(RPs).The first algorithm optimizes the MS’s latency,while the second considers the designing of delay-bound paths,also defined as the number of paths with delay over bound for the MS.Both methods use a weight function and k-means clustering to choose RPs in a way that maximizes efficiency and guarantees network-wide coverage.In addition,a method of using MS scheduling for efficient data collection is provided.Extensive simulations and comparisons to several existing algorithms have shown the effectiveness of the suggested methodologies over a wide range of performance indicators.

Adaptive Butterfly Optimization Algorithm(ABOA)Based Feature Selection and Deep Neural Network(DNN)for Detection of Distributed Denial-of-Service(DDoS)Attacks in Cloud

Cloud computing technology provides flexible,on-demand,and completely controlled computing resources and services are highly desirable.Despite this,with its distributed and dynamic nature and shortcomings in virtualization deployment,the cloud environment is exposed to a wide variety of cyber-attacks and security difficulties.The Intrusion Detection System(IDS)is a specialized security tool that network professionals use for the safety and security of the networks against attacks launched from various sources.DDoS attacks are becoming more frequent and powerful,and their attack pathways are continually changing,which requiring the development of new detection methods.Here the purpose of the study is to improve detection accuracy.Feature Selection(FS)is critical.At the same time,the IDS’s computational problem is limited by focusing on the most relevant elements,and its performance and accuracy increase.In this research work,the suggested Adaptive butterfly optimization algorithm(ABOA)framework is used to assess the effectiveness of a reduced feature subset during the feature selection phase,that was motivated by this motive Candidates.Accurate classification is not compromised by using an ABOA technique.The design of Deep Neural Networks(DNN)has simplified the categorization of network traffic into normal and DDoS threat traffic.DNN’s parameters can be finetuned to detect DDoS attacks better using specially built algorithms.Reduced reconstruction error,no exploding or vanishing gradients,and reduced network are all benefits of the changes outlined in this paper.When it comes to performance criteria like accuracy,precision,recall,and F1-Score are the performance measures that show the suggested architecture outperforms the other existing approaches.Hence the proposed ABOA+DNN is an excellent method for obtaining accurate predictions,with an improved accuracy rate of 99.05%compared to other existing approaches.

Adaptive recurrent neural network for uncertainties estimation in feedback control system

In this paper,a recurrent neural network(RNN)is used to estimate uncertainties and implement feedback control for nonlinear dynamic systems.The neural network approximates the uncertainties related to unmodeled dynamics,parametric variations,and external disturbances.The RNN has a single hidden layer and uses the tracking error and the output as feedback to estimate the disturbance.The RNN weights are online adapted,and the adaptation laws are developed from the stability analysis of the controlled system with the RNN estimation.The used activation function,at the hidden layer,has an expression that simplifies the adaptation laws from the stability analysis.It is found that the adaptive RNN enhances the tracking performance of the feedback controller at the transient and steady state responses.The proposed RNN based feedback control is applied to a DC–DC converter for current regulation.Simulation and experimental results are provided to show its effectiveness.Compared to the feedforward neural network and the conventional feedback control,the RNN based feedback control provides good tracking performance.

Adaptive Control Based on Neural Networks for an Uncertain 2-DOF Helicopter System With Input Deadzone and Output Constraints

In this paper, a study of control for an uncertain2-degree of freedom(DOF) helicopter system is given. The2-DOF helicopter is subject to input deadzone and output constraints. In order to cope with system uncertainties and input deadzone, the neural network technique is introduced because of its capability in approximation. In order to update the weights of the neural network, an adaptive control method is utilized to improve the system adaptability. Furthermore, the integral barrier Lyapunov function(IBLF) is adopt in control design to guarantee the condition of output constraints and boundedness of the corresponding tracking errors. The Lyapunov direct method is applied in the control design to analyze system stability and convergence. Finally, numerical simulations are conducted to prove the feasibility and effectiveness of the proposed control based on the model of Quanser's 2-DOF helicopter.

ADAPTIVE PINNING SYNCHRONIZATION OF COUPLED NEURAL NETWORKS WITH MIXED DELAYS AND VECTOR-FORM STOCHASTIC PERTURBATIONS

In this article, we consider the global chaotic synchronization of general cou- pled neural networks, in which subsystems have both discrete and distributed delays. Stochastic perturbations between subsystems are also considered. On the basis of two sim- ple adaptive pinning feedback control schemes, Lyapunov functional method, and stochas- tic analysis approach, several sufficient conditions are developed to guarantee global syn- chronization of the coupled neural networks with two kinds of delay couplings, even if only partial states of the nodes are coupled. The outer-coupling matrices may be symmetric or asymmetric. Unlike existing results that an isolate node is introduced as the pinning target, we pin to help the network realizing synchronization without introducing any iso- late node when the network is not synchronized. As a by product, sufficient conditions under which the network realizes synchronization without control are derived. Numerical simulations confirm the effectiveness of the obtained results.

Adaptive Backstepping Terminal Sliding Mode Control Method Based on Recurrent Neural Networks for Autonomous Underwater Vehicle

The trajectory tracking control problem is addressed for autonomous underwater vehicle(AUV) in marine environ?ment, with presence of the influence of the uncertain factors including ocean current disturbance, dynamic modeling uncertainty, and thrust model errors. To improve the trajectory tracking accuracy of AUV, an adaptive backstepping terminal sliding mode control based on recurrent neural networks(RNN) is proposed. Firstly, considering the inaccu?rate of thrust model of thruster, a Taylor’s polynomial is used to obtain the thrust model errors. And then, the dynamic modeling uncertainty and thrust model errors are combined into the system model uncertainty(SMU) of AUV; through the RNN, the SMU and ocean current disturbance are classified, approximated online. Finally, the weights of RNN and other control parameters are adjusted online based on the backstepping terminal sliding mode controller. In addition, a chattering?reduction method is proposed based on sigmoid function. In chattering?reduction method, the sigmoid function is used to realize the continuity of the sliding mode switching function, and the sliding mode switching gain is adjusted online based on the exponential form of the sliding mode function. Based on the Lyapu?nov theory and Barbalat’s lemma, it is theoretically proved that the AUV trajectory tracking error can quickly converge to zero in the finite time. This research proposes a trajectory tracking control method of AUV, which can e ectively achieve high?precision trajectory tracking control of AUV under the influence of the uncertain factors. The feasibility and e ectiveness of the proposed method is demonstrated with trajectory tracking simulations and pool?experi?ments of AUV.

Adaptive control of system with hysteresis using neural networks

An adaptive control scheme is developed for a class of single-input nonlinear systems preceded by unknown hysteresis, which is a non-differentiable and multi-value mapping nonlinearity. The controller based on the three-layer neural network （NN）, whose weights are derived from Lyapunov stability analysis, guarantees closed-loop semiglobal stability and convergence of the tracking errors to a small residual set. An example is used to confirm the effectiveness of the proposed control scheme.

Adaptive Neural Network Control with Control Allocation for A Manned Submersible in Deep Sea

This paper thoroughly studies a control system with control allocation for a manned submersible in deep sea being developed in China. The proposed control system consists of a neural-network-based direct adaptive controller and a dynamic control allocation module. A control energy cost function is used as the optimization criteria of the control allocation module, and weighted pseudo-inverse is used to find the solution of the control allocation problem. In the presence of bounded unknown disturbance and neural networks approximation error, stability of the closed-loop control system of manned submersible is proved with Lyaponov theory. The feasibility and validity of the proposed control system is further verified through experiments conducted on a semi-physical simulation platform for the manned submersible in deep sea.

Adaptive Learning with Large Variability of Teaching Signals for Neural Networks and Its Application to Motion Control of an Industrial Robot

Recently, various control methods represented by proportional-integral-derivative （PID） control are used for robotic control. To cope with the requirements for high response and precision, advanced feedforward controllers such as gravity compensator, Coriolis/centrifugal force compensator and friction compensators have been built in the controller. Generally, it causes heavy computational load when calculating the compensating value within a short sampling period. In this paper, integrated recurrent neural networks are applied as a feedforward controller for PUMA560 manipulator. The feedforward controller works instead of gravity and Coriolis/centrifugal force compensators. In the learning process of the neural network by using back propagation algorithm, the learning coefficient and gain of sigmoid function are tuned intuitively and empirically according to teaching signals. The tuning is complicated because it is being conducted by trial and error. Especially, when the scale of teaching signal is large, the problem becomes crucial. To cope with the problem which concerns the learning performance, a simple and adaptive learning technique for large scale teaching signals is proposed. The learning techniques and control effectiveness are evaluated through simulations using the dynamic model of PUMA560 manipulator.

Adaptive Backstepping Output Feedback Control for SISO Nonlinear System Using Fuzzy Neural Networks

In this paper, a new fuzzy-neural adaptive control approach is developed for a class of single-input and single-output （SISO） nonlinear systems with unmeasured states. Using fuzzy neural networks to approximate the unknown nonlinear functions, a fuzzy- neural adaptive observer is introduced for state estimation as well as system identification. Under the framework of the backstepping design, fuzzy-neural adaptive output feedback control is constructed recursively. It is proven that the proposed fuzzy adaptive control approach guarantees the global boundedness property for all the signals, driving the tracking error to a small neighbordhood of the origin. Simulation example is included to illustrate the effectiveness of the proposed approach.

Adaptive synchronization in an array of asymmetric coupled neural networks

This paper investigates the global synchronization in an array of linearly coupled neural networks with constant and delayed coupling. By a simple combination of adaptive control and linear feedback with the updated laws, some sufficient conditions are derived for global synchronization of the coupled neural networks. The coupling configuration matrix is assumed to be asymmetric, which is more coincident with the realistic network. It is shown that the approaches developed here extend and improve the earlier works. Finally, numerical simulations are presented to demonstrate the effectiveness of the theoretical results.