Incorporating Lasso Regression to Physics-Informed Neural Network for Inverse PDE Problem

Partial Differential Equation(PDE)is among the most fundamental tools employed to model dynamic systems.Existing PDE modeling methods are typically derived from established knowledge and known phenomena,which are time-consuming and labor-intensive.Recently,discovering governing PDEs from collected actual data via Physics Informed Neural Networks(PINNs)provides a more efficient way to analyze fresh dynamic systems and establish PEDmodels.This study proposes Sequentially Threshold Least Squares-Lasso(STLasso),a module constructed by incorporating Lasso regression into the Sequentially Threshold Least Squares(STLS)algorithm,which can complete sparse regression of PDE coefficients with the constraints of l0 norm.It further introduces PINN-STLasso,a physics informed neural network combined with Lasso sparse regression,able to find underlying PDEs from data with reduced data requirements and better interpretability.In addition,this research conducts experiments on canonical inverse PDE problems and compares the results to several recent methods.The results demonstrated that the proposed PINN-STLasso outperforms other methods,achieving lower error rates even with less data.

Inverse Molecule Design with Invertible Neural Networks as Generative Models

Using neural networks for supervised learning means learning a function that maps input x to output y. However, in many applications, the inverse learning is also wanted, i.e., inferring y from x, which requires invertibility of the learning. Since the dimension of input is usually much higher than that of the output, there is information loss in the forward learning from input to output. Thus, creating invertible neural networks is a difficult task. However, recent development of invertible learning techniques such as normalizing flows has made invertible neural networks a reality. In this work, we applied flow-based invertible neural networks as generative models to inverse molecule design. In this context, the forward learning is to predict chemical properties given a molecule, and the inverse learning is to infer the molecules given the chemical properties. Trained on 100 and 1000 molecules, respectively, from a benchmark dataset QM9, our model identified novel molecules that had chemical property values well exceeding the limits of the training molecules as well as the limits of the whole QM9 of 133,885 molecules, moreover our generative model could easily sample many molecules (x values) from any one chemical property value (y value). Compared with the previous method in the literature that could only optimize one molecule for one chemical property value at a time, our model could be trained once and then be sampled any multiple times and for any chemical property values without the need of retraining. This advantage comes from treating inverse molecule design as an inverse regression problem. In summary, our main contributions were two: 1) our model could generalize well from the training data and was very data efficient, 2) our model could learn bidirectional correspondence between molecules and their chemical properties, thereby offering the ability to sample any number of molecules from any y values. In conclusion, our findings revealed the efficiency and effectiveness of using invertible neural networks as generative models in inverse molecule design.

Inverse Control of Nonlinear Servo System Based on Neural Networks

This paper presents an inverse controller based on neural networks for servo systems,and gives invertibility criterion for a class of nonlinear servo system.A feedforward multilayer neural network with closed-loop training schemes and Alopex weights learning algorithm is used to identify the inverse dynamic model of the servo system.The weights of the neural network is adjusted online by a feedback error learning method.Simulation results show that the method is effective in identification and control of servo system.

GENERALIZED INVERSE GROUP OF SIGNAL AND ITS IMPLEMENTATION WITH NEURAL NETWORKS

A new concept, the generalized inverse group (GIG) of signal, is firstly proposed and its properties, leaking coefficients and implementation with neural networks are presented. Theoretical analysis and computational simulation have shown that (1) there is a group of finite length of generalized inverse signals for any given finite signal, which forms the GIG; (2) each inverse group has different leaking coefficients, thus different abnormal states; (3) each GIG can be implemented by a grouped and improved single-layer perceptron which appears with fast convergence. When used in deconvolution, the proposed GIG can form a new parallel finite length of filtering deconvolution method. On off-line processing, the computational time is reduced to O(N) from O(N2). And the less the leaking coefficient is, the more reliable the deconvolution will be.

INVERSE KINEMATICS FOR A 6 DOF MANIPULATOR BASED ON NEURAL NETWORK

A methodology is presented whereby a neural network is used to learn the inverse kinematic relationships of the position and orientation of a six joint manipulator. The arm solution for the orientation of a manipulator using a self organizing neural net is studied in this paper. A new training model of the self organizing neural network is proposed by thoroughly studying Martinetz, Ritter and Schulten′s self organizing neural network based on Kohonen′s self organizing mapping algorithm using a Widrow Hoff type error correction rule and closely combining the characters of the inverse kinematic relationship for a robot arm. The computer simulation results for a PUMA 560 robot show that the proposed method has a significant improvement over other methods documented in the references in self organizing capability and precision by training process.

Nonlinear inversion of electrical resistivity imaging using pruning Bayesian neural networks

Conventional artificial neural networks used to solve electrical resistivity imaging （ERI） inversion problem suffer from overfitting and local minima. To solve these problems, we propose to use a pruning Bayesian neural network （PBNN） nonlinear inversion method and a sample design method based on the K-medoids clustering algorithm. In the sample design method, the training samples of the neural network are designed according to the prior information provided by the K-medoids clustering results; thus, the training process of the neural network is well guided. The proposed PBNN, based on Bayesian regularization, is used to select the hidden layer structure by assessing the effect of each hidden neuron to the inversion results. Then, the hyperparameter αk, which is based on the generalized mean, is chosen to guide the pruning process according to the prior distribution of the training samples under the small-sample condition. The proposed algorithm is more efficient than other common adaptive regularization methods in geophysics. The inversion of synthetic data and field data suggests that the proposed method suppresses the noise in the neural network training stage and enhances the generalization. The inversion results with the proposed method are better than those of the BPNN, RBFNN, and RRBFNN inversion methods as well as the conventional least squares inversion.

When linear inversion fails:Neural-network optimization for sparse-ray travel-time tomography of a volcanic edifice

In this study,we present an artificial neural network(ANN)-based approach for travel-time tomography of a volcanic edifice under sparse-ray coverage.We employ ray tracing to simulate the propagation of seismic waves through the heterogeneous medium of a volcanic edifice,and an inverse modeling algorithm that uses an ANN to estimate the velocity structure from the“observed”travel-time data.The performance of the approach is evaluated through a 2-dimensional numerical study that simulates i)an active source seismic experiment with a few(explosive)sources placed on one side of the edifice and a dense line of receivers placed on the other side,and ii)earthquakes located inside the edifice with receivers placed on both sides of the edifice.The results are compared with those obtained from conventional damped linear inversion.The average Root Mean Square Error(RMSE)between the input and output models is approximately 0.03 km/s for the ANN inversions,whereas it is about 0.4 km/s for the linear inversions,demonstrating that the ANN-based approach outperforms the classical approach,particularly in situations with sparse ray coverage.Our study emphasizes the advantages of employing a relatively simple ANN architecture in conjunction with second-order optimizers to minimize the loss function.Compared to using first-order optimizers,our ANN architecture shows a~25%reduction in RMSE.The ANN-based approach is computationally efficient.We observed that even though the ANN is trained based on completely random velocity models,it is still capable of resolving previously unseen anomalous structures within the edifice with about 5%anomalous discrepancies,making it a potentially valuable tool for the detection of low velocity anomalies related to magmatic intrusions or mush.

A robust behavior of Feed Forward Back propagation algorithm of Artificial Neural Networks in the application of vertical electrical sounding data inversion

The applications of intelligent techniques have increased exponentially in recent days to study most of the non-linear parameters. In particular, the behavior of earth resembles the non- linearity applications. An efficient tool is needed for the interpretation of geophysical parameters to study the subsurface of the earth. Artificial Neural Networks （ANN） perform certain tasks if the structure of the network is modified accordingly for the purpose it has been used. The three most robust networks were taken and comparatively analyzed for their performance to choose the appropriate network. The single- layer feed-forward neural network with the back propagation algorithm is chosen as one of the well- suited networks after comparing the results. Initially, certain synthetic data sets of all three-layer curves have been taken tk^r training the network, and the network is validated by the field datasets collected from Tuticorin Coastal Region （78°7＇30＂E and 8°48＇45＂N）, Tamil Nadu, India. The interpretation has been done successfully using the corresponding learning algorithm in the present study. With proper training of back propagation networks, it tends to give the resistivity and thickness of the subsurface layer model of the field resistivity data concerning the synthetic data trained earlier in the appropriate network. The network is trained with more Vertical Electrical Sounding （VES） data, and this trained network is demon- strated by the field data. Groundwater table depth also has been modeled.

Neural Network inverse Adaptive Controller Based on Davidon Least Square

General neural network inverse adaptive controller has two flaws: the first is the slow convergence speed; the second is the invalidation to the non-minimum phase system. These defects limit the scope in which the neural network inverse adaptive controller is used. We employ Davidon least squares in training the multi-layer feedforward neural network used in approximating the inverse model of plant to expedite the convergence, and then through constructing the pseudo-plant, a neural network inverse adaptive controller is put forward which is still effective to the nonlinear non-minimum phase system. The simulation results show the validity of this scheme.

Artificial neural network based inverse design method for circular sliding slopes

Current design method for circular sliding slopes is not so reasonable that it often results in slope (sliding.) As a result, artificial neural network (ANN) is used to establish an artificial neural network based inverse design method for circular sliding slopes. A sample set containing 21 successful circular sliding slopes excavated in the past is used to train the network. A test sample of 3 successful circular sliding slopes excavated in the past is used to test the trained network. The test results show that the ANN based inverse design method is valid and can be applied to the design of circular sliding slopes.

Applying Neural Network Architecture for Inverse Kinematics Problem in Robotics

One of the most important problems in robot kinematics and control is, finding the solution of Inverse Kinematics. Inverse kinematics computation has been one of the main problems in robotics research. As the Complexity of robot increases, obtaining the inverse kinematics is difficult and computationally expensive. Traditional methods such as geometric, iterative and algebraic are inadequate if the joint structure of the manipulator is more complex. As alternative approaches, neural networks and optimal search methods have been widely used for inverse kinematics modeling and control in robotics This paper proposes neural network architecture that consists of 6 sub-neural networks to solve the inverse kinematics problem for robotics manipulators with 2 or higher degrees of freedom. The neural networks utilized are multi-layered perceptron (MLP) with a back-propagation training algorithm. This approach will reduce the complexity of the algorithm and calculation (matrix inversion) faced when using the Inverse Geometric Models implementation (IGM) in robotics. The obtained results are presented and analyzed in order to prove the efficiency of the proposed approach.

APPLICATION OF NEURAL NETWORK INVERSE CONTROL SYSTEM IN TURBO DECODING

Adaptive inverse control system can improve the performance of turbo decoding,and modeling turbo decoder is one of the most important technologies. A neural network model for the inverse model of turbo decoding is proposed in this paper. Compared with linear filter with its revi-sion,the general relationship between the input and output of the inverse model of turbo decoding system can be established exactly by Nonlinear Auto-Regressive eXogeneous input (NARX) filter. Combined with linear inverse system,it has simpler structure and costs less computation,thus can satisfy the demand of real-time turbo decoding. Simulation results show that neural network in-verse control system can improve the performance of turbo decoding further than other linear con-trol system.

DECOUPLING CONTROL OF TWO MOTORS SYSTEM BASED ON NEURAL NETWORK INVERSE SYSTEM

In accordance with the characteristics of two motors system, the unitedmathematic model of two-motors inverter system with v/f variable frequency speed-regulating isgiven. Two-motor inverter system can be decoupled by the neural network invert system, and changedinto a sub-system of speed and a sub-system of tension. Multiple controllers are designed, and goodresults are obtained. Tie system has good static and dynamic performances and high anti-disturbanceof load.

Application of Artificial Neural Network, Kriging, and Inverse Distance Weighting Models for Estimation of Scour Depth around Bridge Pier with Bed Sill

This paper outlines the application of the multi-layer perceptron artificial neural network (ANN), ordinary kriging (OK), and inverse distance weighting (IDW) models in the estimation of local scour depth around bridge piers. As part of this study, bridge piers were installed with bed sills at the bed of an experimental flume. Experimental tests were conducted under different flow conditions and varying distances between bridge pier and bed sill. The ANN, OK and IDW models were applied to the experimental data and it was shown that the artificial neural network model predicts local scour depth more accurately than the kriging and inverse distance weighting models. It was found that the ANN with two hidden layers was the optimum model to predict local scour depth. The results from the sixth test case showed that the ANN with one hidden layer and 17 hidden nodes was the best model to predict local scour depth. Whereas the results from the fifth test case found that the ANN with three hidden layers was the best model to predict local scour depth.

Neural Network Inversion for Multilayer Quaternion Neural Networks

Recently, solutions to inverse problems have been required in various engineering fields. The neural network inversion method has been studied as one of the neural network-based solutions. On the other hand, the extension of the neural network to a higher-dimensional domain, e.g., complex-value or quaternion, has been proposed, and a number of higher-dimensional neural network models have been proposed. Using the quatemion, we have the advantage of expressing 3D （three-dimensional） object attitudes easily. In the quaternion domain, we can define inverse problems where the cause and the result are expressed by the quaternion. In this paper, we extend the neural network inversion method to the quatemion domain. Further, we provide the results of the computer experiments to demonstrate the process and effectiveness of our method.

APPLICATION OF ARTIFICIAL NEURAL NETWORK TO INVERSE PROBLEMS OF ESTIMATING INNER ETCH OF ELASTOPLASTIC PIPE UNDER PRESSURE

To determine a variation of pipe's inner geometric shape as due to etch, the three-layered feedforward artificial neural network is used in the inverse analysis through observing the elastoplastic strains of the outer wall under the working inner pressure. Because of different kinds of inner wail radii and eccentricity. several groups of strains calculated with computational mechanics are used for the network to do learning. Numerical calculation demonstrates that this method is effective and the estimated inner wall geometric parameters have high precision.

PARAMETERS INVERSION OF FLUID-SATURATED POROUS MEDIA BASED ON NEURAL NETWORKS

The multi- layers feedforward neural network is used for inversion ofmaterial constants of fluid-saturated porous media. The direct analysis of fluid-saturated porousmedia is carried out with the boundary element method. The dynamic displacement responses obtainedfrom direct analysis for prescribed material parameters constitute the sample sets training neuralnetwork. By virtue of the effective L-M training algorithm and the Tikhonov regularization method aswell as the GCV method for an appropriate selection of regu-larization parameter, the inversemapping from dynamic displacement responses to material constants is performed. Numerical examplesdemonstrate the validity of the neural network method.

Robot inverse kinematics based on FCM and fuzzy-neural network

Presents a fast and effective method proposed by combining the fuzzy C means (FCM) and the fuzzy neural network for solving robot inverse kinematics, and its successful application to the robot inverse kinematics and concludes from simulation results that this new method not only has high efficiency and accuracy, but also good generalization, and it also overcomes the "dimension disaster" of fuzzy set in a fuzzy neural network fairly well.

Research on inversion method for complex source-term distributions based on deep neural networks

This study proposes a source distribution inversion convolutional neural network (SDICNN), which is deep neural network model for the inversion of complex source distributions, to solve inversion problems involving fixed-source distributions. A function is developed to obtain the distribution information of complex source terms from radiation parameters at individual sampling points in space. The SDICNN comprises two components:a fully connected network and a convolutional neural network. The fully connected network mainly extracts the parameter measurement information from the sampling points,whereas the convolutional neural network mainly completes the fine inversion of the source-term distribution. Finally, the SDICNN obtains a high-resolution source-term distribution image. In this study, the proposed source-term inversion method is evaluated based on typical geometric scenarios. The results show that, unlike the conventional fully connected neural network, the SDICNN model can extract the two-dimensional distribution features of the source terms, and its inversion results are better. In addition, the effects of the shielding mechanism and number of sampling points on the inversion process are examined. In summary, the result of this study can facilitate the accurate assessment of dose distributions in nuclear facilities.