Multi-Head Attention Spatial-Temporal Graph Neural Networks for Traffic Forecasting

Accurate traffic prediction is crucial for an intelligent traffic system (ITS). However, the excessive non-linearity and complexity of the spatial-temporal correlation in traffic flow severely limit the prediction accuracy of most existing models, which simply stack temporal and spatial modules and fail to capture spatial-temporal features effectively. To improve the prediction accuracy, a multi-head attention spatial-temporal graph neural network (MSTNet) is proposed in this paper. First, the traffic data is decomposed into unique time spans that conform to positive rules, and valuable traffic node attributes are mined through an adaptive graph structure. Second, time and spatial features are captured using a multi-head attention spatial-temporal module. Finally, a multi-step prediction module is used to achieve future traffic condition prediction. Numerical experiments were conducted on an open-source dataset, and the results demonstrate that MSTNet performs well in spatial-temporal feature extraction and achieves more positive forecasting results than the baseline methods.

A Lightweight Convolutional Neural Network with Hierarchical Multi-Scale Feature Fusion for Image Classification

Convolutional neural networks (CNNs) are widely used in image classification tasks, but their increasing model size and computation make them challenging to implement on embedded systems with constrained hardware resources. To address this issue, the MobileNetV1 network was developed, which employs depthwise convolution to reduce network complexity. MobileNetV1 employs a stride of 2 in several convolutional layers to decrease the spatial resolution of feature maps, thereby lowering computational costs. However, this stride setting can lead to a loss of spatial information, particularly affecting the detection and representation of smaller objects or finer details in images. To maintain the trade-off between complexity and model performance, a lightweight convolutional neural network with hierarchical multi-scale feature fusion based on the MobileNetV1 network is proposed. The network consists of two main subnetworks. The first subnetwork uses a depthwise dilated separable convolution (DDSC) layer to learn imaging features with fewer parameters, which results in a lightweight and computationally inexpensive network. Furthermore, depthwise dilated convolution in DDSC layer effectively expands the field of view of filters, allowing them to incorporate a larger context. The second subnetwork is a hierarchical multi-scale feature fusion (HMFF) module that uses parallel multi-resolution branches architecture to process the input feature map in order to extract the multi-scale feature information of the input image. Experimental results on the CIFAR-10, Malaria, and KvasirV1 datasets demonstrate that the proposed method is efficient, reducing the network parameters and computational cost by 65.02% and 39.78%, respectively, while maintaining the network performance compared to the MobileNetV1 baseline.

Grid Side Distributed Energy Storage Cloud Group End Region Hierarchical Time-Sharing Configuration Algorithm Based onMulti-Scale and Multi Feature Convolution Neural Network

There is instability in the distributed energy storage cloud group end region on the power grid side.In order to avoid large-scale fluctuating charging and discharging in the power grid environment and make the capacitor components showa continuous and stable charging and discharging state,a hierarchical time-sharing configuration algorithm of distributed energy storage cloud group end region on the power grid side based on multi-scale and multi feature convolution neural network is proposed.Firstly,a voltage stability analysis model based onmulti-scale and multi feature convolution neural network is constructed,and the multi-scale and multi feature convolution neural network is optimized based on Self-OrganizingMaps(SOM)algorithm to analyze the voltage stability of the cloud group end region of distributed energy storage on the grid side under the framework of credibility.According to the optimal scheduling objectives and network size,the distributed robust optimal configuration control model is solved under the framework of coordinated optimal scheduling at multiple time scales;Finally,the time series characteristics of regional power grid load and distributed generation are analyzed.According to the regional hierarchical time-sharing configuration model of“cloud”,“group”and“end”layer,the grid side distributed energy storage cloud group end regional hierarchical time-sharing configuration algorithm is realized.The experimental results show that after applying this algorithm,the best grid side distributed energy storage configuration scheme can be determined,and the stability of grid side distributed energy storage cloud group end region layered timesharing configuration can be improved.

Multi-Objective Optimization and Analysis Model of Sintering Process Based on BP Neural Network

A multi-objective optimization and analysis model of the sintering process based on BP neural network is presented. Genetic algorithms are combined to simplify the BP neural network, which can reduce the learning time and increase the forecasting accuracy of the network model. This model has been experimented in the sintering process, and the production cost, the energy consumption, the quality （revolving intensity）, and the output are considered at the same time. Moreover, the relation between some factors and the multi-objectives has been analyzed, and the results are consistent with the process. Different objectives are emphasized at different practical periods, and this can provide a theoretical basis for the manager.

A Kind of Second-Order Learning Algorithm Based on Generalized Cost Criteria in Multi-Layer Feed-Forward Neural Networks

A kind of second order algorithm--recursive approximate Newton algorithm was given by Karayiannis. The algorithm was simplified when it was formulated. Especially, the simplification to matrix Hessian was very reluctant, which led to the loss of valuable information and affected performance of the algorithm to certain extent. For multi layer feed forward neural networks, the second order back propagation recursive algorithm based generalized cost criteria was proposed. It is proved that it is equivalent to Newton recursive algorithm and has a second order convergent rate. The performance and application prospect are analyzed. Lots of simulation experiments indicate that the calculation of the new algorithm is almost equivalent to the recursive least square multiple algorithm. The algorithm and selection of networks parameters are significant and the performance is more excellent than BP algorithm and the second order learning algorithm that was given by Karayiannis.

Multi-step-prediction of chaotic time series based on co-evolutionary recurrent neural network

This paper proposes a co-evolutionary recurrent neural network （CERNN） for the multi-step-prediction of chaotic time series, it estimates the proper parameters of phase space reconstruction and optimizes the structure of recurrent neural networks by coevolutionary strategy. The searching space was separated into two subspaces and the individuals are trained in a parallel computational procedure. It can dynamically combine the embedding method with the capability of recurrent neural network to incorporate past experience due to internal recurrence. The effectiveness of CERNN is evaluated by using three benchmark chaotic time series data sets： the Lorenz series, Mackey-Glass series and real-world sun spot series. The simulation results show that CERNN improves the performances of multi-step-prediction of chaotic time series.

Hausdorff Dimension of Multi-Layer Neural Networks

This elucidation investigates the Hausdorff dimension of the output space of multi-layer neural networks. When the factor map from the covering space of the output space to the output space has a synchronizing word, the Hausdorff dimension of the output space relates to its topological entropy. This clarifies the geometrical structure of the output space in more details.

Pedestrian attribute classification with multi-scale and multi-label convolutional neural networks

Pedestrian attribute classification from a pedestrian image captured in surveillance scenarios is challenging due to diverse clothing appearances,varied poses and different camera views. A multiscale and multi-label convolutional neural network( MSMLCNN) is proposed to predict multiple pedestrian attributes simultaneously. The pedestrian attribute classification problem is firstly transformed into a multi-label problem including multiple binary attributes needed to be classified. Then,the multi-label problem is solved by fully connecting all binary attributes to multi-scale features with logistic regression functions. Moreover,the multi-scale features are obtained by concatenating those featured maps produced from multiple pooling layers of the MSMLCNN at different scales. Extensive experiment results show that the proposed MSMLCNN outperforms state-of-the-art pedestrian attribute classification methods with a large margin.

Exponential stability of cellular neural networks with multiple time delays and impulsive effects

In this work, the stability issues of the equilibrium points of the cellular neural networks with multiple time delays and impulsive effects are investigated. Based on the stability theory of Lyapunov-Krasovskii, the method of linear matrix inequality （LMI） and parametrized first-order model transformation, several novel conditions guaranteeing the delaydependent and the delay-independent exponential stabilities are obtained. A numerical example is given to illustrate the effectiveness of our results.

Neural Network Based Multi-level Fuzzy Evaluation Model for Mechanical Kinematic Scheme

To implement a quantificational evaluation for mechanical kinematic scheme more effectively,a multi-level and multi-objective evaluation model is presented using neural network and fuzzy theory. Firstly,the structure of evaluation model is constructed according to evaluation indicator system. Then evaluation samples are generated and provided to train this model. Thus it can reflect the relation between attributive value and evaluation result,as well as the weight of evaluation indicator. Once evaluation indicators of each candidate are fuzzily quantified and fed into the trained network model,the corresponding evaluation result is outputted and the best alternative can be selected. Under this model,expert knowledge can be effectively acquired and expressed,and the quantificational evaluation can be implemented for kinematic scheme with multi-level evaluation indicator system. Several key problems on this model are discussed and an illustration has demonstrated that this model is feasible and can be regarded as a new idea for solving kinematic scheme evaluation.

基于长短时记忆神经网络的Multi-GNSS卫星钟差建模预报

针对卫星钟差预报中二次多项式模型存在易受噪声干扰、预报精度不高的问题,构建一种基于长短时记忆神经网络的multi-GNSS卫星钟差预报模型,并分析不同卫星系统、不同钟类型基于不同建模方案的模型精度。为验证该模型的有效性和可行性,利用LSTM模型、QP模型、QP-LSTM模型分别基于12 h和24 h钟差序列进行建模,预报1 h、3 h、6 h、12 h钟差。结果表明,LSTM模型建模24 h、预报1 h精度最高。multi-GNSS卫星钟差LSTM预报模型中Galileo系统精度最高,其次为BDS-2系统和GPS系统,GLONASS系统精度最低,精度分别为0.018 ns、0.069 ns、0.133 ns、0.242 ns。不同原子钟预报精度不同,氢原子钟预报精度优于铷原子钟、铯原子钟。LSTM神经网络模型预报精度相较于QP-LSTM模型提升27%,相较于QP模型提升36%。

Modelling and Multi-Objective Optimal Control of Batch Processes Using Recurrent Neuro-fuzzy Networks

In this paper, the modelling and multi-objective optimal control of batch processes, using a recurrent neuro-fuzzy network, are presented. The recurrent neuro-fuzzy network, forms a ＂global＂ nonlinear long-range prediction model through the fuzzy conjunction of a number of ＂local＂ linear dynamic models. Network output is fed back to network input through one or more time delay units, which ensure that predictions from the recurrent neuro-fuzzy network are long-range. In building a recurrent neural network model, process knowledge is used initially to partition the processes non-linear characteristics into several local operating regions, and to aid in the initialisation of corresponding network weights. Process operational data is then used to train the network. Membership functions of the local regimes are identified, and local models are discovered via network training. Based on a recurrent neuro-fuzzy network model, a multi-objective optimal control policy can be obtained. The proposed technique is applied to a fed-batch reactor.

Identification and Prediction of Internet Traffic Using Artificial Neural Networks

This paper presents the development of an artificial neural network (ANN) model based on the multi-layer perceptron (MLP) for analyzing internet traffic data over IP networks. We applied the ANN to analyze a time series of measured data for network response evaluation. For this reason, we used the input and output data of an internet traffic over IP networks to identify the ANN model, and we studied the performance of some training algorithms used to estimate the weights of the neuron. The comparison between some training algorithms demonstrates the efficiency and the accu-racy of the Levenberg-Marquardt (LM) and the Resilient back propagation (Rp) algorithms in term of statistical crite-ria. Consequently, the obtained results show that the developed models, using the LM and the Rp algorithms, can successfully be used for analyzing internet traffic over IP networks, and can be applied as an excellent and fundamental tool for the management of the internet traffic at different times.

Applying Neural Network Architecture for Inverse Kinematics Problem in Robotics

One of the most important problems in robot kinematics and control is, finding the solution of Inverse Kinematics. Inverse kinematics computation has been one of the main problems in robotics research. As the Complexity of robot increases, obtaining the inverse kinematics is difficult and computationally expensive. Traditional methods such as geometric, iterative and algebraic are inadequate if the joint structure of the manipulator is more complex. As alternative approaches, neural networks and optimal search methods have been widely used for inverse kinematics modeling and control in robotics This paper proposes neural network architecture that consists of 6 sub-neural networks to solve the inverse kinematics problem for robotics manipulators with 2 or higher degrees of freedom. The neural networks utilized are multi-layered perceptron (MLP) with a back-propagation training algorithm. This approach will reduce the complexity of the algorithm and calculation (matrix inversion) faced when using the Inverse Geometric Models implementation (IGM) in robotics. The obtained results are presented and analyzed in order to prove the efficiency of the proposed approach.

Fault diagnosis method of hydraulic system based on fusion of neural network and D-S evidence theory

According to fault type diversity and fault information uncertainty problem of the hydraulic driven rocket launcher servo system（HDRLSS） , the fault diagnosis method based on the evidence theory and neural network ensemble is proposed. In order to overcome the shortcomings of the single neural network, two improved neural network models are set up at the com-mon nodes to simplify the network structure. The initial fault diagnosis is based on the iron spectrum data and the pressure, flow and temperature（PFT） characteristic parameters as the input vectors of the two improved neural network models, and the diagnosis result is taken as the basic probability distribution of the evidence theory. Then the objectivity of assignment is real-ized. The initial diagnosis results of two improved neural networks are fused by D-S evidence theory. The experimental results show that this method can avoid the misdiagnosis of neural network recognition and improve the accuracy of the fault diagnosis of HDRLSS.

Non-Minimum Phase Nonlinear System Predictive Control Based on Local Recurrent Neural Networks

After a recursive multi-step-ahead predictor for nonlinear systems based on local recurrent neural networks is introduced, an intelligent FID controller is adopted to correct the errors including identified model errors and accumulated errors produced in the recursive process. Characterized by predictive control, this method can achieve a good control accuracy and has good robustness. A simulation study shows that this control algorithm is very effective.

Application of artificial neural networks for operating speed prediction at horizontal curves: a case study in Egypt

Horizontal alignment greatly affects the speedof vehicles at rural roads. Therefore, it is necessary toanalyze and predict vehicles speed on curve sections.Numerous studies took rural two-lane as research subjectsand provided models for predicting operating speeds.However, less attention has been paid to multi-lane highwaysespecially in Egypt. In this research, field operatingspeed data of both cars and trucks on 78 curve sections offour multi-lane highways is collected. With the data, correlationbetween operating speed （V85） and alignment isanalyzed. The paper includes two separate relevant analyses.The first analysis uses the regression models toinvestigate the relationships between V85 as dependentvariable, and horizontal alignment and roadway factors asindependent variables. This analysis proposes two predictingmodels for cars and trucks. The second analysisuses the artificial neural networks （ANNs） to explore theprevious relationships. It is found that the ANN modelinggives the best prediction model. The most influential variableon V85 for cars is the radius of curve. Also, for V85 fortrucks, the most influential variable is the median width.Finally, the derived models have statistics within theacceptable regions and they are conceptually reasonable.

Double Glow Plasma Surface Alloying Process Modeling Using Artificial Neural Networks

A model is developed for predicting the correlation between processing parameters and the technical target of double glow by applying artificial neural network (ANN). The input parameters of the neural network (NN) are source voltage, workplace voltage, working pressure and distance between source electrode and workpiece. The output of the NN model is three important technical targets, namely the gross element content, the thickness of surface alloying layer and the absorption rate (the ratio of the mass loss of source materials to the increasing mass of workpiece) in the processing of double glow plasma surface alloying. The processing parameters and technical target are then used as a training set for an artificial neural network. The model is based on multiplayer feedforward neural network. A very good performance of the neural network is achieved and the calculated results are in good agreement with the experimental ones.

Modeling and analysis of resistance spot welding based on neural network

A numerical study on the multi-parameter control method based on nonlinear auto-regressive with exogenous input neural network （NARX） is presented here. Welding current was set as the input parameter; electrode displacement and dynamic resistance were set us the output parameters. The NARX model using these parameters was set up to simulate the multi-parameter resistance spot welding process. By comparing actual experimental data and NARX model output data, it was validated that the results from the model reflect the relationship between input parameter and output parameters correctly under the influence of many affecting factors.

Comparative Appraisal of Response Surface Methodology and Artificial Neural Network Method for Stabilized Turbulent Confined Jet Diffusion Flames Using Bluff-Body Burners

The present study was conducted to present the comparative modeling, predictive and generalization abilities of response surface methodology (RSM) and artificial neural network (ANN) for the thermal structure of stabilized confined jet diffusion flames in the presence of different geometries of bluff-body burners. Two stabilizer disc burners tapered at 30° and 60° and another frustum cone of 60°/30° inclination angle were employed all having the same diameter of 80 (mm) acting as flame holders. The measured radial mean temperature profiles of the developed stabilized flames at different normalized axial distances (x/dj) were considered as the model example of the physical process. The RSM and ANN methods analyze the effect of the two operating parameters namely (r), the radial distance from the center line of the flame, and (x/dj) on the measured temperature of the flames, to find the predicted maximum temperature and the corresponding process variables. A three-layered Feed Forward Neural Network in conjugation with the hyperbolic tangent sigmoid (tansig) as transfer function and the optimized topology of 2:10:1 (input neurons: hidden neurons: output neurons) was developed. Also the ANN method has been employed to illustrate such effects in the three and two dimensions and shows the location of the predicted maximum temperature. The results indicated the superiority of ANN in the prediction capability as the ranges of R2 and F Ratio are 0.868 - 0.947 and 231.7 - 864.1 for RSM method compared to 0.964 - 0.987 and 2878.8 7580.7 for ANN method beside lower values for error analysis terms.