Scale adaptive fitness evaluation‐based particle swarm optimisation for hyperparameter and architecture optimisation in neural networks and deep learning

Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to two issues:Both the hyperparameter and ar-chitecture should be optimised and the optimisation process is computationally expen-sive.To tackle these two issues,this paper focusses on solving the hyperparameter and architecture optimization problem for the NN and proposes a novel light‐weight scale‐adaptive fitness evaluation‐based particle swarm optimisation(SAFE‐PSO)approach.Firstly,the SAFE‐PSO algorithm considers the hyperparameters and architectures together in the optimisation problem and therefore can find their optimal combination for the globally best NN.Secondly,the computational cost can be reduced by using multi‐scale accuracy evaluation methods to evaluate candidates.Thirdly,a stagnation‐based switch strategy is proposed to adaptively switch different evaluation methods to better balance the search performance and computational cost.The SAFE‐PSO algorithm is tested on two widely used datasets:The 10‐category(i.e.,CIFAR10)and the 100−cate-gory(i.e.,CIFAR100).The experimental results show that SAFE‐PSO is very effective and efficient,which can not only find a promising NN automatically but also find a better NN than compared algorithms at the same computational cost.

Quantitative algorithm for airborne gamma spectrum of large sample based on improved shuffled frog leaping-particle swarm optimization convolutional neural network

In airborne gamma ray spectrum processing,different analysis methods,technical requirements,analysis models,and calculation methods need to be established.To meet the engineering practice requirements of airborne gamma-ray measurements and improve computational efficiency,an improved shuffled frog leaping algorithm-particle swarm optimization convolutional neural network(SFLA-PSO CNN)for large-sample quantitative analysis of airborne gamma-ray spectra is proposed herein.This method was used to train the weight of the neural network,optimize the structure of the network,delete redundant connections,and enable the neural network to acquire the capability of quantitative spectrum processing.In full-spectrum data processing,this method can perform the functions of energy spectrum peak searching and peak area calculations.After network training,the mean SNR and RMSE of the spectral lines were 31.27 and 2.75,respectively,satisfying the demand for noise reduction.To test the processing ability of the algorithm in large samples of airborne gamma spectra,this study considered the measured data from the Saihangaobi survey area as an example to conduct data spectral analysis.The results show that calculation of the single-peak area takes only 0.13~0.15 ms,and the average relative errors of the peak area in the U,Th,and K spectra are 3.11,9.50,and 6.18%,indicating the high processing efficiency and accuracy of this algorithm.The performance of the model can be further improved by optimizing related parameters,but it can already meet the requirements of practical engineering measurement.This study provides a new idea for the full-spectrum processing of airborne gamma rays.

Prediction of Flash Point Temperature of Organic Compounds Using a Hybrid Method of Group Contribution ＋ Neural Network ＋ Particle Swarm Optimization

The flash points of organic compounds were estimated using a hybrid method that includes a simple group contribution method (GCM) implemented in an artificial neural network (ANN) with particle swarm optimization (PSO). Different topologies of a multilayer neural network were studied and the optimum architecture was determined. Property data of 350 compounds were used for training the network. To discriminate different substances the molecular structures defined by the concept of the classical group contribution method were given as input variables. The capabilities of the network were tested with 155 substances not considered in the training step. The study shows that the proposed GCM+ANN+PSO method represent an excellent alternative for the estimation of flash points of organic compounds with acceptable accuracy (AARD = 1.8%; AAE = 6.2 K).

A hybrid particle swarm optimization approach with neural network and set pair analysis for transmission network planning

Transmission network planning (TNP) is a large-scale, complex, with more non-linear discrete variables and the multi-objective constrained optimization problem. In the optimization process, the line investment, network reliability and the network loss are the main objective of transmission network planning. Combined with set pair analysis (SPA), particle swarm optimization (PSO), neural network (NN), a hybrid particle swarm optimization model was established with neural network and set pair analysis for transmission network planning (HPNS). Firstly, the contact degree of set pair analysis was introduced, the traditional goal set was converted into the collection of the three indicators including the identity degree, difference agree and contrary degree. On this bases, using shi(H), the three objective optimization problem was converted into single objective optimization problem. Secondly, using the fast and efficient search capabilities of PSO, the transmission network planning model based on set pair analysis was optimized. In the process of optimization, by improving the BP neural network constantly training so that the value of the fitness function of PSO becomes smaller in order to obtain the optimization program fitting the three objectives better. Finally, compared HPNS with PSO algorithm and the classic genetic algorithm, HPNS increased about 23% efficiency than THA, raised about 3.7% than PSO and improved about 2.96% than GA.

A Hybrid Model Based on Back-Propagation Neural Network and Optimized Support Vector Machine with Particle Swarm Algorithm for Assessing Blade Icing on Wind Turbines

With the continuous increase in the proportional use of wind energy across the globe,the reduction of power generation efficiency and safety hazards caused by the icing on wind turbine blades have attracted more consideration for research.Therefore,it is crucial to accurately analyze the thickness of icing on wind turbine blades,which can serve as a basis for formulating corresponding control measures and ensure a safe and stable operation of wind turbines in winter times and/or in high altitude areas.This paper fully utilized the advantages of the support vector machine(SVM)and back-propagation neural network(BPNN),with the incorporation of particle swarm optimization(PSO)algorithms to optimize the parameters of the SVM.The paper proposes a hybrid assessment model of PSO-SVM and BPNN based on dynamic weighting rules.Three sets of icing data under a rotating working state of the wind turbine were used as examples for model verification.Based on a comparative analysis with other models,the results showed that the proposed model has better accuracy and stability in analyzing the icing on wind turbine blades.

Forecasting of Software Reliability Using Neighborhood Fuzzy Particle Swarm Optimization Based Novel Neural Network

This paper proposes an artificial neural network(ANN) based software reliability model trained by novel particle swarm optimization(PSO) algorithm for enhanced forecasting of the reliability of software. The proposed ANN is developed considering the fault generation phenomenon during software testing with the fault complexity of different levels. We demonstrate the proposed model considering three types of faults residing in the software. We propose a neighborhood based fuzzy PSO algorithm for competent learning of the proposed ANN using software failure data. Fitting and prediction performances of the neighborhood fuzzy PSO based proposed neural network model are compared with the standard PSO based proposed neural network model and existing ANN based software reliability models in the literature through three real software failure data sets. We also compare the performance of the proposed PSO algorithm with the standard PSO algorithm through learning of the proposed ANN. Statistical analysis shows that the neighborhood fuzzy PSO based proposed neural network model has comparatively better fitting and predictive ability than the standard PSO based proposed neural network model and other ANN based software reliability models. Faster release of software is achievable by applying the proposed PSO based neural network model during the testing period.

Using particle swarm optimization algorithm in an artificial neural network to forecast the strength of paste filling material

In order to forecast the strength of filling material exactly, the main factors affecting the strength of filling material are analyzed. The model of predicting the strength of filling material was established by applying the theory of artificial neural net- works. Based on cases related to our test data of filling material, the predicted results of the model and measured values are com- pared and analyzed. The results show that the model is feasible and scientifically justified to predict the strength of filling material, which provides a new method for forecasting the strength of filling material for paste filling in coal mines.

Data-based Fault Tolerant Control for Affine Nonlinear Systems Through Particle Swarm Optimized Neural Networks

In this paper, a data-based fault tolerant control(FTC) scheme is investigated for unknown continuous-time(CT)affine nonlinear systems with actuator faults. First, a neural network(NN) identifier based on particle swarm optimization(PSO) is constructed to model the unknown system dynamics. By utilizing the estimated system states, the particle swarm optimized critic neural network(PSOCNN) is employed to solve the Hamilton-Jacobi-Bellman equation(HJBE) more efficiently.Then, a data-based FTC scheme, which consists of the NN identifier and the fault compensator, is proposed to achieve actuator fault tolerance. The stability of the closed-loop system under actuator faults is guaranteed by the Lyapunov stability theorem. Finally, simulations are provided to demonstrate the effectiveness of the developed method.

Surface Quality Evaluation of Fluff Fabric Based on Particle Swarm Optimization Back Propagation Neural Network

Aiming at the problem that back propagation(BP)neural network predicts the low accuracy rate of fluff fabric after fluffing process,a BP neural network model optimized by particle swarm optimization(PSO)algorithm is proposed.The sliced image is obtained by the principle of light-cutting imaging.The fluffy region of the adaptive image segmentation is extracted by the Freeman chain code principle.The upper edge coordinate information of the fabric is subjected to one-dimensional discrete wavelet decomposition to obtain high frequency information and low frequency information.After comparison and analysis,the BP neural network was trained by high frequency information,and the PSO algorithm was used to optimize the BP neural network.The optimized BP neural network has better weights and thresholds.The experimental results show that the accuracy of the optimized BP neural network after applying high-frequency information training is 97.96%,which is 3.79%higher than that of the unoptimized BP neural network,and has higher detection accuracy.

Quantum Particle Swarm Optimization Based Convolutional Neural Network for Handwritten Script Recognition

Even though several advances have been made in recent years,handwritten script recognition is still a challenging task in the pattern recognition domain.This field has gained much interest lately due to its diverse application potentials.Nowadays,different methods are available for automatic script recognition.Among most of the reported script recognition techniques,deep neural networks have achieved impressive results and outperformed the classical machine learning algorithms.However,the process of designing such networks right from scratch intuitively appears to incur a significant amount of trial and error,which renders them unfeasible.This approach often requires manual intervention with domain expertise which consumes substantial time and computational resources.To alleviate this shortcoming,this paper proposes a new neural architecture search approach based on meta-heuristic quantum particle swarm optimization(QPSO),which is capable of automatically evolving the meaningful convolutional neural network(CNN)topologies.The computational experiments have been conducted on eight different datasets belonging to three popular Indic scripts,namely Bangla,Devanagari,and Dogri,consisting of handwritten characters and digits.Empirically,the results imply that the proposed QPSO-CNN algorithm outperforms the classical and state-of-the-art methods with faster prediction and higher accuracy.

A fuzzy neural network evolved by particle swarm optimization

A cooperative system of a fuzzy logic model and a fuzzy neural network(CSFLMFNN)is proposed,in which a fuzzy logic model is acquired from domain experts and a fuzzy neural network is generated and prewired according to the model.Then PSO-CSFLMFNN is constructed by introducing particle swarm optimization(PSO)into the cooperative system instead of the commonly used evolutionary algorithms to evolve the prewired fuzzy neural network.The evolutionary fuzzy neural network implements accuracy fuzzy inference without rule matching.PSO-CSFLMFNN is applied to the intelligent fault diagnosis for a petrochemical engineering equipment,in which the cooperative system is proved to be effective.It is shown by the applied results that the performance of the evolutionary fuzzy neural network outperforms remarkably that of the one evolved by genetic algorithm in the convergence rate and the generalization precision.

Temperature prediction model for a high-speed motorized spindle based on back-propagation neural network optimized by adaptive particle swarm optimization

To predict the temperature of a motorized spindle more accurately,a novel temperature prediction model based on the back-propagation neural network optimized by adaptive particle swarm optimization(APSO-BPNN)is proposed.First,on the basis of the PSO-BPNN algorithm,the adaptive inertia weight is introduced to make the weight change with the fitness of the particle,the adaptive learning factor is used to obtain different search abilities in the early and later stages of the algorithm,the mutation operator is incorporated to increase the diversity of the population and avoid premature convergence,and the APSO-BPNN model is constructed.Then,the temperature of different measurement points of the motorized spindle is forecasted by the BPNN,PSO-BPNN,and APSO-BPNN models.The experimental results demonstrate that the APSO-BPNN model has a significant advantage over the other two methods regarding prediction precision and robustness.The presented algorithm can provide a theoretical basis for intelligently controlling temperature and developing an early warning system for high-speed motorized spindles and machine tools.

Improved wavelet neural network combined with particle swarm optimization algorithm and its application

An improved wavelet neural network algorithm which combines with particle swarm optimization was proposed to avoid encountering the curse of dimensionality and overcome the shortage in the responding speed and learning ability brought about by the traditional models. Based on the operational data provided by a regional power grid in the south of China, the method was used in the actual short term load forecasting. The results show that the average time cost of the proposed method in the experiment process is reduced by 12.2 s, and the precision of the proposed method is increased by 3.43% compared to the traditional wavelet network. Consequently, the improved wavelet neural network forecasting model is better than the traditional wavelet neural network forecasting model in both forecasting effect and network function.

NEURAL NETWORK TRAINING WITH PARALLEL PARTICLE SWARM OPTIMIZER

Objective To reduce the execution time of neural network training. Methods Parallel particle swarm optimization algorithm based on master-slave model is proposed to train radial basis function neural networks, which is implemented on a cluster using MPI libraries for inter-process communication. Results High speed-up factor is achieved and execution time is reduced greatly. On the other hand, the resulting neural network has good classification accuracy not only on training sets but also on test sets. Conclusion Since the fitness evaluation is intensive, parallel particle swarm optimization shows great advantages to speed up neural network training.

Control of Neural Network Feedback Linearization Based on Chaotic Particle Swarm Optimization

A new chaotic particle swarm algorithm is proposed in order to avoid the premature convergence of the particle swarm optimization and the shortcomings of the chaotic optimization, such as slow searching speed and low accuracy when used in the multivariable systems or in large search space. The new algorithm combines the particle swarm algorithm and the chaotic optimization, using randomness and ergodicity of chaos to overcome the premature convergence of the particle swarm optimization. At the same time, a new neural network feedback linearization control system is built to control the single-machine infinite-bus system. The network parameters are trained by the chaos particle swarm algorithm, which makes the control achieve optimization and the control law of prime mover output torque obtained. Finally, numerical simulation and practical application validate the effectiveness of the method.

APSO-CNN-SE:An Adaptive Convolutional Neural Network Approach for IoT Intrusion Detection

The surge in connected devices and massive data aggregation has expanded the scale of the Internet of Things(IoT)networks.The proliferation of unknown attacks and related risks,such as zero-day attacks and Distributed Denial of Service(DDoS)attacks triggered by botnets,have resulted in information leakage and property damage.Therefore,developing an efficient and realistic intrusion detection system(IDS)is critical for ensuring IoT network security.In recent years,traditional machine learning techniques have struggled to learn the complex associations between multidimensional features in network traffic,and the excellent performance of deep learning techniques,as an advanced version of machine learning,has led to their widespread application in intrusion detection.In this paper,we propose an Adaptive Particle Swarm Optimization Convolutional Neural Network Squeeze-andExcitation(APSO-CNN-SE)model for implementing IoT network intrusion detection.A 2D CNN backbone is initially constructed to extract spatial features from network traffic.Subsequently,a squeeze-and-excitation channel attention mechanism is introduced and embedded into the CNN to focus on critical feature channels.Lastly,the weights and biases in the CNN-SE are extracted to initialize the population individuals of the APSO.As the number of iterations increases,the population’s position vector is continuously updated,and the cross-entropy loss function value is minimized to produce the ideal network architecture.We evaluated the models experimentally using binary and multiclassification on the UNSW-NB15 and NSL-KDD datasets,comparing and analyzing the evaluation metrics derived from each model.Compared to the base CNN model,the results demonstrate that APSO-CNNSE enhances the binary classification detection accuracy by 1.84%and 3.53%and the multiclassification detection accuracy by 1.56%and 2.73%on the two datasets,respectively.Additionally,the model outperforms the existing models like DT,KNN,LR,SVM,LSTM,etc.,in terms of accuracy and fitting performance.This means that the model can identify potential attacks or anomalies more precisely,improving the overall security and stability of the IoT environment.

Alternative Method of Constructing Granular Neural Networks

Utilizing granular computing to enhance artificial neural network architecture, a newtype of network emerges—thegranular neural network (GNN). GNNs offer distinct advantages over their traditional counterparts: The ability toprocess both numerical and granular data, leading to improved interpretability. This paper proposes a novel designmethod for constructing GNNs, drawing inspiration from existing interval-valued neural networks built uponNNNs. However, unlike the proposed algorithm in this work, which employs interval values or triangular fuzzynumbers for connections, existing methods rely on a pre-defined numerical network. This new method utilizesa uniform distribution of information granularity to granulate connections with unknown parameters, resultingin independent GNN structures. To quantify the granularity output of the network, the product of two commonperformance indices is adopted: The coverage of numerical data and the specificity of information granules.Optimizing this combined performance index helps determine the optimal parameters for the network. Finally,the paper presents the complete model construction and validates its feasibility through experiments on datasetsfrom the UCIMachine Learning Repository. The results demonstrate the proposed algorithm’s effectiveness andpromising performance.

Fault Diagnosis of Power Electronic Circuits Based on Adaptive Simulated Annealing Particle Swarm Optimization

In the field of energy conversion,the increasing attention on power electronic equipment is fault detection and diagnosis.A power electronic circuit is an essential part of a power electronic system.The state of its internal components affects the performance of the system.The stability and reliability of an energy system can be improved by studying the fault diagnosis of power electronic circuits.Therefore,an algorithm based on adaptive simulated annealing particle swarm optimization(ASAPSO)was used in the present study to optimize a backpropagation(BP)neural network employed for the online fault diagnosis of a power electronic circuit.We built a circuit simulation model in MATLAB to obtain its DC output voltage.Using Fourier analysis,we extracted fault features.These were normalized as training samples and input to an unoptimized BP neural network and BP neural networks optimized by particle swarm optimization(PSO)and the ASAPSO algorithm.The accuracy of fault diagnosis was compared for the three networks.The simulation results demonstrate that a BP neural network optimized with the ASAPSO algorithm has higher fault diagnosis accuracy,better reliability,and adaptability and can more effectively diagnose and locate faults in power electronic circuits.

Closed-loop scheduling optimization strategy based on particle swarm optimization with niche technology and soft sensor method of attributes-applied to gasoline blending process

Gasoline blending scheduling optimization can bring significant economic and efficient benefits to refineries.However,the optimization model is complex and difficult to build,which is a typical mixed integer nonlinear programming(MINLP)problem.Considering the large scale of the MINLP model,in order to improve the efficiency of the solution,the mixed integer linear programming-nonlinear programming(MILP-NLP)strategy is used to solve the problem.This paper uses the linear blending rules plus the blending effect correction to build the gasoline blending model,and a relaxed MILP model is constructed on this basis.The particle swarm optimization algorithm with niche technology(NPSO)is proposed to optimize the solution,and the high-precision soft-sensor method is used to calculate the deviation of gasoline attributes,the blending effect is dynamically corrected to ensure the accuracy of the blending effect and optimization results,thus forming a prediction-verification-reprediction closed-loop scheduling optimization strategy suitable for engineering applications.The optimization result of the MILP model provides a good initial point.By fixing the integer variables to the MILPoptimal value,the approximate MINLP optimal solution can be obtained through a NLP solution.The above solution strategy has been successfully applied to the actual gasoline production case of a refinery(3.5 million tons per year),and the results show that the strategy is effective and feasible.The optimization results based on the closed-loop scheduling optimization strategy have higher reliability.Compared with the standard particle swarm optimization algorithm,NPSO algorithm improves the optimization ability and efficiency to a certain extent,effectively reduces the blending cost while ensuring the convergence speed.

CNN-LSTM: A Novel Hybrid Deep Neural Network Model for Brain Tumor Classification

Current revelations in medical imaging have seen a slew of computer-aided diagnostic(CAD)tools for radiologists developed.Brain tumor classification is essential for radiologists to fully support and better interpret magnetic resonance imaging(MRI).In this work,we reported on new observations based on binary brain tumor categorization using HYBRID CNN-LSTM.Initially,the collected image is pre-processed and augmented using the following steps such as rotation,cropping,zooming,CLAHE(Contrast Limited Adaptive Histogram Equalization),and Random Rotation with panoramic stitching(RRPS).Then,a method called particle swarm optimization(PSO)is used to segment tumor regions in an MR image.After that,a hybrid CNN-LSTM classifier is applied to classify an image as a tumor or normal.In this proposed hybrid model,the CNN classifier is used for generating the feature map and the LSTM classifier is used for the classification process.The effectiveness of the proposed approach is analyzed based on the different metrics and outcomes compared to different methods.