Scale adaptive fitness evaluation‐based particle swarm optimisation for hyperparameter and architecture optimisation in neural networks and deep learning

Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to two issues:Both the hyperparameter and ar-chitecture should be optimised and the optimisation process is computationally expen-sive.To tackle these two issues,this paper focusses on solving the hyperparameter and architecture optimization problem for the NN and proposes a novel light‐weight scale‐adaptive fitness evaluation‐based particle swarm optimisation(SAFE‐PSO)approach.Firstly,the SAFE‐PSO algorithm considers the hyperparameters and architectures together in the optimisation problem and therefore can find their optimal combination for the globally best NN.Secondly,the computational cost can be reduced by using multi‐scale accuracy evaluation methods to evaluate candidates.Thirdly,a stagnation‐based switch strategy is proposed to adaptively switch different evaluation methods to better balance the search performance and computational cost.The SAFE‐PSO algorithm is tested on two widely used datasets:The 10‐category(i.e.,CIFAR10)and the 100−cate-gory(i.e.,CIFAR100).The experimental results show that SAFE‐PSO is very effective and efficient,which can not only find a promising NN automatically but also find a better NN than compared algorithms at the same computational cost.

Adaptive multifactorial particle swarm optimisation

Existing multifactorial particle swarm optimisation(MFPSO)algorithms only explore a relatively narrow area between the inter-task particles.Meanwhile,these algorithms use a fixed inter-task learning probability throughout the evolution process.However,the parameter is problem dependent and can be various at different stages of the evolution.In this work,the authors devise an inter-task learning-based information transferring mechanism to replace the corresponding part in MFPSO.This inter-task learning mechanism transfers the searching step by using a differential term and updates the personal best position by employing an inter-task crossover.By this mean,the particles can explore a broad search space when utilising the additional searching experiences of other tasks.In addition,to enhance the performance on problems with different complementarity,they design a self-adaption strategy to adjust the inter-task learning probability according to the performance feedback.They compared the proposed algorithm with the state-of-the-art algorithms on various benchmark problems.Experimental results demonstrate that the proposed algorithm can transfer inter-task knowledge efficiently and perform well on the problems with different complementarity.

A Chaotic Local Search-Based Particle Swarm Optimizer for Large-Scale Complex Wind Farm Layout Optimization

Wind energy has been widely applied in power generation to alleviate climate problems.The wind turbine layout of a wind farm is a primary factor of impacting power conversion efficiency due to the wake effect that reduces the power outputs of wind turbines located in downstream.Wind farm layout optimization(WFLO)aims to reduce the wake effect for maximizing the power outputs of the wind farm.Nevertheless,the wake effect among wind turbines increases significantly as the number of wind turbines increases in the wind farm,which severely affect power conversion efficiency.Conventional heuristic algorithms suffer from issues of low solution quality and local optimum for large-scale WFLO under complex wind scenarios.Thus,a chaotic local search-based genetic learning particle swarm optimizer(CGPSO)is proposed to optimize large-scale WFLO problems.CGPSO is tested on four larger-scale wind farms under four complex wind scenarios and compares with eight state-of-the-art algorithms.The experiment results indicate that CGPSO significantly outperforms its competitors in terms of performance,stability,and robustness.To be specific,a success and failure memories-based selection is proposed to choose a chaotic map for chaotic search local.It improves the solution quality.The parameter and search pattern of chaotic local search are also analyzed for WFLO problems.

Quantitative algorithm for airborne gamma spectrum of large sample based on improved shuffled frog leaping-particle swarm optimization convolutional neural network

In airborne gamma ray spectrum processing,different analysis methods,technical requirements,analysis models,and calculation methods need to be established.To meet the engineering practice requirements of airborne gamma-ray measurements and improve computational efficiency,an improved shuffled frog leaping algorithm-particle swarm optimization convolutional neural network(SFLA-PSO CNN)for large-sample quantitative analysis of airborne gamma-ray spectra is proposed herein.This method was used to train the weight of the neural network,optimize the structure of the network,delete redundant connections,and enable the neural network to acquire the capability of quantitative spectrum processing.In full-spectrum data processing,this method can perform the functions of energy spectrum peak searching and peak area calculations.After network training,the mean SNR and RMSE of the spectral lines were 31.27 and 2.75,respectively,satisfying the demand for noise reduction.To test the processing ability of the algorithm in large samples of airborne gamma spectra,this study considered the measured data from the Saihangaobi survey area as an example to conduct data spectral analysis.The results show that calculation of the single-peak area takes only 0.13~0.15 ms,and the average relative errors of the peak area in the U,Th,and K spectra are 3.11,9.50,and 6.18%,indicating the high processing efficiency and accuracy of this algorithm.The performance of the model can be further improved by optimizing related parameters,but it can already meet the requirements of practical engineering measurement.This study provides a new idea for the full-spectrum processing of airborne gamma rays.

UAV penetration mission path planning based on improved holonic particle swarm optimization

To meet the requirements of safety, concealment, and timeliness of trajectory planning during the unmanned aerial vehicle(UAV) penetration process, a three-dimensional path planning algorithm is proposed based on improved holonic particle swarm optimization(IHPSO). Firstly, the requirements of terrain threat, radar detection, and penetration time in the process of UAV penetration are quantified. Regarding radar threats, a radar echo analysis method based on radar cross section(RCS)and the spatial situation is proposed to quantify the concealment of UAV penetration. Then the structure-particle swarm optimization(PSO) algorithm is improved from three aspects.First, the conversion ability of the search strategy is enhanced by using the system clustering method and the information entropy grouping strategy instead of random grouping and constructing the state switching conditions based on the fitness function.Second, the unclear setting of iteration numbers is addressed by using particle spacing to create the termination condition of the algorithm. Finally, the trajectory is optimized to meet the intended requirements by building a predictive control model and using the IHPSO for simulation verification. Numerical examples show the superiority of the proposed method over the existing PSO methods.

Neural network hyperparameter optimization based on improved particle swarm optimization

Hyperparameter optimization is considered as one of the most challenges in deep learning and dominates the precision of model in a certain.Recent proposals tried to solve this issue through the particle swarm optimization(PSO),but its native defect may result in the local optima trapped and convergence difficulty.In this paper,the genetic operations are introduced to the PSO,which makes the best hyperparameter combination scheme for specific network architecture be located easier.Spe-cifically,to prevent the troubles caused by the different data types and value scopes,a mixed coding method is used to ensure the effectiveness of particles.Moreover,the crossover and mutation opera-tions are added to the process of particles updating,to increase the diversity of particles and avoid local optima in searching.Verified with three benchmark datasets,MNIST,Fashion-MNIST,and CIFAR10,it is demonstrated that the proposed scheme can achieve accuracies of 99.58%,93.39%,and 78.96%,respectively,improving the accuracy by about 0.1%,0.5%,and 2%,respectively,compared with that of the PSO.

Co-digestion of Waste Coffee and Cocoa Hulls: Modeling of Biogas Production by the Particle Swarm Method

Energy is a crucial material for the development of our economy.Access to sufficient energy remains a major concern for developing countries,particularly those in sub-Saharan Africa.The major challenge lies in access to clean,environmentally friendly,quality and low-cost energy in different households in our municipalities.To cope with this vast energy gap,many households are dependent on fossil fuels.In Cameroon,the consumption of wood for the supply of energy is increasing by 4%per year.Overall,approximately 80%of households in Cameroon depend on woody biomass as the sole main source of energy supply in Cameroon and demand is growing over time.In view of the climatic variations that our countries,particularly Cameroon,undergo through deforestation,the use of wood as a source of energy is expensive and harmful to the environment,hence the urgency of replacing wood with renewable energy.Biogas is one of the most versatile sources of renewable energy.On an industrial scale,it is important to automate the process control.The main objective of the present work is to model the anaerobic digestion of coffee and cocoa hulls using the particle swarm optimisation method.Pretreatment using the organosolv process was done.This resulted in 48%lignin removal and 22%cellulose increase.For the pretreated biomass,the maximum production rate was 21 NmLCH4 per day with a biomethane yield of 90 NmLCH4/gVS.This represents an enhancement of 117%in biomethane yield.A positive flammability test was recorded after the 10th day of retention time.Moreover,the data collected during anaerobic digestion allowed implementation of a two-phase mathematical model.The thirteen parameters of the model were estimated with particle swarm optimisation method in Matlab.The model was able to simulate the biomethane production kinetics and variation of volatile fatty acid concentration.

θ-PSO: a new strategy of particle swarm optimization

Particle swarm optimization (PSO) is an efficient, robust and simple optimization algorithm. Most studies are mainly concentrated on better understanding of the standard PSO control parameters, such as acceleration coefficients, etc. In this paper, a more simple strategy of PSO algorithm called θ-PSO is proposed. In θ-PSO, an increment of phase angle vector replaces the increment of velocity vector and the positions are decided by the mapping of phase angles. Benchmark testing of nonlinear func- tions is described and the results show that the performance of θ-PSO is much more effective than that of the standard PSO.

Optimization of Fairhurst-Cook Model for 2-D Wing Cracks Using Ant Colony Optimization (ACO), Particle Swarm Intelligence (PSO), and Genetic Algorithm (GA)

The common failure mechanism for brittle rocks is known to be axial splitting which happens parallel to the direction of maximum compression. One of the mechanisms proposed for modelling of axial splitting is the sliding crack or so called, “wing crack” model. Fairhurst-Cook model explains this specific type of failure which starts by a pre-crack and finally breaks the rock by propagating 2-D cracks under uniaxial compression. In this paper, optimization of this model has been considered and the process has been done by a complete sensitivity analysis on the main parameters of the model and excluding the trends of their changes and also their limits and “peak points”. Later on this paper, three artificial intelligence algorithms including Particle Swarm Intelligence (PSO), Ant Colony Optimization (ACO) and genetic algorithm (GA) has been used and compared in order to achieve optimized sets of parameters resulting in near-maximum or near-minimum amounts of wedging forces creating a wing crack.

Driving fatigue fusion detection based on T-S fuzzy neural network evolved by subtractive clustering and particle swarm optimization

In order to improve the accuracy and reliability of the driving fatigue detection based on a single feature, a new detection algorithm based on multiple features is proposed. Two direct driver＇s facial features reflecting fatigue and one indirect vehicle behavior feature indicating fatigue are considered. Meanwhile, T-S fuzzy neural network（TSFNN）is adopted to recognize the driving fatigue of drivers. For the structure identification of the TSFNN, subtractive clustering（SC） is used to confirm the fuzzy rules and their correlative parameters. Moreover, the particle swarm optimization （PSO）algorithm is improved to train the TSFNN. Simulation results and experiments on vehicles show that the proposed algorithm can effectively improve the convergence speed and the recognition accuracy of the TSFNN, as well as enhance the correct rate of driving fatigue detection.

Dynamic compensation for sensors based on particle swarm optimization and realization on LabVIEW

In shock wave＇s pressure testing,a dynamic compensation digital filter is designed based on particle swarm optimization （PSO） algorithm.Dynamic calibration experiment and simulation are conducted for the pressure sensor.PSO algorithm is applied on Matlab platform to achieve optimization according to input and output data of the sensor as well as the reference model,and the global optimal values got by optimization become the parameters of the compensator.Finally,the dynamic compensation filter is established on LabVIEW platform.The experimental results show that the data after processing with the compensation filter truly reflects the input signal.

基于PSO−ELM的综采工作面液压支架姿态监测方法

针对基于惯性测量单元的液压支架姿态解算方法会产生累计误差、校正结果不准确的问题,提出一种基于粒子群优化(PSO)−极限学习机(ELM)的综采工作面液压支架姿态监测方法。以液压支架顶梁俯仰角为监测对象,采用倾角传感器和陀螺仪采集液压支架顶梁支护姿态实时信息,对采集到的数据进行预处理,将处理后的数据输入PSO−ELM误差补偿模型中,得到解算误差预测值;同时通过卡尔曼滤波融合进行液压支架姿态解算,得到解算值;再用误差预测值对解算值进行误差补偿,从而求得更加准确的顶梁支护姿态数据。该方法只考虑加速度和角速度数据与解算误差的关系,不依赖具体的物理模型,可有效降低姿态解算累计误差。实验结果表明:液压支架顶梁俯仰角平均绝对误差由补偿前的1.4208°减少到0.0580°,且误差曲线具有良好的收敛性,验证了所提方法可持续稳定地监测液压支架的支护姿态。

一种基于PSO-ELM的低渗透砂岩水淹层测井识别方法

水淹层测井识别对油田开发方案部署及提高采收率有着重要意义。新疆陆梁油田作业区某区块油层水淹类型主要为污水水淹,测井响应特征复杂多变,传统识别图版方法难以对水淹层有效识别。文中基于测井、地质、试油等资料,在水淹层测井响应特征分析基础上,提出了一种利用改进粒子群优化算法(Particle Swarm Optimization,PSO)及极限学习机(Extreme Learning Machine,ELM)的水淹层识别方法。首先,利用相关系数优选6个主控因素:RD,RS,GR,SP,DEN,AC。其次,采用改进粒子群算法对极限学习机模型进行参数寻优;最后,利用优化后的模型对研究区水淹层进行预测。结果表明,利用PSO-ELM模型识别水淹层,识别符合率达到91.7%,应用效果优于ELM模型及传统识别图版,为水淹层测井识别提供了新思路。

基于PSO-Elman神经网络的井底风温预测模型

目前井下风温预测大多采用BP神经网络,但其预测精度受学习样本数量的影响,且容易陷入局部最优,Elman神经网络具备局部记忆能力,提高了网络的稳定性和动态适应能力,但仍然存在收敛速度过慢、易陷入局部最优的问题。针对上述问题,采用粒子群优化(PSO)算法对Elman神经网络的权值和阈值进行优化,建立了基于PSO-Elman神经网络的井底风温预测模型。分析得出入风相对湿度、入风温度、地面大气压力和井筒深度是井底风温的主要影响因素,因此将其作为模型的输入数据,模型的输出数据为井底风温。在相同样本数据集下的实验结果表明:Elman模型迭代90次后收敛,PSO-Elman模型迭代41次后收敛,说明PSO-Elman模型收敛速度更快;与BP神经网络模型、支持向量回归模型和Elman模型相比,PSO-Elman模型的预测误差较低,平均绝对误差、均方误差(MSE)、平均绝对百分比误差分别为0.376 0℃,0.278 3,1.95%,决定系数R^(2)为0.992 4,非常接近1,表明预测模型具有良好的预测效果。实例验证结果表明,PSO-Elman模型的相对误差范围为-4.69%~1.27%,绝对误差范围为-1.06~0.29℃,MSE为0.26,整体预测精度可满足井下实际需要。

基于粗糙集理论与PCA-APSO-SVM的沥青路面使用性能预测

针对传统沥青路面使用性能预测精度较低的问题,建立了基于粗糙集理论(rough set,RS)与主成分分析法(principal compoent analysis,PCA)-自适应粒子群算法(adaptive particle swarm optimization,APSO)-支持向量机(support vector machine,SVM)的沥青路面使用性能预测模型。基于沥青路面的时序指标与影响因素指标,建立了11个初始预测指标(包括前3年的路面使用性能、当量轴次、路龄、养护性质、坑槽率、修补率、年降水量、平均气温、日照时数);通过RS属性约减筛选出9个核心指标;利用PCA提取4个主成分,得到了基于4个主成分的数据集;将APSO引入到SVM中,对数据集进行训练,并优化了SVM模型参数;建立了路面使用性能的PCA-APSO-SVM预测模型,并以G6京藏高速甘肃境内某段道路为例,对路面使用性能进行预测。研究结果表明:PCA-APSO-SVM模型预测精度较PCA-PSO-SVM、APSO-SVM、PSO-SVM有较大提高,预测结果与实际情况更加符合,能为路面养护决策提供相关参考。

基于改进PSO-BP神经网络的热采管柱应力预测

稠油热采过程中,油套管柱由于在温度、地层等多重载荷作用下发生塑性形变进而导致断裂或失效。文中根据热采管柱高温服役工况,引入异步变化学习因子和自适应权重建立输入参数为注汽温度、井深、非均匀系数和水泥环温度,输出参数为套管应力的改进PSO-BP模型。文中以N80热采套管为例,选取260、280、300、320、340℃5种温度工况下有限元模拟结果作为训练数据,对比BP模型、GA-BP模型、MEA-BP模型、PSO-BP模型和改进PSO-BP模型在300℃工况温度下井深200、300、400、500、600、700 m处套管应力的预测值和试验值、有限元计算值。结果表明:改进PSO-BP模型预测的应力与试验值最接近,最大和最小误差分别为2.69%和0.06%。最后从训练数据、预测误差、计算时间等方面对建立的改进PSO-BP模型进行了评价,为热采管柱服役过程中的强度安全分析提供智能高效的模型。

基于PSO-SVM的Φ-OTDR系统模式识别研究

针对相位敏感光时域反射仪(phase sensitive optical time domain reflectometer,Φ-OTDR)系统中误报率高的问题,提出一种多域特征提取与粒子群算法优化支持向量机(particle swarm optimization-support vector machine,PSO-SVM)相结合的模式识别算法。首先,对原始信号进行差分处理后提取时域特征,并利用小波包分解方法,通过验证不同分解层数下的事件分类准确率,设定最优分解层数为6层,提取差分信号的能量特征。然后以SVM分类器为基础,利用PSO算法优化SVM分类器参数,提高光纤振动信号识别准确率。最后利用Φ-OTDR事件数据集进行验证,实验结果表明,该模式识别算法达到了95.6%的振动事件分类准确率。

基于PCA-PSO-ELM模型预测地震死亡人数研究

筛选42个历史地震震例,对地震震级、震源深度、震中烈度、抗震设防烈度、震中烈度与抗震设防烈度之差(ΔL)、人口密度以及发震时刻7个影响指标进行主成分分析(principal components analysis,PCA),构建粒子群优化(particle swarm optimization,PSO)极限学习机(extreme learning machine,ELM)地震死亡人数预测模型。将37个震例数据进行预处理和训练,并使用5个震例数据来检验模型的预测精度。实验结果表明,该PCA-PSO-ELM组合模型的平均误差率为10.87%,相比于PCA-ELM模型和ELM模型,其平均误差率分别降低8.70个百分点和18.38个百分点。因此,采用PCA-PSO-ELM组合模型预测地震死亡人数具有一定的可行性。

基于CEEMDAN-VMD-PSO-LSTM模型的桥梁挠度预测

针对桥梁运行阶段的健康状态监测,构建了CEEMDAN-VMD-PSO-LSTM模型对桥梁挠度进行预测。该模型主要分为二次模态分解平稳化、粒子群优化(PSO)算法和长短期记忆(LSTM)网络预测三大模块,共有5个步骤:①利用自适应噪声完备集合经验模态分解(CEEMDAN)算法对桥梁原始挠度序列进行初次模态分解,分解为若干本征模态分解函数(IMF);②使用样本熵(SampEn/SE)计算各IMF分量的复杂度,并通过K-means聚类为高频、中频和低频3个IMF分量;③通过变分模态分解(VMD)算法对高频IMF分量进行二次模态分解;④分别对各个IMF分量通过PSO算法得出LSTM最优超参数组合;⑤将各最优超参数分别代入LSTM模型进行训练,并将各预测结果融合为最终的预测结果。结果表明:该预测方法具有最高的预测精度,为智慧桥梁的安全监测监控提供了新的技术方法。

Hybrid optimization algorithm based on chaos,cloud and particle swarm optimization algorithm

As for the drop of particle diversity and the slow convergent speed of particle in the late evolution period when particle swarm optimization（PSO） is applied to solve high-dimensional multi-modal functions,a hybrid optimization algorithm based on the cat mapping,the cloud model and PSO is proposed.While the PSO algorithm evolves a certain of generations,this algorithm applies the cat mapping to implement global disturbance of the poorer individuals,and employs the cloud model to execute local search of the better individuals;accordingly,the obtained best individuals form a new swarm.For this new swarm,the evolution operation is maintained with the PSO algorithm,using the parameter of pop distr to balance the global and local search capacity of the algorithm,as well as,adopting the parameter of mix gen to control mixing times of the algorithm.The comparative analysis is carried out on the basis of 4 functions and other algorithms.It indicates that this algorithm shows faster convergent speed and better solving precision for solving functions particularly those high-dimensional multi-modal functions.Finally,the suggested values are proposed for parameters pop distr and mix gen applied to different dimension functions via the comparative analysis of parameters.