Scale adaptive fitness evaluation‐based particle swarm optimisation for hyperparameter and architecture optimisation in neural networks and deep learning

Research into automatically searching for an optimal neural network(NN)by optimi-sation algorithms is a significant research topic in deep learning and artificial intelligence.However,this is still challenging due to two issues:Both the hyperparameter and ar-chitecture should be optimised and the optimisation process is computationally expen-sive.To tackle these two issues,this paper focusses on solving the hyperparameter and architecture optimization problem for the NN and proposes a novel light‐weight scale‐adaptive fitness evaluation‐based particle swarm optimisation(SAFE‐PSO)approach.Firstly,the SAFE‐PSO algorithm considers the hyperparameters and architectures together in the optimisation problem and therefore can find their optimal combination for the globally best NN.Secondly,the computational cost can be reduced by using multi‐scale accuracy evaluation methods to evaluate candidates.Thirdly,a stagnation‐based switch strategy is proposed to adaptively switch different evaluation methods to better balance the search performance and computational cost.The SAFE‐PSO algorithm is tested on two widely used datasets:The 10‐category(i.e.,CIFAR10)and the 100−cate-gory(i.e.,CIFAR100).The experimental results show that SAFE‐PSO is very effective and efficient,which can not only find a promising NN automatically but also find a better NN than compared algorithms at the same computational cost.

Multi-Source Adaptive Selection and Fusion for Pedestrian Dead Reckoning

Accurate multi-source fusion is based on the reliability, quantity, and fusion mode of the sources. The problem of selecting the optimal set for participating in the fusion process is nondeterministic-polynomial-time-hard and is neither sub-modular nor super-modular. Furthermore, in the case of the Kalman filter(KF) fusion algorithm, accurate statistical characteristics of noise are difficult to obtain, and this leads to an unsatisfactory fusion result. To settle the referred cases, a distributed and adaptive weighted fusion algorithm based on KF has been proposed in this paper. In this method, on the basis of the pseudo prior probability of the estimated state of each source, the reliability of the sources is evaluated and the optimal set is selected on a certain threshold. Experiments were performed on multi-source pedestrian dead reckoning for verifying the proposed algorithm. The results obtained from these experiments indicate that the optimal set can be selected accurately with minimal computation, and the fusion error is reduced by 16.6% as compared to the corresponding value resulting from the algorithm without improvements.The proposed adaptive source reliability and fusion weight evaluation is effective against the varied-noise multi-source fusion system, and the fusion error caused by inaccurate statistical characteristics of the noise is reduced by the adaptive weight evaluation.The proposed algorithm exhibits good robustness, adaptability,and value on applications.