可信联邦学习进化优化算法综述

With the development of edge devices and cloud computing,the question of how to accomplish machine learning and optimization tasks in a privacy-preserving and secure way has attracted increased attention over the past decade.As a privacy-preserving distributed machine learning method,federated learning(FL)has become popular in the last few years.However,the data privacy issue also occurs when solving optimization problems,which has received little attention so far.This survey paper is concerned with privacy-preserving optimization,with a focus on privacy-preserving data-driven evolutionary optimization.It aims to provide a roadmap from secure privacy-preserving learning to secure privacy-preserving optimization by summarizing security mechanisms and privacy-preserving approaches that can be employed in machine learning and optimization.We provide a formal definition of security and privacy in learning,followed by a comprehensive review of FL schemes and cryptographic privacy-preserving techniques.Then,we present ideas on the emerging area of privacy-preserving optimization,ranging from privacy-preserving distributed optimization to privacy-preserving evolutionary optimization and privacy-preserving Bayesian optimization(BO).We further provide a thorough security analysis of BO and evolutionary optimization methods from the perspective of inferring attacks and active attacks.On the basis of the above,an in-depth discussion is given to analyze what FL and distributed optimization strategies can be used for the design of federated optimization and what additional requirements are needed for achieving these strategies.Finally,we conclude the survey by outlining open questions and remaining challenges in federated data-driven optimization.We hope this survey can provide insights into the relationship between FL and federated optimization and will promote research interest in secure federated optimization.

Even Search in a Promising Region for Constrained Multi-Objective Optimization

In recent years, a large number of approaches to constrained multi-objective optimization problems(CMOPs) have been proposed, focusing on developing tweaked strategies and techniques for handling constraints. However, an overly finetuned strategy or technique might overfit some problem types,resulting in a lack of versatility. In this article, we propose a generic search strategy that performs an even search in a promising region. The promising region, determined by obtained feasible non-dominated solutions, possesses two general properties.First, the constrained Pareto front(CPF) is included in the promising region. Second, as the number of feasible solutions increases or the convergence performance(i.e., approximation to the CPF) of these solutions improves, the promising region shrinks. Then we develop a new strategy named even search,which utilizes the non-dominated solutions to accelerate convergence and escape from local optima, and the feasible solutions under a constraint relaxation condition to exploit and detect feasible regions. Finally, a diversity measure is adopted to make sure that the individuals in the population evenly cover the valuable areas in the promising region. Experimental results on 45 instances from four benchmark test suites and 14 real-world CMOPs have demonstrated that searching evenly in the promising region can achieve competitive performance and excellent versatility compared to 11 most state-of-the-art methods tailored for CMOPs.

Constrained Multi-Objective Optimization With Deep Reinforcement Learning Assisted Operator Selection

Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention.Various constrained multi-objective optimization evolutionary algorithms(CMOEAs)have been developed with the use of different algorithmic strategies,evolutionary operators,and constraint-handling techniques.The performance of CMOEAs may be heavily dependent on the operators used,however,it is usually difficult to select suitable operators for the problem at hand.Hence,improving operator selection is promising and necessary for CMOEAs.This work proposes an online operator selection framework assisted by Deep Reinforcement Learning.The dynamics of the population,including convergence,diversity,and feasibility,are regarded as the state;the candidate operators are considered as actions;and the improvement of the population state is treated as the reward.By using a Q-network to learn a policy to estimate the Q-values of all actions,the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance.The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems.The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs.

A Survey of Evolutionary Algorithms for Multi-Objective Optimization Problems With Irregular Pareto Fronts

Evolutionary algorithms have been shown to be very successful in solving multi-objective optimization problems(MOPs).However,their performance often deteriorates when solving MOPs with irregular Pareto fronts.To remedy this issue,a large body of research has been performed in recent years and many new algorithms have been proposed.This paper provides a comprehensive survey of the research on MOPs with irregular Pareto fronts.We start with a brief introduction to the basic concepts,followed by a summary of the benchmark test problems with irregular problems,an analysis of the causes of the irregularity,and real-world optimization problems with irregular Pareto fronts.Then,a taxonomy of the existing methodologies for handling irregular problems is given and representative algorithms are reviewed with a discussion of their strengths and weaknesses.Finally,open challenges are pointed out and a few promising future directions are suggested.

Integrating Conjugate Gradients Into Evolutionary Algorithms for Large-Scale Continuous Multi-Objective Optimization

Large-scale multi-objective optimization problems(LSMOPs)pose challenges to existing optimizers since a set of well-converged and diverse solutions should be found in huge search spaces.While evolutionary algorithms are good at solving small-scale multi-objective optimization problems,they are criticized for low efficiency in converging to the optimums of LSMOPs.By contrast,mathematical programming methods offer fast convergence speed on large-scale single-objective optimization problems,but they have difficulties in finding diverse solutions for LSMOPs.Currently,how to integrate evolutionary algorithms with mathematical programming methods to solve LSMOPs remains unexplored.In this paper,a hybrid algorithm is tailored for LSMOPs by coupling differential evolution and a conjugate gradient method.On the one hand,conjugate gradients and differential evolution are used to update different decision variables of a set of solutions,where the former drives the solutions to quickly converge towards the Pareto front and the latter promotes the diversity of the solutions to cover the whole Pareto front.On the other hand,objective decomposition strategy of evolutionary multi-objective optimization is used to differentiate the conjugate gradients of solutions,and the line search strategy of mathematical programming is used to ensure the higher quality of each offspring than its parent.In comparison with state-of-the-art evolutionary algorithms,mathematical programming methods,and hybrid algorithms,the proposed algorithm exhibits better convergence and diversity performance on a variety of benchmark and real-world LSMOPs.

A survey on computationally efficient neural architecture search

Neural architecture search(NAS)has become increasingly popular in the deep learning community recently,mainly because it can provide an opportunity to allow interested users without rich expertise to benefit from the success of deep neural networks(DNNs).However,NAS is still laborious and time-consuming because a large number of performance estimations are required during the search process of NAS,and training DNNs is computationally intensive.To solve this major limitation of NAS,improving the computational efficiency is essential in the design of NAS.However,a systematic overview of computationally efficient NAS(CE-NAS)methods still lacks.To fill this gap,we provide a comprehensive survey of the state-of-the-art on CE-NAS by categorizing the existing work into proxy-based and surrogate-assisted NAS methods,together with a thorough discussion of their design principles and a quantitative comparison of their performances and computational complexities.The remaining challenges and open research questions are also discussed,and promising research topics in this emerging field are suggested.

A comprehensive survey of robust deep learning in computer vision

Deep learning has presented remarkable progress in various tasks.Despite the excellent performance,deep learning models remain not robust,especially to well-designed adversarial examples,limiting deep learning models employed in security-critical applications.Therefore,how to improve the robustness of deep learning has attracted increasing attention from researchers.This paper investigates the progress on the threat of deep learning and the techniques that can enhance the model robustness in computer vision.Unlike previous relevant survey papers summarizing adversarial attacks and defense technologies,this paper also provides an overview of the general robustness of deep learning.Besides,this survey elaborates on the current robustness evaluation approaches,which require further exploration.This paper also reviews the recent literature on making deep learning models resistant to adversarial examples from an architectural perspective,which was rarely mentioned in previous surveys.Finally,interesting directions for future research are listed based on the reviewed literature.This survey is hoped to serve as the basis for future research in this topical field.

Deep Industrial Image Anomaly Detection: A Survey

The recent rapid development of deep learning has laid a milestone in industrial image anomaly detection(IAD).In this pa-per,we provide a comprehensive review of deep learning-based image anomaly detection techniques,from the perspectives of neural net-work architectures,levels of supervision,loss functions,metrics and datasets.In addition,we extract the promising setting from indus-trial manufacturing and review the current IAD approaches under our proposed setting.Moreover,we highlight several opening chal-lenges for image anomaly detection.The merits and downsides of representative network architectures under varying supervision are discussed.Finally,we summarize the research findings and point out future research directions.More resources are available at https://github.com/M-3LAB/awesome-industrial-anomaly-detection.

改进随机森林的集成分类方法预测结直肠癌存活性

癌症是人类死亡的主要原因之一,许多国家在癌症方面的支出占医疗总支出的很大比例。癌症存活性预测作为癌症预后的一项重要工作,可以辅助医生做出更精准的诊疗决策,进而降低癌症治疗成本。近年来,基于数据驱动的癌症存活性预测方法逐渐得到应用,而预测的准确性是评价预测方法性能的主要指标,因此提高癌症存活性预测方法的准确性一直是一个活跃的研究领域。结直肠癌是一种具有高发病率和高死亡率的癌症,为了提高结直肠癌存活性预测的准确性,利用遗传算法对随机森林进行改进,提出基于GA-RF的集成分类方法。该方法通过遗传算法对随机森林中的决策树实行进化搜索,以提高集成分类准确率为目标选出决策树的满意集成。实验分别使用基于GA-RF的集成分类方法、决策树和参数优化的随机森林训练预测模型预测结直肠癌患者的存活性,利用SEER数据库的结直肠癌数据集对3种方法分别进行10折交叉验证,然后用准确性、敏感性和特异性3个指标对它们进行评价。实验结果显示,基于GA-RF的集成分类方法的预测精度最高(88.2%),参数优化的随机森林的预测精度次之(86.4%),但集成复杂度远高于基于GA-RF的集成分类方法,决策树的预测精度最差(74.2%),而基于GA-RF的集成分类方法还表现出了最好的泛化性能。该集成分类方法对随机森林进行了有效的改进,能以更高的运算效率和更好的准确性预测结直肠癌存活性,可以为结直肠癌的预后提供决策参考,弥补经验预测的不足,该方法的提出对节约医疗资源、降低医疗成本、提高患者满意度具有实际意义。

基于结构自适应模糊神经网络的前列腺癌诊断方法

前列腺癌是近年来严重危害男性健康的疾病.利用模糊神经网络方法可以实现前列腺癌诊断,并将诊断模型表示为模糊规则集合.针对模糊神经网络所提取规则解释性差的问题,提出结构自适应模糊神经网络方法,通过改进损失函数,在训练中控制相似隶属度函数的合并,实现模糊神经网络模型结构自适应调整,减少模糊规则数量,在保证诊断准确性情况下,提取出容易理解的可解释性规则.同时该方法在模型的训练过程中引入粒子群优化（PSO）算法进行结构和参数学习,有效减少计算量,提高训练效率.最后,使用临床医学科学数据中心提供的前列腺疾病检查数据进行数值实验,验证了所提出方法在前列腺癌诊断和可解释性规则提取中的有效性.