Dynamic Economic Scheduling with Self-Adaptive Uncertainty in Distribution Network Based on Deep Reinforcement Learning

Traditional optimal scheduling methods are limited to accurate physical models and parameter settings, which aredifficult to adapt to the uncertainty of source and load, and there are problems such as the inability to make dynamicdecisions continuously. This paper proposed a dynamic economic scheduling method for distribution networksbased on deep reinforcement learning. Firstly, the economic scheduling model of the new energy distributionnetwork is established considering the action characteristics of micro-gas turbines, and the dynamic schedulingmodel based on deep reinforcement learning is constructed for the new energy distribution network system with ahigh proportion of new energy, and the Markov decision process of the model is defined. Secondly, Second, for thechanging characteristics of source-load uncertainty, agents are trained interactively with the distributed networkin a data-driven manner. Then, through the proximal policy optimization algorithm, agents adaptively learn thescheduling strategy and realize the dynamic scheduling decision of the new energy distribution network system.Finally, the feasibility and superiority of the proposed method are verified by an improved IEEE 33-node simulationsystem.

Optimizing Deep Learning for Computer-Aided Diagnosis of Lung Diseases: An Automated Method Combining Evolutionary Algorithm, Transfer Learning, and Model Compression

Recent developments in Computer Vision have presented novel opportunities to tackle complex healthcare issues,particularly in the field of lung disease diagnosis.One promising avenue involves the use of chest X-Rays,which are commonly utilized in radiology.To fully exploit their potential,researchers have suggested utilizing deep learning methods to construct computer-aided diagnostic systems.However,constructing and compressing these systems presents a significant challenge,as it relies heavily on the expertise of data scientists.To tackle this issue,we propose an automated approach that utilizes an evolutionary algorithm(EA)to optimize the design and compression of a convolutional neural network(CNN)for X-Ray image classification.Our approach accurately classifies radiography images and detects potential chest abnormalities and infections,including COVID-19.Furthermore,our approach incorporates transfer learning,where a pre-trainedCNNmodel on a vast dataset of chest X-Ray images is fine-tuned for the specific task of detecting COVID-19.This method can help reduce the amount of labeled data required for the task and enhance the overall performance of the model.We have validated our method via a series of experiments against state-of-the-art architectures.

A performance-based hybrid deep learning model for predicting TBM advance rate using Attention-ResNet-LSTM

The technology of tunnel boring machine(TBM)has been widely applied for underground construction worldwide;however,how to ensure the TBM tunneling process safe and efficient remains a major concern.Advance rate is a key parameter of TBM operation and reflects the TBM-ground interaction,for which a reliable prediction helps optimize the TBM performance.Here,we develop a hybrid neural network model,called Attention-ResNet-LSTM,for accurate prediction of the TBM advance rate.A database including geological properties and TBM operational parameters from the Yangtze River Natural Gas Pipeline Project is used to train and test this deep learning model.The evolutionary polynomial regression method is adopted to aid the selection of input parameters.The results of numerical exper-iments show that our Attention-ResNet-LSTM model outperforms other commonly-used intelligent models with a lower root mean square error and a lower mean absolute percentage error.Further,parametric analyses are conducted to explore the effects of the sequence length of historical data and the model architecture on the prediction accuracy.A correlation analysis between the input and output parameters is also implemented to provide guidance for adjusting relevant TBM operational parameters.The performance of our hybrid intelligent model is demonstrated in a case study of TBM tunneling through a complex ground with variable strata.Finally,data collected from the Baimang River Tunnel Project in Shenzhen of China are used to further test the generalization of our model.The results indicate that,compared to the conventional ResNet-LSTM model,our model has a better predictive capability for scenarios with unknown datasets due to its self-adaptive characteristic.

Quafu-RL:The cloud quantum computers based quantum reinforcement learning

With the rapid advancement of quantum computing,hybrid quantum–classical machine learning has shown numerous potential applications at the current stage,with expectations of being achievable in the noisy intermediate-scale quantum(NISQ)era.Quantum reinforcement learning,as an indispensable study,has recently demonstrated its ability to solve standard benchmark environments with formally provable theoretical advantages over classical counterparts.However,despite the progress of quantum processors and the emergence of quantum computing clouds,implementing quantum reinforcement learning algorithms utilizing parameterized quantum circuits(PQCs)on NISQ devices remains infrequent.In this work,we take the first step towards executing benchmark quantum reinforcement problems on real devices equipped with at most 136 qubits on the BAQIS Quafu quantum computing cloud.The experimental results demonstrate that the policy agents can successfully accomplish objectives under modified conditions in both the training and inference phases.Moreover,we design hardware-efficient PQC architectures in the quantum model using a multi-objective evolutionary algorithm and develop a learning algorithm that is adaptable to quantum devices.We hope that the Quafu-RL can be a guiding example to show how to realize machine learning tasks by taking advantage of quantum computers on the quantum cloud platform.

A Two-Layer Encoding Learning Swarm Optimizer Based on Frequent Itemsets for Sparse Large-Scale Multi-Objective Optimization

Traditional large-scale multi-objective optimization algorithms(LSMOEAs)encounter difficulties when dealing with sparse large-scale multi-objective optimization problems(SLM-OPs)where most decision variables are zero.As a result,many algorithms use a two-layer encoding approach to optimize binary variable Mask and real variable Dec separately.Nevertheless,existing optimizers often focus on locating non-zero variable posi-tions to optimize the binary variables Mask.However,approxi-mating the sparse distribution of real Pareto optimal solutions does not necessarily mean that the objective function is optimized.In data mining,it is common to mine frequent itemsets appear-ing together in a dataset to reveal the correlation between data.Inspired by this,we propose a novel two-layer encoding learning swarm optimizer based on frequent itemsets(TELSO)to address these SLMOPs.TELSO mined the frequent terms of multiple particles with better target values to find mask combinations that can obtain better objective values for fast convergence.Experi-mental results on five real-world problems and eight benchmark sets demonstrate that TELSO outperforms existing state-of-the-art sparse large-scale multi-objective evolutionary algorithms(SLMOEAs)in terms of performance and convergence speed.

Adaptive interaction driven by the learning effect in the spatial prisoner's dilemma

We propose a computing model in which individuals can automatically adjust their interaction intensity with their mentor according to the learning effect.This model is designed to investigate the cooperative dynamics of the spatial prisoner's dilemma.More specifically,when the cumulative payoff of a learner is more than his reference earning,he will strengthen his interaction with his mentor;otherwise,he will reduce it.The experimental results indicate that this mechanism can improve the emergence of cooperation in a networked population and that the driving coefficient of interaction intensity plays an important role in promoting cooperation.Interestingly,under a certain social dilemma condition,there exists a minimal driving coefficient that leads to optimal cooperation.This occurs due to a positive feedback effect between the individual's satisfaction frequency and the number of effective neighbors.Moreover,we find that the experimental results are in accord with theoretical predictions obtained from an extension of the classical pair-approximation method.Our conclusions obtained by considering relationships with mentors can provide a new perspective for future investigations into the dynamics of evolutionary games within structured populations.

Constrained Multi-Objective Optimization With Deep Reinforcement Learning Assisted Operator Selection

Solving constrained multi-objective optimization problems with evolutionary algorithms has attracted considerable attention.Various constrained multi-objective optimization evolutionary algorithms(CMOEAs)have been developed with the use of different algorithmic strategies,evolutionary operators,and constraint-handling techniques.The performance of CMOEAs may be heavily dependent on the operators used,however,it is usually difficult to select suitable operators for the problem at hand.Hence,improving operator selection is promising and necessary for CMOEAs.This work proposes an online operator selection framework assisted by Deep Reinforcement Learning.The dynamics of the population,including convergence,diversity,and feasibility,are regarded as the state;the candidate operators are considered as actions;and the improvement of the population state is treated as the reward.By using a Q-network to learn a policy to estimate the Q-values of all actions,the proposed approach can adaptively select an operator that maximizes the improvement of the population according to the current state and thereby improve the algorithmic performance.The framework is embedded into four popular CMOEAs and assessed on 42 benchmark problems.The experimental results reveal that the proposed Deep Reinforcement Learning-assisted operator selection significantly improves the performance of these CMOEAs and the resulting algorithm obtains better versatility compared to nine state-of-the-art CMOEAs.

A single-task and multi-decision evolutionary game model based on multi-agent reinforcement learning

In the evolutionary game of the same task for groups,the changes in game rules,personal interests,the crowd size,and external supervision cause uncertain effects on individual decision-making and game results.In the Markov decision framework,a single-task multi-decision evolutionary game model based on multi-agent reinforcement learning is proposed to explore the evolutionary rules in the process of a game.The model can improve the result of a evolutionary game and facilitate the completion of the task.First,based on the multi-agent theory,to solve the existing problems in the original model,a negative feedback tax penalty mechanism is proposed to guide the strategy selection of individuals in the group.In addition,in order to evaluate the evolutionary game results of the group in the model,a calculation method of the group intelligence level is defined.Secondly,the Q-learning algorithm is used to improve the guiding effect of the negative feedback tax penalty mechanism.In the model,the selection strategy of the Q-learning algorithm is improved and a bounded rationality evolutionary game strategy is proposed based on the rule of evolutionary games and the consideration of the bounded rationality of individuals.Finally,simulation results show that the proposed model can effectively guide individuals to choose cooperation strategies which are beneficial to task completion and stability under different negative feedback factor values and different group sizes,so as to improve the group intelligence level.

An Opposition-Based Learning-Based Search Mechanism for Flying Foxes Optimization Algorithm

The flying foxes optimization(FFO)algorithm,as a newly introduced metaheuristic algorithm,is inspired by the survival tactics of flying foxes in heat wave environments.FFO preferentially selects the best-performing individuals.This tendency will cause the newly generated solution to remain closely tied to the candidate optimal in the search area.To address this issue,the paper introduces an opposition-based learning-based search mechanism for FFO algorithm(IFFO).Firstly,this paper introduces niching techniques to improve the survival list method,which not only focuses on the adaptability of individuals but also considers the population’s crowding degree to enhance the global search capability.Secondly,an initialization strategy of opposition-based learning is used to perturb the initial population and elevate its quality.Finally,to verify the superiority of the improved search mechanism,IFFO,FFO and the cutting-edge metaheuristic algorithms are compared and analyzed using a set of test functions.The results prove that compared with other algorithms,IFFO is characterized by its rapid convergence,precise results and robust stability.

Enhanced self-adaptive evolutionary algorithm for numerical optimization

There are many population-based stochastic search algorithms for solving optimization problems. However, the universality and robustness of these algorithms are still unsatisfactory. This paper proposes an enhanced self-adaptiveevolutionary algorithm （ESEA） to overcome the demerits above. In the ESEA, four evolutionary operators are designed to enhance the evolutionary structure. Besides, the ESEA employs four effective search strategies under the framework of the self-adaptive learning. Four groups of the experiments are done to find out the most suitable parameter values for the ESEA. In order to verify the performance of the proposed algorithm, 26 state-of-the-art test functions are solved by the ESEA and its competitors. The experimental results demonstrate that the universality and robustness of the ESEA out-perform its competitors.

Evolutionary-assisted reinforcement learning for reservoir real-time production optimization under uncertainty

Production optimization has gained increasing attention from the smart oilfield community because it can increase economic benefits and oil recovery substantially.While existing methods could produce high-optimality results,they cannot be applied to real-time optimization for large-scale reservoirs due to high computational demands.In addition,most methods generally assume that the reservoir model is deterministic and ignore the uncertainty of the subsurface environment,making the obtained scheme unreliable for practical deployment.In this work,an efficient and robust method,namely evolutionaryassisted reinforcement learning(EARL),is proposed to achieve real-time production optimization under uncertainty.Specifically,the production optimization problem is modeled as a Markov decision process in which a reinforcement learning agent interacts with the reservoir simulator to train a control policy that maximizes the specified goals.To deal with the problems of brittle convergence properties and lack of efficient exploration strategies of reinforcement learning approaches,a population-based evolutionary algorithm is introduced to assist the training of agents,which provides diverse exploration experiences and promotes stability and robustness due to its inherent redundancy.Compared with prior methods that only optimize a solution for a particular scenario,the proposed approach trains a policy that can adapt to uncertain environments and make real-time decisions to cope with unknown changes.The trained policy,represented by a deep convolutional neural network,can adaptively adjust the well controls based on different reservoir states.Simulation results on two reservoir models show that the proposed approach not only outperforms the RL and EA methods in terms of optimization efficiency but also has strong robustness and real-time decision capacity.

Meta-Learning of Evolutionary Strategy for Stock Trading

Meta-learning algorithms learn about the learning process itself so it can speed up subsequent similar learning tasks with fewer data and iterations. If achieved, these benefits expand the flexibility of traditional machine learning to areas where there are small windows of time or data available. One such area is stock trading, where the relevance of data decreases as time passes, requiring fast results on fewer data points to respond to fast-changing market trends. We, to the best of our knowledge, are the first to apply meta-learning algorithms to an evolutionary strategy for stock trading to decrease learning time by using fewer iterations and to achieve higher trading profits with fewer data points. We found that our meta-learning approach to stock trading earns profits similar to a purely evolutionary algorithm. However, it only requires 50 iterations during test, versus thousands that are typically required without meta-learning, or 50% of the training data during test.

Biometric Finger Vein Recognition Using Evolutionary Algorithm with Deep Learning

In recent years,the demand for biometric-based human recog-nition methods has drastically increased to meet the privacy and security requirements.Palm prints,palm veins,finger veins,fingerprints,hand veins and other anatomic and behavioral features are utilized in the development of different biometric recognition techniques.Amongst the available biometric recognition techniques,Finger Vein Recognition(FVR)is a general technique that analyzes the patterns of finger veins to authenticate the individuals.Deep Learning(DL)-based techniques have gained immense attention in the recent years,since it accomplishes excellent outcomes in various challenging domains such as computer vision,speech detection and Natural Language Processing(NLP).This technique is a natural fit to overcome the ever-increasing biomet-ric detection problems and cell phone authentication issues in airport security techniques.The current study presents an Automated Biometric Finger Vein Recognition using Evolutionary Algorithm with Deep Learning(ABFVR-EADL)model.The presented ABFVR-EADL model aims to accomplish bio-metric recognition using the patterns of the finger veins.Initially,the presented ABFVR-EADL model employs the histogram equalization technique to pre-process the input images.For feature extraction,the Salp Swarm Algorithm(SSA)with Densely-connected Networks(DenseNet-201)model is exploited,showing the proposed method’s novelty.Finally,the Deep-Stacked Denoising Autoencoder(DSAE)is utilized for biometric recognition.The proposed ABFVR-EADL method was experimentally validated using the benchmark databases,and the outcomes confirmed the productive performance of the proposed ABFVR-EADL model over other DL models.

MINIMUM ATTRIBUTE CO-REDUCTION ALGORITHM BASED ON MULTILEVEL EVOLUTIONARY TREE WITH SELF-ADAPTIVE SUBPOPULATIONS

Attribute reduction is an important process in rough set theory.Finding minimum attribute reduction has been proven to help the user-oriented make better knowledge discovery in some cases.In this paper,an efficient minimum attribute reduction algorithm is proposed based on the multilevel evolutionary tree with self-adaptive subpopulations.A model of multilevel evolutionary tree with self-adaptive subpopulations is constructed,and interacting attribute sets are better decomposed into subsets by the self-adaptive mechanism of elitist populations.Moreover it can self-adapt the subpopulation sizes according to the historical performance record so that interacting attribute decision variables are captured into the same grouped subpopulation,which will be extended to better performance in both quality of solution and competitive computation complexity for minimum attribute reduction.The conducted experiments show the proposed algorithm is better on both efficiency and accuracy of minimum attribute reduction than some representative algorithms.Finally the proposed algorithm is applied to magnetic resonance image(MRI)segmentation,and its stronger applicability is further demonstrated by the effective and robust segmentation results.

Online Adaptation of Game AI with Evolutionary Learning

Since the beginning of computer games era, artificial intelligence (AI) has been a standard feature of games. The current emphasis in computer game AI is improving the quality of opponent AI. Our research question reads: How can unsupervised online learning be incorporated in Computer Role Playing Game(CRPG) to improve the strategy of the opponent AI? Our goal is to use online evolutionary learning to design strategies that can defeat the opponent. So we apply a novel technique called dynamic scripting that realizes online adaptation of scripted opponent AI and report on experiments performed in a simulated CRPG to assess the adaptive performance obtained with the technique.

Practical Meta-Reinforcement Learning of Evolutionary Strategy with Quantum Neural Networks for Stock Trading

We show the practicality of two existing meta-learning algorithms Model- Agnostic Meta-Learning and Fast Context Adaptation Via Meta-learning using an evolutionary strategy for parameter optimization, as well as propose two novel quantum adaptations of those algorithms using continuous quantum neural networks, for learning to trade portfolios of stocks on the stock market. The goal of meta-learning is to train a model on a variety of tasks, such that it can solve new learning tasks using only a small number of training samples. In our classical approach, we trained our meta-learning models on a variety of portfolios that contained 5 randomly sampled Consumer Cyclical stocks from a pool of 60. In our quantum approach, we trained our quantum meta-learning models on a simulated quantum computer with portfolios containing 2 randomly sampled Consumer Cyclical stocks. Our findings suggest that both classical models could learn a new portfolio with 0.01% of the number of training samples to learn the original portfolios and can achieve a comparable performance within 0.1% Return on Investment of the Buy and Hold strategy. We also show that our much smaller quantum meta-learned models with only 60 model parameters and 25 training epochs have a similar learning pattern to our much larger classical meta-learned models that have over 250,000 model parameters and 2500 training epochs. Given these findings, we also discuss the benefits of scaling up our experiments from a simulated quantum computer to a real quantum computer. To the best of our knowledge, we are the first to apply the ideas of both classical meta-learning as well as quantum meta-learning to enhance stock trading.

A Study of Learning Strategic Alliances from an Evolutionary Perspective

We present and strategic alliance partnerships from an an exploratory framework based on an amalgamation of organizational learning literatures. It highlights the importance of understanding learning in alliance evolutionary perspective. The research proposes that as alliances progress from awareness through commitment phases, learning priorities change

Evolutionary Learning of Concepts

Concept learning is a kind of classification task that has interesting practical applications in several areas. In this paper, a new evolutionary concept learning algorithm is proposed and a corresponding learning system, called ECL (Evolutionary Concept Learner), is implemented. This system is compared to three traditional learning systems: MLP (Multilayer Perceptron), ID3 (Iterative Dichotomiser) and NB (Naïve Bayes). The comparison takes into account target concepts of varying complexities (e.g., with interacting attributes) and different qualities of training sets (e.g., with imbalanced classes and noisy class labels). The comparison results show that, although no single system is the best in all situations, the proposed system ECL has a very good overall performance.

基于动态演化聚类算法的E-Learning培训搜索研究

随着信息化社会的高速发展,网络学习(E-Learning)培训越来越的应用到人才培训的各个领域,能否准确地利用用户搜索数据的效果关系到长期人才战略的发展。本研究简要分析了E-Learning培训搜索数据结构,利用重叠搜索模块度函数和指标作为目标函数,以演化聚类算法和搜索数据的编码解码为基础,构建了针对E-Learning培训动态网络重叠用户的动态演化聚类算法。最后实验表明提出的动态演化聚类算法能有效的解决用户实时搜索数据问题。

Reconstruction of Gene Regulatory Networks Based on Two-Stage Bayesian Network Structure Learning Algorithm

In the post-genomic biology era,the reconstruction of gene regulatory networks from microarray gene expression data is very important to understand the underlying biological system,and it has been a challenging task in bioinformatics.The Bayesian network model has been used in reconstructing the gene regulatory network for its advantages,but how to determine the network structure and parameters is still important to be explored.This paper proposes a two-stage structure learning algorithm which integrates immune evolution algorithm to build a Bayesian network.The new algorithm is evaluated with the use of both simulated and yeast cell cycle data.The experimental results indicate that the proposed algorithm can find many of the known real regulatory relationships from literature and predict the others unknown with high validity and accuracy.