Distributed Stochastic Optimization with Compression for Non-Strongly Convex Objectives

We are investigating the distributed optimization problem,where a network of nodes works together to minimize a global objective that is a finite sum of their stored local functions.Since nodes exchange optimization parameters through the wireless network,large-scale training models can create communication bottlenecks,resulting in slower training times.To address this issue,CHOCO-SGD was proposed,which allows compressing information with arbitrary precision without reducing the convergence rate for strongly convex objective functions.Nevertheless,most convex functions are not strongly convex(such as logistic regression or Lasso),which raises the question of whether this algorithm can be applied to non-strongly convex functions.In this paper,we provide the first theoretical analysis of the convergence rate of CHOCO-SGD on non-strongly convex objectives.We derive a sufficient condition,which limits the fidelity of compression,to guarantee convergence.Moreover,our analysis demonstrates that within the fidelity threshold,this algorithm can significantly reduce transmission burden while maintaining the same convergence rate order as its no-compression equivalent.Numerical experiments further validate the theoretical findings by demonstrating that CHOCO-SGD improves communication efficiency and keeps the same convergence rate order simultaneously.And experiments also show that the algorithm fails to converge with low compression fidelity and in time-varying topologies.Overall,our study offers valuable insights into the potential applicability of CHOCO-SGD for non-strongly convex objectives.Additionally,we provide practical guidelines for researchers seeking to utilize this algorithm in real-world scenarios.

An Elite-Class Teaching-Learning-Based Optimization for Reentrant Hybrid Flow Shop Scheduling with Bottleneck Stage

Bottleneck stage and reentrance often exist in real-life manufacturing processes;however,the previous research rarely addresses these two processing conditions in a scheduling problem.In this study,a reentrant hybrid flow shop scheduling problem(RHFSP)with a bottleneck stage is considered,and an elite-class teaching-learning-based optimization(ETLBO)algorithm is proposed to minimize maximum completion time.To produce high-quality solutions,teachers are divided into formal ones and substitute ones,and multiple classes are formed.The teacher phase is composed of teacher competition and teacher teaching.The learner phase is replaced with a reinforcement search of the elite class.Adaptive adjustment on teachers and classes is established based on class quality,which is determined by the number of elite solutions in class.Numerous experimental results demonstrate the effectiveness of new strategies,and ETLBO has a significant advantage in solving the considered RHFSP.

Stochastic Maximum Principle for Optimal Advertising Models with Delay and Non-Convex Control Spaces

In this paper we study optimal advertising problems that model the introduction of a new product into the market in the presence of carryover effects of the advertisement and with memory effects in the level of goodwill. In particular, we let the dynamics of the product goodwill to depend on the past, and also on past advertising efforts. We treat the problem by means of the stochastic Pontryagin maximum principle, that here is considered for a class of problems where in the state equation either the state or the control depend on the past. Moreover the control acts on the martingale term and the space of controls U can be chosen to be non-convex but now the space of controls U can be chosen to be non-convex. The maximum principle is thus formulated using a first-order adjoint Backward Stochastic Differential Equations (BSDEs), which can be explicitly computed due to the specific characteristics of the model, and a second-order adjoint relation.

Distributed Momentum-Based Frank-Wolfe Algorithm for Stochastic Optimization

This paper considers distributed stochastic optimization,in which a number of agents cooperate to optimize a global objective function through local computations and information exchanges with neighbors over a network.Stochastic optimization problems are usually tackled by variants of projected stochastic gradient descent.However,projecting a point onto a feasible set is often expensive.The Frank-Wolfe(FW)method has well-documented merits in handling convex constraints,but existing stochastic FW algorithms are basically developed for centralized settings.In this context,the present work puts forth a distributed stochastic Frank-Wolfe solver,by judiciously combining Nesterov's momentum and gradient tracking techniques for stochastic convex and nonconvex optimization over networks.It is shown that the convergence rate of the proposed algorithm is O(k^(-1/2))for convex optimization,and O(1/log_(2)(k))for nonconvex optimization.The efficacy of the algorithm is demonstrated by numerical simulations against a number of competing alternatives.

TopSTO:a 115-line code for topology optimization of structures under stationary stochastic dynamic loading

The use of topology optimization in structural design under dynamic excitation is becoming more prevalent in the literature.While many such applications utilize frequency or time domain formulations,relatively few consider stochastic dynamic excitations.This paper presents an efficient and compact code called TopSTO for structural topology optimization considering stationary stochastic dynamic loading using a method derived from random vibration theory.The theory,described in conjunction with the implementation in the provided code,is illustrated for a seismically excited building.This work demonstrates the efficiency of the approach in terms of both the computational resources and minimal amount of code required.This code is intended to serve as a baseline for understanding the theory and implementation of this topology optimization approach and as a foundation for additional applications and developments.

Scheduling an Energy-Aware Parallel Machine System with Deteriorating and Learning Effects Considering Multiple Optimization Objectives and Stochastic Processing Time

Currently,energy conservation draws wide attention in industrial manufacturing systems.In recent years,many studies have aimed at saving energy consumption in the process of manufacturing and scheduling is regarded as an effective approach.This paper puts forwards a multi-objective stochastic parallel machine scheduling problem with the consideration of deteriorating and learning effects.In it,the real processing time of jobs is calculated by using their processing speed and normal processing time.To describe this problem in a mathematical way,amultiobjective stochastic programming model aiming at realizing makespan and energy consumption minimization is formulated.Furthermore,we develop a multi-objective multi-verse optimization combined with a stochastic simulation method to deal with it.In this approach,the multi-verse optimization is adopted to find favorable solutions from the huge solution domain,while the stochastic simulation method is employed to assess them.By conducting comparison experiments on test problems,it can be verified that the developed approach has better performance in coping with the considered problem,compared to two classic multi-objective evolutionary algorithms.

Forecasting Energy Consumption Using a Novel Hybrid Dipper Throated Optimization and Stochastic Fractal Search Algorithm

The accurate prediction of energy consumption has effective role in decision making and risk management for individuals and governments.Meanwhile,the accurate prediction can be realized using the recent advances in machine learning and predictive models.This research proposes a novel approach for energy consumption forecasting based on a new optimization algorithm and a new forecasting model consisting of a set of long short-term memory(LSTM)units.The proposed optimization algorithm is used to optimize the parameters of the LSTM-based model to boost its forecasting accuracy.This optimization algorithm is based on the recently emerged dipper-throated optimization(DTO)and stochastic fractal search(SFS)algo-rithm and is referred to as dynamic DTOSFS.To prove the effectiveness and superiority of the proposed approach,five standard benchmark algorithms,namely,stochastic fractal search(SFS),dipper throated optimization(DTO),whale optimization algorithm(WOA),particle swarm optimization(PSO),and grey wolf optimization(GWO),are used to optimize the parameters of the LSTM-based model,and the results are compared with that of the proposed approach.Experimental results show that the proposed DDTOSFS+LSTM can accurately forecast the energy consumption with root mean square error RMSE of 0.00013,which is the best among the recorded results of the other methods.In addition,statistical experiments are conducted to prove the statistical difference of the proposed model.The results of these tests confirmed the expected outcomes.

Stochastic Programming for Hub Energy Management Considering Uncertainty Using Two-Point Estimate Method and Optimization Algorithm

To maximize energy profit with the participation of electricity,natural gas,and district heating networks in the day-ahead market,stochastic scheduling of energy hubs taking into account the uncertainty of photovoltaic and wind resources,has been carried out.This has been done using a new meta-heuristic algorithm,improved artificial rabbits optimization(IARO).In this study,the uncertainty of solar and wind energy sources is modeled using Hang’s two-point estimating method(TPEM).The IARO algorithm is applied to calculate the best capacity of hub energy equipment,such as solar and wind renewable energy sources,combined heat and power(CHP)systems,steamboilers,energy storage,and electric cars in the day-aheadmarket.The standard ARO algorithmis developed to mimic the foraging behavior of rabbits,and in this work,the algorithm’s effectiveness in avoiding premature convergence is improved by using the dystudynamic inertia weight technique.The proposed IARO-based scheduling framework’s performance is evaluated against that of traditional ARO,particle swarm optimization(PSO),and salp swarm algorithm(SSA).The findings show that,in comparison to previous approaches,the suggested meta-heuristic scheduling framework based on the IARO has increased energy profit in day-ahead electricity,gas,and heating markets by satisfying the operational and energy hub limitations.Additionally,the results show that TPEM approach dependability consideration decreased hub energy’s profit by 8.995%as compared to deterministic planning.

First Order Convergence Analysis for Sparse Grid Method in Stochastic Two-Stage Linear Optimization Problem

Stochastic two-stage linear optimization is an important and widely used optimization model. Efficiency of numerical integration of the second stage value function is critical. However, the second stage value function is piecewise linear convex, which imposes challenges for applying the modern efficient spare grid method. In this paper, we prove the first order convergence rate of the sparse grid method for this important stochastic optimization model, utilizing convexity analysis and measure theory. The result is two-folded: it establishes a theoretical foundation for applying the sparse grid method in stochastic programming, and extends the convergence theory of sparse grid integration method to piecewise linear and convex functions.

TWO-DIMENSIONAL STOCHASTIC AIRFOIL OPTIMIZATION DESIGN METHOD BASED ON NEURAL NETWORKS

To avoid the aerodynamic performance loss of airfoil at non-design state which often appears in single point design optimization, and to improve the adaptability to the uncertain factors in actual flight environment, a two-dimensional stochastic airfoil optimization design method based on neural networks is presented. To provide highly efficient and credible analysis, four BP neural networks are built as surrogate models to predict the airfoil aerodynamic coefficients and geometry parameter. These networks are combined with the probability density function obeying normal distribution and the genetic algorithm, thus forming an optimization design method. Using the method, for GA（W）-2 airfoil, a stochastic optimization is implemented in a two-dimensional flight area about Mach number and angle of attack. Compared with original airfoil and single point optimization design airfoil, results show that the two-dimensional stochastic method can improve the performance in a specific flight area, and increase the airfoil adaptability to the stochastic changes of multiple flight parameters.

Far and Near Optimization:A New Simple and Effective Metaphor-LessOptimization Algorithm for Solving Engineering Applications

In this article,a novel metaheuristic technique named Far and Near Optimization(FNO)is introduced,offeringversatile applications across various scientific domains for optimization tasks.The core concept behind FNO lies inintegrating global and local search methodologies to update the algorithm population within the problem-solvingspace based on moving each member to the farthest and nearest member to itself.The paper delineates the theoryof FNO,presenting a mathematical model in two phases:(i)exploration based on the simulation of the movementof a population member towards the farthest member from itself and(ii)exploitation based on simulating themovement of a population member towards the nearest member from itself.FNO’s efficacy in tackling optimizationchallenges is assessed through its handling of the CEC 2017 test suite across problem dimensions of 10,30,50,and 100,as well as to address CEC 2020.The optimization results underscore FNO’s adeptness in exploration,exploitation,and maintaining a balance between them throughout the search process to yield viable solutions.Comparative analysis against twelve established metaheuristic algorithms reveals FNO’s superior performance.Simulation findings indicate FNO’s outperformance of competitor algorithms,securing the top rank as the mosteffective optimizer across a majority of benchmark functions.Moreover,the outcomes derived by employing FNOon twenty-two constrained optimization challenges from the CEC 2011 test suite,alongside four engineering designdilemmas,showcase the effectiveness of the suggested method in tackling real-world scenarios.

Enhancing Renewable Energy Integration:A Gaussian-Bare-Bones Levy Cheetah Optimization Approach to Optimal Power Flow in Electrical Networks

In the contemporary era,the global expansion of electrical grids is propelled by various renewable energy sources(RESs).Efficient integration of stochastic RESs and optimal power flow(OPF)management are critical for network optimization.This study introduces an innovative solution,the Gaussian Bare-Bones Levy Cheetah Optimizer(GBBLCO),addressing OPF challenges in power generation systems with stochastic RESs.The primary objective is to minimize the total operating costs of RESs,considering four functions:overall operating costs,voltage deviation management,emissions reduction,voltage stability index(VSI)and power loss mitigation.Additionally,a carbon tax is included in the objective function to reduce carbon emissions.Thorough scrutiny,using modified IEEE 30-bus and IEEE 118-bus systems,validates GBBLCO’s superior performance in achieving optimal solutions.Simulation results demonstrate GBBLCO’s efficacy in six optimization scenarios:total cost with valve point effects,total cost with emission and carbon tax,total cost with prohibited operating zones,active power loss optimization,voltage deviation optimization and enhancing voltage stability index(VSI).GBBLCO outperforms conventional techniques in each scenario,showcasing rapid convergence and superior solution quality.Notably,GBBLCO navigates complexities introduced by valve point effects,adapts to environmental constraints,optimizes costs while considering prohibited operating zones,minimizes active power losses,and optimizes voltage deviation by enhancing the voltage stability index(VSI)effectively.This research significantly contributes to advancing OPF,emphasizing GBBLCO’s improved global search capabilities and ability to address challenges related to local minima.GBBLCO emerges as a versatile and robust optimization tool for diverse challenges in power systems,offering a promising solution for the evolving needs of renewable energy-integrated power grids.

Optimal Quota-Share and Excess-of-Loss Reinsurance and Investment with Heston’s Stochastic Volatility Model

An optimal quota-share and excess-of-loss reinsurance and investment problem is studied for an insurer who is allowed to invest in a risk-free asset and a risky asset.Especially the price process of the risky asset is governed by Heston's stochastic volatility(SV)model.With the objective of maximizing the expected index utility of the terminal wealth of the insurance company,by using the classical tools of stochastic optimal control,the explicit expressions for optimal strategies and optimal value functions are derived.An interesting conclusion is found that it is better to buy one reinsurance than two under the assumption of this paper.Moreover,some numerical simulations and sensitivity analysis are provided.

Multi-source coordinated stochastic restoration for SOP in distribution networks with a two-stage algorithm

After suffering from a grid blackout, distributed energy resources(DERs), such as local renewable energy and controllable distributed generators and energy storage can be used to restore loads enhancing the system’s resilience. In this study, a multi-source coordinated load restoration strategy was investigated for a distribution network with soft open points(SOPs). Here, the flexible regulation ability of the SOPs is fully utilized to improve the load restoration level while mitigating voltage deviations. Owing to the uncertainty, a scenario-based stochastic optimization approach was employed,and the load restoration problem was formulated as a mixed-integer nonlinear programming model. A computationally efficient solution algorithm was developed for the model using convex relaxation and linearization methods. The algorithm is organized into a two-stage structure, in which the energy storage system is dispatched in the first stage by solving a relaxed convex problem. In the second stage, an integer programming problem is calculated to acquire the outputs of both SOPs and power resources. A numerical test was conducted on both IEEE 33-bus and IEEE 123-bus systems to validate the effectiveness of the proposed strategy.

Almost Sure Convergence of Proximal Stochastic Accelerated Gradient Methods

Proximal gradient descent and its accelerated version are resultful methods for solving the sum of smooth and non-smooth problems. When the smooth function can be represented as a sum of multiple functions, the stochastic proximal gradient method performs well. However, research on its accelerated version remains unclear. This paper proposes a proximal stochastic accelerated gradient (PSAG) method to address problems involving a combination of smooth and non-smooth components, where the smooth part corresponds to the average of multiple block sums. Simultaneously, most of convergence analyses hold in expectation. To this end, under some mind conditions, we present an almost sure convergence of unbiased gradient estimation in the non-smooth setting. Moreover, we establish that the minimum of the squared gradient mapping norm arbitrarily converges to zero with probability one.

Stochastic focusing search:a novel optimization algorithm for real-parameter optimization

A novel optimization algorithm called stochastic focusing search （SFS） for the real-parameter optimization is proposed. The new algorithm is a swarm intelligence algorithm, which is based on simulating the act of human randomized searching, and the human searching behaviors. The algorithm＇s performance is studied using a challenging set of typically complex functions with comparison of differential evolution （DE） and three modified particle swarm optimization （PSO） algorithms, and the simulation results show that SFS is competitive to solve most parts of the benchmark problems and will become a promising candidate of search algorithms especially when the existing algorithms have some difficulties in solving certain problems.

Important Issues and Results When Considering the Stochastic Representation of Wind Power Plants in a Generation Optimization Model: An Application to the Large Brazilian Interconnected Power System

Wind power has an increasing share of the Brazilian energy market and may represent 11.6% of total capacity by 2024. For large hydro-thermal systems having high-storage capacity, a complementarity between hydro and wind production could have important effects. The current optimization models are applied to dispatch power plants to meet the market demand and optimize the generation dispatches considering only hydroelectric and thermal power plants. The remaining sources, including wind power, small-hydroelectric plants and biomass plants, are excluded from the optimization model and are included deterministically. This work introduces a general methodology to represent the stochastic behavior of wind production aimed at the planning and operation of large interconnected power systems. In fact, considering the generation of the wind power source stochastically could show the complementarity between the hydro and wind power production, reducing the energy price in the spot market with the reduction of thermal power dispatches. In addition to that, with a reduction in wind power and a simultaneous dry-season occurrence, this model, is able to show the need of thermal power plants dispatches as well as the reduction of the risk of energy shortages.

Stability Analysis for Stochastic Optimization Problems

Stochastic optimization offers a means of considering the objectives and constrains with stochastic parameters. However, it is generally difficult to solve the stochastic optimization problem by employing conventional methods for nonlinear programming when the number of random variables involved is very large. Neural network models and algorithms were applied to solve the stochastic optimization problem on the basis of the stability theory. Stability for stochastic programs was discussed. If random vector sequence converges to the random vector in the original problem in distribution, the optimal value of the corresponding approximation problems converges to the optimal value of the original stochastic optimization problem.

ANew Theoretical Framework forAnalyzing Stochastic Global Optimization Algorithms

In this paper, we develop a new theoretical framework by means of the absorbing Markov process theory for analyzing some stochastic global optimization algorithms. Applying the framework to the pure random search, we prove that the pure random search converges to the global minimum in probability and its time has geometry distribution. We also analyze the pure adaptive search by this framework and turn out that the pure adaptive search converges to the global minimum in probability and its time has Poisson distribution.

A New Stochastic Algorithm of Global Optimization ——Region's Walk and Contraction

This paper presents a new stochastic algorithm for box constrained global optimization problem. Bacause the level set of objective function is always not known, the authors designed a region containing the current minimum point to replace it, and in order to fit the level set well, this region would be walking and contracting in the running process. Thus, the new algorithm is named as region's walk and contraction(RWC). Some numerical experiments for the RWC were conducted, which indicate good property of the algorithm.