Distributed Stochastic Optimization with Compression for Non-Strongly Convex Objectives

We are investigating the distributed optimization problem,where a network of nodes works together to minimize a global objective that is a finite sum of their stored local functions.Since nodes exchange optimization parameters through the wireless network,large-scale training models can create communication bottlenecks,resulting in slower training times.To address this issue,CHOCO-SGD was proposed,which allows compressing information with arbitrary precision without reducing the convergence rate for strongly convex objective functions.Nevertheless,most convex functions are not strongly convex(such as logistic regression or Lasso),which raises the question of whether this algorithm can be applied to non-strongly convex functions.In this paper,we provide the first theoretical analysis of the convergence rate of CHOCO-SGD on non-strongly convex objectives.We derive a sufficient condition,which limits the fidelity of compression,to guarantee convergence.Moreover,our analysis demonstrates that within the fidelity threshold,this algorithm can significantly reduce transmission burden while maintaining the same convergence rate order as its no-compression equivalent.Numerical experiments further validate the theoretical findings by demonstrating that CHOCO-SGD improves communication efficiency and keeps the same convergence rate order simultaneously.And experiments also show that the algorithm fails to converge with low compression fidelity and in time-varying topologies.Overall,our study offers valuable insights into the potential applicability of CHOCO-SGD for non-strongly convex objectives.Additionally,we provide practical guidelines for researchers seeking to utilize this algorithm in real-world scenarios.

Reliable Space Pursuing for Reliability-based Design Optimization with Black-box Performance Functions

Reliability-based design optimization （RBDO） is intrinsically a double-loop procedure since it involves an overall optimization and an iterative reliability assessment at each search point. Due to the double-loop procedure, the computational expense of RBDO is normally very high. Current RBDO research focuses on problems with explicitly expressed performance functions and readily available gradients. This paper addresses a more challenging type of RBDO problem in which the performance functions are computation intensive. These computation intensive functions are often considered as a ＂black-box＂ and their gradients are not available or not reliable. On the basis of the reliable design space （RDS） concept proposed earlier by the authors, this paper proposes a Reliable Space Pursuing （RSP） approach, in which RDS is first identified and then gradually refined while optimization is performed. It fundamentally avoids the nested optimization and probabilistic assessment loop. Three well known RBDO problems from the literature are used for testing and demonstrating the effectiveness of the proposed RSP method.

Application of the optimal Latin hypercube design and radial basis function network to collaborative optimization

Improving the efficiency of ship optimization is crucial for modem ship design. Compared with traditional methods, multidisciplinary design optimization （MDO） is a more promising approach. For this reason, Collaborative Optimization （CO） is discussed and analyzed in this paper. As one of the most frequently applied MDO methods, CO promotes autonomy of disciplines while providing a coordinating mechanism guaranteeing progress toward an optimum and maintaining interdisciplinary compatibility. However, there are some difficulties in applying the conventional CO method, such as difficulties in choosing an initial point and tremendous computational requirements. For the purpose of overcoming these problems, optimal Latin hypercube design and Radial basis function network were applied to CO. Optimal Latin hypercube design is a modified Latin Hypercube design. Radial basis function network approximates the optimization model, and is updated during the optimization process to improve accuracy. It is shown by examples that the computing efficiency and robustness of this CO method are higher than with the conventional CO method.

Two optimization algorithms for solving robotics inverse kinematics with redundancy

The kinematic redundancy in a robot leads to an infinite number of solutions for inverse kinematics, which implies the possibility to select a 'best' solution according to an optimization criterion. In this paper, two optimization objective functions are proposed, aiming at either minimizing extra degrees of freedom (DOFs) or minimizing the total potential energy of a multilink redundant robot. Physical constraints of either equality or inequality types are taken into consideration in the objective functions. Since the closed-form solutions do not exist in general for highly nonlinear and constrained optimization problems, we adopt and develop two numerical methods, which are verified to be effective and precise in solving the two optimization problems associated with the redundant inverse kinematics. We first verify that the well established trajectory following method can precisely solve the two optimization problems, but is computation intensive. To reduce the computation time, a sequential approach that combines the sequential quadratic programming and iterative Newton-Raphson algorithm is developed. A 4-DOF Fujitsu Hoap-1 humanoid robot arm is used as a prototype to validate the effectiveness of the proposed optimization solutions.

OPTIMALITY CONDITIONS AND DUALITY RESULTS FOR NONSMOOTH VECTOR OPTIMIZATION PROBLEMS WITH THE MULTIPLE INTERVAL-VALUED OBJECTIVE FUNCTION

In this paper, both Fritz John and Karush-Kuhn-Tucker necessary optimality conditions are established for a （weakly） LU-efficient solution in the considered nonsmooth multiobjective programming problem with the multiple interval-objective function. Further, the sufficient optimality conditions for a （weakly） LU-efficient solution and several duality results in Mond-Weir sense are proved under assumptions that the functions constituting the considered nondifferentiable multiobjective programming problem with the multiple interval- objective function are convex.

A Hybrid Level Set Optimization Design Method of Functionally Graded Cellular Structures Considering Connectivity

With the continuous advancement in topology optimization and additive manufacturing(AM)technology,the capability to fabricate functionally graded materials and intricate cellular structures with spatially varying microstructures has grown significantly.However,a critical challenge is encountered in the design of these structures–the absence of robust interface connections between adjacent microstructures,potentially resulting in diminished efficiency or macroscopic failure.A Hybrid Level Set Method(HLSM)is proposed,specifically designed to enhance connectivity among non-uniform microstructures,contributing to the design of functionally graded cellular structures.The HLSM introduces a pioneering algorithm for effectively blending heterogeneous microstructure interfaces.Initially,an interpolation algorithm is presented to construct transition microstructures seamlessly connected on both sides.Subsequently,the algorithm enables the morphing of non-uniform unit cells to seamlessly adapt to interconnected adjacent microstructures.The method,seamlessly integrated into a multi-scale topology optimization framework using the level set method,exhibits its efficacy through numerical examples,showcasing its prowess in optimizing 2D and 3D functionally graded materials(FGM)and multi-scale topology optimization.In essence,the pressing issue of interface connections in complex structure design is not only addressed but also a robust methodology is introduced,substantiated by numerical evidence,advancing optimization capabilities in the realm of functionally graded materials and cellular structures.

SELECTION OF OBJECTIVE FUNCTIONS AND APPLICATION OF GENETIC ALGORITHMS IN DAMPING DESIGN OF PIPE SYSTEM

The vibration failure of pipe system of aeroengine seriously influences the safety of aircraft.Its damping design is determined by the selection of the design target,method and their feasibility.Five objective functions for the vibration design of a pipeline or pipe system are introduced,namely,the frequency,amplitude,transfer ratio,curvature and deformation energy as options for the optimization process.The genetic algorithms(GA)are adopted as the opti- mization method,in which the selection of the adaptive genetic operators and the method of implementation of the GA process are crucial.The optimization procedure for all the above ob- jective functions is carried out using GA on the basis of finite element software-MSC/NASTRAN. The optimal solutions of these functions and the stress distribution on the structure are calculated and compared through an example,and their characteristics are analyzed.Finally we put forward two new objective functions,curvature and deformation energy for pipe system optimization.The calculations show that using the curvature as the objective function can reflect the case of minimal stress,and the optimization results using the deformation energy represent lesser and more uni- form stress distribution.The calculation results and process showed that the genetic algorithms can effectively implement damping design of engine pipelines and satisfy the efficient engineering design requirement.

Multidisciplinary Design Optimization of Vehicle Instrument Panel Based on Multi-objective Genetic Algorithm

Typical multidisciplinary design optimization（MDO） has gradually been proposed to balance performances of lightweight, noise, vibration and harshness（NVH） and safety for instrument panel（IP） structure in the automotive development. Nevertheless, plastic constitutive relation of Polypropylene（PP） under different strain rates, has not been taken into consideration in current reliability-based and collaborative IP MDO design. In this paper, based on tensile test under different strain rates, the constitutive relation of Polypropylene material is studied. Impact simulation tests for head and knee bolster are carried out to meet the regulation of FMVSS 201 and FMVSS 208, respectively. NVH analysis is performed to obtain mainly the natural frequencies and corresponding mode shapes, while the crashworthiness analysis is employed to examine the crash behavior of IP structure. With the consideration of lightweight, NVH, head and knee bolster impact performance, design of experiment（DOE）, response surface model（RSM）, and collaborative optimization（CO） are applied to realize the determined and reliability-based optimizations, respectively. Furthermore, based on multi-objective genetic algorithm（MOGA）, the optimal Pareto sets are completed to solve the multi-objective optimization（MOO） problem. The proposed research ensures the smoothness of Pareto set, enhances the ability of engineers to make a comprehensive decision about multi-objectives and choose the optimal design, and improves the quality and efficiency of MDO.

Multi-objective Optimization Conceptual Design of Product Structure Based on Variable Length Gene Expression

It is a complicated problem for the bottom-to-top adaptive conceptual design of complicated products between structure and function. Reliable theories demand to be found in order to determine whether the structure accords with the requirement of design. For the requirement generally is dynamic variety as time passes, new requirements will come, and some initial requirements can no longer be used. The number of product requirements, the gene length expressing requirements, the structure of the product, and the correlation matrix are varied with individuation of customer requirements of the product. By researching on the calculation mechanisms of dynamic variety, the approaches of gene expression and variable length gene expression are proposed. According to the diversity of structure selection in conceptual design and mutual relations between structure and function as well as structure and structure, the correlation matrixes between structure and function as well as structure and structure are defined. By the approach of making the sum of the elements of correlation matrix maximum, the mathematical models of multi-object optimization for structure design are provided based on variable requirements. An improved genetic algorithm called segment genetic algorithm is proposed based on optimization preservation simple genetic algorithm. The models of multi-object optimization are calculated by the segment genetic algorithm and hybrid genetic algorithm. An example for the conceptual design of a washing machine is given to show that the proposed method is able to realize the optimization structure design fitting for variable requirements. In addition, the proposed approach can provide good Pareto optimization solutions, and the individuation customer requirements for structures of products are able to be resolved effectively.

Multi-objective Optimisation Design of Water Distribution Systems:Comparison of Two Evolutionary Algorithms

In order to compare two advanced multi-objective evolutionary algorithms,a multi-objective water distribution problem is formulated in this paper.The multi-objective optimization has received more attention in the water distribution system design.On the one hand the cost of water distribution system including capital,operational,and maintenance cost is mostly concerned issue by the utilities all the time;on the other hand improving the performance of water distribution systems is of equivalent importance,which is often conflicting with the previous goal.Many performance metrics of water networks are developed in recent years,including total or maximum pressure deficit,resilience,inequity,probabilistic robustness,and risk measure.In this paper,a new resilience metric based on the energy analysis of water distribution systems is proposed.Two optimization objectives are comprised of capital cost and the new resilience index.A heuristic algorithm,speedconstrained multi-objective particle swarm optimization( SMPSO) extended on the basis of the multi-objective particle swarm algorithm,is introduced to compare with another state-of-the-art heuristic algorithm,NSGA-II.The solutions are evaluated by two metrics,namely spread and hypervolume.To illustrate the capability of SMPSO to efficiently identify good designs,two benchmark problems( two-loop network and Hanoi network) are employed.From several aspects the results demonstrate that SMPSO is a competitive and potential tool to tackle with the optimization problem of complex systems.

Integrated Building Envelope Design Process Combining Parametric Modelling and Multi-Objective Optimization

As an important element in sustainable building design, the building envelope has been witnessing a constant shift in the design approach. Integrating multi-objective optimization (MOO) into the building envelope design process is very promising, but not easy to realize in an actual project due to several factors, including the complexity of optimization model construction, lack of a dynamic-visualization capacity in the simulation tools and consideration of how to match the optimization with the actual design process. To overcome these difficulties, this study constructed an integrated building envelope design process (IBEDP) based on parametric modelling, which was implemented using Grasshopper platform and interfaces to control the simulation software and optimization algorithm. A railway station was selected as a case study for applying the proposed IBEDP, which also utilized a grid-based variable design approach to achieve flexible optimum fenestrations. To facilitate the stepwise design process, a novel strategy was proposed with a two-step optimization, which optimized various categories of variables separately. Compared with a one-step optimization, though the proposed strategy performed poorly in the diversity of solutions, the quantitative assessment of the qualities of Pareto-optimum solution sets illustrates that it is superior. © 2016, Tianjin University and Springer-Verlag Berlin Heidelberg.

Interface Design and Functional Optimization of Chinese Learning Apps Based on User Experience

This research paper investigates the interface design and functional optimization of Chinese learning apps through the lens of user experience.With the increasing popularity of Chinese language learning apps in the era of rapid mobile internet development,users'demands for enhanced interface design and interaction experience have grown significantly.The study aims to explore the influence of user feedback on the design and functionality of Chinese learning apps,proposing optimization strategies to improve user experience and learning outcomes.By conducting a comprehensive literature review,utilizing methods such as surveys and user interviews for data collection,and analyzing user feedback,this research identifies existing issues in the interface design and interaction experience of Chinese learning apps.The results present user opinions,feedback analysis,identified problems,improvement directions,and specific optimization strategies.The study discusses the potential impact of these optimization strategies on enhancing user experience and learning outcomes,compares findings with previous research,addresses limitations,and suggests future research directions.In conclusion,this research contributes to enriching the design theory of Chinese learning apps,offering practical optimization recommendations for developers,and supporting the continuous advancement of Chinese language learning apps.

Adaptive Learning Rate Optimization BP Algorithm with Logarithmic Objective Function

This paper presents an improved BP algorithm. The approach can reduce the amount of computation by using the logarithmic objective function. The learning rate μ(k) per iteration is determined by dynamic optimization method to accelerate the convergence rate. Since the determination of the learning rate in the proposed BP algorithm only uses the obtained first order derivatives in standard BP algorithm(SBP), the scale of computational and storage burden is like that of SBP algorithm,and the convergence rate is remarkably accelerated. Computer simulations demonstrate the effectiveness of the proposed algorithm

An Objective Penalty Functions Algorithm for Multiobjective Optimization Problem

By using the penalty function method with objective parameters, the paper presents an interactive algorithm to solve the inequality constrained multi-objective programming (MP). The MP is transformed into a single objective optimal problem (SOOP) with inequality constrains;and it is proved that, under some conditions, an optimal solution to SOOP is a Pareto efficient solution to MP. Then, an interactive algorithm of MP is designed accordingly. Numerical examples show that the algorithm can find a satisfactory solution to MP with objective weight value adjusted by decision maker.

A Robust Optimization Approach Considering the Robustness of Design Objectives and Constraints

The problem of robust design is treated as a multi-objective optimization issue in which the performance mean and variation are optimized and minimized respectively, while maintaining the feasibility of design constraints under uncertainty. To effectively address this issue in robust design, this paper presents a novel robust optimization approach which integrates multi-objective optimization concepts with Taguchi’s crossed arrays techniques. In this approach, Pareto-optimal robust design solution sets are obtained with the aid of design of experiment set-ups, which utilize the results of Analysis of Variance to quantify relative dominance and significance of design variables. A beam design problem is used to illustrate the effectiveness of the proposed approach.

Rapid design of secondary deformation-aging parameters for ultra-low Co content Cu-Ni-Co-Si-X alloy via Bayesian optimization machine learning

It is difficult to rapidly design the process parameters of copper alloys by using the traditional trial-and-error method and simultaneously improve the conflicting mechanical and electrical properties.The purpose of this work is to develop a new type of Cu-Ni-Co-Si alloy saving scarce and expensive Co element,in which the Co content is less than half of the lower limit in ASTM standard C70350 alloy,while the properties are as the same level as C70350 alloy.Here we adopted a strategy combining Bayesian optimization machine learning and experimental iteration and quickly designed the secondary deformation-aging parameters(cold rolling deformation 90%,aging temperature 450℃,and aging time 1.25 h)of the new copper alloy with only 32 experiments(27 basic sample data acquisition experiments and 5 iteration experiments),which broke through the barrier of low efficiency and high cost of trial-and-error design of deformation-aging parameters in precipitation strengthened copper alloy.The experimental hardness,tensile strength,and electrical conductivity of the new copper alloy are HV(285±4),(872±3)MPa,and(44.2±0.7)%IACS(international annealed copper standard),reaching the property level of the commercial lead frame C70350 alloy.This work provides a new idea for the rapid design of material process parameters and the simultaneous improvement of mechanical and electrical properties.

Overview of multi-objective optimization methods

To assist readers to have a comprehensive understanding, the classical and intelligent methods roundly based on precursory research achievements are summarized in this paper. First, basic conception and description about multi-objective (MO) optimization are introduced. Then some definitions and related terminologies are given. Furthermore several MO optimization methods including classical and current intelligent methods are discussed one by one succinctly. Finally evaluations on advantages and disadvantages about these methods are made at the end of the paper.

Multiple-response optimization for melting process of aluminum melting furnace based on response surface methodology with desirability function

To reduce the fuel consumption and emissions and also enhance the molten aluminum quality, a mathematical model with user-developed melting model and burning capacity model, were established according to the features of melting process of regenerative aluminum melting furnaces. Based on validating results by heat balance test for an aluminum melting furnace, CFD （computational fluid dynamics） technique, in association with statistical experimental design were used to optimize the melting process of the aluminum melting furnace. Four important factors influencing the melting time, such as horizontal angle between burners, height-to-radius ratio, natural gas mass flow and air preheated temperature, were identified by PLACKETT-BURMAN design. A steepest descent method was undertaken to determine the optimal regions of these factors. Response surface methodology with BOX-BEHNKEN design was adopted to further investigate the mutual interactions between these variables on RSD （relative standard deviation） of aluminum temperature, RSD of furnace temperature and melting time. Multiple-response optimization by desirability function approach was used to determine the optimum melting process parameters. The results indicate that the interaction between the height-to-radius ratio and horizontal angle between burners affects the response variables significantly. The predicted results show that the minimum RSD of aluminum temperature （12.13%）, RSD of furnace temperature （18.50%） and melting time （3.9 h） could be obtained under the optimum conditions of horizontal angle between burners as 64°, height-to-radius ratio as 0.3, natural gas mass flow as 599 m3/h, and air preheated temperature as 639 ℃. These predicted values were further verified by validation experiments. The excellent correlation between the predicted and experimental values confirms the validity and practicability of this statistical optimum strategy.

Well production optimization using streamline features-based objective function and Bayesian adaptive direct search algorithm

Well production optimization is a complex and time-consuming task in the oilfield development.The combination of reservoir numerical simulator with optimization algorithms is usually used to optimize well production.This method spends most of computing time in objective function evaluation by reservoir numerical simulator which limits its optimization efficiency.To improve optimization efficiency,a well production optimization method using streamline features-based objective function and Bayesian adaptive direct search optimization(BADS)algorithm is established.This new objective function,which represents the water flooding potential,is extracted from streamline features.It only needs to call the streamline simulator to run one time step,instead of calling the simulator to calculate the target value at the end of development,which greatly reduces the running time of the simulator.Then the well production optimization model is established and solved by the BADS algorithm.The feasibility of the new objective function and the efficiency of this optimization method are verified by three examples.Results demonstrate that the new objective function is positively correlated with the cumulative oil production.And the BADS algorithm is superior to other common algorithms in convergence speed,solution stability and optimization accuracy.Besides,this method can significantly accelerate the speed of well production optimization process compared with the objective function calculated by other conventional methods.It can provide a more effective basis for determining the optimal well production for actual oilfield development.