Combining reinforcement learning with mathematical programming:An approach for optimal design of heat exchanger networks

Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.

Glia-to-neuron reprogramming to the rescue?

Over the last two decades,the dogma that cell fate is immutable has been increasingly challenged,with important implications for regenerative medicine.The brea kth rough discovery that induced pluripotent stem cells could be generated from adult mouse fibroblasts is powerful proof that cell fate can be changed.An exciting extension of the discovery of cell fate impermanence is the direct cellular reprogram ming hypothesis-that terminally differentiated cells can be reprogrammed into other adult cell fates without first passing through a stem cell state.

Neurocircuit regeneration by extracellular matrix reprogramming

The brain's extracellular matrix(ECM),which is comprised of protein and glycosaminoglycan(GAG)scaffolds,constitutes 20%-40% of the human brain and is considered one of the largest influencers on brain cell functioning(Soles et al.,2023).Synthesized by neural and glial cells,the brain's ECM regulates a myriad of homeostatic cellular processes,including neuronal plasticity and firing(Miyata et al.,2012),cation buffering(Moraws ki et al.,2015),and glia-neuron interactions(Anderson et al.,2016).Considering the diversity of functions,dynamic remodeling of the brain's ECM indicates that this understudied medium is an active participant in both normal physiology and neurological diseases.

A Partial Parallel Airport Gate Assignment Supported by a Knowledge－based System Combined with Mathematical Programming

The gate assignment at an airport is one of the major activities in airport operations.With the increase of passenger traffic volumes and the number of flights, the complexity of this task and the factors to be considered have increased significantly, and an efficient gate utilizationhas received considerable attention. For overcoming the shortcomings of previous gate assignmentapproaches, this paper presents a partial parallel gate assignment approach, by which more factorsconcerning aircraft and gates can be collsidered at the same time. This paper also presents themethod of using a knowledge-based system combined with a mathematical programming method forgetting an optimized feasible assignment solution. By this way, it is more easily to get the solutionthat satisfies both the static and dynamic situations,and thus it may adapt well to meet the needsof actual use to rea-time operations. An experimental prototype has been implemented, and a casestudy is presented at the end of the paper.

Hierarchical On-line Scheduling of Multiproduct Batch Plants with a Combined Approach of Mathematical Programming and Genetic Algorithm

In this contribution we present an online scheduling algorithm for a real world multiproduct batch plant. The overall mixed integer nonlinear programming (MINLP) problem is hierarchically structured into a mixed integer linear programming (MILP) problem first and then a reduced dimensional MINLP problem, which are optimized by mathematical programming (MP) and genetic algorithm (GA) respectively. The basis idea relies on combining MP with GA to exploit their complementary capacity. The key features of the hierarchical model are explained and illustrated with some real world cases from the multiproduct batch plants.

MATHEMATICAL PROGRAMMING ANALYSIS OF BILINEAR RHEOLOGICAL LUBRICATION MECHANICS

Bilinear theological lubrication mechanics provides an important basis for the designs of re- cently developed electrorheological(ER)'smart'journal bearings and those lubricated by mixed fluid-solid lubri- cants.But there is not yet a reliable and efficient numerical method for such a problem of non-Newtonian flu- id mechanics.In the present paper,a finite element method(FEM)together with mat hematical programming solution is successfully used to solve such a problem.A reliable and generalized numerical method for the designs of electrorheological 'smart' journal bearings and the bearings lubricated by mixed fluid- solid lubri- cant is presented.

Neutrosophic Mathematical Programming for Optimization of Multi-Objective Sustainable Biomass Supply Chain Network Design

In this paper,a multi-objective sustainable biomass supply chain network under uncertainty is designed by neutrosophic programming method.In this method,for each objective function of the problem,three functions of truth membership,non-determination and falsehood are considered.Neutrosophic programming method in this paper simultaneously seeks to optimize the total costs of the supply chain network,the amount of greenhouse gas emissions,the number of potential people hired and the time of product transfer along the supply chain network.To achieve the stated objective functions,strategic decisions such as locating potential facilities and tactical decisions such as optimal product flow allocation and vehicle routing must be made.The results of the implementation of neutrosophic programming method show the high efficiency of this method in achieving the optimal values of each objective function.Also,by examining the rate of uncertainty,it was observed that with increasing this rate,the total costs of supply chain network design,greenhouse gas emissions and product transfer times have increased,and in contrast,the potential employment rate of individuals has decreased.

OPTIMIZATION OF THE TAKE-OFF MOVEMENT OF SKI JUMPING WITH THE METHOD OF MATHEMATICAL PROGRAMMING

This paper is based on the finite and dispersed data which were obtained from the experiments of the wind tunnel and of the force measurement and from the high-speed photography. It analyses and optimizes the take-off movement of ski jumping with the theory of dynamics of systems of rigid bodies and with the method of mathematical programming. The paper describes the optimal take-off movement of ski jumping. Furthermore, it presents an example and compares the result with those of other papers published at home and abroad. The comparison shows that our computation and optimization are reasonable and well-grounded.

A FINITE ELEMENT—MATHEMATICAL PROGRAMMING METHOD FOR ELASTOPLASTIC PROBLEMS BASED ON THE PRINCIPLE OF VIRTUAL WORK

By expanding the yielding function according toTaylor series and neglecting the high order terms, the elastoplastic constitutive equation is written in a linear complementary form. Based on this linear complementary form and the principle of virtual work, a finite element-complementary method is derived for elastoplastic problem. This method is available for materials which satisfy either associated or nonassociated flow rule. In addition, the existence and uniqueness oj solution for the method are also discussed and some useful conclusions are given.

A MATHEMATICAL PROGRAMMING ALGORITHM FOR LIMIT ANALYSIS

This paper deals with the limit analyses of perfect rigid-plastic continua.Based on the kinematic theorem of the limit analysis theory,a mathematical programming finite element formula for determining the upper bound load multiplier has been established,and an iteration algorithm proposed accordingly.In this algorithm the plastic and rigid zones are distinguished for every iteration step,and the goal function is modified gradually.The difficulties caused by the nonsmoothness of the goal function are over- come.Some examples solved by this algorithm are presented.

Bi-matrix Games with Fuzzy Strategies and Fuzzy Payoffs and Their Mathematical Programming Equivalents

A fuzzy bi-matrix game（FBG）,namely a two-person non-zero-sum game with fuzzy strategies and fuzzy payoffs is proposed.We have defined and analyzed the optimal strategies of this FBG,and shown that it can be transformed into a corresponding fuzzy mathematical programming issue,for which a ranking function approach can be applied.In addition,optimal strategies of FBG for both Player I and Player II can be gotten.

Mathematical Programming Method as a Means of Calibration of NIR Analyzers

In addition to the conventional methods of the calibration model construction, such as PCR （principal components regression） and PLS （partial least-squares）, a MPM （mathematical programming method） is developed and proposed for practical use in NIR analyses of agricultural and food products. The proposed method involves the mathematical programming techniques to seek the regression coefficients for the calibration model calculation. It is based on the optimization theory used for finding the extremum of the objective function in the given domain of a vector space and employs the method of the complementarity problems solving. The MPM algorithm is described in detail. The MPM was tested on an InfraLUM FT-10 NIR analyzer of Lumex company with samples of dry milk （for fat）, corn （for protein） and rye flour （for moisture）. The obtained results show that the MPM can be used for constructing multivariate calibrations with the qualitative characteristics superior over those of the classical PCR and PLS methods of analysis.

A Numerical Method for Rigid-plastic FEM Analysis Basing on Mathematical Programming

The rigid-plastic analysis of mental forming simulation is formulated as a discrete nonlinear mathematical programming problem with equality and inequality constraints by means of the finite element technique. An iteration algorithm is used to solve this formulation, which distinguishes the integration points of the rigid zones and the plastic zones and solves a series of the quadratic programming to overcome the difficulties caused by the nonsmoothness and the nonlinearity of the objective function. This method has been used to carry out the rigid-plastic FEM analysis. An example is given to demonstrate the effectiveness of this method.

Estimation of CARA Preferences and Positive Mathematical Programming

The purpose of this paper is to combine the estimation of output price risk and positive mathematical programming (PMP). It reconciles the risk programming presented by Freund with a consistent estimate of the constant absolute risk aversion (CARA) coefficient. It extends the PMP approach to calibration of realized production outputs and observed input prices. The results of this specification include 1) uniqueness of the calibrating solution, 2) elimination of the tautological calibration constraints typical of the original PMP procedure, 3) equivalence between a phase I calibrating solution and a solution obtained by combining phase I and phase II of the traditional PMP procedure. In this extended PMP framework, the cost function specification involves output quantities and input prices—contrary to the myopic cost function of the traditional PMP approach. This extension allows for a phase III calibrating model that replaces the usual linear technology with relations corresponding to Shephard lemma (in the primal constraints) and the marginal cost function (in the dual constraints). An empirical example with a sample of farms producing four crops illustrates the novel procedure.

Optimal Solution of Multi-Choice Mathematical Programming Problem Using a New Technique

The study deals with the multi-choice mathematical programming problem, where the right hand side of the constraints is multi-choice in nature. However, the problem of multi-choice linear programming cannot be solved directly by standard linear or nonlinear programming techniques. The aim of this paper is to transform such problems to a standard mathematical linear programming problem. For each constraint, exactly one parameter value is selected out of a multiple number of parameter values. This process of selection can be established in different ways. In this paper, we present a new simple technique enabling us to handle such problem as a mixed integer linear programming problem and consequently solve them by using standard linear programming software. Our main aim depends on inserting a specific number of binary variables and using them to construct a linear combination which gives just one parameter among the multiple choice values for each choice of the values of the binary variables. A numerical example is presented to illustrate our analysis.

On Decompositions of Real Polynomials Using Mathematical Programming Methods

We present a procedure that gives us an SOS (sum of squares) decomposition of a given real polynomial in variables, if there exists such decomposition. For the case of real polynomials in non-commutative variables we extend this procedure to obtain a sum of hermitian squares SOHS) decomposition whenever there exists any. This extended procedure is the main scientific contribution of the paper.

Uncertainty quantification of inverse analysis for geomaterials using probabilistic programming

Uncertainty is an essentially challenging for safe construction and long-term stability of geotechnical engineering.The inverse analysis is commonly utilized to determine the physico-mechanical parameters.However,conventional inverse analysis cannot deal with uncertainty in geotechnical and geological systems.In this study,a framework was developed to evaluate and quantify uncertainty in inverse analysis based on the reduced-order model(ROM)and probabilistic programming.The ROM was utilized to capture the mechanical and deformation properties of surrounding rock mass in geomechanical problems.Probabilistic programming was employed to evaluate uncertainty during construction in geotechnical engineering.A circular tunnel was then used to illustrate the proposed framework using analytical and numerical solution.The results show that the geomechanical parameters and associated uncertainty can be properly obtained and the proposed framework can capture the mechanical behaviors under uncertainty.Then,a slope case was employed to demonstrate the performance of the developed framework.The results prove that the proposed framework provides a scientific,feasible,and effective tool to characterize the properties and physical mechanism of geomaterials under uncertainty in geotechnical engineering problems.

Combined Water-Oxygen Pinch Analysis with Mathematical Programming for Wastewater Treatment

Water-oxygen pinch analysis is an effective method to decrease the wastewater quantity and improve the wastewater quality. But when multiple-contaminants are present, the method is difficult to be carried out. In this paper, the method that combines water-oxygen pinch analysis with mathematical programming is proposed. It obtains the general optimal solution and leads to the reuse stream that cannot be found only by pinch analysis. The new method is illustrated by an example, and the annual cost is reduced by 8.43% compared with the solution of literature.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Mathematical modeling on multi-stage series crushing ratio distribu- tion based on fuzzy physical programming

Double-layer, multi-roller plate crusher is a new device, that uses a multi-stage series crushing style to break particles, with the crushing ratio distribution directly influencing the machine＇s performance. Three crushing ratios of 2.25, 2.15 and 2.01, used for fuzzy physical programming, were determined. The comparison of the optimized result between the double-layer multi-roller plate crusher and a high pressure roll grinder showed that the double-layer multi-roller plate crusher had a better performance, reducing crushing force and wear.