Combining reinforcement learning with mathematical programming:An approach for optimal design of heat exchanger networks

Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Improved design of reconfigurable frequency response masking filters based on second-order cone programming

In order to improve the design results for the reconfigurable frequency response masking FRM filters an improved design method based on second-order cone programming SOCP is proposed.Unlike traditional methods that separately design the proposed method takes all the desired designing modes into consideration when designing all the subfilters. First an initial solution is obtained by separately designing the subfilters and then the initial solution is updated by iteratively solving a SOCP problem. The proposed method is evaluated on a design example and simulation results demonstrate that jointly designing all the subfilters can obtain significantly lower minimax approximation errors compared to the conventional design method.

Multi-gene genetic programming extension of AASHTO M-E for design oflow-volume concrete pavements

The American Association of State Highway and Transportation Officials Mechanistic-Empirical Pavement DesignGuide (AASHTO M-E) offers an opportunity to design more economical and sustainable high-volume rigid pavementscompared to conventional design guidelines. It is achieved through optimizing pavement structural andthickness design under specified climate and traffic conditions using advanced M-E principles, thereby minimizingeconomic costs and environmental impact. However, the implementation of AASHTO M-E design for low-volumeconcrete pavements using AASHTOWare Pavement ME Design (Pavement ME) software is often overly conservative.This is because Pavement ME specifies the minimum design thickness of concrete slab as 152.4 mm (6 in.). Thispaper introduces a novel extension of the AASHTO M-E framework for the design of low-volume joint plain concretepavements (JPCPs) without modification of Pavement ME. It utilizes multi-gene genetic programming (MGGP)-based computational models to obtain rapid solutions for JPCP damage accumulation and long-term performanceanalyses. The developed MGGP models simulate the fatigue damage and differential energy accumulations. Thispermits the prediction of transverse cracking and joint faulting for a wide range of design input parameters and axlespectrum. The developed MGGP-based models match Pavement ME-predicted cracking and faulting for rigidpavements with conventional concrete slab thicknesses and enable rational extrapolation of performance predictionfor thinner JPCPs. This paper demonstrates how the developed computational model enables sustainable lowvolumepavement design using optimized ME solutions for Pittsburgh, PA, conditions.

Evolutionary Design of Fault-Tolerant Digital Circuit Based on Cartesian Genetic Programming

In many areas, reliability of the digital circuits has become the key factor to restrict circuit development. Fault-tolerant design is the commonly used method to improve the reliability of digital circuits. The current fault-tolerant design methods are based on triple modular redundancy( TMR) or multiple modular redundancy( MMR). These redundancy designs rely on the experience of the designers,and the designed circuits have poor adaptabilities to a complex environment. However, evolutionary design of digital circuits does not rely on prior knowledge. During the evolution, some novel and optimal circuit topologies can be found, and the evolved circuits can feature strong adaptive capacities. Based on Cartesian genetic programming( CGP), a novel method for designing fault-tolerant digital circuits by evolution is proposed,key steps of the evolution are introduced,influences of function sets on evolution are investigated,and as a preliminary result,an evolved full adder with high fault-tolerance is shown.

Semi-definite programming based pulse waveform design and its further analysis

In order to provide a judicious pulse waveform design required for ultra-wideband(UWB)communication to enable the UWB spectral mask compatible and coexistent with other existing wireless communication systems,a semi-definite programming(SDP)based pulse waveform design method for UWB radios is introduced and a further analysis is given in this paper.By using Sedumi and Yalmip toolboxes of Matlab,the procedure of solving the SDP problem is simplified.Simulation results show that this SDP based pulse waveform design method can be used to design pulses that fulfill the Federal Communications Commission(FCC)spectral mask strictly and optimize the power efficiency at the same time.This paper also analyzes the influences of the power efficiency duing to the changes of sampling interval and the number of combined pulses,and then the optimal sampling interval that maximizes the transmission power can be found.

Parallel Processing Design for LTE PUSCH Demodulation and Decoding Based on Multi-Core Processor

The Long Term Evolution (LTE) system imposes high requirements for dispatching delay.Moreover,very large air interface rate of LTE requires good processing capability for the devices processing the baseband signals.Consequently,the single-core processor cannot meet the requirements of LTE system.This paper analyzes how to use multi-core processors to achieve parallel processing of uplink demodulation and decoding in LTE systems and designs an approach to parallel processing.The test results prove that this approach works quite well.

Decomposition of Mathematical Programming Models for Aircraft Wing Design Facilitating the Use of Dynamic Programming Approach

Aircraft designers strive to achieve optimal weight-reliability tradeoffs while designing an aircraft. Since aircraft wing skins account for more than fifty percent of their structural weight, aircraft wings must be designed with utmost care and attention in terms of material types and thickness configurations. In particular, the selection of thickness at each location of the aircraft wing skin is the most consequential task for aircraft designers. To accomplish this, we present discrete mathematical programming models to obtain optimal thicknesses either to minimize weight or to maximize reliability. We present theoretical results for the decomposition of these discrete mathematical programming models to reduce computer memory requirements and facilitate the use of dynamic programming for design purposes. In particular, a decomposed version of the weight minimization problem is solved for an aircraft wing with thirty locations (or panels) and fourteen thickness choices for each location to yield an optimal minimum weight design.

A Dynamic Programming Approach to the Design of Composite Aircraft Wings

A light and reliable aircraft has been the major goal of aircraft designers. It is imperative to design the aircraft wing skins as efficiently as possible since the wing skins comprise more than fifty percent of the structural weight of the aircraft wing. The aircraft wing skin consists of many different types of material and thickness configurations at various locations. Selecting a thickness for each location is perhaps the most significant design task. In this paper, we formulate discrete mathematical programming models to determine the optimal thicknesses for three different criteria: maximize reliability, minimize weight, and achieve a trade-off between maximizing reliability and minimizing weight. These three model formulations are generalized discrete resource-allocation problems, which lend themselves well to the dynamic programming approach. Consequently, we use the dynamic programming method to solve these model formulations. To illustrate our approach, an example is solved in which dynamic programming yields a minimum weight design as well as a trade-off curve for weight versus reliability for an aircraft wing with thirty locations (or panels) and fourteen thickness choices for each location.

Nonlinear Programming Based Preamble Design for OFDM Systems

A new preamble structure and design method for orthogonal frequency division multiplexing（OFDM）systems is described,which results a two-symbol long training preamble.The preamble contains four parts,the first part is the same as the third,and the four parts are calculated by using nonlinear programming（NLP）model such that the moving correlation of the preamble results a steep rectangular-like pulse of certain width,whose step-down indicates the timing offset.Simulation results in AWGN channel are given to evaluate the perf o rmance of the proposed preamble design.

Design of Aided Decision-Making Program for Prioritizing Construction Projects in Urban Road Network Planning

The importance and complexity of prioritizing construction projects (PCP) in urban road network planning lead to the necessity to develop an aided decision making program (ADMP). Cost benefit ratio model and stage rolled method are chosen as the theoretical foundations of the program, and then benefit model is improved to accord with the actuality of urban traffic in China. Consequently, program flows, module functions and data structures are designed, and particularly an original data structure of road ...

Solution for integer linear bilevel programming problems using orthogonal genetic algorithm

An integer linear bilevel programming problem is firstly transformed into a binary linear bilevel programming problem, and then converted into a single-level binary implicit programming. An orthogonal genetic algorithm is developed for solving the binary linear implicit programming problem based on the orthogonal design. The orthogonal design with the factor analysis, an experimental design method is applied to the genetic algorithm to make the algorithm more robust, statistical y sound and quickly convergent. A crossover operator formed by the orthogonal array and the factor analysis is presented. First, this crossover operator can generate a smal but representative sample of points as offspring. After al of the better genes of these offspring are selected, a best combination among these offspring is then generated. The simulation results show the effectiveness of the proposed algorithm.

Orthogonal genetic algorithm for solving quadratic bilevel programming problems

A quadratic bilevel programming problem is transformed into a single level complementarity slackness problem by applying Karush-Kuhn-Tucker（KKT） conditions.To cope with the complementarity constraints,a binary encoding scheme is adopted for KKT multipliers,and then the complementarity slackness problem is simplified to successive quadratic programming problems,which can be solved by many algorithms available.Based on 0-1 binary encoding,an orthogonal genetic algorithm,in which the orthogonal experimental design with both two-level orthogonal array and factor analysis is used as crossover operator,is proposed.Numerical experiments on 10 benchmark examples show that the orthogonal genetic algorithm can find global optimal solutions of quadratic bilevel programming problems with high accuracy in a small number of iterations.

Optimization of circulating cooling water systems based on chance constrained programming

Recent research on deterministic methods for circulating cooling water systems optimization has been well developed. However, the actual operating conditions of the system are mostly variable, so the system obtained under deterministic conditions may not be stable and economical. This paper studies the optimization of circulating cooling water systems under uncertain circumstance. To improve the reliability of the system and reduce the water and energy consumption, the influence of different uncertain parameters is taken into consideration. The chance constrained programming method is used to build a model under uncertain conditions, where the confidence level indicates the degree of constraint violation. Probability distribution functions are used to describe the form of uncertain parameters. The objective is to minimize the total cost and obtain the optimal cooling network configuration simultaneously.An algorithm based on Monte Carlo method is proposed, and GAMS software is used to solve the mixed integer nonlinear programming model. A case is optimized to verify the validity of the model. Compared with the deterministic optimization method, the results show that when considering the different types of uncertain parameters, a system with better economy and reliability can be obtained(total cost can be reduced at least 2%).

Policy iteration optimal tracking control for chaotic systems by using an adaptive dynamic programming approach

A policy iteration algorithm of adaptive dynamic programming（ADP） is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking problem is transformed into an optimal regulation one. The policy iteration algorithm for discrete-time chaotic systems is first described. Then,the convergence and admissibility properties of the developed policy iteration algorithm are presented, which show that the transformed chaotic system can be stabilized under an arbitrary iterative control law and the iterative performance index function simultaneously converges to the optimum. By implementing the policy iteration algorithm via neural networks,the developed optimal tracking control scheme for chaotic systems is verified by a simulation.

Detecting Design Patterns in Object-Oriented Program Source Code by Using Metrics and Machine Learning

Detecting well-known design patterns in object-oriented program source code can help maintainers understand the design of a program. Through the detection, the understandability, maintainability, and reusability of object-oriented programs can be improved. There are automated detection techniques;however, many existing techniques are based on static analysis and use strict conditions composed on class structure data. Hence, it is difficult for them to detect and distinguish design patterns in which the class structures are similar. Moreover, it is difficult for them to deal with diversity in design pattern applications. To solve these problems in existing techniques, we propose a design pattern detection technique using source code metrics and machine learning. Our technique judges candidates for the roles that compose design patterns by using machine learning and measurements of several metrics, and it detects design patterns by analyzing the relations between candidates. It suppresses false negatives and distinguishes patterns in which the class structures are similar. As a result of experimental evaluations with a set of programs, we confirmed that our technique is more accurate than two conventional techniques.

Optimal Constrained Self-learning Battery Sequential Management in Microgrid Via Adaptive Dynamic Programming

This paper concerns a novel optimal self-learning battery sequential control scheme for smart home energy systems. The main idea is to use the adaptive dynamic programming U+0028 ADP U+0029 technique to obtain the optimal battery sequential control iteratively. First, the battery energy management system model is established, where the power efficiency of the battery is considered. Next, considering the power constraints of the battery, a new non-quadratic form performance index function is established, which guarantees that the value of the iterative control law cannot exceed the maximum charging/discharging power of the battery to extend the service life of the battery. Then, the convergence properties of the iterative ADP algorithm are analyzed, which guarantees that the iterative value function and the iterative control law both reach the optimums. Finally, simulation and comparison results are given to illustrate the performance of the presented method. © 2017 Chinese Association of Automation.

OBJECT-ORIENTED FINITE ELEMENT ANALYSIS AND PROGRAMMING IN VC++

The design of finite element analysis program using object-oriented programming (OOP) techniques is presented. The objects, classes and the subclasses used in the programming are explained. The system of classes library of finite element analysis program and Windows-type Graphical User Interfaces by VC + + and its MFC are developed. The reliability, reusability and extensibility of program are enhanced. It is a reference to develop the large-scale, versatile and powerful systems of object-oriented finite element software.

A knowledge-to-action approach to food fortification: Guiding principles for the design of fortification programs as a means of effectively addressing micronutrient malnutrition

The gap that exists between research and the dissemination and implementation of research findings has been well established. Food fortification, one of the most cost-effective means of addressing micronutrient malnutrition, is no exception. With decades of implementation experience, there is need to strengthen mechanisms that effectively broadcast proven strategies to promote the successful implementation of fortification programs in changing, challenging, and dynamic environments. This requires clear channels of communication, well-defined in-country leadership, and a streamlined and focused approach that can be adapted to country-specific contexts. Based on experience designing and implementing fortification programs throughout Africa and a broad understanding of past successes and failures, a model is proposed that articulates often over-looked program elements critical to design and implementation.