Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

A Survey on Type-3 Fuzzy Logic Systems and Their Control Applications

In this paper,we offer a review of type-3 fuzzy logic systems and their applications in control.The main objective of this work is to observe and analyze in detail the applications in the control area using type-3 fuzzy logic systems.In this case,we review their most important applications in control and other related topics with type-3 fuzzy systems.Intelligent algorithms have been receiving increasing attention in control and for this reason a review in this area is important.This paper reviews the main applications that make use of Intelligent Computing methods.Specifically,type-3 fuzzy logic systems.The aim of this research is to be able to appreciate,in detail,the applications in control systems and to point out the scientific trends in the use of Intelligent Computing techniques.This is done with the construction and visualization of bibliometric networks,developed with VosViewer Software,which it is a free Java-based program,mainly intended to be used for analyzing and visualizing bibliometric networks.With this tool,we can create maps of publications,authors,or journals based on a co-citation network or construct maps of keywords,countries based on a co-occurrence networks,research groups,etc.

Energy Storage Systems Technologies, Evolution and Applications

Energy in its varied forms and applications has become the main driver of today’s modern society. However, recent changes in power demand and climatic changes (decarbonization policy) has awakened the need to rethink through the current energy generating and distribution system. This led to the exploration of other energy sources of which renewable energy (like thermal, solar and wind energy) is fast becoming an integral part of most energy system. However, this innovative and promising energy source is highly unreliable in maintaining a constant peak power that matches demand. Energy storage systems have thus been highlighted as a solution in managing such imbalances and maintaining the stability of supply. Energy storage technologies absorb and store energy, and release it on demand. This includes gravitational potential energy (pumped hydroelectric), chemical energy (batteries), kinetic energy (flywheels or compressed air), and energy in the form of electrical (capacitors) and magnetic fields. This paper provides a detailed and comprehensive overview of some of the state-of-the-art energy storage technologies, its evolution, classification, and comparison along with various area of applications. Also highlighted in this paper is a plethora of power electronic Interface technologies that plays a significant role in enabling optimum performance and utilization of energy storage systems in different areas of application.

NEW SYSTEMS OF GENERALIZED QUASI-VARIATIONAL INCLUSIONS IN FC-SPACES AND APPLICATIONS

In this paper, we study some new systems of generalized quasi-variational inclusion problems in FC-spaces without convexity structure.By applying an existence theorem of maximal elements of set-valued mappings due to the author, some new existence theorems of solutions for the systems of generalized quasi-variational inclusion problems are proved in noncompact FC-spaces. As applications, some existence results of solutions for the system of quasi-optimization problems and mathematical programs with the systems of generalized quasi-variational inclusion constraints are obtained in FC-spaces.

Inference of General Mass Action-Based State Equations for Oscillatory Biochemical Reaction Systems Using <i>k</i>-Step Genetic Programming

Systems biology requires the development of algorithms that use omics data to infer interaction networks among biomolecules working within an organism. One major type of evolutionary algorithm, genetic programming (GP), is useful for its high heuristic ability as a search method for obtaining suitable solutions expressed as tree structures. However, because GP determines the values of parameters such as coefficients by random values, it is difficult to apply in the inference of state equations that describe oscillatory biochemical reaction systems with high nonlinearity. Accordingly, in this study, we propose a new GP procedure called “k-step GP” intended for inferring the state equations of oscillatory biochemical reaction systems. The k-step GP procedure consists of two algorithms: 1) Parameter optimization using the modified Powell method—after genetic operations such as crossover and mutation, the values of parameters such as coefficients are optimized by applying the modified Powell method with secondary convergence. 2) GP using divided learning data—to improve the inference efficiency, imposes perturbations through the addition of learning data at various intervals and adaptations to these changes result in state equations with higher fitness. We are confident that k-step GP is an algorithm that is particularly well suited to inferring state equations for oscillatory biochemical reaction systems and contributes to solving inverse problems in systems biology.

Max-plus-linear model-based predictive control for constrained hybrid systems: linear programming solution

In this paper, a linear programming method is proposed to solve model predictive control for a class of hybrid systems. Firstly, using the （max, ＋） algebra, a typical subclass of hybrid systems called max-plus-linear （MPL） systems is obtained. And then, model predictive control （MPC） framework is extended to MPL systems. In general, the nonlinear optimization approach or extended linear complementarity problem （ELCP） were applied to solve the MPL-MPC optimization problem. A new optimization method based on canonical forms for max-min-plus-scaling （MMPS） functions （using the operations maximization, minimization, addition and scalar multiplication） with linear constraints on the inputs is presented. The proposed approach consists in solving several linear programming problems and is more efficient than nonlinear optimization. The validity of the algorithm is illustrated by an example.

Visual Programming of Stochastic Weather Generator and Future Applications on Agroecological Study

Based on former studies on weather simulator modules in IPMist laboratory, study on visual programming of stochastic weather generator(VS-WGEN)was continued. In this study, Markov Chain, Monte Carlo, Fourier Series, and weak stationary process were used to generate the daily weather data in software Matlab 6. 0, with the data input from 40 years' weather data recorded by Beijing Weather Station. The generated data includes daily maximum temperature, minimum temperature, precipitation and solar radiation. It has been verified that the weather data generated by the VS-WGEN were more accurate than that by the old WGEN, when twenty new model parameters were included. VS-WGEN has wide software applications, such as pest risk analysis, pest forecast and management. It can be implemented in hardware development as well, such as weather control in weather chamber and greenhouse for researches on ecological adaptation of crop varieties to a given location over time and space. Overall, VS-WGEN is a very useful tool for studies on theoretical and applied ecology.

Modeling Dynamic Systems by Using the Nonlinear Difference Equations Based on Genetic Programming

When acquaintances of a model are little or the model is too complicate to build by using traditional time series methods, it is convenient for us to take advantage of genetic programming (GP) to build the model. Considering the complexity of nonlinear dynamic systems, this paper proposes modeling dynamic systems by using the nonlinear difference e-quation based on GP technique. First it gives the method, criteria and evaluation of modeling. Then it describes the modeling algorithm using GP. Finally two typical examples of time series are used to perform the numerical experiments. The result shows that this algorithm can successfully establish the difference equation model of dynamic systems and its predictive result is also satisfactory.

MULTI-HOP RELAYING IN 5G： FROM RESEARCH TO SYSTEMS, STANDARDS, AND APPLICATIONS

In recent years,the demands of high traffic transmission motivate the rapid development of wireless access techniques,and it becomes promising to design the fifth generation(5G)wireless networks.Essential requirements for 5G involve higher traffic volume,indoor or hotspot traffic,and spectrum,energy,and cost efficien-

第六届IEEE International Conference on Dependable Systems and Their Applications(DSA)征稿启事

第六届IEEE International Conference on Dependable Systems and Their Applications(DSA)将于2020年1月3日~6日在美丽的哈尔滨举行。我们诚挚地邀请您投稿参与。除了regular(12页)和short papers(6页),DSA也接受Fast abstract(2页)的投稿。截稿日期2019年10月31日。

ROS-responsive drug delivery systems for biomedical applications

In the field of biomedicine, stimuli-responsive drug delivery systems(DDSs) have become increasingly popular due to their site-specific release ability in response to a certain physiological stimulus, which may result in both enhanced treatment outcome and reduced side effects. Reactive oxygen species(ROS) are the unavoidable consequence of cell oxidative metabolism. ROS play a crucial part in regulating biological and physiological processes,whereas excessive intracellular ROS usually lead to the oxidation stress which has implications in several typical diseases such as cancer, inflammation and atherosclerosis. Therefore,ROS-responsive DDSs have elicited widespread popularity for their promising applications in a series of biomedical research because the payload is only released in targeted cells or tissues that overproduce ROS. According to the design of ROS-responsive DDSs, the main release mechanisms of therapeutic agents can be ascribed to ROS-induced carrier solubility change, ROS-induced carrier cleavage or ROS-induced prodrug linker cleavage. This review summarized the latest development and novel design of ROS-responsive DDSs and discussed their design concepts and the applications in the biomedical field.

QoE Modeling and Applications for Multimedia Systems

Ireproving the quality and experience perceived by the user is fundamental when de- veloping multimedia technologies, products, and services. Quality of experience （QoE） involves subjective perception, user behavior and needs, appropriateness, con- text, and usability of delivered content. Modeling QoE is critical for enhancing QoE in various nmhimedia applications. In this special issue,

Stability of Nonlinear Systems Using Optimal Fuzzy Controllers and Its Simulation by Java Programming

In this paper, at first, the single input rule modules(SIRMs) dynamically connected fuzzy inference model is used to stabilize a double inverted pendulum system. Then, a multiobjective particle swarm optimization(MOPSO) is implemented to optimize the fuzzy controller parameters in order to decrease the distance error of the cart and summation of the angle errors of the pendulums, simultaneously. The feasibility and efficiency of the proposed Pareto front is assessed in comparison with results reported in literature and obtained from other algorithms.Finally, the Java programming with applets is utilized to simulate the stability of the nonlinear system and explain the internetbased control.

Special Issue on New Trends in Theory and Applications of Complex Control Systems

Complexity is one of the leading features of modem control systems. It is caused by the complex properties of controlled plants and the varied requirements in controller designs. In dealing with the control problems with complexity, in the past two decades, a lot of papers have been published that reported fruitful results on different theoretical backgrounds and with different methodologies, such as differential geometry-based design methods, hybrid system theory, switching control, neural network-based intelligent control, etc.

Call for papers Journal of Control Theory and Applications Special issue on Approximate dynamic programming and reinforcement learning

Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain and nonconvex environments over time.

Combining Geographic Information Systems for Transportation and Mixed Integer Linear Programming in Facility Location-Allocation Problems

In this study, we aimed to assess the solution quality for location-allocation problems from facilities generated by the software TransCAD&reg;?, a Geographic Information System for Transportation (GIS-T). Such facilities were obtained after using two routines together: Facility Location and Transportation Problem, when compared with optimal solutions from exact mathematical models, based on Mixed Integer Linear Programming (MILP), developed externally for the GIS. The models were applied to three simulations: the first one proposes opening factories and customer allocation in the state of Sao Paulo, Brazil;the second involves a wholesaler and a study of location and allocation of distribution centres for retail customers;and the third one involves the location of day-care centers and allocation of demand (0 - 3 years old children). The results showed that when considering facility capacity, the MILP optimising model presents results up to 37% better than the GIS and proposes different locations to open new facilities.

Optimal Redundancy Allocation in Hierarchical Series-Parallel Systems Using Mixed Integer Programming

Reliability optimization plays an important role in design, operation and management of the industrial systems. System reliability can be easily enhanced by improving the reliability of unreliable components and/or by using redundant configuration with subsystems/components in parallel. Redundancy Allocation Problem (RAP) was studied in this research. A mixed integer programming model was proposed to solve the problem, which considers simultaneously two objectives under several resource constraints. The model is only for the hierarchical series-parallel systems in which the elements of any subset of subsystems or components are connected in series or parallel and constitute a larger subsystem or total system. At the end of the study, the performance of the proposed approach was evaluated by a numerical example.

Fund budget model for multipurpose transit smart card systems

The fund budget of multipurpose transit smart card systems is studied by stochastic programming to assign limited funds to different applications reasonably. Under the constraints of a gross fund, models of chance-constrained and dependentchance for the fund budget of multipurpose transit smart card systems are established with application scale and social demand as random variables, respectively aiming to maximize earnings and satisfy the service requirements the furthest; and the genetic algorithm based on stochastic simulation is adopted for model solution. The calculation results show that the fund budget differs greatly with different system objectives which can cause the systems to have distinct expansibilities, and the application scales of some applications may not satisfy user demands with limited funds. The analysis results indicate that the forecast of application scales and application future demands should be done first, and then the system objective is determined according to the system mission, which can help reduce the risks of fund budgets.

流通式-时间分辨分析技术在地质领域的应用进展

流通式-时间分辨分析系统(简称FT-TRA)是二十一世纪初新发展起来的一种快速反应(溶解)-在线分析系统,由淋洗液混合单元、反应单元与分析单元组成,核心功能是通过特定流动相淋洗样品池中的微量样品,分离或去除样品中的特定组分,并监测样品不同元素和矿物组分的出溶特征,实现高分辨的在线过程分析。本文综述了FT-TRA系统的技术原理与软硬件组成、实验方法与操作要点及地质应用发展过程,重点对该系统在地质应用过程中出现的争议点进行阐释与分析,并基于其发展现状展望其未来发展方向与潜力。FT-TRA系统目前主要的地质应用包括古海洋学与古环境学研究代用指标的验证(如有孔虫、介形虫淋洗)、矿物溶解过程与反应动力学研究、环境样品的元素相态分析等。FT-TRA系统以溶解并提取有孔虫/介形虫壳体的元素组成信号作为还原古海洋指标的重要手段,与传统批处理法相比,该方法被认为具有实时监测清洁程度、降低损失率并实现差异溶解的优势,能够获取更精细的壳体化学组成信息;测试矿物溶解态与溶解参数也是FT-TRA系统的重要功能之一,该系统的实验室模拟能够与模型结合探究不同类型矿物在稳态下的溶解动力学,为研究矿物在自然状态下的溶解过程提供启示;近些年来该系统还逐渐被用于矿物反应性测试,其中将气相CO_(2)作为淋滤液与矿物反应的研究可能在全球变暖及CO_(2)的人工捕捉课题上具有潜在应用价值。FT-TRA系统运行中涉及的不同组分的溶解机制是其应用过程中亟待解决的重要问题,进一步完善其溶解动力学原理必然将为该系统的未来发展提供更多如多类型地质样品溶解、矿物的模拟合成等新思路。