Reinforcement Learning in Process Industries:Review and Perspective

This survey paper provides a review and perspective on intermediate and advanced reinforcement learning(RL)techniques in process industries. It offers a holistic approach by covering all levels of the process control hierarchy. The survey paper presents a comprehensive overview of RL algorithms,including fundamental concepts like Markov decision processes and different approaches to RL, such as value-based, policy-based, and actor-critic methods, while also discussing the relationship between classical control and RL. It further reviews the wide-ranging applications of RL in process industries, such as soft sensors, low-level control, high-level control, distributed process control, fault detection and fault tolerant control, optimization,planning, scheduling, and supply chain. The survey paper discusses the limitations and advantages, trends and new applications, and opportunities and future prospects for RL in process industries. Moreover, it highlights the need for a holistic approach in complex systems due to the growing importance of digitalization in the process industries.

Optimization study of station track utilization in high-speed railroad based on constraints of control in random origin and process

Purpose-The purpose of this paper is to eliminate the fluctuations in train arrival and departure times caused by skewed distributions in interval operation times.These fluctuations arise from random origin and process factors during interval operations and can accumulate over multiple intervals.The aim is to enhance the robustness of high-speed rail station arrival and departure track utilization schemes.Design/methodologylapproach-To achieve this objective,the paper simulates actual train operations,incorporating the fluctuations in interval operation times into the utilization of arrival and departure tracks at the station.The Monte Carlo simulation method is adopted to solve this problem.This approach transforms a nonlinear model,which includes constraints from probability distribution functions and is difficult to solve directly,into a linear programming model that is easier to handle.The method then linearly weights two objectives to optimize the solution.Findings-Through the application of Monte Carlo simulation,the study successfully converts the complex nonlinear model with probability distribution function constraints into a manageable linear programming model.By continuously adjusting the weighting coefficients of the linear objectives,the method is able to optimize the Pareto solution.Notably,this approach does not require extensive scene data to obtain a satisfactory Pareto solution set.Originality/value-The paper contributes to the field by introducing a novel method for optimizing high-speed rail station arrival and departure track utilization in the presence of fluctuations in interval operation times.The use of Monte Carlo simulation to transform the problem into a tractable linear programming model represents a significant advancement.Furthermore,the method's ability to produce satisfactory Pareto solutions without relying on extensive data sets adds to its practical value and applicability in real-world scenarios.

Study on the Interactivity of Controlled and Automatic Information Processing in L2 Reading

L2 reading is not only an important channel for people to obtain information and knowledge,but also the main way for people to learn a foreign language.Reading information processing can be divided into controlled processing and automatic processing.Controlled information processing is a conscious and resource-intensive processing model,while automatic information processing is an unconscious and automatic processing model.This study investigates the characteristics and interactivity of controlled and automatic information processing in L2 reading,and explores the roles of controlled and automatic information processing strategies in improving L2 reading ability.The findings are as follows:(a)controlled and automatic information processing is interactive in L2 reading;and(b)the uses of controlled and automatic information processing strategies are beneficial to the improvement of the reading ability of L2 learners.This study has important theoretical and practical value in improving the efficiency of L2 reading teaching and learning.

A master-slave generalized predictive synchronization control for preheating process of multi-cavity hot runner system

As a key component of injection molding,multi-cavity hot runner(MCHR)system faces the crucial problem of polymer melt filling imbalance among the cavities.The thermal imbalance in the system has been considered as the leading cause.Hence,the solution may rest with the synchronization of those heating processes in MCHR system.This paper proposes a’Master-Slave’generalized predictive synchronization control(MS-GPSC)method with’Mr.Slowest’strategy for preheating stage of MCHR system.The core of the proposed method is choosing the heating process with slowest dynamics as the’Master’to track the setpoint,while the other heating processes are treated as‘Slaves’tracking the output of’Master’.This proposed method is shown to have the good ability of temperature synchronization.The corresponding analysis is conducted on parameters tuning and stability,simulations and experiments show the strategy is effective.

Integration Between Enterprise Process Monitoring and Controlling System and Enterprise Application

The relationships and the features of integration between Enterprise ProcessMonitoring and Controlling System (EPMCS) and Enterprise Process Related Applications (EPRA) wereanalyzed. An integration architecture centered on EPMCS was presented, in which there were fourlayers to connect from EPMCS to EPRA: EPMCS, application integration layer, transport layer andEPRA, and there were four layers used to etstablish integration: presentation layer, function layer,data layer and system layer. The frameworks to connect EPMCS and EPRA were designed, thatEnterprise-Independent Model (EIM), Enterprise-Specific Model (ESM) and meta-model to describe thesetwo models were defined. The method to integrate data based on XML was designed to exchange datafrom EPMCS to EPRA according to the mapping between EIM and ESM. The approches are suitable forintegrating EPMCS and systems in Product Data Management (PDM), project management and enterprisebusiness management.

Experimental Study of Monitoring and Controlling of Composite Cure Process in Autoclave Featured with Fiber Optic Sensor

With the aid of the latest fiber optic sensing technology parameters in the cure process of ther- mosetting resin-matrix composite, such as temperature, viscosity,void and residual stress, can be monitored entirely and efficiently.In this paper, experiment results of viscosity measurement in composite cure process in autoclave using fiber optic sensors are presented. Based on the sensed information, a computer program is utilized to control the cure process. With this technology, the cure process becomes more apparent and controllable, which will greatly improve the cured products and reduce the cost.

Application of single neuron adaptive PID controller during the process of timber drying

The paper presents a method of using single neuron adaptive PID control for adjusting system or servo system to implement timber drying process control, which combines the thought of parameter adaptive PID control and the character of neural network on exactly describing nonlinear and uncertainty dynamic process organically. The method implements functions of adaptive and self-learning by adjusting weighting parameters. Adaptive neural network can make some output trail given hoping value to decouple in static state. The simulation result indicates the validity, veracity and robustness of the method used in the timber drying process

Energy-saving design and optimization of pressure-swing-assisted ternary heterogenous azeotropic distillations

A huge amount of energy is always consumed to separate the ternary azeotropic mixtures by distillations.The heterogeneous azeotropic distillation and the pressure-swing distillation are two kinds of effective technologies to separate heterogeneous azeotropes without entrainer addition.To give better play to the synergistic energy-saving effect of these two processes,a novel pressure-swing-assisted ternary heterogeneous azeotropic distillation(THAD)process is proposed firstly.In this process,the ternary heterogeneous azeotrope is decanted into two liquid phases before being refluxed into the azeotropic distillation column to avoid the aqueous phase remixing,and three columns'pressures are modified to decrease the flowrates of the recycle streams.Then the dividing wall column and heat integration technologies are introduced to further reduce its energy consumption,and the pressureswing-assisted ternary heterogeneous azeotropic dividing-wall column and its heat integration structure are achieved.A genetic algorithm procedure is used to optimize the proposed processes.The design results show that the proposed processes have higher energy efficiencies and lower CO_(2)emissions than the published THAD process.

Intelligent casting:Empowering the future foundry industry

Emerging technological advances are reshaping the casting sector in latest decades.Casting technology is evolving towards intelligent casting paradigm that involves automation,greenization and intelligentization,which attracts more and more attention from the academic and industry communities.In this paper,the main features of casting technology were briefly summarized and forecasted,and the recent developments of key technologies and the innovative efforts made in promoting intelligent casting process were discussed.Moreover,the technical visions of intelligent casting process were also put forward.The key technologies for intelligent casting process comprise 3D printing technologies,intelligent mold technologies and intelligent process control technologies.In future,the intelligent mold that derived from mold with sensors,control devices and actuators will probably incorporate the Internet of Things,online inspection,embedded simulation,decision-making and control system,and other technologies to form intelligent cyber-physical casting system,which may pave the way to realize intelligent casting.It is promising that the intelligent casting process will eventually achieve the goal of real-time process optimization and full-scale control,with the defects,microstructure,performance,and service life of the fabricated castings can be accurately predicted and tailored.

Chemical simulation teaching system based on virtual reality and gesture interaction

Background Most existing chemical experiment teaching systems lack solid immersive experiences,making it difficult to engage students.To address these challenges,we propose a chemical simulation teaching system based on virtual reality and gesture interaction.Methods The parameters of the models were obtained through actual investigation,whereby Blender and 3DS MAX were used to model and import these parameters into a physics engine.By establishing an interface for the physics engine,gesture interaction hardware,and virtual reality(VR)helmet,a highly realistic chemical experiment environment was created.Using code script logic,particle systems,as well as other systems,chemical phenomena were simulated.Furthermore,we created an online teaching platform using streaming media and databases to address the problems of distance teaching.Results The proposed system was evaluated against two mainstream products in the market.In the experiments,the proposed system outperformed the other products in terms of fidelity and practicality.Conclusions The proposed system which offers realistic simulations and practicability,can help improve the high school chemistry experimental education.

Research on Quality Management Measures of Green Building Construction

Urbanization has led to the rapid development of the construction industry.However,this has also led to higher requirements for the construction engineering management.Other than the quality monitoring of engineering construction,the energy-saving properties of the building should also be considered.Therefore,a scientific management approach should be adopted to improve green building management.This paper primarily examines the importance of quality management in green building construction,along with the factors influencing it.It also identifies the quality issues present in current green building construction.Finally,it proposes measures for quality management in the green building construction process to facilitate the industry’s healthy development.

Multivariable Decoupling Predictive Control with Input Constraints and Its Application on Chemical Process

A constrained decoupling (generalized predictive control) GPC algorithm is proposed for MIMO (malti-input multi-output) system. This algorithm takes account of all constraints of inputs and their increments. By solving matrix equations, the multi-step predictive decoupling controllers are realized. This algorithm need not solve Diophantine functions, and weakens the cross-coupling of the variables. At last the simulation results demon- strate the effectiveness of this proposed strategy.

Process Control and Optimization for Heterologous Protein Production by Methylotrophic Pichia pastoris

The methylotrophic yeast Pichia pastoris is a highly successful system for production of a variety of heterologous proteins due to its unique features/abilities for effective protein expression, and tremendous efforts have been made to increase heterologous protein productivity by P. pastoris in recent years. When new engineered yeast strains are constructed and are ready to use tot industrial protein production, process control and optimization techniques should be applied to improve the fermentation performance in the following aspects： （1） increase recombinant cell concentrations in fermentor to high density during growth phase; （2） effectively induce heterologous proteins by enhancing/stabilizing titers or concentrations of the proteins during induction phase; （3） decrease operation costs by relieving the working loads of heat-exchange and oxygen supply. This article reviews and discusses the key and commonly used techniques in heterologous protein production by P. pastoris, with the focus on optimizations of fermentation media and basic operation conditions, development of optimal glycerol feeding strategies for achieving high density cultivation of P. pastoris and effective heterologous protein induction methods by regulating specific growth rate, methanol concentration, temperatures, mixture ratio of multi-carbon substrates, etc. Metabolic analysis for recombinant protein production by P. pastoris is also introduced to interpret the mechanism of sub-optimal heterologous protein production and to explore further optimal expression methods.

A New Strategy of Integrated Control and On-line Optimization on High-purity Distillation Process

For high-purity distillation processes,it is difficult to achieve a good direct product quality control using traditional proportional-integral-differential(PID)control or multivariable predictive control technique due to some difficulties,such as long response time,many un-measurable disturbances,and the reliability and precision issues of product quality soft-sensors.In this paper,based on the first principle analysis and dynamic simulation of a distillation process,a new predictive control scheme is proposed by using the split ratio of distillate flow rate to that of bottoms as an essential controlled variable.Correspondingly,a new strategy with integrated control and on-line optimization is developed,which consists of model predictive control of the split ratio,surrogate model based on radial basis function neural network for optimization,and modified differential evolution optimization algorithm. With the strategy,the process achieves its steady state quickly,so more profit can be obtained.The proposed strategy has been successfully applied to a gas separation plant for more than three years,which shows that the strategy is feasible and effective.

Hybrid Neural Network Model for RH Vacuum Refining Process Control

A hybrid neural network model,in which RH process(theoretical)model is combined organically with neural network(NN)and case-base reasoning(CBR),was established.The CBR method was used to select the operation mode and the RH operational guide parameters for different steel grades according to the initial conditions of molten steel,and a three-layer BP neural network was adopted to deal with nonlinear factors for improving and compensating the limitations of technological model for RH process control and end-point prediction.It was verified that the hybrid neural network is effective for improving the precision and calculation efficiency of the model.

Recent Developments in the Crystallization Process： Toward the Pharmaceutical Industry

Crystallization is one of the oldest separation and purification unit operations, and has recently contributed to significant improvements in producing higher-value products with specific properties and in building efficient manufacturing processes. In this paper, we review recent developments in crystal engineering and crystallization process design and control in the pharmaceutical industry. We systematically summarize recent methods for understanding and developing new types of crystals such as co-crystals, polymorphs, and solvates, and include several milestones such as the launch of the first co-crystal drug, Entresto （No- vartis）, and the continuous manufacture of Orkambi （Vertex）. Conventional batch and continuous processes, which are becoming increasingly mature, are being coupled with various control strategies and the recently developed crystallizers are thus adapting to the needs of the pharmaceutical industry. The development of crystallization process design and control has led to the appearance of several new and innovative crystal- lizer geometries for continuous operation and improved performance. This paper also reviews major recent orogress in the area of process analytical technology.

Modeling, Optimization, and Control of Solution Purification Process in Zinc Hydrometallurgy

The solution purification process is an essential step in zinc hydrometallurgy. The performance of solution purification directly affects the normal functioning and economical benefits of zinc hydrometallurgy. This paper summarizes the authors' recent work on the modeling, optimization, and control of solution purification process. The online measurable property of the oxidation reduction potential(ORP) and the multiple reactors, multiple running statuses characteristic of the solution purification process are extensively utilized in this research. The absence of reliable online equipment for detecting the impurity ion concentration is circumvented by introducing the oxidationreduction potential into the kinetic model. A steady-state multiple reactors gradient optimization, unsteady-state operationalpattern adjustment strategy, and a process evaluation strategy based on the oxidation-reduction potential are proposed. The effectiveness of the proposed research is demonstrated by its industrial experiment.

High-Order Volterra Model Predictive Control and Its Application to a Nonlinear Polymerisation Process

Model Predictive Control (MPC) has recently found wide acceptance in the process industry, but existing design and implementation methods are restricted to linear process models. A chemical process, however, involves severe nonlinearity which cannot be ignored in practice. This paper aims to solve this nonlinear control problem by extending MPC to accommodate nonlinear models. It develops an analytical framework for nonlinear model predictive control (NMPC). It also offers a third-order Volterra series based nonparametric nonlinear modelling technique for NMPC design, which relieves practising engineers from the need for deriving a physical-principles based model first. An on-line realisation technique for implementing NMPC is then developed and applied to a Mitsubishi Chemicals polymerisation reaction process. Results show that this nonlinear MPC technique is feasible and very effective. It considerably outperforms linear and low-order Volterra model based methods. The advantages of the developed approach lie not only in control performance superior to existing NMPC methods, but also in eliminating the need for converting an analytical model and then convert it to a Volterra model obtainable only up to the second order. Keywords Model predictive control - Volterra series - process control - nonlinear control Yun Li is a senior lecturer at University of Glasgow, UK, where has taught and researched in evolutionary computation and control engineering since 1991. He worked in the UK National Engineering Laboratory and Industrial Systems and Control Ltd, Glasgow in 1989 and 1990. In 1998, he established the IEEE CACSD Evolutionary Computation Working Group and the European Network of Excellence in Evolutionary Computing (EvoNet) Workgroup on Systems, Control, and Drives. In summer 2002, he served as a visiting professor to Kumamoto University, Japan. He is also a visiting professor at University of Electronic Science and Technology of China. His research interests are in parallel processing, design automation and discovery of engineering systems using evolutionary learning and intelligent search techniques. Applications include control, system modelling and prediction, circuit design, microwave engineering, and operations management. He has advised 12 Ph.D.s in evolutionary computation and has 140 publications.Hiroshi Kashiwagi received B.E, M.E. and Ph.D. degrees in measurement and control engineering from the University of Tokyo, Japan, in 1962, 1964 and 1967 respectively. In 1967 he became an Associate Professor and in 1976 a Professor at Kumamoto University. From 1973 to 1974, he served as a visiting Associate Professor at Purdue University, Indiana, USA. From 1990 to 1994, he was the Director at Computer Center of Kumamoto University. He has also served as a member of Board of Trustees of Society of Instrument and Control Engineers (SICE), Japan, Chairman of Kyushu Branch of SICE and General Chair of many international conferences held in Japan, Korea, Chin and India. In 1994, he was awarded SICE Fellow for his contributions to the field of measurement and control engineering through his various academic activities. He also received the Gold Medal Prize at ICAUTO’95 held in India. In 1997, he received the “Best Book Award” from SICE for his new book entitled “M-sequence and its application” written in Japanese and published in 1996 by Shoukoudou Publishing Co. in Japan. In 1999, he received the “Best Paper Award” from SICE for his paper “M-transform and its application to system identification”. His research interests include signal processing and applications, especially pseudorandom sequence and its applications to measurement and control engineering.

Dynamic Modeling and Simulation of a Commercial Naphtha Catalytic Reforming Process

A first principles-based dynamic model for a continuous catalyst regeneration (CCR) platforming process, the UOP commercial naphtha catalytic reforming process, is developed in this paper. The lumping details of the naphtha feed and reaction scheme of the reaction model are given. The process model is composed of the reforming reaction model with catalyst deactivation, the furnace model and the separator model, which is capable of capturing the major dynamics that occurs in this process system. Dynamic simulations are performed based on Gear numerical algorithm and method of lines (MOL), a numerical technique dealing with partial differential equations (PDEs). The results of simulation are also presented. Dynamic responses caused by disturbances in the process system can be correctly predicted through simulations.

Multimodal process monitoring based on transition-constrained Gaussian mixture model

Reliable process monitoring is important for ensuring process safety and product quality.A production process is generally characterized bymultiple operation modes,and monitoring thesemultimodal processes is challenging.Most multimodal monitoring methods rely on the assumption that the modes are independent of each other,which may not be appropriate for practical application.This study proposes a transition-constrained Gaussian mixture model method for efficient multimodal process monitoring.This technique can reduce falsely and frequently occurring mode transitions by considering the time series information in the mode identification of historical and online data.This process enables the identified modes to reflect the stability of actual working conditions,improve mode identification accuracy,and enhance monitoring reliability in cases of mode overlap.Case studies on a numerical simulation example and simulation of the penicillin fermentation process are provided to verify the effectiveness of the proposed approach inmultimodal process monitoring with mode overlap.