A deep reinforcement learning approach to gasoline blending real-time optimization under uncertainty

The gasoline inline blending process has widely used real-time optimization techniques to achieve optimization objectives,such as minimizing the cost of production.However,the effectiveness of real-time optimization in gasoline blending relies on accurate blending models and is challenged by stochastic disturbances.Thus,we propose a real-time optimization algorithm based on the soft actor-critic(SAC)deep reinforcement learning strategy to optimize gasoline blending without relying on a single blending model and to be robust against disturbances.Our approach constructs the environment using nonlinear blending models and feedstocks with disturbances.The algorithm incorporates the Lagrange multiplier and path constraints in reward design to manage sparse product constraints.Carefully abstracted states facilitate algorithm convergence,and the normalized action vector in each optimization period allows the agent to generalize to some extent across different target production scenarios.Through these well-designed components,the algorithm based on the SAC outperforms real-time optimization methods based on either nonlinear or linear programming.It even demonstrates comparable performance with the time-horizon based real-time optimization method,which requires knowledge of uncertainty models,confirming its capability to handle uncertainty without accurate models.Our simulation illustrates a promising approach to free real-time optimization of the gasoline blending process from uncertainty models that are difficult to acquire in practice.

Evolutionary-assisted reinforcement learning for reservoir real-time production optimization under uncertainty

Production optimization has gained increasing attention from the smart oilfield community because it can increase economic benefits and oil recovery substantially.While existing methods could produce high-optimality results,they cannot be applied to real-time optimization for large-scale reservoirs due to high computational demands.In addition,most methods generally assume that the reservoir model is deterministic and ignore the uncertainty of the subsurface environment,making the obtained scheme unreliable for practical deployment.In this work,an efficient and robust method,namely evolutionaryassisted reinforcement learning(EARL),is proposed to achieve real-time production optimization under uncertainty.Specifically,the production optimization problem is modeled as a Markov decision process in which a reinforcement learning agent interacts with the reservoir simulator to train a control policy that maximizes the specified goals.To deal with the problems of brittle convergence properties and lack of efficient exploration strategies of reinforcement learning approaches,a population-based evolutionary algorithm is introduced to assist the training of agents,which provides diverse exploration experiences and promotes stability and robustness due to its inherent redundancy.Compared with prior methods that only optimize a solution for a particular scenario,the proposed approach trains a policy that can adapt to uncertain environments and make real-time decisions to cope with unknown changes.The trained policy,represented by a deep convolutional neural network,can adaptively adjust the well controls based on different reservoir states.Simulation results on two reservoir models show that the proposed approach not only outperforms the RL and EA methods in terms of optimization efficiency but also has strong robustness and real-time decision capacity.

ROS2 Real-time Performance Optimization and Evaluation

Real-time interaction with uncertain and dynamic environments is essential for robotic systems to achieve functions such as visual perception,force interaction,spatial obstacle avoidance,and motion planning.To ensure the reliability and determinism of system execution,a flexible real-time control system architecture and interaction algorithm are required.The ROS framework was designed to improve the reusability of robotic software development by providing a distributed structure,hardware abstraction,message-passing mechanism,and application prototypes.Rich ecosystems for robotic development have been built around ROS1 and ROS2 architectures based on the Linux system.However,because of the fairness scheduling principle of the default Linux system design and the complexity of the kernel,the system does not have real-time computing.To achieve a balance between real-time and non-real-time computing,this paper uses the transmission mechanism of ROS2,combines it with the scheduling mechanism of the Linux operating system,and uses Preempt_RT to enhance the real-time computing of ROS1 and ROS2.The real-time performance evaluation of ROS1 and ROS2 is conducted from multiple perspectives,including throughput,transmission mode,QoS service quality,frequency,number of subscription nodes and EtherCAT master.This paper makes two significant contributions:firstly,it employs Preempt_RT to optimize the native ROS2 system,effectively enhancing the real-time performance of native ROS2 message transmission;secondly,it conducts a comprehensive evaluation of the real-time performance of both native and optimized ROS2 systems.This comparison elucidates the benefits of the optimized ROS2 architecture regarding real-time performance,with results vividly demonstrated through illustrative figures.

Efficient Route Planning for Real-Time Demand-Responsive Transit

Demand Responsive Transit (DRT) responds to the dynamic users’ requests without any fixed routes and timetablesand determines the stop and the start according to the demands. This study explores the optimization of dynamicvehicle scheduling and real-time route planning in urban public transportation systems, with a focus on busservices. It addresses the limitations of current shared mobility routing algorithms, which are primarily designedfor simpler, single origin/destination scenarios, and do not meet the complex demands of bus transit systems. Theresearch introduces an route planning algorithm designed to dynamically accommodate passenger travel needsand enable real-time route modifications. Unlike traditional methods, this algorithm leverages a queue-based,multi-objective heuristic A∗ approach, offering a solution to the inflexibility and limited coverage of suburbanbus routes. Also, this study conducts a comparative analysis of the proposed algorithm with solutions based onGenetic Algorithm (GA) and Ant Colony Optimization Algorithm (ACO), focusing on calculation time, routelength, passenger waiting time, boarding time, and detour rate. The findings demonstrate that the proposedalgorithmsignificantly enhances route planning speed, achieving an 80–100-fold increase in efficiency over existingmodels, thereby supporting the real-time demands of Demand-Responsive Transportation (DRT) systems. Thestudy concludes that this algorithm not only optimizes route planning in bus transit but also presents a scalablesolution for improving urban mobility.

Research on Optimal Configuration of Energy Storage in Wind-Solar Microgrid Considering Real-Time Electricity Price

Capacity allocation and energy management strategies for energy storage are critical to the safety and economical operation of microgrids.In this paper,an improved energymanagement strategy based on real-time electricity price combined with state of charge is proposed to optimize the economic operation of wind and solar microgrids,and the optimal allocation of energy storage capacity is carried out by using this strategy.Firstly,the structure and model of microgrid are analyzed,and the outputmodel of wind power,photovoltaic and energy storage is established.Then,considering the interactive power cost between the microgrid and the main grid and the charge-discharge penalty cost of energy storage,an optimization objective function is established,and an improved energy management strategy is proposed on this basis.Finally,a physicalmodel is built inMATLAB/Simulink for simulation verification,and the energy management strategy is compared and analyzed on sunny and rainy days.The initial configuration cost function of energy storage is added to optimize the allocation of energy storage capacity.The simulation results show that the improved energy management strategy can make the battery charge-discharge response to real-time electricity price and state of charge better than the traditional strategy on sunny or rainy days,reduce the interactive power cost between the microgrid system and the power grid.After analyzing the change of energy storage power with cost,we obtain the best energy storage capacity and energy storage power.

Data network traffic analysis and optimization strategy of real-time power grid dynamic monitoring system for wide-frequency measurements

The application and development of a wide-area measurement system(WAMS)has enabled many applications and led to several requirements based on dynamic measurement data.Such data are transmitted as big data information flow.To ensure effective transmission of wide-frequency electrical information by the communication protocol of a WAMS,this study performs real-time traffic monitoring and analysis of the data network of a power information system,and establishes corresponding network optimization strategies to solve existing transmission problems.This study utilizes the traffic analysis results obtained using the current real-time dynamic monitoring system to design an optimization strategy,covering the optimization in three progressive levels:the underlying communication protocol,source data,and transmission process.Optimization of the system structure and scheduling optimization of data information are validated to be feasible and practical via tests.

Multi-objective Optimization of Multi-Agent Elevator Group Control System Based on Real-time Particle Swarm Optimization Algorithm

In order to get a globally optimized solution for the Elevator Group Control System (EGCS) scheduling problem, an algorithm with an overall optimization function is needed. In this study, Real-time Particle Swarm Optimization (RPSO) is proposed to find an optimal solution to the EGCS scheduling problem. Different traffic patterns and controller mechanisms for EGCS are analyzed. This study focuses on up-peak traffic because of its critical importance to modern office buildings. Simulation results show that EGCS based on Multi-Agent Systems (MAS) using RPSO gives good results for up-peak EGCS scheduling problem. Besides, the elevator real-time scheduling and reallocation functions are realized based on RPSO in case new information is available or the elevator becomes busy because it is unavailable or full. This study contributes a new scheduling algorithm for EGCS, and expands the application of PSO.

Real-Time Optimization Model for Continuous Reforming Regenerator

An approach for the simulation and optimization of continuous catalyst-regenerative process of reforming is proposed in this paper.Compared to traditional method such as finite difference method,the orthogonal collocation method is less time-consuming and more accurate,which can meet the requirement of real-time optimization(RTO).In this paper,the equation-oriented method combined with the orthogonal collocation method and the finite difference method is adopted to build the RTO model for catalytic reforming regenerator.The orthogonal collocation method was adopted to discretize the differential equations and sequential quadratic programming(SQP)algorithm was used to solve the algebraic equations.The rate constants,active energy and reaction order were estimated,with the sum of relative errors between actual value and simulated value serving as optimization objective function.The model can quickly predict the fields of component concentration,temperature and pressure inside the regenerator under different conditions,as well as the real-time optimized conditions for industrial reforming regenerator.

A real-time prediction method for tunnel boring machine cutter-head torque using bidirectional long short-term memory networks optimized by multi-algorithm

Based on data from the Jilin Water Diversion Tunnels from the Songhua River(China),an improved and real-time prediction method optimized by multi-algorithm for tunnel boring machine(TBM)cutter-head torque is presented.Firstly,a function excluding invalid and abnormal data is established to distinguish TBM operating state,and a feature selection method based on the SelectKBest algorithm is proposed.Accordingly,ten features that are most closely related to the cutter-head torque are selected as input variables,which,in descending order of influence,include the sum of motor torque,cutter-head power,sum of motor power,sum of motor current,advance rate,cutter-head pressure,total thrust force,penetration rate,cutter-head rotational velocity,and field penetration index.Secondly,a real-time cutterhead torque prediction model’s structure is developed,based on the bidirectional long short-term memory(BLSTM)network integrating the dropout algorithm to prevent overfitting.Then,an algorithm to optimize hyperparameters of model based on Bayesian and cross-validation is proposed.Early stopping and checkpoint algorithms are integrated to optimize the training process.Finally,a BLSTMbased real-time cutter-head torque prediction model is developed,which fully utilizes the previous time-series tunneling information.The mean absolute percentage error(MAPE)of the model in the verification section is 7.3%,implying that the presented model is suitable for real-time cutter-head torque prediction.Furthermore,an incremental learning method based on the above base model is introduced to improve the adaptability of the model during the TBM tunneling.Comparison of the prediction performance between the base and incremental learning models in the same tunneling section shows that:(1)the MAPE of the predicted results of the BLSTM-based real-time cutter-head torque prediction model remains below 10%,and both the coefficient of determination(R^(2))and correlation coefficient(r)between measured and predicted values exceed 0.95;and(2)the incremental learning method is suitable for realtime cutter-head torque prediction and can effectively improve the prediction accuracy and generalization capacity of the model during the excavation process.

Adaptive Resource Planning for AI Workloads with Variable Real-Time Tasks

AI(Artificial Intelligence)workloads are proliferating in modernreal-time systems.As the tasks of AI workloads fluctuate over time,resourceplanning policies used for traditional fixed real-time tasks should be reexamined.In particular,it is difficult to immediately handle changes inreal-time tasks without violating the deadline constraints.To cope with thissituation,this paper analyzes the task situations of AI workloads and findsthe following two observations.First,resource planning for AI workloadsis a complicated search problem that requires much time for optimization.Second,although the task set of an AI workload may change over time,thepossible combinations of the task sets are known in advance.Based on theseobservations,this paper proposes a new resource planning scheme for AIworkloads that supports the re-planning of resources.Instead of generatingresource plans on the fly,the proposed scheme pre-determines resourceplans for various combinations of tasks.Thus,in any case,the workload isimmediately executed according to the resource plan maintained.Specifically,the proposed scheme maintains an optimized CPU(Central Processing Unit)and memory resource plan using genetic algorithms and applies it as soonas the workload changes.The proposed scheme is implemented in the opensourcesimulator SimRTS for the validation of its effectiveness.Simulationexperiments show that the proposed scheme reduces the energy consumptionof CPU and memory by 45.5%on average without deadline misses.

LDA-ID:An LDA-Based Framework for Real-Time Network Intrusion Detection

Network intrusion poses a severe threat to the Internet.However,existing intrusion detection models cannot effectively distinguish different intrusions with high-degree feature overlap.In addition,efficient real-time detection is an urgent problem.To address the two above problems,we propose a Latent Dirichlet Allocation topic model-based framework for real-time network Intrusion Detection(LDA-ID),consisting of static and online LDA-ID.The problem of feature overlap is transformed into static LDA-ID topic number optimization and topic selection.Thus,the detection is based on the latent topic features.To achieve efficient real-time detection,we design an online computing mode for static LDA-ID,in which a parameter iteration method based on momentum is proposed to balance the contribution of prior knowledge and new information.Furthermore,we design two matching mechanisms to accommodate the static and online LDA-ID,respectively.Experimental results on the public NSL-KDD and UNSW-NB15 datasets show that our framework gets higher accuracy than the others.

Multi-Objective Adaptive Optimization Model Predictive Control:Decreasing Carbon Emissions from a Zinc Oxide Rotary Kiln

The zinc oxide rotary kiln,as an essential piece of equipment in the zinc smelting industrial process,is presenting new challenges in process control.China’s strategy of achieving a carbon peak and carbon neutrality is putting new demands on the industry,including green production and the use of fewer resources;thus,traditional stability control is no longer suitable for multi-objective control tasks.Although researchers have revealed the principle of the rotary kiln and set up computational fluid dynamics(CFD)simulation models to study its dynamics,these models cannot be directly applied to process control due to their high computational complexity.To address these issues,this paper proposes a multi-objective adaptive optimization model predictive control(MAO-MPC)method based on sparse identification.More specifically,with a large amount of data collected from a CFD model,a sparse regression problem is first formulated and solved to obtain a reduction model.Then,a two-layered control framework including real-time optimization(RTO)and model predictive control(MPC)is designed.In the RTO layer,an optimization problem with the goal of achieving optimal operation performance and the lowest possible resource consumption is set up.By solving the optimization problem in real time,a suitable setting value is sent to the MPC layer to ensure that the zinc oxide rotary kiln always functions in an optimal state.Our experiments show the strength and reliability of the proposed method,which reduces the usage of coal while maintaining high profits.

A Time Pattern-Based Intelligent Cache Optimization Policy on Korea Advanced Research Network

Data is growing quickly due to a significant increase in social media applications.Today,billions of people use an enormous amount of data to access the Internet.The backbone network experiences a substantial load as a result of an increase in users.Users in the same region or company frequently ask for similar material,especially on social media platforms.The subsequent request for the same content can be satisfied from the edge if stored in proximity to the user.Applications that require relatively low latency can use Content Delivery Network(CDN)technology to meet their requirements.An edge and the data center con-stitute the CDN architecture.To fulfill requests from the edge and minimize the impact on the network,the requested content can be buffered closer to the user device.Which content should be kept on the edge is the primary concern.The cache policy has been optimized using various conventional and unconventional methods,but they have yet to include the timestamp beside a video request.The 24-h content request pattern was obtained from publicly available datasets.The popularity of a video is influenced by the time of day,as shown by a time-based video profile.We present a cache optimization method based on a time-based pat-tern of requests.The problem is described as a cache hit ratio maximization pro-blem emphasizing a relevance score and machine learning model accuracy.A model predicts the video to be cached in the next time stamp,and the relevance score identifies the video to be removed from the cache.Afterwards,we gather the logs and generate the content requests using an extracted video request pattern.These logs are pre-processed to create a dataset divided into three-time slots per day.A Long short-term memory(LSTM)model is trained on this dataset to forecast the video at the next time interval.The proposed optimized caching policy is evaluated on our CDN architecture deployed on the Korean Advanced Research Network(KOREN)infrastructure.Our findings demonstrate how add-ing time-based request patterns impacts the system by increasing the cache hit rate.To show the effectiveness of the proposed model,we compare the results with state-of-the-art techniques.

Optimizing Real-Time Performance of 3D Virtual Mining Environment with MultiGen Creator

System optimization plays a crucial role in developing VR system after 3D modeling, affecting the system's Immersion and Interaction performance enormously. In this article, several key techniques of optimizing a virtual mining system were discussed: optimizing 3D models to keep the polygon number in VR system within target hardware's processing ability; optimizing texture database to save texture memory with perfect visual effect; optimizing database hierarchy structure to accelerate model retrieval; and optimizing LOD hierarchy structure to speed up rendering.

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

The coordinated optimization problem of the electricity-gas-heat integrated energy system(IES)has the characteristics of strong coupling,non-convexity,and nonlinearity.The centralized optimization method has a high cost of communication and complex modeling.Meanwhile,the traditional numerical iterative solution cannot deal with uncertainty and solution efficiency,which is difficult to apply online.For the coordinated optimization problem of the electricity-gas-heat IES in this study,we constructed a model for the distributed IES with a dynamic distribution factor and transformed the centralized optimization problem into a distributed optimization problem in the multi-agent reinforcement learning environment using multi-agent deep deterministic policy gradient.Introducing the dynamic distribution factor allows the system to consider the impact of changes in real-time supply and demand on system optimization,dynamically coordinating different energy sources for complementary utilization and effectively improving the system economy.Compared with centralized optimization,the distributed model with multiple decision centers can achieve similar results while easing the pressure on system communication.The proposed method considers the dual uncertainty of renewable energy and load in the training.Compared with the traditional iterative solution method,it can better cope with uncertainty and realize real-time decision making of the system,which is conducive to the online application.Finally,we verify the effectiveness of the proposed method using an example of an IES coupled with three energy hub agents.

Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

Application of the asynchronous advantage actor–critic machine learning algorithm to real-time accelerator tuning

This paper describes a real-time beam tuning method with an improved asynchronous advantage actor–critic(A3C)algorithm for accelerator systems.The operating parameters of devices are usually inconsistent with the predictions of physical designs because of errors in mechanical matching and installation.Therefore,parameter optimization methods such as pointwise scanning,evolutionary algorithms(EAs),and robust conjugate direction search are widely used in beam tuning to compensate for this inconsistency.However,it is difficult for them to deal with a large number of discrete local optima.The A3C algorithm,which has been applied in the automated control field,provides an approach for improving multi-dimensional optimization.The A3C algorithm is introduced and improved for the real-time beam tuning code for accelerators.Experiments in which optimization is achieved by using pointwise scanning,the genetic algorithm(one kind of EAs),and the A3C-algorithm are conducted and compared to optimize the currents of four steering magnets and two solenoids in the low-energy beam transport section(LEBT)of the Xi’an Proton Application Facility.Optimal currents are determined when the highest transmission of a radio frequency quadrupole(RFQ)accelerator downstream of the LEBT is achieved.The optimal work points of the tuned accelerator were obtained with currents of 0 A,0 A,0 A,and 0.1 A,for the four steering magnets,and 107 A and 96 A for the two solenoids.Furthermore,the highest transmission of the RFQ was 91.2%.Meanwhile,the lower time required for the optimization with the A3C algorithm was successfully verified.Optimization with the A3C algorithm consumed 42%and 78%less time than pointwise scanning with random initialization and pre-trained initialization of weights,respectively.

A Model Predictive Scheduling Algorithm inReal-Time Control Systems

The ineffective utilization of power resources has attracted much attention in current years. This paper proposes a real-time distributed load scheduling algorithm considering constraints of power supply. Firstly, an objective function is designed based on the constraint, and a base load forecasting model is established when aggregating renewable generation and non-deferrable load into a power system, which aims to transform the problem of deferrable loads scheduling into a distributed optimal control problem. Then, to optimize the objective function, a real-time scheduling algorithm is presented to solve the proposed control problem. At every time step, the purpose is to minimize the variance of differences between power supply and aggregate load, which can thus ensure the effective utilization of power resources. Finally, simulation examples are provided to illustrate the effectiveness of the proposed algorithm.

Efficiency analysis of numerical integrations for finite element substructure in real-time hybrid simulation

Finite element（FE） is a powerful tool and has been applied by investigators to real-time hybrid simulations（RTHSs）. This study focuses on the computational efficiency, including the computational time and accuracy, of numerical integrations in solving FE numerical substructure in RTHSs. First, sparse matrix storage schemes are adopted to decrease the computational time of FE numerical substructure. In this way, the task execution time（TET） decreases such that the scale of the numerical substructure model increases. Subsequently, several commonly used explicit numerical integration algorithms, including the central difference method（CDM）, the Newmark explicit method, the Chang method and the Gui-λ method, are comprehensively compared to evaluate their computational time in solving FE numerical substructure. CDM is better than the other explicit integration algorithms when the damping matrix is diagonal, while the Gui-λ（λ = 4） method is advantageous when the damping matrix is non-diagonal. Finally, the effect of time delay on the computational accuracy of RTHSs is investigated by simulating structure-foundation systems. Simulation results show that the influences of time delay on the displacement response become obvious with the mass ratio increasing, and delay compensation methods may reduce the relative error of the displacement peak value to less than 5% even under the large time-step and large time delay.

Sensor placement of long-term health monitoring for large bridges based on the real-time correction of finite element model

The process of optimized placement of long-term health monitoring sensors for large bridges generally begins with finite element models, but there will arise great discrepancies between theoretically-calculated results and actual measurements.Therefore, rectified finite element models need to be rectified by virtue of model rectifying technology. Firstly, the result of construction monitoring and finished state load test is used to real-time modification of finite element model. Subsequently, an accurate finite element model is established. Secondly, the optimizing the layout of sensor with following orthogonality guarantees orthogonal property and linear independence for the measured data. Lastly, the effectiveness and feasibility of method in the paper is tested by real-time modifying finite element model and optimizing the layout of sensor for Nujiang Bridge.