A deep reinforcement learning approach to gasoline blending real-time optimization under uncertainty

The gasoline inline blending process has widely used real-time optimization techniques to achieve optimization objectives,such as minimizing the cost of production.However,the effectiveness of real-time optimization in gasoline blending relies on accurate blending models and is challenged by stochastic disturbances.Thus,we propose a real-time optimization algorithm based on the soft actor-critic(SAC)deep reinforcement learning strategy to optimize gasoline blending without relying on a single blending model and to be robust against disturbances.Our approach constructs the environment using nonlinear blending models and feedstocks with disturbances.The algorithm incorporates the Lagrange multiplier and path constraints in reward design to manage sparse product constraints.Carefully abstracted states facilitate algorithm convergence,and the normalized action vector in each optimization period allows the agent to generalize to some extent across different target production scenarios.Through these well-designed components,the algorithm based on the SAC outperforms real-time optimization methods based on either nonlinear or linear programming.It even demonstrates comparable performance with the time-horizon based real-time optimization method,which requires knowledge of uncertainty models,confirming its capability to handle uncertainty without accurate models.Our simulation illustrates a promising approach to free real-time optimization of the gasoline blending process from uncertainty models that are difficult to acquire in practice.

Distributed Optimal Variational GNE Seeking in Merely Monotone Games

In this paper, the optimal variational generalized Nash equilibrium(v-GNE) seeking problem in merely monotone games with linearly coupled cost functions is investigated, in which the feasible strategy domain of each agent is coupled through an affine constraint. A distributed algorithm based on the hybrid steepest descent method is first proposed to seek the optimal v-GNE. Then, an accelerated algorithm with relaxation is proposed and analyzed, which has the potential to further improve the convergence speed to the optimal v-GNE. Some sufficient conditions in both algorithms are obtained to ensure the global convergence towards the optimal v-GNE. To illustrate the performance of the algorithms, numerical simulation is conducted based on a networked Nash-Cournot game with bounded market capacities.

Secure Impulsive Synchronization in Lipschitz-Type Multi-Agent Systems Subject to Deception Attacks

Cyber attacks pose severe threats on synchronization of multi-agent systems.Deception attack,as a typical type of cyber attack,can bypass the surveillance of the attack detection mechanism silently,resulting in a heavy loss.Therefore,the problem of mean-square bounded synchronization in multi-agent systems subject to deception attacks is investigated in this paper.The control signals can be replaced with false data from controllerto-actuator channels or the controller.The success of the attack is measured through a stochastic variable.A distributed impulsive controller using a pinning strategy is redesigned,which ensures that mean-square bounded synchronization is achieved in the presence of deception attacks.Some sufficient conditions are derived,in which upper bounds of the synchronization error are given.Finally,two numerical simulations with symmetric and asymmetric network topologies are given to illustrate the theoretical results.

A Linear Algorithm for Quantized Event-Triggered Optimization Over Directed Networks

Dear Editor,This letter investigates a class of distributed optimization problems with constrained communication.A quantized discrete-time eventtriggered zero-gradient-sum algorithm(QDE-ZGS)is developed to optimize the sum of local functions over weight-balanced directed networks.Based on an encoder-decoder scheme and a zooming-in technique,an event-triggered quantization communication is designed.Theoretical analysis shows that the exact convergence to the global optimal solution is guaranteed when the triggering threshold is bounded and the scaled sequence introduced by the zooming-in technique is quadratic summable.When the scaled sequence is bounded by an exponential decay function,QDE-ZGS converges linearly to the unique global optimal solution.Numerical simulations are conducted to demonstrate the theoretical results.