Reinforcement learning for wind-farm flow control:Current state and future actions

Wind-farm flow control stands at the forefront of grand challenges in wind-energy science.The central issue is that current algorithms are based on simplified models and,thus,fall short of capturing the complex physics of wind farms associated with the high-dimensional nature of turbulence and multiscale wind-farm-atmosphere interactions.Reinforcement learning(RL),as a subset of machine learning,has demonstrated its effectiveness in solving high-dimensional problems in various domains,and the studies performed in the last decade prove that it can be exploited in the development of the next generation of algorithms for wind-farm flow control.This review has two main objectives.Firstly,it aims to provide an up-to-date overview of works focusing on the development of wind-farm flow control schemes utilizing RL methods.By examining the latest research in this area,the review seeks to offer a comprehensive understanding of the advancements made in wind-farm flow control through the application of RL techniques.Secondly,it aims to shed light on the obstacles that researchers face when implementing wind-farm flow control based on RL.By highlighting these challenges,the review aims to identify areas requiring further exploration and potential opportunities for future research.

Experimental investigation of dynamic stall flow control using a microsecond-pulsed plasma actuator

To alleviate the performance deterioration caused by dynamic stall of a wind turbine airfoil,the flow control by a microsecond-pulsed dielectric barrier discharge(MP-DBD) actuator on the dynamic stall of a periodically pitching NACA0012 airfoil was investigated experimentally.Unsteady pressure measurements with high temporal accuracy were employed in this study,and the unsteady characteristics of the boundary layer were investigated by wavelet packet analysis and the moving root mean square method based on the acquired pressure.The experimental Mach number was 0.2,and the chord-based Reynolds number was 870 000.The dimensionless actuation frequencies F+ were chosen to be 0.5,1,2,and 3,respectively.For the light dynamic regime,the MP-DBD plasma actuator plays the role of suppressing flow separation from the trial edge and accelerating the flow reattachment due to the high-momentum freestream flow being entrained into the boundary layer.Meanwhile,actuation effects were promoted with the increasing dimensionless actuation frequency F+.The control effects of the deep dynamic stall were to delay the onset and reduce the strength of the dynamic stall vortex due to the accumulating vorticity near the leading edge being removed by the induced coherent vortex structures.The laminar fluctuation and Kelvin-Helmholtz(K-H) instabilities of transition and relaminarization were also mitigated by the MP-DBD actuation,and the alleviated K-H rolls led to the delay of the transition onset and earlier laminar reattachment,which improved the hysteresis effect of the dynamic stall.For the controlled cases of F+=2,and F+=3,the laminar fluctuation was replaced by relatively low frequency band disturbances corresponding to the harmonic responses of the MP-DBD actuation frequency.

Flow control performance evaluation of a tri-electrode sliding discharge plasma actuator

Tri-electrode sliding discharge(TED)plasma actuators are formed by adding a direct current(DC)exposed electrode to conventional dielectric barrier discharge(DBD)plasma actuators.There are three TED modes depending on the polarity and amplitude of the DC supply:DBD discharge,extended discharge and sliding discharge.This paper evaluates the electrical,aerodynamic and mechanical characteristics of a TED plasma actuator based on energy analysis,particle image velocimetry experiments and calculations using the Navier-Stokes equation.The flow control performances of different discharge modes are quantitatively analyzed based on characteristic parameters.The results show that flow control performance in both extended discharge and sliding discharge is more significant than that of DBD,mainly because of the significantly higher(up to 141%)body force of TED compared with DBD.However,conductivity loss is the primary power loss caused by the DC polarity for TED discharge.Therefore,power consumption can be reduced by optimizing the dielectric material and thickness,thus improving the flow control performance of plasma actuators.

Model Predictive Control Strategy of Multi-Port Interline DC Power Flow Controller

There are issues with flexible DC transmission system such as a lack of control freedom over power flow.In order to tackle these issues,a DC power flow controller(DCPFC)is incorporated into a multi-terminal,flexible DC power grid.In recent years,a multi-port DC power flow controller based on a modular multi-level converter has become a focal point of research due to its simple structure and robust scalability.This work proposes a model predictive control(MPC)strategy for multi-port interline DC power flow controllers in order to improve their steady-state dynamic performance.Initially,the mathematical model of a multi-terminal DC power grid with a multi-port interline DC power flow controller is developed,and the relationship between each regulated variable and control variable is determined;The power flow controller is then discretized,and the cost function and weight factor are built with numerous control objectives.Sub module sorting method and nearest level approximation modulation regulate the power flow controller;Lastly,theMATLAB/Simulink simulation platformis used to verify the correctness of the establishedmathematicalmodel and the control performance of the suggestedMPC strategy.Finally,it is demonstrated that the control strategy possesses the benefits of robust dynamic performance,multiobjective control,and a simple structure.

A Fuzzy Flow Control Approach for ABR Service in ATM Networks

The explicit rate flow control mechanisms for ABR service are used to sharethe available bandwidth of a bottleneck link fairly and reasonably among many competitive users andto maintain the buffer queue length of a bottleneck switch connected to the link at a desired levelin order to avoid and control congestion in ATM networks. However, designing effective flow controlmechanisms for the service is known to be difficult because of the variety of dynamic parametersinvolved such as available link bandwidth, burst of the traffic, the distances between ABR sourcesand switches. In this paper, we present a fuzzy explicit rate flow control mechanism for ABRservice. The mechanism has a simple structure and is robust in the sense that the mechanism'sstability is not sensitive to the change in the number of active virtual connections (VCs). Manysimulations show that this mechanism can not only effectively avoid network congestion, but alsoensure fair share of the bandwidth for all active VCs regardless of the number of hops theytraverse. Additionally, it has the advantages of fast convergence, low oscillation, and high linkbandwidth utilization.

Theoretical Analysis and Numerical Simulation of the Static and Dynamic Characteristics of Hydrostatic Guides Based on Progressive Mengen Flow Controller

The oil film thickness of oil hydrostatic guide with constant pressure supply based on capillary restrictor is greatly affected by load, and this kind of hydrostatic guide is usually applied to the machine tools with moderate load. The static and dynamic characteristics of the guide have been studied by using some theoretical, numerical and experimental approaches, and some methods and measures have been proposed to improve its performances. The hydrostatic guide based on progressive mengen（PM） flow controller is especially suitable for the heavy numerical control（NC） machine tools. However, few literatures about the research on the static and dynamic characteristics of the hydrostatic guides based on PM flow controller are reported. In this paper, the formulae are derived for analyzing the static and dynamic characteristics of hydrostatic guides with rectangle pockets and PM flow controller according to the theory of hydrostatic bearing. On the basis of the analysis of hydrostatic bearing with circular pocket, some equations are derived for solving the static pressure, volume pressure and squeezing pressure which influence the dynamic characteristics of hydrostatic guides with rectangle pocket. The function and the influencing factors of three pressures are clarified. The formulae of amplitude-frequency characteristics and dynamic stiffness of the hydrostatic guide system are derived. With the help of software MATLAB, programs are coded with C＋＋ language to simulate numerically the static and dynamic characteristics of the hydrostatic guide based on PM flow controller. The simulation results indicate that the sensitive oil volume between the outlet of the PM flow controller and the guide pocket has the greatest influence on the characteristics of the guide, and it should be reduced as small as possible when the field working condition is met. Choosing the oil with a greater viscosity is also helpful in improving the dynamic performance of hydrostatic guides. The research work has instructing significance for analyzing and designing the guide with PM flow controller.

Optimization of flow control devices in a single-strand slab continuous casting tundish

The optimization of flow control devices in a single-slab continuous casting tundish was carried out by physical modeling, and the optimized scheme was presented. With the optimal tundish configuration, the minimum residence time of liquid steel was increased by 1.4 times, the peak concentration time was increased by 97%, and the dead volume fraction was decreased by 72%. A mathematical model for molten steel in the tundish was established by using the fluid dynamics package Fluent. The velocity field, concentration field, and the resi-dence time distribution （RTD） curves of molten steel flow before and after optimization were obtained. Experimental results showed that the reasonable configuration with flow control devices can improve the fluid flow characteristics in the tundish. The results of industrial applica-tion show that the nonmetallic inclusion area ratio in casting slabs is decreased by 32% with the optimal tundish configuration.

Orthogonal analysis of water model study on the optimization of flow control devices in a six-strand tundish

Improper flow control devices in a multi-strand tundish can cause some problems, for example, liquid steel cannot reach every nozzle at the same time and the liquid steel in nozzles far away from the entry zone has a lower temperature. The water model experiment of a six-strand tundish of Tianjin Iron ＆ Steel Co. Ltd. was performed, a new ＂U＂ type baffle was obtained, and its parameters were defined by perpendicular analysis. The ＂U＂ baffle can not only improve those imperfections, but also prolong the residence time of nonmetallic inclusions, which is good for their flotation and separation.

Improvement of tundish shape and optimization of flow control devices for sequence casting heavy steel ingots

The metallurgical effect of a round tundish used to cast heavy steel ingots in machine works at present was evaluated through water modeling experiments. The flow control devices of the improved oval tundish, which was used instead of the round tundish, had been optimized. The results show that the residence time of the round tundish is short, its inclusion removal efficiency is too low, and it has more dead zones and an unreasonable flow field. Compared with the round tundish, the improved oval tundish with the optimized weir and dam has a better effect： its minimum residence time is prolonged by 38.1 s, the average residence time is prolonged by 233.4 s, its dead volume fraction decreases from 26% to 15%, and the ratio of plug volume fraction to dead volume fraction increases from 0.54 to 1.27. The inclusion removal efficiency also increases by 17.5%.

Metro passenger flow control with station-to-station cooperation based on stop-skipping and boarding limiting

Metro passenger flow control problem is studied under given total inbound demand in this work,which considers passenger demand control and train capacity supply.Relevant connotations are analyzed and a mathematical model is developed.The decision variables are boarding limiting and stop-skipping strategies and the objective is the maximal passenger profit.And a passenger original station choice model based on utility theory is built to modify the inbound passenger distribution among stations.Algorithm of metro passenger flow control scheme is designed,where two key technologies of stopping-station choice and headway adjustment are given and boarding limiting and train stopping-station scheme are optimized.Finally,a real case of Beijing metro is taken for example to verify validity.The results show that in the three scenarios with different ratios of normal trains to stop-skipping trains,the total limited passenger volume is the smallest and the systematic profit is the largest in scenario 3.

Numerical Investigation of Electrohydrodynamic (EHD) Flow Control in an S-Shaped Duct

An electrohydrodynamic （EHD） method, which is based on glow discharge plasma, is presented for flow control in an S-shaped duct. The research subject is an expanding channel with a constant width and a rectangular cross section. An equivalent divergence angle and basic function are introduced to build the three-dimensional model. Subsequently, the plasma physical models are simplified as the effects of electrical body force and work （done by the force） on the fluid near the wall. With the aid of FLUENT software, the source terms of momentum and energy are added to the Navier-Stokes equation. Finally, the original performance of three models （A, B and C） is studied, in which model A demonstrates better performance. Then EHD control based on model A is discussed. The results show that the EHD method is an effective way of reducing flow loss and improving uniformity at the duct exit. The innovation in this study is the assessment of the EHD control effect on the flow in an S-shaped duct. Both the parametric modeling of the S-shaped duct and the simplified models of plasma provide valuable information for future research on aircraft inlet ducts.

Flow Control for a Two-Stage Proportional Valve with Hydraulic Position Feedback

The current research mainly focuses on the flow control for the two-stage proportional valve with hydraulic position feedback which is named as Valvistor valve.Essentially,the Valvistor valve is a proportional throttle valve and the flow fluctuates with the change of load pressure.The flow fluctuation severely restricts the application of the Valvistor valve.In this paper,a novel flow control method the Valvistor valve is provided to suppress the flow fluctuation and develop a high performance proportional flow valve.The mathematical model of this valve is established and linearized.Fuzzy proportional-integral-derivative(PID)controller is adopted in the closed-loop flow control system.The feedback is obtained by the flow inference with back-propagation neural network(BPNN)based on the spool displacement in the pilot stage and the pressure differential across the main orifice.The results show that inference with BPNN can obtain the flow data fast and accurately.With the flow control method,the flow can keep at the set point when the pressure differential across the main orifice changes.The flow control method is effective and the Valvistor valve changes from proportional throttle valve to proportional flow valve.For the developed proportional flow valve,the settling time of the flow is very short when the load pressure changes abruptly.The performances of hysteresis,linearity and bandwidth are in a high range.The linear mathematical model can be verified and the assumptions in the system modeling is reasonable.

Experimental Study on Surface Dielectric Barrier Discharge Plasma Actuator with Different Encapsulated Electrode Widths for Airflow Control at Atmospheric Pressure

The surface dielectric barrier discharge （SDBD） plasma actuator has shown great promise as an aerodynamic flow control device. In this paper, the encapsulated electrode width of a SDBD actuator is changed to study the airflow acceleration behavior. The effects of encapsulated electrode width on the actuator performance are experimentally investigated by measuring the dielectric layer surface potential, time-averaged ionic wind velocity and thrust force. Experimental results show that the airflow velocity and thrust force increase with the encapsulated electrode width. The results can be attributed to the distinct plasma distribution at different encapsulated electrode widths.

Material flow control technology of ironmaking and steelmaking interface

In order to achieve higher efficient cohesion match of procedure and equipment between ironmaking and steelmaking interface, the theory of multi-dimensional material flow control was applied to analyze torpedo ladle-iron ladle transportation process between blast furnace and basic oxygen furnace. Moreover, basic parameters of material flow were analyzed and optimized, such as time, temperature and material quantity. Based on operating principles of material flow, control methods were optimized, such as product organization mode, scheduling discipline and scheduling plan of hot metal ladle. Finally, the material flow control technology of ironmaking and steelmaking interface was integrated. Satisfactory effects are obtained after applying the technology in practice. The total turnover number of torpedo ladle decreases from 20 to 18, the hot metal temperature of 1# BF torpedo ladle decreases from 36 °C to 19.5 °C, the hot metal temperature of 2# BF torpedo ladle decreases from 36.6 °C to 19.8 °C, the temperature drop of desulfurization hot metal decreases by 4 °C, and the temperature drop of non-desulfurization hot metal decreases by 2.8 °C. Furthermore, the ironmaking and steelmaking interface system will realize high-efficiency control by using this control technology.

Flow control of micro-ramps on supersonic forward-facing step flow

The effects of the micro-ramps on supersonic turbulent flow over a forward-facing step（FFS） was experimentally investigated in a supersonic low-noise wind tunnel at Mach number 3 using nano-tracer planar laser scattering（NPLS）and particle image velocimetry（PIV） techniques. High spatiotemporal resolution images and velocity fields of supersonic flow over the testing model were captured. The fine structures and their spatial evolutionary characteristics without and with the micro-ramps were revealed and compared. The large-scale structures generated by the micro-ramps can survive the downstream FFS flowfield. The micro-ramps control on the flow separation and the separation shock unsteadiness was investigated by PIV results. With the micro-ramps, the reduction in the range of the reversal flow zone in streamwise direction is 50% and the turbulence intensity is also reduced. Moreover, the reduction in the average separated region and in separation shock unsteadiness are 47% and 26%, respectively. The results indicate that the micro-ramps are effective in reducing the flow separation and the separation shock unsteadiness.

Optimization of flow control devices in a tenstrand billet caster tundish

The physical model of a ten-strand billet caster tundish was established to study the effects of various flow control devices on the melt flow. Before and after the optimization of the melt flow, the inclusion removal in the tundish was evaluated by plant trials. The physical modeling results show that when combined with a baffle, the turbulence inhibitor, instead of the impact pad, can significantly improve the melt flow. A turbulence inhibitor with a longer length of inner cavity and without an extending lip at the top of the sidewall seems to be efficient in the improvement of the melt flow. Various types and designs of baffles all influence the flow characteristics significantly. The "V" type baffles are better than the straight baffles for flow control. The "V" type baffle with four inclined holes at the sidewall away from the stopper rods is better in melt flow control than the one with one inclined hole at each sidewall. The combination of a well-designed turbulence inhibitor and an appropriate baffle shows high efficiency on improving the melt flow and an optimal proposal was presented. Plant trials indicate that, compared with the original tundish configuration in prototype, the inclusions reduce by 42% and the inclusion distribution of individual strands is more similar with the optimal one. The optimal tundish configuration effectively improves the melt flow in the ten-strand billet caster tundish.

A Distributionally Robust Optimization Method for Passenger Flow Control Strategy and Train Scheduling on an Urban Rail Transit Line

Regular coronavirus disease 2019(COVID-19)epidemic prevention and control have raised new require-ments that necessitate operation-strategy innovation in urban rail transit.To alleviate increasingly seri-ous congestion and further reduce the risk of cross-infection,a novel two-stage distributionally robust optimization(DRO)model is explicitly constructed,in which the probability distribution of stochastic scenarios is only partially known in advance.In the proposed model,the mean-conditional value-at-risk(CVaR)criterion is employed to obtain a tradeoff between the expected number of waiting passen-gers and the risk of congestion on an urban rail transit line.The relationship between the proposed DRO model and the traditional two-stage stochastic programming(SP)model is also depicted.Furthermore,to overcome the obstacle of model solvability resulting from imprecise probability distributions,a discrepancy-based ambiguity set is used to transform the robust counterpart into its computationally tractable form.A hybrid algorithm that combines a local search algorithm with a mixed-integer linear programming(MILP)solver is developed to improve the computational efficiency of large-scale instances.Finally,a series of numerical examples with real-world operation data are executed to validate the pro-posed approaches.

Flow Control over a Conical Forebody by Duty-Cycle Actuations

Duty-cycle modulation alternately blowing from two opposite-facing plasma actu- ators on the leeward surface near the apex of a cone with a 10° semi-apex angle is adopted to control mean lateral force and moment, and the flow control mechanisms are presented. Pressure distributions over the forebody of the cone are measured by steady pressure tappings. The experiments are performed in a 3.0×1.6 m open-circuit wind tunnel at a wind speed of 20 m/s, a 45° angle of attack and a Reynolds number of 2×10^5, based on the diameter of the base of the cone. Almost linearly proportional control of the lateral forces and moments over a slender conical forebody at a high angle of attack has been demonstrated by employing a pair of single dielectric barrier discharge plasma actuators near the apex of the cone, combined with a duty-cycle tech- nique. The pressure distribution measurements indicate that the hi-stable vortex pattern appears to be shifted in the opposite direction when the port or starboard actuator is activated, while the other is kept off during the test. It is shown that the reduced pulse-repetition frequency based on the local diameter at the plasma actuator equal to one yields the highest effectiveness among the cases considered.

Experimental study on surface arc plasma actuation-based hypersonic boundary layer transition flow control

Effective control of hypersonic transition is essential.In order to avoid affecting the structural proflle of the aircraft,as well as reducing power consumption and electromagnetic interference,a low-frequency surface arc plasma disturbance experiment to promote hypersonic transition was carried out in theΦ0.25 m double-throat Ludwieg tube wind tunnel at Huazhong University of Science and Technology.Contacting printed circuit board sensors and non-contact focused laser differential interferometry testing technology were used in combination.Experimental results showed that the low-frequency surface arc plasma actuation had obvious stimulation effects on the second-mode unstable wave and could promote boundary layer transition by changing the spectral characteristics of the second-mode unstable wave.At the same time,the plasma actuation could promote energy exchange between the second-mode unstable wave and other unstable waves.Finally,the corresponding control mechanism is discussed.

Convergence and stability of the Newton-Like algorithm with estimation error in optimization flow control

The Newton-Like algorithm with price estimation error in optimization flow control in network is analyzed. The estimation error is treated as inexactness of the gradient and the inexact descent direction is analyzed. Based on the optimization theory, a sufficient condition for convergence of this algorithm with bounded price estimation error is obtained. Furthermore, even when this sufficient condition doesn＇t hold, this algorithm can also converge, provided a modified step size, and an attraction region is obtained. Based on Lasalle＇s invariance principle applied to a suitable Lyapunov function, the dynamic system described by this algorithm is proved to be global stability if the error is zero. And the Newton-Like algorithm with bounded price estimation error is also globally stable if the error satisfies the sufficient condition for convergence. All trajectories ultimately converge to the equilibrium point.