Safety-Constrained Multi-Agent Reinforcement Learning for Power Quality Control in Distributed Renewable Energy Networks

This paper examines the difficulties of managing distributed power systems,notably due to the increasing use of renewable energy sources,and focuses on voltage control challenges exacerbated by their variable nature in modern power grids.To tackle the unique challenges of voltage control in distributed renewable energy networks,researchers are increasingly turning towards multi-agent reinforcement learning(MARL).However,MARL raises safety concerns due to the unpredictability in agent actions during their exploration phase.This unpredictability can lead to unsafe control measures.To mitigate these safety concerns in MARL-based voltage control,our study introduces a novel approach:Safety-ConstrainedMulti-Agent Reinforcement Learning(SC-MARL).This approach incorporates a specialized safety constraint module specifically designed for voltage control within the MARL framework.This module ensures that the MARL agents carry out voltage control actions safely.The experiments demonstrate that,in the 33-buses,141-buses,and 322-buses power systems,employing SC-MARL for voltage control resulted in a reduction of the Voltage Out of Control Rate(%V.out)from0.43,0.24,and 2.95 to 0,0.01,and 0.03,respectively.Additionally,the Reactive Power Loss(Q loss)decreased from 0.095,0.547,and 0.017 to 0.062,0.452,and 0.016 in the corresponding systems.

A Universal Portable Appliance for Stellarator W7-X Power Supply Controlling

In the project Wendelstein 7-X (W7-X), the popular fieldbus Profibus has been determined as a uniform connection between the central control system and all the subordinate systems. A universal embedded control system has been developed for W7-X power supply controlling. Siemens 80C167CR microcontroller is used as the central control unit of the system. With a user-defined printed circuit board (PCB) several control buses, i.e., Profibus, CAN, IEEE 488, RS485 and RS 232 have been connected to the microcontroller. The corresponding hardware interfaces for the control buses have been designed. A graphic liquid crystal display(LCD) and a user-defined keyboard are used as user interface. The control software will be developed with a C-like language, i.e., C166 for the controller.

A Data Driven Security Correction Method for Power Systems with UPFC

The access of unified power flow controllers(UPFC)has changed the structure and operation mode of power grids all across the world,and it has brought severe challenges to the traditional real-time calculation of security correction based on traditionalmodels.Considering the limitation of computational efficiency regarding complex,physical models,a data-driven power system security correction method with UPFC is,in this paper,proposed.Based on the complex mapping relationship between the operation state data and the security correction strategy,a two-stage deep neural network(DNN)learning framework is proposed,which divides the offline training task of security correction into two stages:in the first stage,the stacked auto-encoder(SAE)classification model is established,and the node correction state(0/1)output based on the fault information;in the second stage,the DNN learningmodel is established,and the correction amount of each action node is obtained based on the action nodes output in the previous stage.In this paper,the UPFC demonstration project of NanjingWest Ring Network is taken as a case study to validate the proposed method.The results show that the proposed method can fully meet the real-time security correction time requirements of power grids,and avoid the inherent defects of the traditional model method without an iterative solution and can also provide reasonable security correction strategies for N-1 and N-2 faults.

A novel fractional uplink power control framework for self-organizing networks

Internet of things and network densification bring significant challenges to uplink management.Only depending on optimization algorithm enhancements is not enough for uplink transmission.To control intercell interference,Fractional Uplink Power Control(FUPC)should be optimized from network-wide perspective,which has to find a better traffic distribution model.Conventionally,traffic distribution is geographic-based,and ineffective due to tricky locating efforts.This paper proposes a novel uplink power management framework for Self-Organizing Networks(SON),which firstly builds up pathloss-based traffic distribution model and then makes the decision of FUPC based on the model.PathLoss-based Traffic Distribution(PLTD)aggregates traffic based on the propagation condition of traffic that is defined as the pathloss between the position generating the traffic and surrounding cells.Simulations show that the improvement in optimization efficiency of FUPC with PLTD can be up to 40%compared to conventional GeoGraphic-based Traffic Distribution(GGTD).

Distributed robust power control in two-tier vehicle networks under uncertain channel environments

This paper proposes a novel optimization scheme to support stable and reliable vehicle-to-everything connections in two-tier networks,where the uplink channel of the cellular user is reused by underlay vehicle-to-vehicle communications.However,considering complex channel fading and high-speed vehicle movement,the cer-tainty assumption is impractical and fails to maintain power control strategy in reality in the traditional statical vehicular networks.Rather than the perfect channel state information assumption,the first-order Gauss-Markov process which is a probabilistic model affected by vehicle speed and fading is introduced to describe imperfect channel gains.Moreover,interference management is a major challenge in reusing communications,especially in uncertain channel environments.Power control is an effective way to realize interference management,and optimal power allocation can ensure that interference of the user meets the communication requirements.In this study,the sum-rate-oriented power control scheme and minimum-rate-oriented power control scheme were implemented to manage interference and satisfy different design objectives.Since both of these schemes are non-convex and intractable,the Bernstein approximation and successive convex approximation methods were adopted to transform the original problems into convex ones.Furthermore,a novel distributed robust power control al-gorithm was developed to determine the optimal solutions.The performance of the algorithm was evaluated through numerical simulations,and the results indicate that the proposed algorithm is effective in vehicular communication networks with uncertain channel environments.

An Adaptive Wireless Power Sharing Control for Multiterminal HVDC

Power sharing among multiterminal high voltage direct current terminals(MT-HVDC)is mainly developed based on a priority or sequential manners,which uses to prevent the problem of overloading due to a predefined controller coefficient.Furthermore,fixed power sharing control also suffers from an inability to identify power availability at a rectification station.There is a need for a controller that ensures an efficient power sharing among the MT-HVDC terminals,prevents the possibility of overloading,and utilizes the available power sharing.A new adaptive wireless control for active power sharing among multiterminal(MT-HVDC)systems,including power availability and power management policy,is proposed in this paper.The proposed control strategy solves these issues and,this proposed controller strategy is a generic method that can be applied for unlimited number of converter stations.The rational of this proposed controller is to increase the system reliability by avoiding the necessity of fast communication links.The test system in this paper consists of four converter stations based on three phase-two AC voltage levels.The proposed control strategy for a multiterminal HVDC system is conducted in the power systems computer aided design/electromagnetic transient design and control(PSCAD/EMTDC)simulation environment.The simulation results significantly show the flexibility and usefulness of the proposed power sharing control provided by the new adaptive wireless method.

A Deep Reinforcement Learning-Based Power Control Scheme for the 5G Wireless Systems

In the fifth generation(5G)wireless system,a closed-loop power control(CLPC)scheme based on deep Q learning network(DQN)is introduced to intelligently adjust the transmit power of the base station(BS),which can improve the user equipment(UE)received signal to interference plus noise ratio(SINR)to a target threshold range.However,the selected power control(PC)action in DQN is not accurately matched the fluctuations of the wireless environment.Since the experience replay characteristic of the conventional DQN scheme leads to a possibility of insufficient training in the target deep neural network(DNN).As a result,the Q-value of the sub-optimal PC action exceed the optimal one.To solve this problem,we propose the improved DQN scheme.In the proposed scheme,we add an additional DNN to the conventional DQN,and set a shorter training interval to speed up the training of the DNN in order to fully train it.Finally,the proposed scheme can ensure that the Q value of the optimal action remains maximum.After multiple episodes of training,the proposed scheme can generate more accurate PC actions to match the fluctuations of the wireless environment.As a result,the UE received SINR can achieve the target threshold range faster and keep more stable.The simulation results prove that the proposed scheme outperforms the conventional schemes.

Model Predictive Control Strategy of Multi-Port Interline DC Power Flow Controller

There are issues with flexible DC transmission system such as a lack of control freedom over power flow.In order to tackle these issues,a DC power flow controller(DCPFC)is incorporated into a multi-terminal,flexible DC power grid.In recent years,a multi-port DC power flow controller based on a modular multi-level converter has become a focal point of research due to its simple structure and robust scalability.This work proposes a model predictive control(MPC)strategy for multi-port interline DC power flow controllers in order to improve their steady-state dynamic performance.Initially,the mathematical model of a multi-terminal DC power grid with a multi-port interline DC power flow controller is developed,and the relationship between each regulated variable and control variable is determined;The power flow controller is then discretized,and the cost function and weight factor are built with numerous control objectives.Sub module sorting method and nearest level approximation modulation regulate the power flow controller;Lastly,theMATLAB/Simulink simulation platformis used to verify the correctness of the establishedmathematicalmodel and the control performance of the suggestedMPC strategy.Finally,it is demonstrated that the control strategy possesses the benefits of robust dynamic performance,multiobjective control,and a simple structure.

Cooperative Rate Splitting Transmit Design for Full-Duplex-Enabled Multiple Multicast Communication Systems

This paper examines the performance of Full-Duplex Cooperative Rate Splitting(FD-CRS)with Simultaneous Wireless Information and Power Transfer(SWIPT)support in Multiple Input Single Output(MISO)networks.In a Rate Splitting Multiple Access(RSMA)multicast system with two local users and one remote user,the common data stream contains the needs of all users,and all users can decode the common data stream.Therefore,each user can receive some information that other users need,and local users with better channel conditions can use this information to further enhance the reception reliability and data rate of users with poor channel quality.Even using Cell-Center-Users(CCUs)as a cooperative relay to assist the transmission of common data can improve the average system speed.To maximize the minimum achievable rate,we optimize the beamforming vector of Base Station(BS),the common streamsplitting vector,the cooperative distributed beamvector and the strong user transmission power under the power budget constraints of BS and relay devices and the service quality requirements constraints of users.Since the whole problem is not convex,we cannot solve it directly.Therefore,we propose a low complexity algorithm based on Successive Convex Approximation(SCA)technology to find the optimal solution to the problemunder consideration.The simulation results show that FD C-RSMA has better gain andmore powerful than FD C-NOMA,HD C-RSMA,RSMA and NOMA.

Design of Controller for New EAST Fast Control Power Supply

The effectiveness of the magnetic confinement of plasma can be improved by elongat- ing the plasma cross-section in tokamak devices. But elongated plasma has vertical displacement instability, so a feedback control system is needed to restrain the plasma＇s vertical displacement. A fast control power supply is needed to excite the active feedback coils, which produces a magnetic field to control the plasma＇s displacement. With the development of EAST, the fast control power supply needs to keep on enhancing the fast response and output current. The structure of a new power supply is introduced in this paper. The method of multiple inverters paralleled with the current sharing reactor is presented to meet the need for large current and fast control. According to the design demands of the EAST fast control power supply, the adjuster of the current close loop is applied to the inverter, which can advance its ability to restrain the loop current in low frequency and DC output. The result of the experiment confirms the validity of the proposed scheme and control strategy.

Application of a Current and Voltage Mixed Control Mode for the New Fast Control Power Supply at EAST

A feedback control system is needed to restrain plasma vertical displacement in EAST （Experimental Advanced Superconducting Toknmak）. A fast control power supply excites active feedback coils, which produces a magnetic field to control the plasma＇s displacement. With the development of EAST, new demands on the new fast control power supply have led to an enhanced ability of fast response and output current, as well as a new control mode. The structure of cascaded and paralleled H-bridges can meet the demand of extended capacity, and digital control can reMize current and voltage mixed control mode. The validity of the proposed scheme is confirmed by experiments.

Distributed power control algorithm based on game theory for wireless sensor networks

Energy saving is the most important issue in research and development for wireless sensor networks. A power control mechanism can reduce the power consumption of the whole network. Because the character of wireless sensor networks is restrictive energy, this paper proposes a distributed power control algorithm based on game theory for wireless sensor networks which objects of which are reducing power consumption and decreasing overhead and increasing network lifetime. The game theory and OPNET simulation shows that the power control algorithm converges to a Nash Equilibrium when decisions are updated according to a better response dynamic.

A Combining Call Admission Control and Power Control Scheme for D2D Communications Underlaying Cellular Networks

As device-to-device(D2D) communications usually reuses the resource of cellular networks, call admission control(CAC) and power control are crucial problems. However in most power control schemes, total data rates or throughput are regarded as optimization criterion. In this paper, a combining call admission control(CAC) and power control scheme under guaranteeing QoS of every user equipment(UE) is proposed. First, a simple CAC scheme is introduced. Then based on the CAC scheme, a combining call admission control and power control scheme is proposed. Next, the performance of the proposed scheme is evaluated. Finally, maximum DUE pair number and average transmitting power is calculated. Simulation results show that D2 D communications with the proposed combining call admission control and power control scheme can effectively improve the maximum DUE pair number under the premise of meeting necessary QoS.

Joint Trajectory and Power Optimization for Securing UAV Communications Against Active Eavesdropping

Due to both of jamming and eavesdropping,active eavesdroppers can induce more serious security threats to unmanned aerial vehicle(UAV)-enabled communications.This paper considers a secure UAV communication system including both the downlink(DL)and uplink(UL)transmissions,where the confidential information is transmitted between a UAV and a ground node in the presence of an active eavesdropper.We aim to maximize the average secrecy rates of the DL and UL communications,respectively,by jointly optimizing the UAV trajectory and the UAV/ground node’s transmit power control over a given flight period.Due to the non-convexity of the formulated problems,it is difficult to obtain globally optimal solutions.However,we propose efficient iterative algorithms to obtain high-quality suboptimal solutions by applying the block coordinate descent and successive convex optimization methods.Simulation results show that the joint optimization algorithms can effectively improve the secrecy rate performance for both the DL and UL communications,as compared with other baseline schemes.The proposed schemes can be considered as special cases of UAV-assisted non-orthogonal multiple access(NOMA)networks.

Review on grid-forming converter control methods in high-proportion renewable energy power systems

Grid-forming(GFM)converters can provide inertia support for power grids through control technology,stabilize voltage and frequency,and improve system stability,unlike traditional grid-following(GFL)converters.Therefore,in future“double high”power systems,research on the control technology of GFM converters will become an urgent demand.In this paper,we first introduce the basic principle of GFM control and then present five currently used control strategies for GFM converters:droop control,power synchronization control(PSC),virtual synchronous machine control(VSM),direct power control(DPC),and virtual oscillator control(VOC).These five strategies can independently establish voltage phasors to provide inertia to the system.Among these,droop control is the most widely used strategy.PSC and VSM are strategies that simulate the mechanical characteristics of synchronous generators;thus,they are more accurate than droop control.DPC regulates the active power and reactive power directly,with no inner current controller,and VOC is a novel method under study using an oscillator circuit to realize synchronization.Finally,we highlight key technologies and research directions to be addressed in the future.

DS-CDMA system outer loop power control and improvement for multi-service

When a new user accesses the CDMA system, the load will change drastically, and therefore, the advanced outer loop power control （OLPC） technology has to be adopted to enrich the target signal interference ratio （Silt） and improve the system performance. The existing problems about DS-CDMA outer loop power control for multi-service are introduced and the power control theoretical model is analyzed. System simulation is adopted on how to obtain the theoretical performance and parameter optimization of the power control algorithm. The OLPC algorithm is improved and the performance comparisons between the old algorithm and the improved algorithm are given. The results show good performance of the improved OLPC algorithm and prove the validity of the improved method for multi-service.

Low complexity power control approach based on MMSE detection for V-BLAST

A low complexity Per-Antenna Power Control （PAPC） approach based on Minimum Mean Squared Error （MMSE） detection for V-BLAST is proposed in this paper. The PAPC approach is developed for minimizing the Bit Error Rate （BER） averaged over all substreams when the data throughput and the total transmit power keep constant over time. Simulation results show that the Power-controlled V-BLAST （P-BLAST） outperforms the conventional V-BLAST in terms of BER performance with MMSE detector, especially in presence of high spatial correlation between antennas. However, the additional complexity for P-BLAST is not high. When MMSE detector is adopted, the P-BLAST can achieve a comparable BER performance to that of conventional V-BLAST with Maximum Likelihood （ML） detector but with low complexity.

Difference between grid connections of large-scale wind power and conventional synchronous generation

In China, regions with abundant wind energy resources are generally located at the end of power grids. The power grid architecture in these regions is typically not sufficiently strong, and the energy structure is relatively simple. Thus, connecting large-capacity wind power units complicates the peak load regulation and stable operation of the power grids in these regions. Most wind turbines use power electronic converter technology, which affects the safety and stability of the power grid differently compared with conventional synchronous generators. Furthermore, fluctuations in wind power cause fluctuations in the output of wind farms, making it difficult to create and implement suitable power generation plans for wind farms. The generation technology and grid connection scheme for wind power and conventional thermal power generation differ considerably. Moreover, the active and reactive power control abilities of wind turbines are weaker than those of thermal power units, necessitating additional equipment to control wind turbines. Hence, to address the aforementioned issues with large-scale wind power generation, this study analyzes the differences between the grid connection and collection strategies for wind power bases and thermal power plants. Based on this analysis, the differences in the power control modes of wind power and thermal power are further investigated. Finally, the stability of different control modes is analyzed through simulation. The findings can be beneficial for the planning and development of large-scale wind power generation farms.

Power control and channel allocation optimization game algorithm with low energy consumption for wireless sensor network

In a wireless sensor network （WSN）, the energy of nodes is limited and cannot be charged. Hence, it is necessary to reduce energy consumption. Both the transmission power of nodes and the interference among nodes influence energy consumption. In this paper, we design a power control and channel allocation game model with low energy consumption （PCCAGM）. This model contains transmission power, node interference, and residual energy. Besides, the interaction between power and channel is considered. The Nash equilibrium has been proved to exist. Based on this model, a power control and channel allocation optimization algorithm with low energy consumption （PCCAA） is proposed. Theoretical analysis shows that PCCAA can converge to the Pareto Optimal. Simulation results demonstrate that this algorithm can reduce transmission power and interference effectively. Therefore, this algorithm can reduce energy consumption and prolong the network lifetime.

Game-theoretic approach to power and admission control in hierarchical wireless sensor networks

Power efficiency and link reliability are of great impor- tance in hierarchical wireless sensor networks （HWSNs）, espe- cially at the key level, which consists of sensor nodes located only one hop away from the sink node called OHS. The power and admission control problem in HWSNs is comsidered to improve its power efficiency and link reliability. This problem is modeled as a non-cooperative game in which the active OHSs are con- sidered as players. By applying a double-pricing scheme in the definition of OHSs＇ utility function, a Nash Equilibrium solution with network properties is derived. Besides, a distributed algorithm is also proposed to show the dynamic processes to achieve Nash Equilibrium. Finally, the simulation results demonstrate the effec- tiveness of the proposed algorithm.