Deep Reinforcement Learning Based Joint Cooperation Clustering and Downlink Power Control for Cell-Free Massive MIMO

In recent times,various power control and clustering approaches have been proposed to enhance overall performance for cell-free massive multipleinput multiple-output(CF-mMIMO)networks.With the emergence of deep reinforcement learning(DRL),significant progress has been made in the field of network optimization as DRL holds great promise for improving network performance and efficiency.In this work,our focus delves into the intricate challenge of joint cooperation clustering and downlink power control within CF-mMIMO networks.Leveraging the potent deep deterministic policy gradient(DDPG)algorithm,our objective is to maximize the proportional fairness(PF)for user rates,thereby aiming to achieve optimal network performance and resource utilization.Moreover,we harness the concept of“divide and conquer”strategy,introducing two innovative methods termed alternating DDPG(A-DDPG)and hierarchical DDPG(H-DDPG).These approaches aim to decompose the intricate joint optimization problem into more manageable sub-problems,thereby facilitating a more efficient resolution process.Our findings unequivo-cally showcase the superior efficacy of our proposed DDPG approach over the baseline schemes in both clustering and downlink power control.Furthermore,the A-DDPG and H-DDPG obtain higher performance gain than DDPG with lower computational complexity.

Safety-Constrained Multi-Agent Reinforcement Learning for Power Quality Control in Distributed Renewable Energy Networks

This paper examines the difficulties of managing distributed power systems,notably due to the increasing use of renewable energy sources,and focuses on voltage control challenges exacerbated by their variable nature in modern power grids.To tackle the unique challenges of voltage control in distributed renewable energy networks,researchers are increasingly turning towards multi-agent reinforcement learning(MARL).However,MARL raises safety concerns due to the unpredictability in agent actions during their exploration phase.This unpredictability can lead to unsafe control measures.To mitigate these safety concerns in MARL-based voltage control,our study introduces a novel approach:Safety-ConstrainedMulti-Agent Reinforcement Learning(SC-MARL).This approach incorporates a specialized safety constraint module specifically designed for voltage control within the MARL framework.This module ensures that the MARL agents carry out voltage control actions safely.The experiments demonstrate that,in the 33-buses,141-buses,and 322-buses power systems,employing SC-MARL for voltage control resulted in a reduction of the Voltage Out of Control Rate(%V.out)from0.43,0.24,and 2.95 to 0,0.01,and 0.03,respectively.Additionally,the Reactive Power Loss(Q loss)decreased from 0.095,0.547,and 0.017 to 0.062,0.452,and 0.016 in the corresponding systems.

Mathematical Modeling and Control Algorithm Development for Bidirectional Power Flow in CCS-CNT System

As the demand for more efficient and adaptable power distribution systems intensifies, especially in rural areas, innovative solutions like the Capacitor-Coupled Substation with a Controllable Network Transformer (CCS-CNT) are becoming increasingly critical. Traditional power distribution networks, often limited by unidirectional flow capabilities and inflexibility, struggle to meet the complex demands of modern energy systems. The CCS-CNT system offers a transformative approach by enabling bidirectional power flow between high-voltage transmission lines and local distribution networks, a feature that is essential for integrating renewable energy sources and ensuring reliable electrification in underserved regions. This paper presents a detailed mathematical representation of power flow within the CCS-CNT system, emphasizing the control of both active and reactive power through the adjustment of voltage levels and phase angles. A control algorithm is developed to dynamically manage power flow, ensuring optimal performance by minimizing losses and maintaining voltage stability across the network. The proposed CCS-CNT system demonstrates significant potential in enhancing the efficiency and reliability of power distribution, making it particularly suited for rural electrification and other applications where traditional methods fall short. The findings underscore the system's capability to adapt to varying operational conditions, offering a robust solution for modern power distribution challenges.

Online Learning Control for Harmonics Reduction Based on Current Controlled Voltage Source Power Inverters

Nonlinear loads in the power distribution system cause non-sinusoidal currents and voltages with harmonic components.Shunt active filters(SAF) with current controlled voltage source inverters(CCVSI) are usually used to obtain balanced and sinusoidal source currents by injecting compensation currents.However,CCVSI with traditional controllers have a limited transient and steady state performance.In this paper,we propose an adaptive dynamic programming(ADP) controller with online learning capability to improve transient response and harmonics.The proposed controller works alongside existing proportional integral(PI) controllers to efficiently track the reference currents in the d-q domain.It can generate adaptive control actions to compensate the PI controller.The proposed system was simulated under different nonlinear(three-phase full wave rectifier) load conditions.The performance of the proposed approach was compared with the traditional approach.We have also included the simulation results without connecting the traditional PI control based power inverter for reference comparison.The online learning based ADP controller not only reduced average total harmonic distortion by 18.41%,but also outperformed traditional PI controllers during transients.

A Deep Reinforcement Learning-Based Power Control Scheme for the 5G Wireless Systems

In the fifth generation(5G)wireless system,a closed-loop power control(CLPC)scheme based on deep Q learning network(DQN)is introduced to intelligently adjust the transmit power of the base station(BS),which can improve the user equipment(UE)received signal to interference plus noise ratio(SINR)to a target threshold range.However,the selected power control(PC)action in DQN is not accurately matched the fluctuations of the wireless environment.Since the experience replay characteristic of the conventional DQN scheme leads to a possibility of insufficient training in the target deep neural network(DNN).As a result,the Q-value of the sub-optimal PC action exceed the optimal one.To solve this problem,we propose the improved DQN scheme.In the proposed scheme,we add an additional DNN to the conventional DQN,and set a shorter training interval to speed up the training of the DNN in order to fully train it.Finally,the proposed scheme can ensure that the Q value of the optimal action remains maximum.After multiple episodes of training,the proposed scheme can generate more accurate PC actions to match the fluctuations of the wireless environment.As a result,the UE received SINR can achieve the target threshold range faster and keep more stable.The simulation results prove that the proposed scheme outperforms the conventional schemes.

A Data Driven Security Correction Method for Power Systems with UPFC

The access of unified power flow controllers(UPFC)has changed the structure and operation mode of power grids all across the world,and it has brought severe challenges to the traditional real-time calculation of security correction based on traditionalmodels.Considering the limitation of computational efficiency regarding complex,physical models,a data-driven power system security correction method with UPFC is,in this paper,proposed.Based on the complex mapping relationship between the operation state data and the security correction strategy,a two-stage deep neural network(DNN)learning framework is proposed,which divides the offline training task of security correction into two stages:in the first stage,the stacked auto-encoder(SAE)classification model is established,and the node correction state(0/1)output based on the fault information;in the second stage,the DNN learningmodel is established,and the correction amount of each action node is obtained based on the action nodes output in the previous stage.In this paper,the UPFC demonstration project of NanjingWest Ring Network is taken as a case study to validate the proposed method.The results show that the proposed method can fully meet the real-time security correction time requirements of power grids,and avoid the inherent defects of the traditional model method without an iterative solution and can also provide reasonable security correction strategies for N-1 and N-2 faults.

A novel fractional uplink power control framework for self-organizing networks

Internet of things and network densification bring significant challenges to uplink management.Only depending on optimization algorithm enhancements is not enough for uplink transmission.To control intercell interference,Fractional Uplink Power Control(FUPC)should be optimized from network-wide perspective,which has to find a better traffic distribution model.Conventionally,traffic distribution is geographic-based,and ineffective due to tricky locating efforts.This paper proposes a novel uplink power management framework for Self-Organizing Networks(SON),which firstly builds up pathloss-based traffic distribution model and then makes the decision of FUPC based on the model.PathLoss-based Traffic Distribution(PLTD)aggregates traffic based on the propagation condition of traffic that is defined as the pathloss between the position generating the traffic and surrounding cells.Simulations show that the improvement in optimization efficiency of FUPC with PLTD can be up to 40%compared to conventional GeoGraphic-based Traffic Distribution(GGTD).

Distributed robust power control in two-tier vehicle networks under uncertain channel environments

This paper proposes a novel optimization scheme to support stable and reliable vehicle-to-everything connections in two-tier networks,where the uplink channel of the cellular user is reused by underlay vehicle-to-vehicle communications.However,considering complex channel fading and high-speed vehicle movement,the cer-tainty assumption is impractical and fails to maintain power control strategy in reality in the traditional statical vehicular networks.Rather than the perfect channel state information assumption,the first-order Gauss-Markov process which is a probabilistic model affected by vehicle speed and fading is introduced to describe imperfect channel gains.Moreover,interference management is a major challenge in reusing communications,especially in uncertain channel environments.Power control is an effective way to realize interference management,and optimal power allocation can ensure that interference of the user meets the communication requirements.In this study,the sum-rate-oriented power control scheme and minimum-rate-oriented power control scheme were implemented to manage interference and satisfy different design objectives.Since both of these schemes are non-convex and intractable,the Bernstein approximation and successive convex approximation methods were adopted to transform the original problems into convex ones.Furthermore,a novel distributed robust power control al-gorithm was developed to determine the optimal solutions.The performance of the algorithm was evaluated through numerical simulations,and the results indicate that the proposed algorithm is effective in vehicular communication networks with uncertain channel environments.

An Adaptive Wireless Power Sharing Control for Multiterminal HVDC

Power sharing among multiterminal high voltage direct current terminals(MT-HVDC)is mainly developed based on a priority or sequential manners,which uses to prevent the problem of overloading due to a predefined controller coefficient.Furthermore,fixed power sharing control also suffers from an inability to identify power availability at a rectification station.There is a need for a controller that ensures an efficient power sharing among the MT-HVDC terminals,prevents the possibility of overloading,and utilizes the available power sharing.A new adaptive wireless control for active power sharing among multiterminal(MT-HVDC)systems,including power availability and power management policy,is proposed in this paper.The proposed control strategy solves these issues and,this proposed controller strategy is a generic method that can be applied for unlimited number of converter stations.The rational of this proposed controller is to increase the system reliability by avoiding the necessity of fast communication links.The test system in this paper consists of four converter stations based on three phase-two AC voltage levels.The proposed control strategy for a multiterminal HVDC system is conducted in the power systems computer aided design/electromagnetic transient design and control(PSCAD/EMTDC)simulation environment.The simulation results significantly show the flexibility and usefulness of the proposed power sharing control provided by the new adaptive wireless method.

Model Predictive Control Strategy of Multi-Port Interline DC Power Flow Controller

There are issues with flexible DC transmission system such as a lack of control freedom over power flow.In order to tackle these issues,a DC power flow controller(DCPFC)is incorporated into a multi-terminal,flexible DC power grid.In recent years,a multi-port DC power flow controller based on a modular multi-level converter has become a focal point of research due to its simple structure and robust scalability.This work proposes a model predictive control(MPC)strategy for multi-port interline DC power flow controllers in order to improve their steady-state dynamic performance.Initially,the mathematical model of a multi-terminal DC power grid with a multi-port interline DC power flow controller is developed,and the relationship between each regulated variable and control variable is determined;The power flow controller is then discretized,and the cost function and weight factor are built with numerous control objectives.Sub module sorting method and nearest level approximation modulation regulate the power flow controller;Lastly,theMATLAB/Simulink simulation platformis used to verify the correctness of the establishedmathematicalmodel and the control performance of the suggestedMPC strategy.Finally,it is demonstrated that the control strategy possesses the benefits of robust dynamic performance,multiobjective control,and a simple structure.

REDUNDANCY TECHNOLOGY FOR AIRCRAFT MULTI-CHANNEL DC POWER SUPPLY SYSTEM

The redundancy technology for the aircraft multi-channel DC electrical power supply system is studied. In this system, the key loads can obtain power from seven sources. The direct current bus power control unit （DC BPCU） is put forward to manage the power supply system automatically. The redundancy innovation is also applied in both hardware and software of DC BPCU. Furthermore, redundancy fault diagnosis is discussed through the existing parts. Experiments and applications show that the proposed aircraft DC power supply system possesses many advantages of high reliability, high automation and so on.

Cooperative Rate Splitting Transmit Design for Full-Duplex-Enabled Multiple Multicast Communication Systems

This paper examines the performance of Full-Duplex Cooperative Rate Splitting(FD-CRS)with Simultaneous Wireless Information and Power Transfer(SWIPT)support in Multiple Input Single Output(MISO)networks.In a Rate Splitting Multiple Access(RSMA)multicast system with two local users and one remote user,the common data stream contains the needs of all users,and all users can decode the common data stream.Therefore,each user can receive some information that other users need,and local users with better channel conditions can use this information to further enhance the reception reliability and data rate of users with poor channel quality.Even using Cell-Center-Users(CCUs)as a cooperative relay to assist the transmission of common data can improve the average system speed.To maximize the minimum achievable rate,we optimize the beamforming vector of Base Station(BS),the common streamsplitting vector,the cooperative distributed beamvector and the strong user transmission power under the power budget constraints of BS and relay devices and the service quality requirements constraints of users.Since the whole problem is not convex,we cannot solve it directly.Therefore,we propose a low complexity algorithm based on Successive Convex Approximation(SCA)technology to find the optimal solution to the problemunder consideration.The simulation results show that FD C-RSMA has better gain andmore powerful than FD C-NOMA,HD C-RSMA,RSMA and NOMA.

Resource allocation for D2D-assisted haptic communications

Haptic communications is recognized as a promising enabler of extensive services by enabling real-time haptic control and feedback in remote environments,e.g.,teleoperation and autonomous driving.Considering the strict transmission requirements on reliability and latency,Device-to-Device(D2D)communications is introduced to assist haptic communications.In particular,the teleoperators with poor channel quality are assisted by auxiliaries,and each auxiliary and its corresponding teleoperator constitute a D2D pair.However,the haptic interaction and the scarcity of radio resources pose severe challenges to the resource allocation,especially facing the sporadic packet arrivals.First,the contentionbased access scheme is applied to achieve low-latency transmission,where the resource scheduling latency is omitted and users can directly access available resources.In this context,we derive the reliability index of D2D pairs under the contention-based access scheme,i.e.,closed-loop packet error probability.Then,the reliability performance is guaranteed by bidirectional power control,which aims to minimize the sum packet error probability of all D2D pairs.Potential game theory is introduced to solve the problem with low complexity.Accordingly,a distributed power control algorithm based on synchronous log-linear learning is proposed to converge to the optimal Nash Equilibrium.Experimental results demonstrate the superiority of the proposed learning algorithm.

Learning-based user association and dynamic resource allocation in multi-connectivity enabled unmanned aerial vehicle networks

Unmanned Aerial Vehicles(UAvs)as aerial base stations to provide communication services for ground users is a flexible and cost-effective paradigm in B5G.Besides,dynamic resource allocation and multi-connectivity can be adopted to further harness the potentials of UAVs in improving communication capacity,in such situations such that the interference among users becomes a pivotal disincentive requiring effective solutions.To this end,we investigate the Joint UAV-User Association,Channel Allocation,and transmission Power Control(J-UACAPC)problem in a multi-connectivity-enabled UAV network with constrained backhaul links,where each UAV can determine the reusable channels and transmission power to serve the selected ground users.The goal was to mitigate co-channel interference while maximizing long-term system utility.The problem was modeled as a cooperative stochastic game with hybrid discrete-continuous action space.A Multi-Agent Hybrid Deep Reinforcement Learning(MAHDRL)algorithm was proposed to address this problem.Extensive simulation results demonstrated the effectiveness of the proposed algorithm and showed that it has a higher system utility than the baseline methods.

Graph neural network-based scheduling for multi-UAV-enabled communications in D2D networks

In this paper,we jointly design the power control and position dispatch for Multi-Unmanned Aerial Vehicle(UAV)-enabled communication in Device-to-Device(D2D)networks.Our objective is to maximize the total transmission rate of Downlink Users(DUs).Meanwhile,the Quality of Service(QoS)of all D2D users must be satisfied.We comprehensively considered the interference among D2D communications and downlink transmissions.The original problem is strongly non-convex,which requires high computational complexity for traditional optimization methods.And to make matters worse,the results are not necessarily globally optimal.In this paper,we propose a novel Graph Neural Networks(GNN)based approach that can map the considered system into a specific graph structure and achieve the optimal solution in a low complexity manner.Particularly,we first construct a GNN-based model for the proposed network,in which the transmission links and interference links are formulated as vertexes and edges,respectively.Then,by taking the channel state information and the coordinates of ground users as the inputs,as well as the location of UAVs and the transmission power of all transmitters as outputs,we obtain the mapping from inputs to outputs through training the parameters of GNN.Simulation results verified that the way to maximize the total transmission rate of DUs can be extracted effectively via the training on samples.Moreover,it also shows that the performance of proposed GNN-based method is better than that of traditional means.

Adaptive and distance-driven power control scheme in mobile ad hoc networks

In order to save the energy and reduce the latency of the end-to-end transmission in mobile ad hoc networks an adaptive and distance-driven power control ADPC scheme is proposed by means of distance research in random geometrics. Through mathematical proof the optimal number of relay nodes and the optimal location of each node for data transmission can be obtained when a distance is given.In the ADPC first the source node computes the optimal number and the sites of the relay nodes between the source and the destination nodes.Then it searches feasible relay nodes around the optimal virtual relay-sites and selects one link with the minimal total transmission energy consumption for data transmission.Simulation results show that the ADPC can reduce both the energy dissipation and the end-to-end latency of the transmission.

An Uplink Power Control Scheme in CDMA Mobile Communication System

A brief introduction of principles and algorithm realization of uplink power control in CDMA mobile communication system based on IS 95 are given, and then the blocking probability and Erlang capacity under the condition of perfect and imperfect uplink power control are presented and analyzed. Finally the uplink power control algorithms are simulated, and the optimum uplink power control algorithm that maximizes system Erlang capacity is acquired.

The Research and Application of the Anti-Interference Techniques of Electronic Device in Power System

This paper presents the study and application of the electronic device anti-interference techniques underhigh voltage and/or heavy current electro-magnetic circumstance in power system.[

Distributed Power Control for Energy Conservation in Hybrid Cellular Network with Device-to-Device Communication

Device-to-Device(D2D) communication has been proposed as a promising implementation of green communication to benefit the existed cellular network.In order to limit cross-tier interference while explore the gain of short-range communication,we devise a series of distributed power control(DPC) schemes for energy conservation(EC)and enhancement of radio resource utilization in the hybrid system.Firstly,a constrained opportunistic power control model is built up to take advantage of the interference avoidance methodology in the presence of service requirement and power constraint.Then,biasing scheme and admission control are added to evade ineffective power consumption and maintain the feasibility of the system.Upon feasibility,a non-cooperative game is further formulated to exploit the profit in EC with minor influence on spectral efficiency(SE).The convergence of the DPC schemes is validated and their performance is confirmed via simulation results.

Mobile Jammer-Aided Secure UAV Communications via Trajectory Design and Power Control

To further promote the achievable average secrecy rate for UAV-ground communications, a UAV-aided mobile jamming strategy was proposed in this paper. Specifically, an additional cooperative UAV is employed as a mobile jammer to transmit the jamming signal to help keep the source UAV closer to the ground destination, thus establishing more favorable legitimate link and enhancing the secrecy performance. We aimed to maximize the achievable secrecy rate by jointly optimizing the trajectories and transmit power of both source UAV and jammer UAV. To solve the considered non-convex optimization problem, we presented a block coordinate descent based iterative algorithm to address a sequence of approximated convex problems for the optimized parameter block by block to find a local optimal solution. Numerical results verify that the proposed algorithm can achieve significant secrecy rate gain compared to all the benchmark schemes.