Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

A Hybrid Integrated and Low-Cost Multi-Chip Broadband Doherty Power Amplifier Module for 5G Massive MIMO Application

In this paper,a hybrid integrated broadband Doherty power amplifier(DPA)based on a multi-chip module(MCM),whose active devices are fabricated using the gallium nitride(GaN)process and whose passive circuits are fabricated using the gallium arsenide(GaAs)integrated passive device(IPD)process,is proposed for 5G massive multiple-input multiple-output(MIMO)application.An inverted DPA structure with a low-Q output network is proposed to achieve better bandwidth performance,and a single-driver architecture is adopted for a chip with high gain and small area.The proposed DPA has a bandwidth of 4.4-5.0 GHz that can achieve a saturation of more than 45.0 dBm.The gain compression from 37 dBm to saturation power is less than 4 dB,and the average power-added efficiency(PAE)is 36.3%with an 8.5 dB peak-to-average power ratio(PAPR)in 4.5-5.0 GHz.The measured adjacent channel power ratio(ACPR)is better than50 dBc after digital predistortion(DPD),exhibiting satisfactory linearity.

Hybrid model based on K-means++ algorithm, optimal similar day approach, and long short-term memory neural network for short-term photovoltaic power prediction

Photovoltaic(PV) power generation is characterized by randomness and intermittency due to weather changes.Consequently, large-scale PV power connections to the grid can threaten the stable operation of the power system. An effective method to resolve this problem is to accurately predict PV power. In this study, an innovative short-term hybrid prediction model(i.e., HKSL) of PV power is established. The model combines K-means++, optimal similar day approach,and long short-term memory(LSTM) network. Historical power data and meteorological factors are utilized. This model searches for the best similar day based on the results of classifying weather types. Then, the data of similar day are inputted into the LSTM network to predict PV power. The validity of the hybrid model is verified based on the datasets from a PV power station in Shandong Province, China. Four evaluation indices, mean absolute error, root mean square error(RMSE),normalized RMSE, and mean absolute deviation, are employed to assess the performance of the HKSL model. The RMSE of the proposed model compared with those of Elman, LSTM, HSE(hybrid model combining similar day approach and Elman), HSL(hybrid model combining similar day approach and LSTM), and HKSE(hybrid model combining K-means++,similar day approach, and LSTM) decreases by 66.73%, 70.22%, 65.59%, 70.51%, and 18.40%, respectively. This proves the reliability and excellent performance of the proposed hybrid model in predicting power.

Dynamic and Power Generation Features of A Wind-Wave Hybrid System Consisting of A Spar-Type Wind Turbine and An Annular Wave Energy Converter in Irregular Waves

Combining wave energy converters(WECs)with floating offshore wind turbines proves a potential strategy to achieve better use of marine renewable energy.The full coupling investigation on the dynamic and power generation features of the hybrid systems under operational sea states is necessary but limited by numerical simulation tools.Here an aero-hydro-servo-elastic coupling numerical tool is developed and applied to investigate the motion,mooring tension,and energy conversion performance of a hybrid system consisting of a spar-type floating wind turbine and an annular wave energy converter.Results show that the addition of the WEC has no significant negative effect on the dynamic performance of the platform and even enhances the rotational stability of the platform.For surge and pitch motion,the peak of the spectra is originated from the dominating wave component,whereas for the heave motion,the peak of the spectrum is the superposed effect of the dominating wave component and the resonance of the system.The addition of the annular WEC can slightly improve the wind power by making the rotor to be in a better position to face the incoming wind and provide considerable wave energy production,which can compensate for the downtime of the offshore wind.

Power Quality Improvement Using ANN Controller For Hybrid Power Distribution Systems

In this work,an Artificial Neural Network(ANN)based technique is suggested for classifying the faults which occur in hybrid power distribution systems.Power,which is generated by the solar and wind energy-based hybrid system,is given to the grid at the Point of Common Coupling(PCC).A boost converter along with perturb and observe(P&O)algorithm is utilized in this system to obtain a constant link voltage.In contrast,the link voltage of the wind energy conversion system(WECS)is retained with the assistance of a Proportional Integral(PI)controller.The grid synchronization is tainted with the assis-tance of the d-q theory.For the analysis of faults like islanding,line-ground,and line-line fault,the ANN is utilized.The voltage signal is observed at the PCC,and the Discrete Wavelet Transform(DWT)is employed to obtain different features.Based on the collected features,the ANN classifies the faults in an effi-cient manner.The simulation is done in MATLAB and the results are also validated through the hardware implementation.Detailed fault analysis is carried out and the results are compared with the existing techniques.Finally,the Total harmonic distortion(THD)is lessened by 4.3%by using the proposed methodology.

Enhanced Electric Power Adaptability Using Hybrid Pumped-Hydro Technology with Wind and Photovoltaic Integration

The integration of solar and wind energy into the electrical grid has received global research attention due to their unpredictable characteristics.Because wind energy varies across all timescales of utility activity,renewable energy generation should be supplemented and enhanced,from real-time,minute-to-minute variations to annual alterations influencing long-termstrategy.Wind energy generation does not only fluctuate but is also challenging to accurately forecast the timeframes of significance to electricity decision makers;day-ahead and long-term making plans of framework sufficiency such as meeting the network peak load annually.A utility that integrates wind and solar energy into its electricity mix would understand how to adapt to uncertainty and variability in operations while sustaining grid stability.Due to hydropower’s adaptability,a system using hydropower as one of its generating resources could be precisely adapted to absorb the variability of wind and solar energy.The objective of this research study is to create a hybrid system comprising hydro-wind and solar(Hybrid-HWS)integration for power balancing in an isolated electrical network in Klipkop village,Pretoria region,South Africa.The desirability of designing and building goaf storage tank in regard to capability,the fullness of line throughoutwater pumping,dispensing,storage tank spillage,and pressure difference throughout liquid flow within the storage tanks were preliminary assessed using geotechnical and weather forecasting data from a distinctive area of Klipkop town in Pretoria,South Africa.Different facility hours premised on daylight accessibility are scheduled to balance maximum load at early and late hours.However,in the scenario of electrical power,time shift requiring storage for extended periods of time,such as in terms of hours,Hybrid-HWS has been found to have a crucial role.The results of simulations showed a coordinated process design for Hybrid-HWS Energy Storage(ES)to determine everyday strategic planning in reducing the variability of the system resulting from wind-solar-pumped hydro ES output inadequacies and satisfy daily load demands.It could be recommended that by considering the adaptability characteristics,extremely rapidly,ramping,peaking support and maximum stabilizing aid of the system could be archived with pump-hydro into the energy mix which can provide specific guidelines for energy policymakers.

Hybrid Power Bank Deployment Model for Energy Supply Coverage Optimization in Industrial Wireless Sensor Network

Energy supply is one of the most critical challenges of wireless sensor networks(WSNs)and industrial wireless sensor networks(IWSNs).While research on coverage optimization problem(COP)centers on the network’s monitoring coverage,this research focuses on the power banks’energy supply coverage.The study of 2-D and 3-D spaces is typical in IWSN,with the realistic environment being more complex with obstacles(i.e.,machines).A 3-D surface is the field of interest(FOI)in this work with the established hybrid power bank deployment model for the energy supply COP optimization of IWSN.The hybrid power bank deployment model is highly adaptive and flexible for new or existing plants already using the IWSN system.The model improves the power supply to a more considerable extent with the least number of power bank deployments.The main innovation in this work is the utilization of a more practical surface model with obstacles and training while improving the convergence speed and quality of the heuristic algorithm.An overall probabilistic coverage rate analysis of every point on the FOI is provided,not limiting the scope to target points or areas.Bresenham’s algorithm is extended from 2-D to 3-D surface to enhance the probabilistic covering model for coverage measurement.A dynamic search strategy(DSS)is proposed to modify the artificial bee colony(ABC)and balance the exploration and exploitation ability for better convergence toward eliminating NP-hard deployment problems.Further,the cellular automata(CA)is utilized to enhance the convergence speed.The case study based on two typical FOI in the IWSN shows that the CA scheme effectively speeds up the optimization process.Comparative experiments are conducted on four benchmark functions to validate the effectiveness of the proposed method.The experimental results show that the proposed algorithm outperforms the ABC and gbest-guided ABC(GABC)algorithms.The results show that the proposed energy coverage optimization method based on the hybrid power bank deployment model generates more accurate results than the results obtained by similar algorithms(i.e.,ABC,GABC).The proposed model is,therefore,effective and efficient for optimization in the IWSN.

Hybrid Block Diagonalization Precoding for Multi-User Weighted Sum-Rate Maximization

This paper studies large-scale multi-input multi-output(MIMO)orthogonal frequency division multiplexing(OFDM)communications in a broadband frequency-selective channel,where a massive MIMO base station(BS)communicates with multiple users equipped with multi-antenna.We develop a hybrid precoding design to maximize the weighted sum-rate(WSR)of the users by optimizing the digital and the analog precoders alternately.For the digital part,we employ block-diagonalization to eliminate inter-user interference and apply water-filling power allocation to maximize the WSR.For the analog part,the optimization of the PSN is formulated as an unconstrained problem,which can be efficiently solved by a gradient descent method.Numerical results show that the proposed block-diagonal hybrid precoding algorithm can outperform the existing works.

Research on the IL-Bagging-DHKELM Short-Term Wind Power Prediction Algorithm Based on Error AP Clustering Analysis

With the continuous advancement of China’s“peak carbon dioxide emissions and Carbon Neutrality”process,the proportion of wind power is increasing.In the current research,aiming at the problem that the forecasting model is outdated due to the continuous updating of wind power data,a short-term wind power forecasting algorithm based on Incremental Learning-Bagging Deep Hybrid Kernel Extreme Learning Machine(IL-Bagging-DHKELM)error affinity propagation cluster analysis is proposed.The algorithm effectively combines deep hybrid kernel extreme learning machine(DHKELM)with incremental learning(IL).Firstly,an initial wind power prediction model is trained using the Bagging-DHKELM model.Secondly,Euclidean morphological distance affinity propagation AP clustering algorithm is used to cluster and analyze the prediction error of wind power obtained from the initial training model.Finally,the correlation between wind power prediction errors and Numerical Weather Prediction(NWP)data is introduced as incremental updates to the initial wind power prediction model.During the incremental learning process,multiple error performance indicators are used to measure the overall model performance,thereby enabling incremental updates of wind power models.Practical examples show the method proposed in this article reduces the root mean square error of the initial model by 1.9 percentage points,indicating that this method can be better adapted to the current scenario of the continuous increase in wind power penetration rate.The accuracy and precision of wind power generation prediction are effectively improved through the method.

Reliability-BasedModel for Incomplete Preventive ReplacementMaintenance of Photovoltaic Power Systems

At present,the operation and maintenance of photovoltaic power generation systems mainly comprise regular maintenance,breakdown maintenance,and condition-based maintenance,which is very likely to lead to over-or under-repair of equipment.Therefore,a preventive maintenance and replacement strategy for PV power generation systems based on reliability as a constraint is proposed.First,a hybrid failure function with a decreasing service age factor and an increasing failure rate factor is introduced to describe the deterioration of PV power generation equipment,and the equipment is replaced when its reliability drops to the replacement threshold in the last cycle.Then,based on the reliability as a constraint,the average maintenance cost and availability of the equipment are considered,and the non-periodic incomplete maintenance model of the PV power generation system is established to obtain the optimal number of repairs,each maintenance cycle and the replacement cycle of the PV power generation system components.Next,the inverter of a PV power plant is used as a research object.The model in this paper is compared and analyzed with the equal cycle maintenance model without considering reliability and the maintenance model without considering the equipment replacement threshold,Through model comparison,when the optimal maintenance strategy is(0.80,4),the average maintenance cost of this paper’s model are decreased by 20.3%and 5.54%and the availability is increased by 0.2395% and 0.0337%,respectively,compared with the equal-cycle maintenance model without considering the reliability constraint and the maintenance model without considering the equipment replacement threshold.Therefore,this maintenance model can ensure the high reliability of PV plant operation while increasing the equipment availability to improve the system economy.

Proposal of a Deuterium-Deuterium Fusion/PWR Fission Hybrid Reactor

This article proposes to associate a Deuterium-Deuterium (D-D) fusion reactor with a PWR (fission Pressurized Water Reactor) in a hybrid reactor. Even if the mechanical gain (Q factor) of the D-D fusion reactor is below the unity and consequently consumes more energy than it supplies, due to the high energy amplification factor of the PWR fission reactor, the global yield is widely superior to 1. As the energy supplied by the fusion reactor is relatively low and as the neutrons supplied are mainly issued from D-D fusions (at 2.45 MeV), the problems of heat flux and neutrons damage connected with materials, as with D-T fusion reactors are reduced. Of course, there is no need to produce Tritium with this D-D fusion reactor. This type of reactor is able to incinerate any mixture of natural Uranium, natural Thorium and depleted Uranium (waste issued from enrichment plants), with natural Thorium being the best choice. No enriched fuel is needed. So, this type of reactor could constitute a source of energy for several thousands of years because it is about 90 more efficient than a standard fission reactor, such as a PWR or a Candu one, by extracting almost completely the energy from the fertile materials U238 and Th232. For the fission part, PWR technology is mature. For the fusion part, it is based on a reasonable hypothesis done on present Stellarators projects. The working of this reactor is continuous, 24 hours a day. In this paper, it will be targeted a reactor able to provide net electric power of about 1400 MWe, as a big fission power plant.

High-Power and Ultralong-Life Aqueous Zinc-Ion Hybrid Capacitors Based on Pseudocapacitive Charge Storage

Rechargeable aqueous zinc-ion hybrid capacitors and zincion batteries are promising safe energy storage systems.In this study,amorphous RuO2·H2O for the first time was employed to achieve fast and ultralong-life Zn2+storage based on a pseudocapacitive storage mechanism.In the RuO2·H2O||Zn zinc-ion hybrid capacitors with Zn(CF3SO3)2 aqueous electrolyte,the RuO2·H2O cathode can reversibly store Zn2+in a voltage window of 0.4-1.6 V(vs.Zn/Zn2+),delivering a high discharge capacity of 122 mAh g?1.In particular,the zinc-ion hybrid capacitors can be rapidly charged/discharged within 36 s with a very high power density of 16.74 kW kg?1 and a high energy density of 82 Wh kg?1.Besides,the zinc-ion hybrid capacitors demonstrate an ultralong cycle life(over 10,000 charge/discharge cycles).The kinetic analysis elucidates that the ultrafast Zn2+storage in the RuO2·H2O cathode originates from redox pseudocapacitive reactions.This work could greatly facilitate the development of high-power and safe electrochemical energy storage.

Synthetical Efficiency-based Optimization for the Power Distribution of Power-split Hybrid Electric Vehicles

Now the optimization strategies for power distribution are researched widely, and most of them are aiming to the optimal fuel economy and the driving cycle must be preknown. Thus if the actual driving condition deviates from the scheduled driving cycle, the effect of optimal results will be declined greatly. Therefore, the instantaneous optimization strategy carried out on-line is studied in this paper. The power split path and the transmission efficiency are analyzed based on a special power-split scheme and the efficiency models of the power transmitting components are established. The synthetical efficiency optimization model is established for enhancing the transmission efficiency and the fuel economy. The identification of the synthetical efficiency as the optimization objective and the constrain group are discussed emphatically. The optimization is calculated by the adaptive simulated annealing （ASA） algorithm and realized on-line by the radial basis function （RBF）-based similar models. The optimization for power distribution of the hybrid vehicle in an actual driving condition is carried out and the road test results are presented. The test results indicate that the synthetical efficiency optimization method can enhance the transmission efficiency and the fuel economy of the power-split hybrid electric vehicle （HEV） observably. Compared to the rules-based strategy the optimization strategy is optimal and achieves the approximate global optimization solution for the power distribution. The synthetical efficiency optimization solved by ASA algorithm can give attentions to both optimization quality and calculation efficiency, thus it has good application foreground for the power distribution of power-split HEV.

Economic feasibility of large-scale hydro–solar hybrid power including long distance transmission

Solar PV is expected to become the most cost-competitive renewable energy owing to the rapidly decreasing cost of the system. On the other hand, hydropower is a high-quality and reliable regulating power source that can be bundled with solar PV to improve the economic feasibility of long-distance transmitted power. In this paper, a quantification model is established taking into account the regulating capacity of the reservoir, the characteristics of solar generation, and cost of hydro and solar PV with long-distance transmission based on the installed capacity ratio of hydro–solar hybrid power. Results indicate that for hydropower stations with high regulating capacity and generation factor of approximately 0.5, a hydro–solar installed capacity ratio of 1:1 will yield overall optimal economic performance, whereas for hydropower stations with daily regulating capacity reservoir and capacity factor of approximately 0.65, the optimal hydro–solar installed capacity ratio is approximately 1:0.3. In addition, the accuracy of the approach used in this study is verified through operation simulation of a hydro–solar hybrid system including ultra high-voltage direct current(UHVDC) transmission using two case studies in Africa.

Study on integrated development and hybrid operation mode of nuclear power plant and pumped-storage power station

The nuclear power plant is suitable for base-load operation, while the pumped-storage unit mainly gives play to capacity benefit in the electric power system;hence, the integrated development and hybrid operation mode of the two can better meet the needs of the electric power system. This article first presents an analysis of the necessity and superiority of such mode, then explains its meaning and analyzes the working routes. Finally, it proposes the business modes as follows: low price pumping water electricity plus nuclear power in the near term;nuclear power shifted to pumped storage power participating in market competition in the middle term;and, in the long term, nuclear power shifted to pumped storage power as primary and serving as an electric power system when needed.

Study on ultracapacitor-battery hybrid power system for PHEV applications

For the battery only power system is hard to meet the energy and power requirements reasonably, a hybrid power system with uhracapacitor and battery is studied. A Topology structure is analyzed that the uhracapacitor system is connected with battery pack parallel after a bidirectional DC/DC converter. The ultracapacitor, battery and the hybrid power system are modeled. For the plug-in hybrid electric vehicle （PHEV） application, the control target and control strategy of the hybrid power system are put forward. From the simulation results based on the Chinese urban driving cycle, the hybrid power system could meet the peak power requirements reasonably while the battery pack＇ s current is controlled in a reasonable limit which will be helpful to optimize the battery pack＇ s working conditions to get long cycling life and high efficiency.

SER analysis and power allocation for hybrid cooperative transmission system

The symbol-error-rate（SER） and power allocation for hybrid cooperative（HC） transmission system are investigated.Closed-form SER expression is derived by using the moment generating function（MGF）-based approach.However,the resultant SER contains an MGF of the harmonic mean of two independent random variables（RVs）,which is not tractable in SER analysis.We present a simple MGF expression of the harmonic mean of two independent RVs which avoids the hypergeometric functions used commonly in previous studies.Using the simple MGF,closed-form SER for HC system with M-ary phase shift keying（M-PSK） signals is provided.Further,an approximation as well as an upper bound of the SER is presented.It is shown that the SER approximation is asymptotically tight.Based on the tight SER approximation,the power allocation of the HC system is investigated.It is shown that the optimal power allocation does not depend on the fading parameters of the source-destination（SD） channel and it only depends on the source-relay（SR） and relay-destination（RD） channels.Moreover,the performance gain of the power allocation depends on the ratio of the channel quality between RD and SR.With the increase of this ratio,more performance gain can be acquired.

A Numerical Convex Lens for the State-Discretized Modeling and Simulation of Megawatt Power Electronics Systems as Generalized Hybrid Systems

Modeling and simulation have emerged as an indispensable approach to create numerical experiment platforms and study engineering systems.However,the increasingly complicated systems that engineers face today dramatically challenge state-of-the-art modeling and simulation approaches.Such complicated systems,which are composed of not only continuous states but also discrete events,and which contain complex dynamics across multiple timescales,are defined as generalized hybrid systems(GHSs)in this paper.As a representative GHS,megawatt power electronics(MPE)systems have been largely integrated into the modern power grid,but MPE simulation remains a bottleneck due to its unacceptable time cost and poor convergence.To address this challenge,this paper proposes the numerical convex lens approach to achieve state-discretized modeling and simulation of GHSs.This approach transforms conventional time-discretized passive simulations designed for pure-continuous systems into state-discretized selective simulations designed for GHSs.When this approach was applied to a largescale MPE-based renewable energy system,a 1000-fold increase in simulation speed was achieved,in comparison with existing software.Furthermore,the proposed approach uniquely enables the switching transient simulation of a largescale megawatt system with high accuracy,compared with experimental results,and with no convergence concerns.The numerical convex lens approach leads to the highly efficient simulation of intricate GHSs across multiple timescales,and thus significantly extends engineers’capability to study systems with numerical experiments.

Research on power electronic transformer applied in AC/DC hybrid distribution networks

The AC/DC hybrid distribution network is one of the trends in distribution network development, which poses great challenges to the traditional distribution transformer. In this paper, a new topology suitable for AC/DC hybrid distribution network is put forward according to the demands of power grid, with advantages of accepting DG and DC loads, while clearing DC fault by blocking the clamping double sub-module(CDSM) of input stage. Then, this paper shows the typical structure of AC/DC distribution network that is hand in hand. Based on the new topology, this paper designs the control and modulation strategies of each stage, where the outer loop controller of input stage is emphasized for its twocontrol mode. At last, the rationality of new topology and the validity of control strategies are verified by the steady and dynamic state simulation. At the same time, the simulation results highlight the role of PET in energy regulation.

Short-term power generation scheduling rules for cascade hydropower stations based on hybrid algorithm

Power generation dispatching is a large complex system problem with multi-dimensional and nonlinear characteristics. A mathematical model was established based on the principle of reservoir operation. A large quantity of optimal scheduling processes were obtained by calculating the daily runoff process within three typical years, and a large number of simulated daily runoff processes were obtained using the progressive optimality algorithm （POA） in combination with the genetic algorithm （GA）. After analyzing the optimal scheduling processes, the corresponding scheduling rules were determined, and the practical formulas were obtained. These rules can make full use of the rolling runoff forecast and carry out the rolling scheduling. Compared with the optimized results, the maximum relative difference of the annual power generation obtained by the scheduling rules is no more than 1%. The effectiveness and practical applicability of the scheduling rules are demonstrated by a case study. This study provides a new perspective for formulating the rules of power generation dispatching.