Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

Study on ultracapacitor-battery hybrid power system for PHEV applications

For the battery only power system is hard to meet the energy and power requirements reasonably, a hybrid power system with uhracapacitor and battery is studied. A Topology structure is analyzed that the uhracapacitor system is connected with battery pack parallel after a bidirectional DC/DC converter. The ultracapacitor, battery and the hybrid power system are modeled. For the plug-in hybrid electric vehicle （PHEV） application, the control target and control strategy of the hybrid power system are put forward. From the simulation results based on the Chinese urban driving cycle, the hybrid power system could meet the peak power requirements reasonably while the battery pack＇ s current is controlled in a reasonable limit which will be helpful to optimize the battery pack＇ s working conditions to get long cycling life and high efficiency.

Damping controller design based on FO-PID-EMA in VSC HVDC system to improve stability of hybrid power system

Wind energy sources have different structures and functions from conventional power plants in the power system.These resources can affect the exchange of active and reactive power of the network.Therefore,power system stability will be affected by the performance of wind power plants,especially in the event of a fault.In this paper,the improvement of the dynamic stability in power system equipped by wind farm is examined through the supplementary controller design in the high voltage direct current(HVDC)based on voltage source converter(VSC)transmission system.In this regard,impacts of the VSC HVDC system and wind farm on the improvement of system stability are considered.Also,an algorithm based on controllability(observability)concept is proposed to select most appropriate and effective coupling between inputs-outputs(IO)signals of system in different work conditions.The selected coupling is used to apply damping controller signal.Finally,a fractional order PID controller(FO-PID)based on exchange market algorithm(EMA)is designed as damping controller.The analysis of the results shows that the wind farm does not directly contribute to the improvement of the dynamic stability of power system.However,it can increase the controllability of the oscillatory mode and improve the performance of the supplementary controller.

Research and practice of designing hydro/ photovoltaic hybrid power system in microgrid

Small-hydro power station is often used in remote areas beside a river,but it doesn't match electricity demand so well,especially in dry seasons. A photovoltaic (PV) system with battery is a suitable option to complement the electricity gap. In this paper,a new structure of megawatt-class PV system integrating battery at DC-bus (DC: direct current) is proposed to be used in hydro/PV hybrid power system,and 4 main designing considerations and several key equipments are discussed. In 2011,a 2 MWp PV station with the proposed structure was built up in Yushu,China. From stability analysis,the station shows a strong stability under load cut-in/off and solar irradiance's fluctuation.

Hybrid Power Systems Energy Controller Based on Neural Network and Fuzzy Logic

This paper presents a novel adaptive scheme for energy management in stand-alone hybrid power systems. The proposed management system is designed to manage the power flow between the hybrid power system and energy storage elements in order to satisfy the load requirements based on artificial neural network (ANN) and fuzzy logic controllers. The neural network controller is employed to achieve the maximum power point (MPP) for different types of photovoltaic (PV) panels. The advance fuzzy logic controller is developed to distribute the power among the hybrid system and to manage the charge and discharge current flow for performance optimization. The developed management system performance was assessed using a hybrid system comprised PV panels, wind turbine (WT), battery storage, and proton exchange membrane fuel cell (PEMFC). To improve the generating performance of the PEMFC and prolong its life, stack temperature is controlled by a fuzzy logic controller. The dynamic behavior of the proposed model is examined under different operating conditions. Real-time measured parameters are used as inputs for the developed system. The proposed model and its control strategy offer a proper tool for optimizing hybrid power system performance, such as that used in smart-house applications.

Performance of a Wind-Diesel Hybrid Power System with STATCOM as a Reactive Power Compensator

The paper deals with automatic reactive power control of an isolated wind-diesel hybrid power system. The power is generated by diesel engine and wind turbine as prime movers with electrical power conversion by permanent-magnet synchronous generator (PMSG) and permanent-magnet induction generator (PMIG) respectively. The mathematical model of the system developed is based on reactive power flow equations. The paper investigates the dynamic performance of the hybrid system for 1% step increase in reactive power load with 1% step increase in input wind power.

Hybrid Power System Power Flow Analysis

Hybrid power supply system consists of a number of independent and different sources of electrical energy with different operating times during different seasons and with energy storage system. Deployment of a hybrid power system is expected in places outside the normal distribution network. For the further research and improvements it is necessary to know in detail the power flow from various sources to the load or to storage battery depending on different seasons. The paper presents data analysis results computed by application developed for detailed analysis of power flows within hybrid power system. Developed application analyses the data from the monitoring system. Data has been acquired and recorded within last year. This data is visualized as power flows in the individual hybrid power system circuits. Together with electrical power the effectiveness and performance parameters of rectifier and DC/AC converter are evaluated. The paper describes achieved results and needs for further improvements of such solution.

Effect of Probabilistic Pattern on System Voltage Stability in Decentralized Hybrid Power System

This paper presents an proportional integral (PI) based voltage-reactive power control for wind diesel based decentralized hybrid power system with wide range of disturbances to demonstrate the compensation effect on system with intelligent tuning methods such as genetic algorithm (GA), artificial neural network (ANN) and adaptive neuro fuzzy inference system (ANFIS). The effect of probabilistic load and/or input power pattern is introduced which is incorporated in MATLAB simulink model developed for the study of decentralized hybrid power system. Results show how tuning method becomes important with high percentage of probabilistic pattern in system. Testing of all tuning methods shows that GA, ANN and ANFIS can preserve optimal performances over wide range of disturbances with superiority to GA in terms of settling time using Integral of Square of Errors (ISE) criterion as fitness function.

Hybrid Power System Options for Off-Grid Rural Electrification in Northern Kenya

For domestic consumers in the rural areas of northern Kenya, as in other developing countries, the typical source of electrical supply is diesel generators. However, diesel generators are associated with both CO2 emissions, which adversely affect the environment and increase diesel fuel prices, which inflate the prices of consumer goods. The Kenya government has taken steps towards addressing this issue by proposing The Hybrid Mini-Grid Project, which involves the installation of 3 MW of wind and solar energy systems in facilities with existing diesel generators. However, this project has not yet been implemented. As a contribution to this effort, this study proposes, simulates and analyzes five different configurations of hybrid energy systems incorporating wind energy, solar energy and battery storage to replace the stand-alone diesel power systems servicing six remote villages in northern Kenya. If implemented, the systems proposed here would reduce Kenya’s dependency on diesel fuel, leading to reductions in its carbon footprint. This analysis confirms the feasibility of these hybrid systems with many configurations being profitable. A Multi-Attribute Trade-Off Analysis is employed to determine the best hybrid system configuration option that would reduce diesel fuel consumption and jointly minimize CO2 emissions and net present cost. This analysis determined that a wind-diesel-battery configuration consisting of two 500 kW turbines, 1200 kW diesel capacity and 95,040 Ah battery capacity is the best option to replace a 3200 kW stand-alone diesel system providing electricity to a village with a peak demand of 839 kW. It has the potential to reduce diesel fuel consumption and CO2 emissions by up to 98.8%.

EID-based Load Frequency Control for Interconnected Hybrid Power System Integrated with RESs

By considering the influence of renewable energy sources(RESs)integration on multi-area interconnected hybrid power systems,this paper proposes an equivalent input disturbance(EID)-based load frequency control(LFC)strategy,which can effectively overcome the factors of random disturbance,model uncertainties and communication delay.First,an equivalent mathematical LFC model of an interconnected system is constructed.Then,the proposed robust controllers,based on the idea of EID,are designed to suppress the randomness and volatility of the renewable energy grid connection and coordinate the frequency fluctuation of the interconnected power system.Finally,the validity and superiority of the established topology structure and the superiority of the proposed strategy are demonstrated by dynamic time domain response experiments under the condition of high penetration of renewable energy.

Machine learning modeling for fuel cell-battery hybrid power system dynamics in a Toyota Mirai 2 vehicle under various drive cycles

Electrification is considered essential for the decarbonization of mobility sector, and understanding and modeling the complex behavior of modern fuel cell-battery electric-electric hybrid power systems is challenging, especially for product development and diagnostics requiring quick turnaround and fast computation. In this study, a novel modeling approach is developed, utilizing supervised machine learning algorithms, to replicate the dynamic characteristics of the fuel cell-battery hybrid power system in a 2021 Toyota Mirai 2nd generation (Mirai 2) vehicle under various drive cycles. The entire data for this study is collected by instrumenting the Mirai vehicle with in-house data acquisition devices and tapping into the Mirai controller area network bus during chassis dynamometer tests. A multi-input - multi-output, feed-forward artificial neural network architecture is designed to predict not only the fuel cell attributes, such as average minimum cell voltage, coolant and cathode air outlet temperatures, but also the battery hybrid system attributes, including lithium-ion battery pack voltage and temperature with the help of 15 system operating parameters. Over 21,0000 data points on various drive cycles having combinations of transient and near steady-state driving conditions are collected, out of which around 15,000 points are used for training the network and 6,000 for the evaluation of the model performance. Various data filtration techniques and neural network calibration processes are explored to condition the data and understand the impact on model performance. The calibrated neural network accurately predicts the hybrid power system dynamics with an R-squared value greater than 0.98, demonstrating the potential of machine learning algorithms for system development and diagnostics.

Optimal power flow calculation in AC/DC hybrid power system based on adaptive simplified human learning optimization algorithm

This paper employs an efficacious analytical tool,adaptive simplified human learning optimization(ASHLO)algorithm,to solve optimal power flow(OPF)problem in AC/DC hybrid power system,considering valve-point loading effects of generators,carbon tax,and prohibited operating zones of generators,respectively.ASHLO algorithm,involves random learning operator,individual learning operator,social learning operator and adaptive strategies.To compare and analyze the computation performance of the ASHLO method,the proposed ASHLO method and other heuristic intelligent optimization methods are employed to solve OPF problem on the modified IEEE 30-bus and 118-bus AC/DC hybrid test system.Numerical results indicate that the ASHLO method has good convergent property and robustness.Meanwhile,the impacts of wind speeds and locations of HVDC transmission line integrated into the AC network on the OPF results are systematically analyzed.

Estimation of composite load model with aggregate induction motor dynamic load for an isolated hybrid power system

It is well recognized that the voltage stability of a power system is affected by the load model and hence, to effectively analyze the reactive power compensation of an isolated hybrid wind-diesel based power system, the loads need to be considered along with the generators in a transient analysis. This paper gives a detailed mathematical modeling to compute the reactive power response with small voltage perturbation for composite load. The composite load is a combination of the static and dynamic load model. To develop this composite load model, the exponential load is used as a static load model and induction motors （IMs） are used as a dynamic load model. To analyze the dynamics of IM load, the fifth, third and first order model of IM are formulated and compared using differential equations solver in Matlab coding. Since the decentralized areas have many small consumers which may consist large numbers of IMs of small rating, it is not realistic to model either a single large rating unit or all small rating IMs together that are placed in the system. In place of using a single large rating IM, a group of motors are considered and then the aggregate model of IM is developed using the law of energy conservation. This aggregate model is used as a dynamic load model. For different simulation studies, especially in the area of voltage stability with reactive power compensation of an isolated hybrid power system, the transfer function AQ/AV of the composite load is required. The transfer function of the composite load is derived in this paper by successive derivation for the exponential model of static load and for the fifth and third order IM dynamic load model using state space model.

Capacity Allocation of Hybrid Power System with Hot Dry Rock Geothermal Energy,Thermal Storage,and PV Based on Game Approaches

This study utilizes hot dry rock(HDR)geothermal energy,which is not affected by climate,to address the capacity allocation of photovoltaic(PV)-storage hybrid power systems(HPSs)in frigid plateau regions.The study replaces the conventional electrochemical energy storage system with a stable HDR plant assisted by a flexible thermal storage(TS)plant.An HPS consisting of an HDR plant,a TS plant,and a PV plant is proposed.Game approaches are introduced to establish the game pattern model of the proposed HPS as the players.The annualized income of each player is used as the payoff function.Furthermore,non-cooperative game and cooperative game approaches for capacity allocation are proposed according to the interests of each player in the proposed HPS.Finally,the proposed model and approaches are validated by performing calculations for an HPS in the Gonghe Basin,Qinghai,China as a case study.The results show that in the proposed non-cooperative game approach,the players focus only on the individual payoff and neglect the overall system optimality.The proposed cooperative game approach for capacity allocation improves the flexibility of the HPS as well as the payoff of each game player.Thereby,the HPS can better satisfy the power fluctuation rate requirements of the grid and increase the equivalent firm capacity(EFC)of PV plants,which in turn indirectly guarantees the reliability of grid operation.

System modeling and dynamic analysis of ISG hybrid power shafting

A system model is established to analyze the dynamic performance of an integrated starter and generator （ISG） hybrid power shafting. The model couples the electromechanical coupling shaft dynamics, the bearing hydrodynamic lubrication and the engine block stiffness. The model is com- pared with the model based on ADAMS or the model neglecting the bearing hydrodynamics. The bearing eccentricity and the oil film pressure have been calculated under different hybrid conditions or at the different motor power levels. It＇ s found that the bearing hydrodynamics decreases the cal- culation results of the bearing peak load. Changes of the hybrid conditions or the motor power have no significant effect on the main bearing, but have impact on the motor bearing. A hybrid power sys- tem composed of a 1.6 L engine and a 45 kW ISG motor can operate safely.

Hierarchical Frequency-dependent Chance Constrained Unit Commitment for Bulk AC/DC Hybrid Power Systems with Wind Power Generation

As the steady-state frequency of an actual power system decreases from its nominal value,the composite load of the system generally responds positively to lower power consumption,and vice versa.It is believed that this load frequency damping(LFD)effect will be artificially enhanced,i.e.,sensitivities of loads with respect to operational frequency will increase,in future power systems.Thus,for wind-integrated power systems,this paper proposes a frequency-dependent chance constrained unit commitment(FDCCUC)model that employs the operational frequency as a dispatching variable so that the LFD effect-based load power can act as a supplemental reserve.Because the frequency deviation is safely restricted,this low-cost reserve can be sufficiently exerted to upgrade the wind power accommodation capability of a power system that is normally confined by an inadequate reserve to cope with uncertain wind power forecasting error.Moreover,when the FDCCUC model is applied to a bulk AC/DC hybrid power system consisting of several independently operated regional AC grids interconnected by DC tie-lines,a hierarchically implemented searching algorithm is proposed to protect private scheduling information of the regional AC grids.Simulations on a 2-area 6-bus system and a 3-area 354-bus system verify the effectiveness of the FDCCUC model and hierarchical searching algorithm.

Effective Control of Smart Hybrid Power Systems:Cooperation of Robust LFC and Virtual Inertia Control Systems

A modern power system is expected to consist primarily of renewables,which either lack or have less rotating masses(i.e.,source of inertia)compared to the traditional generation sources.However,the growth of renewables generation,based on power electronics,can substantially decrease the inertia levels of renewable power grids,which can create several frequency stability issues,resulting in power system degradation.To address this issue,this paper presents a recent virtual inertia scheme predicated on electric vehicles(EVs)to mimic the necessary inertia power in low-inertia smart hybrid power systems(SHPSs),thus regulating the system frequency and avoiding system instability.Moreover,to guarantee robust performance and more stability for SHPSs against multiple perturbations,system uncertainties,and physical constraints,this paper also proposes a robust control strategy relying on a coefficient diagram method(CDM)for the load frequency control(LFC)of SHPSs considering high renewables penetration and EVs.The efficacy of the proposed system(i.e.,robust LFC with the proposed VIC strategy)is validated by comparison with a conventional LFC with/without the proposed VIC system.In addition,the simulation outcomes show that the proposed system can considerably support smart low-inertia hybrid power systems for many different contingencies.

Energy-Saving and Control of General Indoor Lighting Elements Fed from a Micro-Controlled Hybrid Power Generation System

In this study, an off grid wind-solar hybrid power generation system was established at Afyon Kocatepe University to meet the energy need of lighting system of three different laboratories. It is planned to efficiently use the energy obtained from the designed hybrid power generation system. For this purpose, PIC 16F877 was used in controlling of lighting load of laboratories. The off-grid wind-solar hybrid power generation system consists of 570 W 24 V mono crystal solar panels, 600 W wind power generation system and accumulator groups. The load control circuit made with PIC 16F877 is designed in a manner that will control the lighting armature groups individually activate and deactivate the armature groups according to intensity of illumination in environment. Besides, separately from generation and storing units constituting the hybrid power generation system, data in kWh are recorded by means of software in 10 seconds intervals. With the obtained power generation and storing data, analyzing of power consumption data when the load control system in active or passive position is made. According to analysis results, with controlling of lighting load and using of energy obtained from off grid wind-solar hybrid power generation system, 20.6% energy saving has been ensured.

Electricity Price of Hybrid Power System and Decision Making of Renewable Energy Investment Capacity

In the existing electricity market,the traditional power suppliers and renewable energy generators coexist in the power supply side. In the power supply side,renewable energy generators generate power by wind and other natural conditions,leading renewable energy output a certain randomness. However,the low marginal generating cost and the reduction of carbon emissions,and thus brings a certain advantage for renewable energy compared to alternative energy. Electricity,as a special commodity,stable and adequate power supply is a necessary guarantee for economic and social development. Power shortage situation is not allowed in the power system,and the extra power needs to be handled for the purpose of safety. In this paper,the hybrid power generated by renewable energy generators and traditional energy generators is used as power supply,and then the electricity market sells hybrid power to electricity consumers,the hybrid power system determines the optimal daytimeprice,nighttime price,and the optimal installed capacity of the renewable energy suppliers. We find that the installed capacity of renewable energy increases first and then decreases with the increase of the price sensitivity coefficient of traditional energy supply. Electricity demand is negatively related to electricity price in the current period,and is positively related to price in the other period. The average price of day and night is only related to the total potential demand of day and night and the total generation probability of renewable energy. The price difference between daytime and nighttime is positively related to potential electricity demand,and negatively related to the sensitivity coefficient of electricity price.

Power Quality Improvement Using ANN Controller For Hybrid Power Distribution Systems

In this work,an Artificial Neural Network(ANN)based technique is suggested for classifying the faults which occur in hybrid power distribution systems.Power,which is generated by the solar and wind energy-based hybrid system,is given to the grid at the Point of Common Coupling(PCC).A boost converter along with perturb and observe(P&O)algorithm is utilized in this system to obtain a constant link voltage.In contrast,the link voltage of the wind energy conversion system(WECS)is retained with the assistance of a Proportional Integral(PI)controller.The grid synchronization is tainted with the assis-tance of the d-q theory.For the analysis of faults like islanding,line-ground,and line-line fault,the ANN is utilized.The voltage signal is observed at the PCC,and the Discrete Wavelet Transform(DWT)is employed to obtain different features.Based on the collected features,the ANN classifies the faults in an effi-cient manner.The simulation is done in MATLAB and the results are also validated through the hardware implementation.Detailed fault analysis is carried out and the results are compared with the existing techniques.Finally,the Total harmonic distortion(THD)is lessened by 4.3%by using the proposed methodology.