Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

Study on ultracapacitor-battery hybrid power system for PHEV applications

For the battery only power system is hard to meet the energy and power requirements reasonably, a hybrid power system with uhracapacitor and battery is studied. A Topology structure is analyzed that the uhracapacitor system is connected with battery pack parallel after a bidirectional DC/DC converter. The ultracapacitor, battery and the hybrid power system are modeled. For the plug-in hybrid electric vehicle （PHEV） application, the control target and control strategy of the hybrid power system are put forward. From the simulation results based on the Chinese urban driving cycle, the hybrid power system could meet the peak power requirements reasonably while the battery pack＇ s current is controlled in a reasonable limit which will be helpful to optimize the battery pack＇ s working conditions to get long cycling life and high efficiency.

Damping controller design based on FO-PID-EMA in VSC HVDC system to improve stability of hybrid power system

Wind energy sources have different structures and functions from conventional power plants in the power system.These resources can affect the exchange of active and reactive power of the network.Therefore,power system stability will be affected by the performance of wind power plants,especially in the event of a fault.In this paper,the improvement of the dynamic stability in power system equipped by wind farm is examined through the supplementary controller design in the high voltage direct current(HVDC)based on voltage source converter(VSC)transmission system.In this regard,impacts of the VSC HVDC system and wind farm on the improvement of system stability are considered.Also,an algorithm based on controllability(observability)concept is proposed to select most appropriate and effective coupling between inputs-outputs(IO)signals of system in different work conditions.The selected coupling is used to apply damping controller signal.Finally,a fractional order PID controller(FO-PID)based on exchange market algorithm(EMA)is designed as damping controller.The analysis of the results shows that the wind farm does not directly contribute to the improvement of the dynamic stability of power system.However,it can increase the controllability of the oscillatory mode and improve the performance of the supplementary controller.

Research and practice of designing hydro/ photovoltaic hybrid power system in microgrid

Small-hydro power station is often used in remote areas beside a river,but it doesn't match electricity demand so well,especially in dry seasons. A photovoltaic (PV) system with battery is a suitable option to complement the electricity gap. In this paper,a new structure of megawatt-class PV system integrating battery at DC-bus (DC: direct current) is proposed to be used in hydro/PV hybrid power system,and 4 main designing considerations and several key equipments are discussed. In 2011,a 2 MWp PV station with the proposed structure was built up in Yushu,China. From stability analysis,the station shows a strong stability under load cut-in/off and solar irradiance's fluctuation.

System modeling and dynamic analysis of ISG hybrid power shafting

A system model is established to analyze the dynamic performance of an integrated starter and generator （ISG） hybrid power shafting. The model couples the electromechanical coupling shaft dynamics, the bearing hydrodynamic lubrication and the engine block stiffness. The model is com- pared with the model based on ADAMS or the model neglecting the bearing hydrodynamics. The bearing eccentricity and the oil film pressure have been calculated under different hybrid conditions or at the different motor power levels. It＇ s found that the bearing hydrodynamics decreases the cal- culation results of the bearing peak load. Changes of the hybrid conditions or the motor power have no significant effect on the main bearing, but have impact on the motor bearing. A hybrid power sys- tem composed of a 1.6 L engine and a 45 kW ISG motor can operate safely.

Low complexity hybrid power distribution combined with subcarrier allocation algorithm for OFDMA

To improve the total throughput of the uplink orthogonal frequency division multiple access system,a low complexity hybrid power distribution（HPD） combined with subcarrier allocation scheme is proposed.For the fairness mechanism for the subcarrier,the inter-cell interference is first analyzed to calculate the capacity of the multi-cell.The user selects the subcarrier with the largest channel gain.Based on the above subcarrier allocation scheme,a new kind of HPD scheme is proposed,which adopts the waterfilling-power-distributed scheme and the equal-power-distributed scheme in the cell-boundary and the cellcenter,respectively.Simulation results show that compared with the waterfilling-power-distributed scheme in the whole cell,the proposed HPD scheme decreases the system complexity significantly,meanwhile its capacity is 2% higher than that of the equal-powerdistributed scheme over the same subcarrier allocation.

Research on hybrid power filter of 6 kV power grid in coal mine

Studied the harmonic control of the 6 kV power grid in a coal mine substation.Taking harmonic suppression and reactive power compensation into account, and complyingwith the economic and efficient technical line of the smart grid, a new hybrid activefilter was proposed and applied to the power grid in the coal mine with the advantagessuch as large capacity, low cost and low loss.In order to improve detection speed and reducethe succeeding errors to improve the filtering performance of the active power filter,the DFT (Discrete Fourier Transform) sliding window algorithm based on coordinatetransformation and improved hysteresis control method was proposed.The Matlab simulationresults show that the hybrid active filter is satisfactory, can improve the grid powerfactor and can meet the requirements of improving the power quality in the coal mine.

Power Quality Improvement Using ANN Controller For Hybrid Power Distribution Systems

In this work,an Artificial Neural Network(ANN)based technique is suggested for classifying the faults which occur in hybrid power distribution systems.Power,which is generated by the solar and wind energy-based hybrid system,is given to the grid at the Point of Common Coupling(PCC).A boost converter along with perturb and observe(P&O)algorithm is utilized in this system to obtain a constant link voltage.In contrast,the link voltage of the wind energy conversion system(WECS)is retained with the assistance of a Proportional Integral(PI)controller.The grid synchronization is tainted with the assis-tance of the d-q theory.For the analysis of faults like islanding,line-ground,and line-line fault,the ANN is utilized.The voltage signal is observed at the PCC,and the Discrete Wavelet Transform(DWT)is employed to obtain different features.Based on the collected features,the ANN classifies the faults in an effi-cient manner.The simulation is done in MATLAB and the results are also validated through the hardware implementation.Detailed fault analysis is carried out and the results are compared with the existing techniques.Finally,the Total harmonic distortion(THD)is lessened by 4.3%by using the proposed methodology.

Hybrid Power Bank Deployment Model for Energy Supply Coverage Optimization in Industrial Wireless Sensor Network

Energy supply is one of the most critical challenges of wireless sensor networks(WSNs)and industrial wireless sensor networks(IWSNs).While research on coverage optimization problem(COP)centers on the network’s monitoring coverage,this research focuses on the power banks’energy supply coverage.The study of 2-D and 3-D spaces is typical in IWSN,with the realistic environment being more complex with obstacles(i.e.,machines).A 3-D surface is the field of interest(FOI)in this work with the established hybrid power bank deployment model for the energy supply COP optimization of IWSN.The hybrid power bank deployment model is highly adaptive and flexible for new or existing plants already using the IWSN system.The model improves the power supply to a more considerable extent with the least number of power bank deployments.The main innovation in this work is the utilization of a more practical surface model with obstacles and training while improving the convergence speed and quality of the heuristic algorithm.An overall probabilistic coverage rate analysis of every point on the FOI is provided,not limiting the scope to target points or areas.Bresenham’s algorithm is extended from 2-D to 3-D surface to enhance the probabilistic covering model for coverage measurement.A dynamic search strategy(DSS)is proposed to modify the artificial bee colony(ABC)and balance the exploration and exploitation ability for better convergence toward eliminating NP-hard deployment problems.Further,the cellular automata(CA)is utilized to enhance the convergence speed.The case study based on two typical FOI in the IWSN shows that the CA scheme effectively speeds up the optimization process.Comparative experiments are conducted on four benchmark functions to validate the effectiveness of the proposed method.The experimental results show that the proposed algorithm outperforms the ABC and gbest-guided ABC(GABC)algorithms.The results show that the proposed energy coverage optimization method based on the hybrid power bank deployment model generates more accurate results than the results obtained by similar algorithms(i.e.,ABC,GABC).The proposed model is,therefore,effective and efficient for optimization in the IWSN.

Performance and Stability of Supercapacitor Modules based on Porous Carbon Electrodes in Hybrid Powertrain

Hybrid power sources have attracted much attention in the electric vehicle area. Particularly, electric-electric hybrid powertrain system consisting of supercapacitor modules and lithium-ion batteries has been widely applied because of the high power density of supercapacitors. In this study, we design a hybrid powertrain system containing two porous carbon electrode-based supercapacitor modules in parallel and one lithium ion battery pack. With the construction of the testing station, the performance and stability of the used supercapacitor modules are investigated in correlation with the structure of the supercapacitor and the nature of the electrode materials applied. It has been shown that the responding time for voltage vibration from 20 V to 48.5 V during charging or discharging process decreases from about 490 s to 94 s with the increase in applied current from 20 A to 100 A. The capacitance of the capacitor modules is nearly independent on the applied current. With the designed setup, the energy efficiency can reach as high as 0.99. The results described here provide a guidance for material selection of supercapacitors and optimized controlling strategy for hybrid power system applied in electric vehicles.

Energy-Saving and Control of General Indoor Lighting Elements Fed from a Micro-Controlled Hybrid Power Generation System

In this study, an off grid wind-solar hybrid power generation system was established at Afyon Kocatepe University to meet the energy need of lighting system of three different laboratories. It is planned to efficiently use the energy obtained from the designed hybrid power generation system. For this purpose, PIC 16F877 was used in controlling of lighting load of laboratories. The off-grid wind-solar hybrid power generation system consists of 570 W 24 V mono crystal solar panels, 600 W wind power generation system and accumulator groups. The load control circuit made with PIC 16F877 is designed in a manner that will control the lighting armature groups individually activate and deactivate the armature groups according to intensity of illumination in environment. Besides, separately from generation and storing units constituting the hybrid power generation system, data in kWh are recorded by means of software in 10 seconds intervals. With the obtained power generation and storing data, analyzing of power consumption data when the load control system in active or passive position is made. According to analysis results, with controlling of lighting load and using of energy obtained from off grid wind-solar hybrid power generation system, 20.6% energy saving has been ensured.

Economic feasibility of large-scale hydro–solar hybrid power including long distance transmission

Solar PV is expected to become the most cost-competitive renewable energy owing to the rapidly decreasing cost of the system. On the other hand, hydropower is a high-quality and reliable regulating power source that can be bundled with solar PV to improve the economic feasibility of long-distance transmitted power. In this paper, a quantification model is established taking into account the regulating capacity of the reservoir, the characteristics of solar generation, and cost of hydro and solar PV with long-distance transmission based on the installed capacity ratio of hydro–solar hybrid power. Results indicate that for hydropower stations with high regulating capacity and generation factor of approximately 0.5, a hydro–solar installed capacity ratio of 1:1 will yield overall optimal economic performance, whereas for hydropower stations with daily regulating capacity reservoir and capacity factor of approximately 0.65, the optimal hydro–solar installed capacity ratio is approximately 1:0.3. In addition, the accuracy of the approach used in this study is verified through operation simulation of a hydro–solar hybrid system including ultra high-voltage direct current(UHVDC) transmission using two case studies in Africa.

Hybrid Power Systems Energy Controller Based on Neural Network and Fuzzy Logic

This paper presents a novel adaptive scheme for energy management in stand-alone hybrid power systems. The proposed management system is designed to manage the power flow between the hybrid power system and energy storage elements in order to satisfy the load requirements based on artificial neural network (ANN) and fuzzy logic controllers. The neural network controller is employed to achieve the maximum power point (MPP) for different types of photovoltaic (PV) panels. The advance fuzzy logic controller is developed to distribute the power among the hybrid system and to manage the charge and discharge current flow for performance optimization. The developed management system performance was assessed using a hybrid system comprised PV panels, wind turbine (WT), battery storage, and proton exchange membrane fuel cell (PEMFC). To improve the generating performance of the PEMFC and prolong its life, stack temperature is controlled by a fuzzy logic controller. The dynamic behavior of the proposed model is examined under different operating conditions. Real-time measured parameters are used as inputs for the developed system. The proposed model and its control strategy offer a proper tool for optimizing hybrid power system performance, such as that used in smart-house applications.

Performance of a Wind-Diesel Hybrid Power System with STATCOM as a Reactive Power Compensator

The paper deals with automatic reactive power control of an isolated wind-diesel hybrid power system. The power is generated by diesel engine and wind turbine as prime movers with electrical power conversion by permanent-magnet synchronous generator (PMSG) and permanent-magnet induction generator (PMIG) respectively. The mathematical model of the system developed is based on reactive power flow equations. The paper investigates the dynamic performance of the hybrid system for 1% step increase in reactive power load with 1% step increase in input wind power.

Hybrid Power System Power Flow Analysis

Hybrid power supply system consists of a number of independent and different sources of electrical energy with different operating times during different seasons and with energy storage system. Deployment of a hybrid power system is expected in places outside the normal distribution network. For the further research and improvements it is necessary to know in detail the power flow from various sources to the load or to storage battery depending on different seasons. The paper presents data analysis results computed by application developed for detailed analysis of power flows within hybrid power system. Developed application analyses the data from the monitoring system. Data has been acquired and recorded within last year. This data is visualized as power flows in the individual hybrid power system circuits. Together with electrical power the effectiveness and performance parameters of rectifier and DC/AC converter are evaluated. The paper describes achieved results and needs for further improvements of such solution.

Effect of Probabilistic Pattern on System Voltage Stability in Decentralized Hybrid Power System

This paper presents an proportional integral (PI) based voltage-reactive power control for wind diesel based decentralized hybrid power system with wide range of disturbances to demonstrate the compensation effect on system with intelligent tuning methods such as genetic algorithm (GA), artificial neural network (ANN) and adaptive neuro fuzzy inference system (ANFIS). The effect of probabilistic load and/or input power pattern is introduced which is incorporated in MATLAB simulink model developed for the study of decentralized hybrid power system. Results show how tuning method becomes important with high percentage of probabilistic pattern in system. Testing of all tuning methods shows that GA, ANN and ANFIS can preserve optimal performances over wide range of disturbances with superiority to GA in terms of settling time using Integral of Square of Errors (ISE) criterion as fitness function.

Practice and Prospect of AC/DC Hybrid Power Grid in Southern China

China Southern Power Grid is a unique EHV AC/DC hybrid transmission network that operates in China. In its service area, the distribution of energy resources and the development of economy are extremely unbalanced, so long-distance and bulk power transmission are needed; besides, the geography and climate conditions are serious, rains, fogs, lightning and typhoon as well as high temperature are common all the year round. Facing these challenges, the power grid enhanced stability control, improved the equipment and strengthen the network structure. In the future, the power grid plans to optimize the disposition of power sources and build digitalized power system.

Hybrid Power System Options for Off-Grid Rural Electrification in Northern Kenya

For domestic consumers in the rural areas of northern Kenya, as in other developing countries, the typical source of electrical supply is diesel generators. However, diesel generators are associated with both CO2 emissions, which adversely affect the environment and increase diesel fuel prices, which inflate the prices of consumer goods. The Kenya government has taken steps towards addressing this issue by proposing The Hybrid Mini-Grid Project, which involves the installation of 3 MW of wind and solar energy systems in facilities with existing diesel generators. However, this project has not yet been implemented. As a contribution to this effort, this study proposes, simulates and analyzes five different configurations of hybrid energy systems incorporating wind energy, solar energy and battery storage to replace the stand-alone diesel power systems servicing six remote villages in northern Kenya. If implemented, the systems proposed here would reduce Kenya’s dependency on diesel fuel, leading to reductions in its carbon footprint. This analysis confirms the feasibility of these hybrid systems with many configurations being profitable. A Multi-Attribute Trade-Off Analysis is employed to determine the best hybrid system configuration option that would reduce diesel fuel consumption and jointly minimize CO2 emissions and net present cost. This analysis determined that a wind-diesel-battery configuration consisting of two 500 kW turbines, 1200 kW diesel capacity and 95,040 Ah battery capacity is the best option to replace a 3200 kW stand-alone diesel system providing electricity to a village with a peak demand of 839 kW. It has the potential to reduce diesel fuel consumption and CO2 emissions by up to 98.8%.

EID-based Load Frequency Control for Interconnected Hybrid Power System Integrated with RESs

By considering the influence of renewable energy sources(RESs)integration on multi-area interconnected hybrid power systems,this paper proposes an equivalent input disturbance(EID)-based load frequency control(LFC)strategy,which can effectively overcome the factors of random disturbance,model uncertainties and communication delay.First,an equivalent mathematical LFC model of an interconnected system is constructed.Then,the proposed robust controllers,based on the idea of EID,are designed to suppress the randomness and volatility of the renewable energy grid connection and coordinate the frequency fluctuation of the interconnected power system.Finally,the validity and superiority of the established topology structure and the superiority of the proposed strategy are demonstrated by dynamic time domain response experiments under the condition of high penetration of renewable energy.