Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

High Energy Efficiency Dynamic Connected Hybrid Precoding for mm Wave Massive MIMO Systems

This paper considers a high energy efficiency dynamic connected(HEDC)structure,which promotes the practicability and reduces the power consumption of hybrid precoding system by lowresolution phase shifters(PSs).Based on the proposed structure,a new hybrid precoding algorithm is presented to optimize the energy efficiency,namely,HP-HEDC algorithm.Firstly,via a new defined effective optimal precoding matrix,the problem of optimizing the analog switch precoding matrix is formulated as a sparse representation problem.Thus,the optimal analog switch precoding matrix can be readily obtained by the branch-and-bound method.Then,the digital precoding matrix optimization problem is modeled as a dictionary update problem and solved by the method of optimal direction(MOD).Finally,the diagonal entries of the analog PS precoding matrix are optimized by exhaustive search independently since PS and antenna is one-to-one.Simulation results show that the HEDC structure enjoys low power consumption and satisfactory spectral efficiency.The proposed algorithm presents at least 50%energy efficiency improvement compared with other algorithms when the PS resolution is set as 3-bit.

Optimal Multi-Timescale Scheduling of Integrated Energy Systems with Hybrid Energy Storage System Based on Lyapunov Optimization

The economic operation of integrated energy system(IES)faces new challenges such as multi-timescale characteristics of heterogeneous energy sources,and cooperative operation of hybrid energy storage system(HESS).To this end,this paper investigates the multi-timescale rolling opti-mization problem for IES integrated with HESS.Firstly,the architecture of IES with HESS is established,a comparative analysis is conducted to evaluate the advantages of the HESS over a single energy storage system(SESS)in stabilizing power fluctuations.Secondly,the dayahead and real-time scheduling cost functions of IES are established,the day-ahead scheduling mainly depends on operation costs of the components in IES,the real-time optimal scheduling adopts the Lya-punov optimization method to schedule the battery and hydrogen energy storage in each time slot,so as to minimize the real-time average scheduling operation cost,and the problem of day-ahead and real-time scheduling error,which caused by the uncertainty of the energy storage is solved by online optimization.Finally,the proposed model is verified to reduce the scheduling operation cost and the dispatching error by performing an arithmetic example analysis of the IES in Shanghai,which provides a reference for the safe and stable operation of the IES.

Energy-Efficient Traffic Offloading for RSMA-Based Hybrid Satellite Terrestrial Networks with Deep Reinforcement Learning

As the demands of massive connections and vast coverage rapidly grow in the next wireless communication networks, rate splitting multiple access(RSMA) is considered to be the new promising access scheme since it can provide higher efficiency with limited spectrum resources. In this paper, combining spectrum splitting with rate splitting, we propose to allocate resources with traffic offloading in hybrid satellite terrestrial networks. A novel deep reinforcement learning method is adopted to solve this challenging non-convex problem. However, the neverending learning process could prohibit its practical implementation. Therefore, we introduce the switch mechanism to avoid unnecessary learning. Additionally, the QoS constraint in the scheme can rule out unsuccessful transmission. The simulation results validates the energy efficiency performance and the convergence speed of the proposed algorithm.

Enhanced Deep Reinforcement Learning Strategy for Energy Management in Plug-in Hybrid Electric Vehicles with Entropy Regularization and Prioritized Experience Replay

Plug-in Hybrid Electric Vehicles(PHEVs)represent an innovative breed of transportation,harnessing diverse power sources for enhanced performance.Energy management strategies(EMSs)that coordinate and control different energy sources is a critical component of PHEV control technology,directly impacting overall vehicle performance.This study proposes an improved deep reinforcement learning(DRL)-based EMSthat optimizes realtime energy allocation and coordinates the operation of multiple power sources.Conventional DRL algorithms struggle to effectively explore all possible state-action combinations within high-dimensional state and action spaces.They often fail to strike an optimal balance between exploration and exploitation,and their assumption of a static environment limits their ability to adapt to changing conditions.Moreover,these algorithms suffer from low sample efficiency.Collectively,these factors contribute to convergence difficulties,low learning efficiency,and instability.To address these challenges,the Deep Deterministic Policy Gradient(DDPG)algorithm is enhanced using entropy regularization and a summation tree-based Prioritized Experience Replay(PER)method,aiming to improve exploration performance and learning efficiency from experience samples.Additionally,the correspondingMarkovDecision Process(MDP)is established.Finally,an EMSbased on the improvedDRLmodel is presented.Comparative simulation experiments are conducted against rule-based,optimization-based,andDRL-based EMSs.The proposed strategy exhibitsminimal deviation fromthe optimal solution obtained by the dynamic programming(DP)strategy that requires global information.In the typical driving scenarios based onWorld Light Vehicle Test Cycle(WLTC)and New European Driving Cycle(NEDC),the proposed method achieved a fuel consumption of 2698.65 g and an Equivalent Fuel Consumption(EFC)of 2696.77 g.Compared to the DP strategy baseline,the proposed method improved the fuel efficiency variances(FEV)by 18.13%,15.1%,and 8.37%over the Deep QNetwork(DQN),Double DRL(DDRL),and original DDPG methods,respectively.The observational outcomes demonstrate that the proposed EMS based on improved DRL framework possesses good real-time performance,stability,and reliability,effectively optimizing vehicle economy and fuel consumption.

Numerical Modeling and Technico-Economic Analysis of a Hybrid Energy Production System for Self-Consumption: Case of Rural Area in the Comoros

This study aims to provide electricity to a remote village in the Union of Comoros that has been affected by energy problems for over 40 years. The study uses a 50 kW diesel generator, a 10 kW wind turbine, 1500 kW photovoltaic solar panels, a converter, and storage batteries as the proposed sources. The main objective of this study is to conduct a detailed analysis and optimization of a hybrid diesel and renewable energy system to meet the electricity demand of a remote area village of 800 to 1500 inhabitants located in the north of Ngazidja Island in Comoros. The study uses the Hybrid Optimization Model for Electric Renewable (HOMER) Pro to conduct simulations and optimize the analysis using meteorological data from Comoros. The results show that hybrid combination is more profitable in terms of margin on economic cost with a less expensive investment. With a diesel cost of $1/L, an average wind speed of 5.09 m/s and a solar irradiation value of 6.14 kWh/m2/day, the system works well with a proportion of renewable energy production of 99.44% with an emission quantity of 1311.407 kg/year. 99.2% of the production comes from renewable sources with an estimated energy surplus of 2,125,344 kWh/year with the cost of electricity (COE) estimated at $0.18/kWh, presenting a cost-effective alternative compared to current market rates. These results present better optimization of the used hybrid energy system, satisfying energy demand and reducing the environmental impact.

Energy saving analysis of a hybrid ground-coupled heat pump project

A hybrid ground-coupled heat pump（HGCHP）project in Nanjing,China is chosen to analyze the building energy-consumption properties in terms of different control strategies,building envelope and the terminal air-conditioning system.The HGCHP uses a supplemental heat rejecter to dissipate extra thermal energy to guarantee underground soil heat balance.The software EnergyPlus is employed to simulate the project and design the heat flow of the cooling tower and the borehole heat exchanger（BHE）.Then two feasible control strategies for the cooling tower and the borehole heat exchanger are proposed.The energy-saving potential of the building envelope is analyzed in terms of the surface color of the wall/roof.With the same terminal system,it is found that in the cooling season the heat flow of the insulated building with black wall/roof is 1.2 times more than that with white wall/roof.With the same insulated building and gray wall/roof,it is concluded that the heat pump units for a primary air fan-coil system show an annual energy consumption increase of 44.7 GJ compared with a radiant floor system.

Effects of Light Stress on Oxygen Evolution and Photochemical Energy Stor age of Hybrid Poplar Clones Determined by Photoacoustic Technique

The oxygen evolution, thermal dissipation, and photochemical energy storage of three hybrid poplar clones, namely the triploid clone B342, the diploid clone B11 [(Populus alba×P. glandulosa)×(P.tomentosa×P.bolleana)], and the triploid clone B346 [(P.tomentosa×P. bolleana)×(P. alba×P.glandulosa)], under light stress were studied using photoacoustics. The oxygen evolution signal and photochemical energy storage varied negatively with the pretreatment_PFD (photon flux density), whereas the thermal signal varied positively with the pretreatment_PFD. Photochemical energy storage was reallocated to PSⅡ more than to PSⅠ, while the photochemical energy storage in PSⅠ was more stable than that in PSⅡ when subjected to light stress. The inhibitors streptomycin (SM), dithiothreitol (DTT) and sodium fluoride (NaF) could all affect the oxygen evolution signal. Clones B11 and B342 were more resistant to light stress than clone B346.

Hydraulic Accumulator-Motor-Generator Energy Regeneration System for a Hybrid Hydraulic Excavator

Though the traditional energy regeneration system（ERS） which used a hydraulic motor and a generator in hybrid excavators can regenerate part of the energy, the power of the motor and the generator should be larger and the time for regenerating energy is so short. At first, the structure of new ERS that combines the advantages of an electric and hydraulic accumulator is analyzed. The energy can be converted into both the electric energy and the hydraulic energy at the lowering of the boom and the generator can still works when the boom stops going down. Then, a method how to set the working pressure of the hydraulic accumulator is proposed. To avoid the excess loss, extra noise and shock pressure, a two-level pressure threshold method that the generator starts to work at the rising edge of the high pressure threshold and stops working at the falling edge of the low pressure threshold is presented to characterize the working mode of the generator. The control strategies on how to control the boom velocity at the lowering of the boom and how to improve the recovery efficiency when the boom stops going down are presented. The test bench of hybrid excavator with ERS is constructed, with which the studies on the influences of ERS on energy conversion efficiency and control performance are carried out. Experimental results show that the proposed ERS features better speed control performance of the boom than traditional ERS. It is also observed that an estimated 45% of the total potential energy could be regenerated at the lowering of the boom in the proposed ERS, and the power level of the generator and the hydraulic motor could be reduced by 60%. Hence, the proposed ERS has obvious advantages over the traditional ERS on the improvement of energy regeneration time, energy efficiency, control performance and economy.

Power-balancing Instantaneous Optimization Energy Management for a Novel Series-parallel Hybrid Electric Bus

Energy management（EM） is a core technique of hybrid electric bus（HEB） in order to advance fuel economy performance optimization and is unique for the corresponding configuration. There are existing algorithms of control strategy seldom take battery power management into account with international combustion engine power management. In this paper, a type of power-balancing instantaneous optimization（PBIO） energy management control strategy is proposed for a novel series-parallel hybrid electric bus. According to the characteristic of the novel series-parallel architecture, the switching boundary condition between series and parallel mode as well as the control rules of the power-balancing strategy are developed. The equivalent fuel model of battery is implemented and combined with the fuel of engine to constitute the objective function which is to minimize the fuel consumption at each sampled time and to coordinate the power distribution in real-time between the engine and battery. To validate the proposed strategy effective and reasonable, a forward model is built based on Matlab/Simulink for the simulation and the dSPACE autobox is applied to act as a controller for hardware in-the-loop integrated with bench test. Both the results of simulation and hardware-in-the-loop demonstrate that the proposed strategy not only enable to sustain the battery SOC within its operational range and keep the engine operation point locating the peak efficiency region, but also the fuel economy of series-parallel hybrid electric bus（SPHEB） dramatically advanced up to 30.73% via comparing with the prototype bus and a similar improvement for PBIO strategy relative to rule-based strategy, the reduction of fuel consumption is up to 12.38%. The proposed research ensures the algorithm of PBIO is real-time applicability, improves the efficiency of SPHEB system, as well as suite to complicated configuration perfectly.

Hybrid reinforced thermoset polymer composite in energy absorption tube application:A review

The custom of hybridization fibre composite in energy absorption tube application has gained the attention of structural crashworthiness in composite material industry. Thus, the approach of this review is to understand the effect in hybridization within metal/synthetic fibre composite, synthetic/synthetic fibre composite and nature/synthetic fibre composite as energy absorption tube, which reflects on the energy absorption characteristics and crashworthiness behaviors in previous the study. By way of instance, a wide range of methodology and particular parameter in previous study such as the effect in fibre arrangement, matrix polymer, technique of fabrication, fibre treatment(natural fibre), design in geometry/cross-section and others mechanism of hybrid fibre composite tube are highlighted which to comprehend the capability of the mechanical performance and collapsible behavior as sacrificial structure in high-performance structure applications. Moreover, in the recently studies there have been many of the research regarding structural materials as energy absorption tube has been introduced such as metal/matrix composites, new alloy metals and polymer composites which intended to evaluate the performance of these materials into circumstance in loading and impact characteristic. Therefore, this review article is trying to explore the research articles related to the effect of hybridization fibres and thermoset polymer as reinforcement for energy absorption tube research and expected would provide an information and idea which to expend the knowledge in future study of hybridization effect for energy absorption tube, moreover the development for future potential as new hybrid composite fibre materials from the natural/synthetic fibres reinforced composite material in employing of high-performance energy absorption tube application is still less discover and highlighted.

An optimal energy management development for various configuration of plug-in and hybrid electric vehicle

Due to soaring fuel prices and environmental concerns, hybrid electric vehicle(HEV) technology attracts more attentions in last decade. Energy management system, configuration of HEV and traffic conditions are the main factors which affect HEV's fuel consumption, emission and performance. Therefore, optimal management of the energy components is a key element for the success of a HEV. An optimal energy management system is developed for HEV based on genetic algorithm. Then, different powertrain system component combinations effects are investigated in various driving cycles. HEV simulation results are compared for default rule-based, fuzzy and GA-fuzzy controllers by using ADVISOR. The results indicate the effectiveness of proposed optimal controller over real world driving cycles. Also, an optimal powertrain configuration to improve fuel consumption and emission efficiency is proposed for each driving condition. Finally, the effects of batteries in initial state of charge and hybridization factor are investigated on HEV performance to evaluate fuel consumption and emissions. Fuel consumption average reduction of about 14% is obtained for optimal configuration data in contrast to default configuration. Also results indicate that proposed controller has reduced emission of about 10% in various traffic conditions.

CGE model-based analysis of the neutralized hybrid carbon policy and its decomposed effects on economic growth,carbon reduction,and energy utilization costs

The hybrid policy is a flexible policy tool that combines features of carbon trading and carbon taxation.Its economic and environmental effects under China's background are still not studied in detail.Given the exogenous carbon reduction targets,carbon prices,and carbon tax-rates,by computable general equilibrium modeling methods and factor decomposition methods,this article investigates direct and cascaded effects of the hybrid policy on economic growth,energy utilization,and carbon emission on the national level and the sector level,with China's national input-output data-set.Stepwisely,policy scenarios with irrational estimated results are selectively excluded based on comprehensive evaluation among economic,carbon reduction and other policy targets.As a result,against national economic conditions in 2007,the hybrid policy,with a carbon reduction target of -10%,a carbon tax-rate of around $10,and a ceiling carbon price of $40,is highly recommended,because of its significant lower economic loss,lower energy utilization cost,and practical robustness against fluctuation of energy market and carbon market.Furthermore,by decomposition analysis,carbon reduction-related costs are decomposed into a direct part that includes carbon allowance price and carbon tax,and an indirect part as the energy price incremental induced by direct carbon costs.Gross carbon reduction may be decomposed into three parts such as energy intensity,economic scale,and technical progress.And,carbon taxation is the main policy tool that stimulates to improve the energy efficiency.

ENERGY MANAGEMENT STRATEGY FOR PARALLEL HYBRID ELECTRIC VEHICLES

Energy management strategy (EMS) is the core of the real-time controlalgorithm of the hybrid electric vehicle (HEV). A novel EMS using the logic threshold approach withincorporation of a stand-by optimization algorithm is proposed. The aim of it is to minimize theengine fuel consumption and maintain the battery state of charge (SOC) in its operation range, whilesatisfying the vehicle performance and drivability requirements. The hybrid powertrain bench testis carried out to collect data of the engine, motor and battery pack, which are used in the EMS tocontrol the powertrain. Computer simulation model of the HEV is established in the MATLAB/Simulinkenvironment according to the bench test results. Simulation results are presented for behaviors ofthe engine, motor and battery. The proposed EMS is implemented for a real parallel hybrid carcontrol system and validated by vehicle field tests.

Hybrid Triboelectric-Electromagnetic Nanogenerators for Mechanical Energy Harvesting:A Review

Motion-driven electromagnetictriboelectric energy generators(E-TENGs) hold a Hybrid triboelectric-electromagnetic generators C A Bgreat potential to provide higher voltages,higher currents and wider operating bandwidths than Cantilever generators1 μm both electromagnetic and triboelectric generators Rotating generatorsSliding generators standing alone. Therefore,they are promising solutions to autonomously supply a broad range of Flexible blade generators Magnetoelectric highly sophisticated devices. This paper provides generators a thorough review focused on major recent breakLinear generators Multidimensional generators throughs in the area of electromagnetic-triboelectric vibrational energy harvesting. A detailed Pendulum generators analysis was conducted on various architectures including rotational,pendulum,linear,sliding,cantilever,flexible blade,multidimensional and 1 cm magnetoelectric,and the following hybrid technologies. They enable highly e cient ways to harvest electric energy from many forms of vibrational,rotational,biomechanical,wave,wind and thermal sources,among others. Open-circuit voltages up to 75 V,short-circuit currents up to 60 mA and instantaneous power up to 144 mW were already achieved by these nanogenerators. Their transduction mechanisms,including proposed models to make intelligible the involved physical phenomena,are also overviewed here. A comprehensive analysis was performed to compare their respective construction designs,external excitations and electric outputs. The results highlight the potential of hybrid E-TENGs to convert unused mechanical motion into electric energy for both large-and small-scale applications.Finally,this paper proposes future research directions toward optimization of energy conversion e ciency,power management,durability and stability,packaging,energy storage,operation input,research of transduction mechanisms,quantitative standardization,system integration,miniaturization and multi-energy hybrid cells.

A Hybrid Biofuel and Triboelectric Nanogenerator for Bioenergy Harvesting

Various types of energy exist everywhere around us,and these energies can be harvested from multiple sources to power micro-/nanoelectronic system and even personal electronic products.In this work,we proposed a hybrid energy-harvesting system(HEHS)for potential in vivo applications.The HEHS consisted of a triboelectric nanogenerator and a glucose fuel cell for simultaneously harvesting biomechanical energy and biochemical energy in simulated body fluid.These two energy-harvesting units can work individually as a single power source or work simultaneously as an integrated system.This design strengthened the flexibility of harvesting multiple energies and enhanced corresponding electric output.Compared with any individual device,the integrated HEHS outputs a superimposed current and has a faster charging rate.Using the harvested energy,HEHS can power a calculator or a green light-emitting diode pattern.Considering the widely existed biomechanical energy and glucose molecules in the body,the developed HEHS can be a promising candidate for building in vivo self-powered healthcare monitoring system.

Influence of Different Degrees of Complementarity of Solar and Hydro Energy Availability on the Performance of Hybrid Hydro PV Generating Plants

The complementarity of energy resources used in hybrid power generation can result in optimization of power capacity and reservation capabilities. This article is dedicated to the study of hybrid hydro PV systems. The goal is to establish the relationship between system performance and complementarity of energy resources. The study was carried out with computer simulations based on a method that uses ideal functions developed to describe the energy resources and determines a limit of performance. The results confirm expectations that performance, as measured by the total time of failure to meet demand, will be better as energy resources are complementary. Charts relating energy complementarity with failures are presented. The subsequent research work shall proceed to at least two different phases. In the first one, the method exposed in the present work shall be applied to real data and compared to the operation of existing hybrid plants. In the second phase, results shall be confronted with design parameters of hydro PV plants based on complementary resources. A next stage would be the enlargement of the method applied in this work for systems based on other energy resources, such as wind energy and ocean wave energy.

A new hybrid energy absorption mechanism subjected to axial loading

This study aims to introduce a novel hybrid design with a combination of two more common mechanisms for improving the capacity of systems in absorbing the kinetic energy of moving vehicles or devices. This new model consists of two individual mechanisms, i.e., expansion of a circular tube accompanied by crushing of an inner tube, which dissipate the energy through friction, plastic deformations and failures of inner tube. This study comprises 24 case studies surveyed under two different design controls, constant mass and constant volume, for comparing purposes. Finite element simulations are utilized so as to investigate models’ deformations and to extract some crashworthiness parameters in aid of representing the efficiency of the mechanism as well as conducting a parametric study between three different profiles of inner tube. This study shows that models with inner circular and hexagonal tube profile absorb higher amount of energy due to experiencing three different modes of energy dissipation systems, including folding, shear and ductile damages.

DEVELOPMENT OF THE ENERGY MANAGEMENT STRATEGY FOR PARALLEL HYBRID ELECTRIC URBAN BUSES

A novel parallel hybrid electrical urban bus （PHEUB） configuration consisting of an extra one-way clutch and an automatic mechanical transmission （AMT） is taken as the study subject. An energy management strategy combining a logic threshold approach and an instantaneous optimization algorithm is proposed for the investigated PHEUB. The objective of the energy management strategy is to achieve acceptable vehicle performance and drivability requirements while simultaneously maximizing the engine fuel consumption and maintaining the battery state of charge in its operation range at all times. Under the environment of Matlab/Simulink, a computer simulation model for the PHEUB is constructed by using the model building method combining theoretical analysis and bench test data. Simulation and experiment results for China Typical Bus Driving Schedule at Urban District （CTBDS_UD） are obtained, and the results indicate that the proposed control strategy not only controls the hybrid system efficiently but also improves the fuel economy significantly.