Enhanced Deep Reinforcement Learning Strategy for Energy Management in Plug-in Hybrid Electric Vehicles with Entropy Regularization and Prioritized Experience Replay

Plug-in Hybrid Electric Vehicles(PHEVs)represent an innovative breed of transportation,harnessing diverse power sources for enhanced performance.Energy management strategies(EMSs)that coordinate and control different energy sources is a critical component of PHEV control technology,directly impacting overall vehicle performance.This study proposes an improved deep reinforcement learning(DRL)-based EMSthat optimizes realtime energy allocation and coordinates the operation of multiple power sources.Conventional DRL algorithms struggle to effectively explore all possible state-action combinations within high-dimensional state and action spaces.They often fail to strike an optimal balance between exploration and exploitation,and their assumption of a static environment limits their ability to adapt to changing conditions.Moreover,these algorithms suffer from low sample efficiency.Collectively,these factors contribute to convergence difficulties,low learning efficiency,and instability.To address these challenges,the Deep Deterministic Policy Gradient(DDPG)algorithm is enhanced using entropy regularization and a summation tree-based Prioritized Experience Replay(PER)method,aiming to improve exploration performance and learning efficiency from experience samples.Additionally,the correspondingMarkovDecision Process(MDP)is established.Finally,an EMSbased on the improvedDRLmodel is presented.Comparative simulation experiments are conducted against rule-based,optimization-based,andDRL-based EMSs.The proposed strategy exhibitsminimal deviation fromthe optimal solution obtained by the dynamic programming(DP)strategy that requires global information.In the typical driving scenarios based onWorld Light Vehicle Test Cycle(WLTC)and New European Driving Cycle(NEDC),the proposed method achieved a fuel consumption of 2698.65 g and an Equivalent Fuel Consumption(EFC)of 2696.77 g.Compared to the DP strategy baseline,the proposed method improved the fuel efficiency variances(FEV)by 18.13%,15.1%,and 8.37%over the Deep QNetwork(DQN),Double DRL(DDRL),and original DDPG methods,respectively.The observational outcomes demonstrate that the proposed EMS based on improved DRL framework possesses good real-time performance,stability,and reliability,effectively optimizing vehicle economy and fuel consumption.

Energy Control of Plug-In Hybrid Electric Vehicles Using Model Predictive Control With Route Preview

The paper proposes an adoption of slope,elevation,speed and route distance preview to achieve optimal energymanagement of plug-in hybrid electric vehicles(PHEVs).Theapproach is to identify route features from historical and real-time traffic data,in which information fusion model and trafficprediction model are used to improve the information accuracy.Then,dynamic programming combined with equivalent con-sumption minimization strategy is used to compute an optimalsolution for real-time energy management.The solution is thereference for PHEV energy management control along the route.To improve the system's ability of handling changing situation,the study further explores predictive control model in the real-time control of the energy.A simulation is performed to modelPHEV under above energy control strategy with route preview.The results show that the average fuel consumption of PHEValong the previewed route with model predictive control(MPC)strategy can be reduced compared with optimal strategy andbase control strategy.

Novel flexible hybrid electric system and adaptive online-optimal energy management controller for plug-in hybrid electric vehicles

In order to achieve the improvement of the driving comfort and energy efficiency,an new e-CVT flexible full hybrid electric system(E2FHS) is proposed,which uses an integrated main drive motor and generator to take the place of the original automatic or manual transmission to realize the functions of continuously variable transmission(e-CVT).The design and prototype realization of the E2FHS system for a plug-in hybrid vehicle(PHEV) is performed.In order to analyze and optimize the parameters and the power flux between different parts of the E2FHS,simulation software is developed.Especially,in order to optimize the performance of the energy economy improvement of the E2FHS,the effect of the different energy management controllers is investigated,and an adaptive online-optimal energy management controller for the E2FHS is built and validated by the prototype PHEV.

Analysis of Hybrid Rechargeable Energy Storage Systems in Series Plug-In Hybrid Electric Vehicles Based on Simulations

In this paper, an extended analysis of the performance of different hybrid Rechargeable Energy Storage Systems (RESS) for use in Plug-in Hybrid Electric Vehicle (PHEV) with a series drivetrain topology is analyzed, based on simulations with three different driving cycles. The investigated hybrid energy storage topologies are an energy optimized lithium-ion battery (HE) in combination with an Electrical Double-Layer Capacitor (EDLC) system, in combination with a power optimized lithium-ion battery (HP) system or in combination with a Lithium-ion Capacitor (LiCap) system, that act as a Peak Power System. From the simulation results it was observed that hybridization of the HE lithium-ion based energy storage system resulted from the three topologies in an increased overall energy efficiency of the RESS, in an extended all electric range of the PHEV and in a reduced average current through the HE battery. The lowest consumption during the three driving cycles was obtained for the HE-LiCap topology, where fuel savings of respectively 6.0%, 10.3% and 6.8% compared with the battery stand-alone system were achieved. The largest extension of the range was achieved for the HE-HP configuration (17% based on FTP-75 driving cycle). HP batteries however have a large internal resistance in comparison to EDLC and LiCap systems, which resulted in a reduced overall energy efficiency of the hybrid RESS. Additionally, it was observed that the HP and LiCap systems both offer significant benefits for the integration of a peak power system in the drivetrain of a Plug-in Hybrid Electric Vehicle due to their low volume and weight in comparison to that of the EDLC system.

Powertrain design and energy management of a novel coaxial series-parallel plug-in hybrid electric vehicle

For a series plug-in hybrid electric vehicle,higher working efficiency can be achieved by the drive system with two small motors in parallel than that with one big motor alone.However,the overly complex structure will inevitably lead to a substantial increase in the development cost.To improve the system price-performance ratio,a new kind of series-parallel hybrid system evolved from the series plug-in hybrid system is designed.According to the technical parameters of the selected components,the system model is established,and the vehicle dynamic property and pure electric drive economy are evaluated.Based on the dynamic programming,the energy management strategy for the drive system under the city driving cycle is developed,and the superiority validation of the system is completed.For the studied vehicle driven by the designed series-parallel plug-in hybrid system,compared with the one driven by the described series plug-in hybrid system,the dynamic property is significantly improved because of the multi-power coupling,and the fuel consumption is reduced by 11.4%with 10 city driving cycles.In a word,with the flexible configuration of the designed hybrid system and the optimized control strategy of the energy management,the vehicle performance can be obviously improved.

Microgrid economic operation considering plug-in hybrid electric vehicles integration

In this paper,the microgrid economic scheduling mathematical model considering the integration of plug-in hybrid electric vehicles(PHEVs)is presented and the influence of different charging and discharging modes on microgrid economic operation is analyzed.The generic algorithm is used to find an economically optimal solution for the microgrid and PHEV owners.The scheduling of PHEVs and the microgrid are optimized to reduce daily electricity cost and the potential benefits of controlled charging/discharging are explored systematically.Constraints caused by vehicle utilization as well as technical limitations of distributed generation and energy storage system are taken into account.The proposed economic scheduling is evaluated through a simulation by using a typical grid-connected microgrid model.

A multivariable output neural network approach for simulation of plug-in hybrid electric vehicle fuel consumption

This study is laser focused on the simulation of fuel consumption and fuel economy label parameters of plug-in hybrid electric vehicles.While fuel economy is a key factor in the design of plug-in hybrid electric vehicles,a fuel economy label can educate customers about the economic advantage of purchasing a particular car.The fuel economy label of a PHEV consists of parameters like driving range,electrical energy consumption,fuel economy for city,highway,and combined use,battery recharge time,and fuel consumption rates.The study used an inverse function model of an artificial neural network to simulate and calculate the parameters of the fuel economy labels of PHEVs.Firstly,the selected parameters of the fuel economy label of plug-in hybrid electric vehicles were used to develop a single output model.The output variable of the single output model was then merged with dummy functions to form input variables for the inverse function model.The output variables simulated were engine size in litres;estimated driving range when the battery is fully charged in km,battery recharged time in hours,city fuel consumption(L/100 km),highway fuel consumption(L/100 km),combined fuel consumption(L/100 km),estimated driving range when the tank is full,carbon dioxide(CO_(2))emission in grams/km,electric motor power in kW,number of cylinders,and electrical charges consumed in kWh/100 km.Different cases of input variables were considered for the inverse function model.The accuracy of the model was 29.1 times greater than that of the conventional inverse artificial neural network model.

Implementation of Fuzzy Logic Control into an Equivalent Minimization Strategy for Adaptive Energy Management of A Parallel Hybrid Electric Vehicle

As government agencies continue to tighten emissions regulations due to the continued increase in greenhouse gas production, automotive industries are seeking to produce increasingly efficient vehicle technology. Hybrid electric vehicles (HEVs) have been introduced to mitigate problems while improving fuel economy. HEVs have led to the demand of creating more advanced controls software to consider multiple components for propulsive power in a vehicle. A large section in the software development process is the implementation of an optimal energy management strategy meant to improve the overall fuel efficiency of the vehicle. Optimal strategies can be implemented when driving conditions are known a prior. The Equivalent Consumption Minimization Strategy (ECMS) is an optimal control strategy that uses an equivalence factor to equate electrical to mechanical power when performing torque split determination between the internal combustion engine and electric motor for propulsive and regenerative torque. This equivalence factor is determined from offline vehicle simulations using a sensitivity analysis to provide optimal fuel economy results while maintaining predetermined high voltage battery state of charge (SOC) constraints. When the control hierarchy is modified or different driving styles are applied, the analysis must be redone to update the equivalence factor. The goal of this work is to implement a fuzzy logic controller that dynamically updates the equivalence factor to improve fuel economy, maintain a strict charge sustaining window of operation for the high voltage battery, and reduce computational time required during algorithm development. The adaptive algorithm is validated against global optimum fuel economy and charge sustaining results from a sensitivity analysis performed for multiple drive cycles. Results show a maximum fuel economy improvement of 9.82% when using a mild driving style and a 95% success rate when maintaining an ending SOC within 5% of the desired SOC regardless of starting SOC.

An optimal energy management development for various configuration of plug-in and hybrid electric vehicle

Due to soaring fuel prices and environmental concerns, hybrid electric vehicle(HEV) technology attracts more attentions in last decade. Energy management system, configuration of HEV and traffic conditions are the main factors which affect HEV's fuel consumption, emission and performance. Therefore, optimal management of the energy components is a key element for the success of a HEV. An optimal energy management system is developed for HEV based on genetic algorithm. Then, different powertrain system component combinations effects are investigated in various driving cycles. HEV simulation results are compared for default rule-based, fuzzy and GA-fuzzy controllers by using ADVISOR. The results indicate the effectiveness of proposed optimal controller over real world driving cycles. Also, an optimal powertrain configuration to improve fuel consumption and emission efficiency is proposed for each driving condition. Finally, the effects of batteries in initial state of charge and hybridization factor are investigated on HEV performance to evaluate fuel consumption and emissions. Fuel consumption average reduction of about 14% is obtained for optimal configuration data in contrast to default configuration. Also results indicate that proposed controller has reduced emission of about 10% in various traffic conditions.

Simulation research of energy management strategy for dual mode plug-in hybrid electrical vehicles

In this paper, a plug-in hybrid electrical vehicle(PHEV) is taken as the research object, and its dynamic performance and economic performance are taken as the research goals. Battery charge-sustaining(CS) period is divided into power mode and economy mode. Energy management strategy designing methods of power mode and economy mode are proposed. Maximum velocity, acceleration performance and fuel consumption are simulated during the CS period in the AVL CRUISE simulation environment. The simulation results indicate that the maximum velocity and acceleration time of the power mode are better than those in the economy mode. Fuel consumption of the economy mode is better than that in the power mode. Fuel consumption of PHEV during the CS period is further improved by using the methods proposed in this paper, and this is meaningful for research and development of PHEV.

The Future Trend of E-Mobility in Terms of Battery Electric Vehicles and Their Impact on Climate Change: A Case Study Applied in Hungary

The transportation sector is responsible for 25% of the total Carbon dioxide (CO2) emissions, whereas 60.6% of this sector represents small and medium passenger cars. However, as noted by the European Union Long-term strategy, there are two ways to reduce the amount of CO2 emissions in the transportation sector. The first way is characterized by creating more efficient vehicles. In contrast, the second way is characterized by changing the fuel used. The current study addressed the second way, changing the fuel type. The study examined the potential of battery electric vehicles (BEVs) as an alternative fuel type to reduce CO2 emissions in Hungarys transportation sector. The study used secondary data retrieved from Statista and stata.com to analyze the future trends of BEVs in Hungary. The results showed that the percentage of BEVs in Hungary in 2022 was 0.4% compared to the total number of registered passenger cars, which is 3.8 million. The simple exponential smoothing (SES) time series forecast revealed that the number of BEVs is expected to reach 84,192 in 2030, indicating a percentage increase of 2.21% in the next eight years. The study suggests that increasing the number of BEVs is necessary to address the negative impact of CO2 emissions on society. The Hungarian Ministry of Innovation and Technologys strategy to reduce the cost of BEVs may increase the percentage of BEVs by 10%, resulting in a potential average reduction of 76,957,600 g/km of CO2 compared to gasoline, diesel, hybrid electric vehicles (HEVs), and plug-in hybrid vehicles (PHEVs).

Design and Analysis of Electro-mechanical Hybrid Anti-lock Braking System for Hybrid Electric Vehicle Utilizing Motor Regenerative Braking

Braking on low adhesion-coefficient roads, hybrid electric vehicle＇s motor regenerative torque is switched off to safeguard the normal anti-lock braking system （ABS） fimction. When the ABS control is terminated, the motor regenerative braking is readmitted. Aiming at avoiding permanent cycles from hydraulic anti-lock braking to motor regenerative braking, a novel electro-mechanical hybrid anti-lock braking system using fuzzy logic is designed. Different from the traditional single control structure, this system has a two-layered hierarchical structure, The first layer is responsible for harmonious adjustment or interaction between regenerative system and anti-lock braking system. The second layer is responsible for braking torque distribution and adjustment. The closed-loop simulation model is built. Control strategy and method for coordination between regenerative and anti-lock braking are developed. Simulation braking on low adhesion-coefficient roads with fuzzy logic control and real vehicle braking field test are presented. The results from simulating analysis and experiment show braking performance of the vehicle is perfect, harmonious coordination between regenerative and anti-lock braking function, significant amount of braking energy can be recovered and the proposed control strategy and method are effective.

Synthetical Efficiency-based Optimization for the Power Distribution of Power-split Hybrid Electric Vehicles

Now the optimization strategies for power distribution are researched widely, and most of them are aiming to the optimal fuel economy and the driving cycle must be preknown. Thus if the actual driving condition deviates from the scheduled driving cycle, the effect of optimal results will be declined greatly. Therefore, the instantaneous optimization strategy carried out on-line is studied in this paper. The power split path and the transmission efficiency are analyzed based on a special power-split scheme and the efficiency models of the power transmitting components are established. The synthetical efficiency optimization model is established for enhancing the transmission efficiency and the fuel economy. The identification of the synthetical efficiency as the optimization objective and the constrain group are discussed emphatically. The optimization is calculated by the adaptive simulated annealing （ASA） algorithm and realized on-line by the radial basis function （RBF）-based similar models. The optimization for power distribution of the hybrid vehicle in an actual driving condition is carried out and the road test results are presented. The test results indicate that the synthetical efficiency optimization method can enhance the transmission efficiency and the fuel economy of the power-split hybrid electric vehicle （HEV） observably. Compared to the rules-based strategy the optimization strategy is optimal and achieves the approximate global optimization solution for the power distribution. The synthetical efficiency optimization solved by ASA algorithm can give attentions to both optimization quality and calculation efficiency, thus it has good application foreground for the power distribution of power-split HEV.

OPTIMIZATION APPROACH FOR HYBRID ELECTRIC VEHICLE POWERTRAIN DESIGN

According to bench test results of fuel economy and engine emission for thereal power-train system of EQ7200HEV car. a 3-D performance map oriented quasi-linear model isdeveloped for the configuration of the powertrain components such as internal combustion engine,traction electric motor, transmission, main retarder and energy storage unit. A genetic algorithmbased on optimization procedure is proposed and applied for parametric optimization of the keycomponents by consideration of requirements of some driving cycles. Through comparison of numericalresults obtained by the genetic algorithm with those by traditional optimization methods, it isshown that the present approach is quite effective and efficient in emission reduction and fueleconomy for the design of the hybrid electric car powertrain.

Online Learning Control for Hybrid Electric Vehicle

Improvements in hybrid electric vehicle (HEV) fuel economy and emissions heavily depend on an efficient energy management strategy (EMS). However, the uncertainty of future driving conditions generally cannot be easily tackled in EMS design. Most existing EMSs act upon fixed parameters and cannot adapt to varying driving conditions. Therefore, they usually fail to fully explore the potential of these advanced vehicles. In this paper, a novel EMS design procedure based on neural dynamic programming (NDP) is proposed. The NDP is a generic online learning algorithm, which combines stochastic dynamic programming (SDP) and the temporal difference (TD) method. Instead of computing the utility function and optimal control actions through Bellman equations, the NDP algorithm uses two neural networks to approximate them. The weights of these neural networks are updated online by the TD method. It avoids the high computational cost that SDP suffers from and is suitable for real-time implementation. The main advantages of NDP EMS is that it does not rely on prior information related to future driving conditions, and can self-tune with a wide variance in operating conditions. The NDP EMS has been applied to “Qianghua-I”, a prototype of a parallel HEV, using a revolving drum test bench for verification. Experiment results illustrate the potential of the proposed EMS in terms of fuel economy and in keeping state of charge (SOC) deviations at a low level. The proposed research ensures the optimality of NDP EMS, as well as real-time applicability.

A comprehensive review on hybrid electric vehicles: architectures and components

The rapid consumption of fossil fuel and increased environmental damage caused by it have given a strong impetus to the growth and development of fuelefficient vehicles. Hybrid electric vehicles (HEVs) have evolved from their inchoate state and are proving to be a promising solution to the serious existential problem posed to the planet earth. Not only do HEVs provide better fuel economy and lower emissions satisfying environmental legislations, but also they dampen the effect of rising fuel prices on consumers. HEVs combine the drive powers of an internal combustion engine and an electrical machine. The main components of HEVs are energy storage system, motor, bidirectional converter and maximum power point trackers (MPPT, in case of solar-powered HEVs). The performance of HEVs greatly depends on these components and its architecture. This paper presents an extensive review on essential components used in HEVs such as their architectures with advantages and disadvantages, choice of bidirectional converter to obtain high efficiency, combining ultracapacitor with battery to extend the battery life, traction motors’ role and their suitability for a particular application. Inclusion of photovoltaic cell in HEVs is a fairly new concept and has been discussed in detail. Various MPPT techniques used for solar-driven HEVs are also discussed in this paper with their suitability.

Realization and Analysis of Good Fuel Economy and Kinetic Performance of a Low-cost Hybrid Electric Vehicle

By using high-power and high-efficiency propulsion systems,current hybrid electric vehicles（HEVs） in market can achieve excellent fuel economy and kinetic performance.However,it is the cost of current HEVs that hinders HEVs coming into widespread use.A novel hybrid electric propulsion system is designed to balance HEV cost and performance for developing markets.A battery/supercapacitor-based hybrid energy storage system（HESS） is used to improve energy conversion efficiency and reduce battery size and cost.An all-in-one-controller（AIOC） which integrates engine electronic control unit（ECU）,motor ECU,and HESS management system is developed to save materials and energy,and reduce the influence of distribution parameters on circuit.As for the powertrain configuration,four schemes are presented：belt-driven starter generator（BSG） scheme,four-wheel drive HEV scheme,full HEV scheme,and ranger-extender electric vehicle（EV） scheme.Component selection and parameter matching for the propulsion system are performed,and an energy management strategy is developed based on powertrain configuration and selected components.Forward-facing simulation models are built,comprehending the control strategy based on the optimal engine torque for the low-cost hybrid electric propulsion system.Co-simulation of AVL CRUISE and Matlab/Simulink is presented and the best scheme is selected.The simulation results indicate that,for the best design,fuel consumption in urban driving condition is 4.11 L/（100 km） and 0-50 km/h accelerating time is 10.95 s.The proposed research can realize low-cost concept for HEV while achieving satisfactory fuel economy and kinetic performance,and help to improve commercialization of HEVs.

STEADY-STATE AND IDLE OPTIMIZA-TION OF INTERNAL COMBUSTION ENGINE CONTROL STRATEGIES FOR HYBRID ELECTRIC VEHICLES

A novel steady-state optimization （SSO） of internal combustion engine （ICE） strategy is proposed to maximize the efficiency of the overall powertrain for hybrid electric vehicles, in which the ICE efficiency, the efficiencies of the electric motor （EM） and the energy storage device are all explicitly taken into account. In addition, a novel idle optimization of ICE strategy is implemented to obtain the optimal idle operating point of the ICE and corresponding optimal parking generation power of the EM using the view of the novel SSO of ICE strategy. Simulations results show that potential fuel economy improvement is achieved relative to the conventional one which only optimized the ICE efficiency by the novel SSO of ICE strategy, and fuel consumption per voltage increment decreases a lot during the parking charge by the novel idle optimization of ICE strategy.

ENERGY MANAGEMENT STRATEGY FOR PARALLEL HYBRID ELECTRIC VEHICLES

Energy management strategy (EMS) is the core of the real-time controlalgorithm of the hybrid electric vehicle (HEV). A novel EMS using the logic threshold approach withincorporation of a stand-by optimization algorithm is proposed. The aim of it is to minimize theengine fuel consumption and maintain the battery state of charge (SOC) in its operation range, whilesatisfying the vehicle performance and drivability requirements. The hybrid powertrain bench testis carried out to collect data of the engine, motor and battery pack, which are used in the EMS tocontrol the powertrain. Computer simulation model of the HEV is established in the MATLAB/Simulinkenvironment according to the bench test results. Simulation results are presented for behaviors ofthe engine, motor and battery. The proposed EMS is implemented for a real parallel hybrid carcontrol system and validated by vehicle field tests.

An Investigation into Regenerative Braking Control Strategy for Hybrid Electric Vehicle

Energy regeneration during braking is an important technique for hybrid electric vehicle (HEV) to improve their fuel economy and extend their driving range. Due to the effect of regenerative braking torque which is added by electric motor, the braking torque distribution between front and rear axles should be changed and the control logic of anti-lock braking system (ABS) ought to be adjusted according to the regenerative braking torque. This paper put forward a braking control strategy for hybrid electric vehicle; the control strategy is implemented with eight DOFs (Degree-of-Freedom) nonlinear vehicle forward simulation model which is built under the environment of Matlab/Simulink. Based on target wheel slip ratio, a fuzzy logic approach was applied to maintain the optimal target slip ratio so that best compromise between hydraulic torque and regenerative torque can be obtained for the vehicle.