Enhanced Deep Reinforcement Learning Strategy for Energy Management in Plug-in Hybrid Electric Vehicles with Entropy Regularization and Prioritized Experience Replay

Plug-in Hybrid Electric Vehicles(PHEVs)represent an innovative breed of transportation,harnessing diverse power sources for enhanced performance.Energy management strategies(EMSs)that coordinate and control different energy sources is a critical component of PHEV control technology,directly impacting overall vehicle performance.This study proposes an improved deep reinforcement learning(DRL)-based EMSthat optimizes realtime energy allocation and coordinates the operation of multiple power sources.Conventional DRL algorithms struggle to effectively explore all possible state-action combinations within high-dimensional state and action spaces.They often fail to strike an optimal balance between exploration and exploitation,and their assumption of a static environment limits their ability to adapt to changing conditions.Moreover,these algorithms suffer from low sample efficiency.Collectively,these factors contribute to convergence difficulties,low learning efficiency,and instability.To address these challenges,the Deep Deterministic Policy Gradient(DDPG)algorithm is enhanced using entropy regularization and a summation tree-based Prioritized Experience Replay(PER)method,aiming to improve exploration performance and learning efficiency from experience samples.Additionally,the correspondingMarkovDecision Process(MDP)is established.Finally,an EMSbased on the improvedDRLmodel is presented.Comparative simulation experiments are conducted against rule-based,optimization-based,andDRL-based EMSs.The proposed strategy exhibitsminimal deviation fromthe optimal solution obtained by the dynamic programming(DP)strategy that requires global information.In the typical driving scenarios based onWorld Light Vehicle Test Cycle(WLTC)and New European Driving Cycle(NEDC),the proposed method achieved a fuel consumption of 2698.65 g and an Equivalent Fuel Consumption(EFC)of 2696.77 g.Compared to the DP strategy baseline,the proposed method improved the fuel efficiency variances(FEV)by 18.13%,15.1%,and 8.37%over the Deep QNetwork(DQN),Double DRL(DDRL),and original DDPG methods,respectively.The observational outcomes demonstrate that the proposed EMS based on improved DRL framework possesses good real-time performance,stability,and reliability,effectively optimizing vehicle economy and fuel consumption.

The Future Trend of E-Mobility in Terms of Battery Electric Vehicles and Their Impact on Climate Change: A Case Study Applied in Hungary

The transportation sector is responsible for 25% of the total Carbon dioxide (CO2) emissions, whereas 60.6% of this sector represents small and medium passenger cars. However, as noted by the European Union Long-term strategy, there are two ways to reduce the amount of CO2 emissions in the transportation sector. The first way is characterized by creating more efficient vehicles. In contrast, the second way is characterized by changing the fuel used. The current study addressed the second way, changing the fuel type. The study examined the potential of battery electric vehicles (BEVs) as an alternative fuel type to reduce CO2 emissions in Hungarys transportation sector. The study used secondary data retrieved from Statista and stata.com to analyze the future trends of BEVs in Hungary. The results showed that the percentage of BEVs in Hungary in 2022 was 0.4% compared to the total number of registered passenger cars, which is 3.8 million. The simple exponential smoothing (SES) time series forecast revealed that the number of BEVs is expected to reach 84,192 in 2030, indicating a percentage increase of 2.21% in the next eight years. The study suggests that increasing the number of BEVs is necessary to address the negative impact of CO2 emissions on society. The Hungarian Ministry of Innovation and Technologys strategy to reduce the cost of BEVs may increase the percentage of BEVs by 10%, resulting in a potential average reduction of 76,957,600 g/km of CO2 compared to gasoline, diesel, hybrid electric vehicles (HEVs), and plug-in hybrid vehicles (PHEVs).

An optimal energy management development for various configuration of plug-in and hybrid electric vehicle

Due to soaring fuel prices and environmental concerns, hybrid electric vehicle(HEV) technology attracts more attentions in last decade. Energy management system, configuration of HEV and traffic conditions are the main factors which affect HEV's fuel consumption, emission and performance. Therefore, optimal management of the energy components is a key element for the success of a HEV. An optimal energy management system is developed for HEV based on genetic algorithm. Then, different powertrain system component combinations effects are investigated in various driving cycles. HEV simulation results are compared for default rule-based, fuzzy and GA-fuzzy controllers by using ADVISOR. The results indicate the effectiveness of proposed optimal controller over real world driving cycles. Also, an optimal powertrain configuration to improve fuel consumption and emission efficiency is proposed for each driving condition. Finally, the effects of batteries in initial state of charge and hybridization factor are investigated on HEV performance to evaluate fuel consumption and emissions. Fuel consumption average reduction of about 14% is obtained for optimal configuration data in contrast to default configuration. Also results indicate that proposed controller has reduced emission of about 10% in various traffic conditions.

ENERGY MANAGEMENT STRATEGY FOR PARALLEL HYBRID ELECTRIC VEHICLES

Energy management strategy (EMS) is the core of the real-time controlalgorithm of the hybrid electric vehicle (HEV). A novel EMS using the logic threshold approach withincorporation of a stand-by optimization algorithm is proposed. The aim of it is to minimize theengine fuel consumption and maintain the battery state of charge (SOC) in its operation range, whilesatisfying the vehicle performance and drivability requirements. The hybrid powertrain bench testis carried out to collect data of the engine, motor and battery pack, which are used in the EMS tocontrol the powertrain. Computer simulation model of the HEV is established in the MATLAB/Simulinkenvironment according to the bench test results. Simulation results are presented for behaviors ofthe engine, motor and battery. The proposed EMS is implemented for a real parallel hybrid carcontrol system and validated by vehicle field tests.

Energy Control of Plug-In Hybrid Electric Vehicles Using Model Predictive Control With Route Preview

The paper proposes an adoption of slope,elevation,speed and route distance preview to achieve optimal energymanagement of plug-in hybrid electric vehicles(PHEVs).Theapproach is to identify route features from historical and real-time traffic data,in which information fusion model and trafficprediction model are used to improve the information accuracy.Then,dynamic programming combined with equivalent con-sumption minimization strategy is used to compute an optimalsolution for real-time energy management.The solution is thereference for PHEV energy management control along the route.To improve the system's ability of handling changing situation,the study further explores predictive control model in the real-time control of the energy.A simulation is performed to modelPHEV under above energy control strategy with route preview.The results show that the average fuel consumption of PHEValong the previewed route with model predictive control(MPC)strategy can be reduced compared with optimal strategy andbase control strategy.

Identification of dynamic model parameters for lithium-ion batteries used in hybrid electric vehicles

An electrical equivalent circuit model for lithium-ion batteries used for hybrid electric vehicles （HEV） is presented. The model has two RC networks characterizing battery activation and concentration polarization process. The parameters of the model are identified using combined experimental and extended Kalman filter （EKF） recursive methods. The open-circuit voltage and ohmic resistance of the battery are directly measured and calculated from experimental measurements, respectively. The rest of the coupled dynamic parameters, i.e. the RC network parameters, are estimated using the EKF method. Experimental and simulation results are presented to demonstrate the efficacy of the proposed circuit model and parameter identification techniques for simulating battery dynamics.

Simulation research of energy management strategy for dual mode plug-in hybrid electrical vehicles

In this paper, a plug-in hybrid electrical vehicle(PHEV) is taken as the research object, and its dynamic performance and economic performance are taken as the research goals. Battery charge-sustaining(CS) period is divided into power mode and economy mode. Energy management strategy designing methods of power mode and economy mode are proposed. Maximum velocity, acceleration performance and fuel consumption are simulated during the CS period in the AVL CRUISE simulation environment. The simulation results indicate that the maximum velocity and acceleration time of the power mode are better than those in the economy mode. Fuel consumption of the economy mode is better than that in the power mode. Fuel consumption of PHEV during the CS period is further improved by using the methods proposed in this paper, and this is meaningful for research and development of PHEV.

Reinforcement Learning-Based Electric Vehicles Energy Management Strategy with Battery Thermal Model

The promotion of electric vehicles(EVs)is restricted due to their short cruising range.It is desirable to design an effective energy management strategy to improve their energy efficiency.Most existing work concerning energy management strategies focused on hybrids rather than the EVs.The work focusing on the energy management strategy for EVs mainly uses the traditional optimization strategies,thereby limiting the advantages of energy economy.To this end,a novel energy management strategy that considered the impact of battery thermal effects was proposed with the help of reinforcement learning.The main idea was to first analyze the energy flow path of EVs,further formulize the energy management as an optimization problem,and finally propose an online strategy based on reinforcement learning to obtain the optimal strategy.Additionally,extensive simulation results have demonstrated that our strategy reduces energy consumption by at least 27.4%compared to the existing methods.

Analysis of Hybrid Rechargeable Energy Storage Systems in Series Plug-In Hybrid Electric Vehicles Based on Simulations

In this paper, an extended analysis of the performance of different hybrid Rechargeable Energy Storage Systems (RESS) for use in Plug-in Hybrid Electric Vehicle (PHEV) with a series drivetrain topology is analyzed, based on simulations with three different driving cycles. The investigated hybrid energy storage topologies are an energy optimized lithium-ion battery (HE) in combination with an Electrical Double-Layer Capacitor (EDLC) system, in combination with a power optimized lithium-ion battery (HP) system or in combination with a Lithium-ion Capacitor (LiCap) system, that act as a Peak Power System. From the simulation results it was observed that hybridization of the HE lithium-ion based energy storage system resulted from the three topologies in an increased overall energy efficiency of the RESS, in an extended all electric range of the PHEV and in a reduced average current through the HE battery. The lowest consumption during the three driving cycles was obtained for the HE-LiCap topology, where fuel savings of respectively 6.0%, 10.3% and 6.8% compared with the battery stand-alone system were achieved. The largest extension of the range was achieved for the HE-HP configuration (17% based on FTP-75 driving cycle). HP batteries however have a large internal resistance in comparison to EDLC and LiCap systems, which resulted in a reduced overall energy efficiency of the hybrid RESS. Additionally, it was observed that the HP and LiCap systems both offer significant benefits for the integration of a peak power system in the drivetrain of a Plug-in Hybrid Electric Vehicle due to their low volume and weight in comparison to that of the EDLC system.

Towards Realistic Vibration Testing of Large Floor Batteries for Battery Electric Vehicles (BEV)

This contribution shows an analysis of vibration measurement on large floor-mounted traction batteries of Battery Electric Vehicles(BEV).The focus lies on the requirements for a realistic replication of the mechanical environments in a testing laboratory.Especially the analysis on global bending transfer functions and local corner bending coherence indicate that neither a fully stiff fixation of the battery nor a completely independent movement on the four corners yields a realistic and conservative test scenario.The contribution will further show what implication these findings have on future vibration&shock testing equipment for large traction batteries.Additionally,it will cover an outlook on how vibration behavior of highly integrated approaches(cell2car)changes the mechanical loads on the cells.

A Study of Advanced Efficient Hybrid Electric Vehicles, Electric Propulsion and Energy Source

Recently global warming and the depletion of fuel resources have accelerated researchers’ efforts to produce more efficient and clean alternatives. This research presents a comprehensive review of the different adjustments/con figurations of electric vehicles (EVs) and hybrid electric vehicles (HEVs), traction motors for power systems, and wireless speed control of traction drive. Electronic installation of technology can reduce pollution efficiently and effectively. The efficient operation has always been one of the most common investigators’ objectives in the automotive industry and academic areas. There are several renewable energy resources for hybrid vehicles that will replace depleted gasoline worldwide. The purpose of this paper is the development of more efficient pure EVs, HEVs, and fuel cell electric vehicles (FCEV) present both a challenge and a definite solution to current mobility issues. Fuel consumption in cars is a concern due to the harmful effects on the environment. Among other battery sources, fuel cells (FC), super capacitors (SC), and photovoltaic cells are studied for vehicle application. A combination of these renewable energy sources can be used for hybrid electric vehicles (HEV) in the next generation of transportation. With the significant progress of automobile technology, the hybrid electric vehicle has already become the main achievement of transportation electrification due to its excellent fuel-saving performance.

Online Learning Control for Hybrid Electric Vehicle

Improvements in hybrid electric vehicle (HEV) fuel economy and emissions heavily depend on an efficient energy management strategy (EMS). However, the uncertainty of future driving conditions generally cannot be easily tackled in EMS design. Most existing EMSs act upon fixed parameters and cannot adapt to varying driving conditions. Therefore, they usually fail to fully explore the potential of these advanced vehicles. In this paper, a novel EMS design procedure based on neural dynamic programming (NDP) is proposed. The NDP is a generic online learning algorithm, which combines stochastic dynamic programming (SDP) and the temporal difference (TD) method. Instead of computing the utility function and optimal control actions through Bellman equations, the NDP algorithm uses two neural networks to approximate them. The weights of these neural networks are updated online by the TD method. It avoids the high computational cost that SDP suffers from and is suitable for real-time implementation. The main advantages of NDP EMS is that it does not rely on prior information related to future driving conditions, and can self-tune with a wide variance in operating conditions. The NDP EMS has been applied to “Qianghua-I”, a prototype of a parallel HEV, using a revolving drum test bench for verification. Experiment results illustrate the potential of the proposed EMS in terms of fuel economy and in keeping state of charge (SOC) deviations at a low level. The proposed research ensures the optimality of NDP EMS, as well as real-time applicability.

A comprehensive review on hybrid electric vehicles: architectures and components

The rapid consumption of fossil fuel and increased environmental damage caused by it have given a strong impetus to the growth and development of fuelefficient vehicles. Hybrid electric vehicles (HEVs) have evolved from their inchoate state and are proving to be a promising solution to the serious existential problem posed to the planet earth. Not only do HEVs provide better fuel economy and lower emissions satisfying environmental legislations, but also they dampen the effect of rising fuel prices on consumers. HEVs combine the drive powers of an internal combustion engine and an electrical machine. The main components of HEVs are energy storage system, motor, bidirectional converter and maximum power point trackers (MPPT, in case of solar-powered HEVs). The performance of HEVs greatly depends on these components and its architecture. This paper presents an extensive review on essential components used in HEVs such as their architectures with advantages and disadvantages, choice of bidirectional converter to obtain high efficiency, combining ultracapacitor with battery to extend the battery life, traction motors’ role and their suitability for a particular application. Inclusion of photovoltaic cell in HEVs is a fairly new concept and has been discussed in detail. Various MPPT techniques used for solar-driven HEVs are also discussed in this paper with their suitability.

Realization and Analysis of Good Fuel Economy and Kinetic Performance of a Low-cost Hybrid Electric Vehicle

By using high-power and high-efficiency propulsion systems,current hybrid electric vehicles（HEVs） in market can achieve excellent fuel economy and kinetic performance.However,it is the cost of current HEVs that hinders HEVs coming into widespread use.A novel hybrid electric propulsion system is designed to balance HEV cost and performance for developing markets.A battery/supercapacitor-based hybrid energy storage system（HESS） is used to improve energy conversion efficiency and reduce battery size and cost.An all-in-one-controller（AIOC） which integrates engine electronic control unit（ECU）,motor ECU,and HESS management system is developed to save materials and energy,and reduce the influence of distribution parameters on circuit.As for the powertrain configuration,four schemes are presented：belt-driven starter generator（BSG） scheme,four-wheel drive HEV scheme,full HEV scheme,and ranger-extender electric vehicle（EV） scheme.Component selection and parameter matching for the propulsion system are performed,and an energy management strategy is developed based on powertrain configuration and selected components.Forward-facing simulation models are built,comprehending the control strategy based on the optimal engine torque for the low-cost hybrid electric propulsion system.Co-simulation of AVL CRUISE and Matlab/Simulink is presented and the best scheme is selected.The simulation results indicate that,for the best design,fuel consumption in urban driving condition is 4.11 L/（100 km） and 0-50 km/h accelerating time is 10.95 s.The proposed research can realize low-cost concept for HEV while achieving satisfactory fuel economy and kinetic performance,and help to improve commercialization of HEVs.

STEADY-STATE AND IDLE OPTIMIZA-TION OF INTERNAL COMBUSTION ENGINE CONTROL STRATEGIES FOR HYBRID ELECTRIC VEHICLES

A novel steady-state optimization （SSO） of internal combustion engine （ICE） strategy is proposed to maximize the efficiency of the overall powertrain for hybrid electric vehicles, in which the ICE efficiency, the efficiencies of the electric motor （EM） and the energy storage device are all explicitly taken into account. In addition, a novel idle optimization of ICE strategy is implemented to obtain the optimal idle operating point of the ICE and corresponding optimal parking generation power of the EM using the view of the novel SSO of ICE strategy. Simulations results show that potential fuel economy improvement is achieved relative to the conventional one which only optimized the ICE efficiency by the novel SSO of ICE strategy, and fuel consumption per voltage increment decreases a lot during the parking charge by the novel idle optimization of ICE strategy.

Cloud Computing Based Optimal Driving for a Parallel Hybrid Electric Vehicle

A cloud computing based optimal driving method is proposed and its feasibility is validated through a real-world scenario simulation.Based on principles of vehicle dynamics,the driving optimization problem has been formulated into an optimal control problem constrained by traffic rules,directed at achieving lower equivalent fuel consumption and shorter travel time.In order to conveniently specify the constraints and facilitate the application of the dynamic programming(DP)algorithm,the driving optimization problem is transformed into spatial domain and discretized properly.Considering the heavy computational costs of the DP algorithm,a cloud computing based platform structure is proposed to solve the optimal driving problem in real-time.A case study is simulated based on a real-world traffic scenario in Matlab.Simulation results demonstrate that the cloud computing framework is promising toward realizing the real-time energy management for hybrid electric vehicles.

A comparative study of hybrid electric vehicle fuel consumption over diverse driving cycles

Environmental pollution and declining resources of fossil fuels in recent years,have increased demand for better fuel economy and less pollution for ground transportation.Among the alternative solutions provided by researchers in recent decades,hybrid electric vehicles consisted of an internal combustion engine and an electric motor have been considered as a promising solution in the short-term.In the present study,fuel economy characteristics of a parallel hybrid electric vehicle are investigated by using numerical simulation.The simulation methodology is based on a fast forward facing simulation model of a parallel hybrid and an internal combustion engine powertrains.The objective of this study is to present the main parameters which result in an optimum combination of hybrid powertrain components in order to obtain a better fuel economy of hybrid powertrains regarding different driven cycles and hybridization factors.Then,the fuel consumption of the parallel hybrid electric vehicles are compared considering various driven cycles and hybridization factors.The results showed that the better fuel economy of hybrid powertrains increases by decreasing average load of the test cycle and the point of the best fuel economy for a particular average load of the cycle moves towards higher hybridization factors when the average load of the test cycle is reduced.

Multi-objective comprehensive optimization of fuel consumption and emission for hybrid electric vehicles

Aiming to reduce fuel consumption and emissions of a dual-clutch hybrid electric vehicle during cold start, multiobjective optimization for fuel consumption and HC/CO emission from a TWC(three-way catalytic converter) outlet is presented in this paper. DP(dynamic programming) considering dual-state variables is proposed based on the Bellman optimality principle. Both the battery SOC(state of charge) and the temperature of TWC monolith are considered in the algorithm simultaneously. In this way the global optimal control strategy and the Pareto optimal solution of multi-objective function are derived. Simulation results show that the proposed method is able to promote the TWC light-off significantly by decreasing the engine's load and improving exhaust temperature from the outlet of the engine, in comparison with original DP considering the single battery SOC. Compared to the results achieved by rule-based control strategy, fuel economy and emission of TWC outlet for cold start are optimized comprehensively. Each indicator of Pareto solution set shows the significant improvement.

Design and Implementation of a Battery Big Data Platform Through Intelligent Connected Electric Vehicles

The development of a battery management algorithm is highly dependent on high-quality battery operation data,especially the data in extreme conditions such as low temperatures.The data in faults are also essential for failure and safety management research.This study developed a battery big data platform to realize vehicle operation,energy interaction and data management.First,we developed an electric vehicle with vehicle navigation and position detection and designed an environmental cabin that allows the vehicle to operate autonomously.Second,charging and heating systems based on wireless energy transfer were developed and equipped on the vehicle to investigate optimal charging and heating methods of the batteries in the vehicle.Third,the data transmission network was designed,a real-time monitoring interface was developed,and the self-developed battery management system was used to measure,collect,upload,and store battery operation data in real time.Finally,experimental validation was performed on the platform.Results demonstrate the efficiency and reliability of the platform.Battery state of charge estimation is used as an example to illustrate the availability of battery operation data.

A Novel Method for the Application of the ECMS(Equivalent Consumption Minimization Strategy)to Reduce Hydrogen Consumption in Fuel Cell Hybrid Electric Vehicles

Fuel cell hybrid electric vehicles are currently being considered as ideal means to solve the energy crisis and global warming in today’s society.In this context,this paper proposes a method to solve the problem related to the dependence of the so-called optimal equivalent factor(determined in the framework of the equivalent consumption minimum strategy-ECMS)on the working conditions.The simulation results show that under typical conditions(some representative cities being considered),the proposed strategy can maintain the power balance;for different initial battery’s states of charge(SOC),after the SOC stabilizes,the fuel consumption is 5.25 L/100 km.