Enhanced Deep Reinforcement Learning Strategy for Energy Management in Plug-in Hybrid Electric Vehicles with Entropy Regularization and Prioritized Experience Replay

Plug-in Hybrid Electric Vehicles(PHEVs)represent an innovative breed of transportation,harnessing diverse power sources for enhanced performance.Energy management strategies(EMSs)that coordinate and control different energy sources is a critical component of PHEV control technology,directly impacting overall vehicle performance.This study proposes an improved deep reinforcement learning(DRL)-based EMSthat optimizes realtime energy allocation and coordinates the operation of multiple power sources.Conventional DRL algorithms struggle to effectively explore all possible state-action combinations within high-dimensional state and action spaces.They often fail to strike an optimal balance between exploration and exploitation,and their assumption of a static environment limits their ability to adapt to changing conditions.Moreover,these algorithms suffer from low sample efficiency.Collectively,these factors contribute to convergence difficulties,low learning efficiency,and instability.To address these challenges,the Deep Deterministic Policy Gradient(DDPG)algorithm is enhanced using entropy regularization and a summation tree-based Prioritized Experience Replay(PER)method,aiming to improve exploration performance and learning efficiency from experience samples.Additionally,the correspondingMarkovDecision Process(MDP)is established.Finally,an EMSbased on the improvedDRLmodel is presented.Comparative simulation experiments are conducted against rule-based,optimization-based,andDRL-based EMSs.The proposed strategy exhibitsminimal deviation fromthe optimal solution obtained by the dynamic programming(DP)strategy that requires global information.In the typical driving scenarios based onWorld Light Vehicle Test Cycle(WLTC)and New European Driving Cycle(NEDC),the proposed method achieved a fuel consumption of 2698.65 g and an Equivalent Fuel Consumption(EFC)of 2696.77 g.Compared to the DP strategy baseline,the proposed method improved the fuel efficiency variances(FEV)by 18.13%,15.1%,and 8.37%over the Deep QNetwork(DQN),Double DRL(DDRL),and original DDPG methods,respectively.The observational outcomes demonstrate that the proposed EMS based on improved DRL framework possesses good real-time performance,stability,and reliability,effectively optimizing vehicle economy and fuel consumption.

Implementation of Fuzzy Logic Control into an Equivalent Minimization Strategy for Adaptive Energy Management of A Parallel Hybrid Electric Vehicle

As government agencies continue to tighten emissions regulations due to the continued increase in greenhouse gas production, automotive industries are seeking to produce increasingly efficient vehicle technology. Hybrid electric vehicles (HEVs) have been introduced to mitigate problems while improving fuel economy. HEVs have led to the demand of creating more advanced controls software to consider multiple components for propulsive power in a vehicle. A large section in the software development process is the implementation of an optimal energy management strategy meant to improve the overall fuel efficiency of the vehicle. Optimal strategies can be implemented when driving conditions are known a prior. The Equivalent Consumption Minimization Strategy (ECMS) is an optimal control strategy that uses an equivalence factor to equate electrical to mechanical power when performing torque split determination between the internal combustion engine and electric motor for propulsive and regenerative torque. This equivalence factor is determined from offline vehicle simulations using a sensitivity analysis to provide optimal fuel economy results while maintaining predetermined high voltage battery state of charge (SOC) constraints. When the control hierarchy is modified or different driving styles are applied, the analysis must be redone to update the equivalence factor. The goal of this work is to implement a fuzzy logic controller that dynamically updates the equivalence factor to improve fuel economy, maintain a strict charge sustaining window of operation for the high voltage battery, and reduce computational time required during algorithm development. The adaptive algorithm is validated against global optimum fuel economy and charge sustaining results from a sensitivity analysis performed for multiple drive cycles. Results show a maximum fuel economy improvement of 9.82% when using a mild driving style and a 95% success rate when maintaining an ending SOC within 5% of the desired SOC regardless of starting SOC.

Modeling and Control of Parallel Hybrid Electric Vehicle Using Sea-Lion Optimization

This paper develops a parallel hybrid electric vehicle(PHEV)propor-tional integral controller with driving cycle.To improve fuel efficiency and reduce hazardous emissions in hybrid electric vehicles(HEVs)combine an electric motor(EM),a battery and an internal combustion engine(ICE).The electric motor assists the engine when accelerating,driving longer highways or climbing hills.This enables the use of a smaller,more efficient engine.It also makes use of the concept of regenerative braking to maximize energy efficiency.In a Hybrid Electric Vehicle(HEV),energy dissipated while braking is utilized to charge the battery.The proportional integral controller was used in this paper to analyze engine,motor performance and the New European Driving Cycle(NEDC)was used in the vehicle driving test using Matlab/Simulink.The proportional integral controllers were designed to track the desired vehicle speed and manage the vehi-cle’s energyflow.The Sea Lion Optimization(SLnO)methods were created to reduce fuel consumption in a parallel hybrid electric vehicle and the results were obtained for the New European Driving Cycle.

An optimal energy management development for various configuration of plug-in and hybrid electric vehicle

Due to soaring fuel prices and environmental concerns, hybrid electric vehicle(HEV) technology attracts more attentions in last decade. Energy management system, configuration of HEV and traffic conditions are the main factors which affect HEV's fuel consumption, emission and performance. Therefore, optimal management of the energy components is a key element for the success of a HEV. An optimal energy management system is developed for HEV based on genetic algorithm. Then, different powertrain system component combinations effects are investigated in various driving cycles. HEV simulation results are compared for default rule-based, fuzzy and GA-fuzzy controllers by using ADVISOR. The results indicate the effectiveness of proposed optimal controller over real world driving cycles. Also, an optimal powertrain configuration to improve fuel consumption and emission efficiency is proposed for each driving condition. Finally, the effects of batteries in initial state of charge and hybridization factor are investigated on HEV performance to evaluate fuel consumption and emissions. Fuel consumption average reduction of about 14% is obtained for optimal configuration data in contrast to default configuration. Also results indicate that proposed controller has reduced emission of about 10% in various traffic conditions.

Realization and Analysis of Good Fuel Economy and Kinetic Performance of a Low-cost Hybrid Electric Vehicle

By using high-power and high-efficiency propulsion systems,current hybrid electric vehicles（HEVs） in market can achieve excellent fuel economy and kinetic performance.However,it is the cost of current HEVs that hinders HEVs coming into widespread use.A novel hybrid electric propulsion system is designed to balance HEV cost and performance for developing markets.A battery/supercapacitor-based hybrid energy storage system（HESS） is used to improve energy conversion efficiency and reduce battery size and cost.An all-in-one-controller（AIOC） which integrates engine electronic control unit（ECU）,motor ECU,and HESS management system is developed to save materials and energy,and reduce the influence of distribution parameters on circuit.As for the powertrain configuration,four schemes are presented：belt-driven starter generator（BSG） scheme,four-wheel drive HEV scheme,full HEV scheme,and ranger-extender electric vehicle（EV） scheme.Component selection and parameter matching for the propulsion system are performed,and an energy management strategy is developed based on powertrain configuration and selected components.Forward-facing simulation models are built,comprehending the control strategy based on the optimal engine torque for the low-cost hybrid electric propulsion system.Co-simulation of AVL CRUISE and Matlab/Simulink is presented and the best scheme is selected.The simulation results indicate that,for the best design,fuel consumption in urban driving condition is 4.11 L/（100 km） and 0-50 km/h accelerating time is 10.95 s.The proposed research can realize low-cost concept for HEV while achieving satisfactory fuel economy and kinetic performance,and help to improve commercialization of HEVs.

ENERGY MANAGEMENT STRATEGY FOR PARALLEL HYBRID ELECTRIC VEHICLES

Energy management strategy (EMS) is the core of the real-time controlalgorithm of the hybrid electric vehicle (HEV). A novel EMS using the logic threshold approach withincorporation of a stand-by optimization algorithm is proposed. The aim of it is to minimize theengine fuel consumption and maintain the battery state of charge (SOC) in its operation range, whilesatisfying the vehicle performance and drivability requirements. The hybrid powertrain bench testis carried out to collect data of the engine, motor and battery pack, which are used in the EMS tocontrol the powertrain. Computer simulation model of the HEV is established in the MATLAB/Simulinkenvironment according to the bench test results. Simulation results are presented for behaviors ofthe engine, motor and battery. The proposed EMS is implemented for a real parallel hybrid carcontrol system and validated by vehicle field tests.

Estimation Method of State-of-Charge For Lithium-ion Battery Used in Hybrid Electric Vehicles Based on Variable Structure Extended Kalman Filter

Since the main power source of hybrid electric vehicle（HEV） is supplied by the power battery,the predicted performance of power battery,especially the state-of-charge（SOC） estimation has attracted great attention in the area of HEV.However,the value of SOC estimation could not be greatly precise so that the running performance of HEV is greatly affected.A variable structure extended kalman filter（VSEKF）-based estimation method,which could be used to analyze the SOC of lithium-ion battery in the fixed driving condition,is presented.First,the general lower-order battery equivalent circuit model（GLM）,which includes column accumulation model,open circuit voltage model and the SOC output model,is established,and the off-line and online model parameters are calculated with hybrid pulse power characteristics（HPPC） test data.Next,a VSEKF estimation method of SOC,which integrates the ampere-hour（Ah） integration method and the extended Kalman filter（EKF） method,is executed with different adaptive weighting coefficients,which are determined according to the different values of open-circuit voltage obtained in the corresponding charging or discharging processes.According to the experimental analysis,the faster convergence speed and more accurate simulating results could be obtained using the VSEKF method in the running performance of HEV.The error rate of SOC estimation with the VSEKF method is focused in the range of 5% to 10% comparing with the range of 20% to 30% using the EKF method and the Ah integration method.In Summary,the accuracy of the SOC estimation in the lithium-ion battery cell and the pack of lithium-ion battery system,which is obtained utilizing the VSEKF method has been significantly improved comparing with the Ah integration method and the EKF method.The VSEKF method utilizing in the SOC estimation in the lithium-ion pack of HEV can be widely used in practical driving conditions.

Energy Control of Plug-In Hybrid Electric Vehicles Using Model Predictive Control With Route Preview

The paper proposes an adoption of slope,elevation,speed and route distance preview to achieve optimal energymanagement of plug-in hybrid electric vehicles(PHEVs).Theapproach is to identify route features from historical and real-time traffic data,in which information fusion model and trafficprediction model are used to improve the information accuracy.Then,dynamic programming combined with equivalent con-sumption minimization strategy is used to compute an optimalsolution for real-time energy management.The solution is thereference for PHEV energy management control along the route.To improve the system's ability of handling changing situation,the study further explores predictive control model in the real-time control of the energy.A simulation is performed to modelPHEV under above energy control strategy with route preview.The results show that the average fuel consumption of PHEValong the previewed route with model predictive control(MPC)strategy can be reduced compared with optimal strategy andbase control strategy.

OPTIMAL TORQUE CONTROL STRATEGY FOR PARALLEL HYBRID ELECTRIC VEHICLE WITH AUTOMATIC MECHANICAL TRANSMISSION

In parallel hybrid electrical vehicle （PHEV） equipped with automatic mechanical transmission （AMT）, the driving smoothness and the clutch abrasion are the primary considerations for powertrain control during gearshift and clutch operation. To improve these performance indexes of PHEV, a coordinated control system is proposed through the analyzing of HEV powertrain dynamic characteristics. Using the method of minimum principle, the input torque of transmission is optimized to improve the driving smoothness of vehicle. Using the methods of fuzzy logic and fuzzy-PID, the engaging speed of clutch and the throttle opening of engine are manipulated to ensure the smoothness of clutch engagement and reduce the abrasion of clutch friction plates. The motor provides the difference between the required input torque of transmission and the torque transmitted through clutch plates. Results of simulation and experiments show that the proposed control strategy performs better than the contrastive control system, the smoothness of driving and the abrasion of clutch can be improved simultaneously.

The Combination of Two Control Strategies for Series Hybrid Electric Vehicles

With most countries paying attention to the environment protection, hybrid electric vehicles have become a focus of automobile research and development due to the characteristics of energy saving and low emission. Power follower control strategy(PFCS) and DC-link voltage control strategy are two sorts of control strategies for series hybrid electric vehicles(HEVs). Combining those two control strategies is a new idea for control strategy of series hybrid electric vehicles. By tuning essential parameters which are the defined constants under DClink voltage control and under PFCS, the points of minimum mass of equivalent fuel consumption(EFC) corresponding to a series of variables are marked for worldwide harmonized light vehicles test procedure(WLTP). The fuel economy of series HEVs with the combination control schemes performs better compared with individual control scheme. The results show the effects of the combination control schemes for series HEVs driving in an urban environment.

A Novel Method for the Application of the ECMS(Equivalent Consumption Minimization Strategy)to Reduce Hydrogen Consumption in Fuel Cell Hybrid Electric Vehicles

Fuel cell hybrid electric vehicles are currently being considered as ideal means to solve the energy crisis and global warming in today’s society.In this context,this paper proposes a method to solve the problem related to the dependence of the so-called optimal equivalent factor(determined in the framework of the equivalent consumption minimum strategy-ECMS)on the working conditions.The simulation results show that under typical conditions(some representative cities being considered),the proposed strategy can maintain the power balance;for different initial battery’s states of charge(SOC),after the SOC stabilizes,the fuel consumption is 5.25 L/100 km.

MODELING AND IMPLEMENTATION OF SPEED GOVERNOR FOR THE HYBRID ELECTRIC VEHICLE ENGINE

A speed control analysis for an in-line gasoline fueled internal combustion （IC） engine is presented for the purpose of alleviation of high frequency oscillations in engine revolutions. A dynamic cylinder-by-cylinder model is proposed, base on slider-crank mechanism, which is extended to develop a digital governor providing a high fidelity estimation of rotary speed oscillation for hybrid vehicle engines. A modified PID controller that P and I gain is placed in feedback path is also described for hybrid electric vehicle （HEV） engine speed regulation, By comparison between measured and estimated signals, it is demonstrated that a good agreement has been achieved and the governor behaves an excellent damping speed ripple.

Study on load performance of electric motor system used in hybrid electric vehicle

The performance of the power assist, global optimization solved by dynamic programming （DP） method, Chery and Insight control strategies are analyzed using the mild parallel hybrid electric vehicle （PHEV） model based on Insight structure. The influence of the four control strategies to the load power of the electric motor system used on parallel hybrid electric vehicle is studied. It is found that 80 percent of the motor load power points are under 1/5 of the electric peak power. The motor load power of the power assist control strategy is distributed in the widest range during generating operation, and the motor load power of the global optimization control strategy has the smallest one.

Design and Analysis of Electro-mechanical Hybrid Anti-lock Braking System for Hybrid Electric Vehicle Utilizing Motor Regenerative Braking

Braking on low adhesion-coefficient roads, hybrid electric vehicle＇s motor regenerative torque is switched off to safeguard the normal anti-lock braking system （ABS） fimction. When the ABS control is terminated, the motor regenerative braking is readmitted. Aiming at avoiding permanent cycles from hydraulic anti-lock braking to motor regenerative braking, a novel electro-mechanical hybrid anti-lock braking system using fuzzy logic is designed. Different from the traditional single control structure, this system has a two-layered hierarchical structure, The first layer is responsible for harmonious adjustment or interaction between regenerative system and anti-lock braking system. The second layer is responsible for braking torque distribution and adjustment. The closed-loop simulation model is built. Control strategy and method for coordination between regenerative and anti-lock braking are developed. Simulation braking on low adhesion-coefficient roads with fuzzy logic control and real vehicle braking field test are presented. The results from simulating analysis and experiment show braking performance of the vehicle is perfect, harmonious coordination between regenerative and anti-lock braking function, significant amount of braking energy can be recovered and the proposed control strategy and method are effective.

Synthetical Efficiency-based Optimization for the Power Distribution of Power-split Hybrid Electric Vehicles

Now the optimization strategies for power distribution are researched widely, and most of them are aiming to the optimal fuel economy and the driving cycle must be preknown. Thus if the actual driving condition deviates from the scheduled driving cycle, the effect of optimal results will be declined greatly. Therefore, the instantaneous optimization strategy carried out on-line is studied in this paper. The power split path and the transmission efficiency are analyzed based on a special power-split scheme and the efficiency models of the power transmitting components are established. The synthetical efficiency optimization model is established for enhancing the transmission efficiency and the fuel economy. The identification of the synthetical efficiency as the optimization objective and the constrain group are discussed emphatically. The optimization is calculated by the adaptive simulated annealing （ASA） algorithm and realized on-line by the radial basis function （RBF）-based similar models. The optimization for power distribution of the hybrid vehicle in an actual driving condition is carried out and the road test results are presented. The test results indicate that the synthetical efficiency optimization method can enhance the transmission efficiency and the fuel economy of the power-split hybrid electric vehicle （HEV） observably. Compared to the rules-based strategy the optimization strategy is optimal and achieves the approximate global optimization solution for the power distribution. The synthetical efficiency optimization solved by ASA algorithm can give attentions to both optimization quality and calculation efficiency, thus it has good application foreground for the power distribution of power-split HEV.

OPTIMIZATION APPROACH FOR HYBRID ELECTRIC VEHICLE POWERTRAIN DESIGN

According to bench test results of fuel economy and engine emission for thereal power-train system of EQ7200HEV car. a 3-D performance map oriented quasi-linear model isdeveloped for the configuration of the powertrain components such as internal combustion engine,traction electric motor, transmission, main retarder and energy storage unit. A genetic algorithmbased on optimization procedure is proposed and applied for parametric optimization of the keycomponents by consideration of requirements of some driving cycles. Through comparison of numericalresults obtained by the genetic algorithm with those by traditional optimization methods, it isshown that the present approach is quite effective and efficient in emission reduction and fueleconomy for the design of the hybrid electric car powertrain.

Online Learning Control for Hybrid Electric Vehicle

Improvements in hybrid electric vehicle (HEV) fuel economy and emissions heavily depend on an efficient energy management strategy (EMS). However, the uncertainty of future driving conditions generally cannot be easily tackled in EMS design. Most existing EMSs act upon fixed parameters and cannot adapt to varying driving conditions. Therefore, they usually fail to fully explore the potential of these advanced vehicles. In this paper, a novel EMS design procedure based on neural dynamic programming (NDP) is proposed. The NDP is a generic online learning algorithm, which combines stochastic dynamic programming (SDP) and the temporal difference (TD) method. Instead of computing the utility function and optimal control actions through Bellman equations, the NDP algorithm uses two neural networks to approximate them. The weights of these neural networks are updated online by the TD method. It avoids the high computational cost that SDP suffers from and is suitable for real-time implementation. The main advantages of NDP EMS is that it does not rely on prior information related to future driving conditions, and can self-tune with a wide variance in operating conditions. The NDP EMS has been applied to “Qianghua-I”, a prototype of a parallel HEV, using a revolving drum test bench for verification. Experiment results illustrate the potential of the proposed EMS in terms of fuel economy and in keeping state of charge (SOC) deviations at a low level. The proposed research ensures the optimality of NDP EMS, as well as real-time applicability.

A comprehensive review on hybrid electric vehicles: architectures and components

The rapid consumption of fossil fuel and increased environmental damage caused by it have given a strong impetus to the growth and development of fuelefficient vehicles. Hybrid electric vehicles (HEVs) have evolved from their inchoate state and are proving to be a promising solution to the serious existential problem posed to the planet earth. Not only do HEVs provide better fuel economy and lower emissions satisfying environmental legislations, but also they dampen the effect of rising fuel prices on consumers. HEVs combine the drive powers of an internal combustion engine and an electrical machine. The main components of HEVs are energy storage system, motor, bidirectional converter and maximum power point trackers (MPPT, in case of solar-powered HEVs). The performance of HEVs greatly depends on these components and its architecture. This paper presents an extensive review on essential components used in HEVs such as their architectures with advantages and disadvantages, choice of bidirectional converter to obtain high efficiency, combining ultracapacitor with battery to extend the battery life, traction motors’ role and their suitability for a particular application. Inclusion of photovoltaic cell in HEVs is a fairly new concept and has been discussed in detail. Various MPPT techniques used for solar-driven HEVs are also discussed in this paper with their suitability.

STEADY-STATE AND IDLE OPTIMIZA-TION OF INTERNAL COMBUSTION ENGINE CONTROL STRATEGIES FOR HYBRID ELECTRIC VEHICLES

A novel steady-state optimization （SSO） of internal combustion engine （ICE） strategy is proposed to maximize the efficiency of the overall powertrain for hybrid electric vehicles, in which the ICE efficiency, the efficiencies of the electric motor （EM） and the energy storage device are all explicitly taken into account. In addition, a novel idle optimization of ICE strategy is implemented to obtain the optimal idle operating point of the ICE and corresponding optimal parking generation power of the EM using the view of the novel SSO of ICE strategy. Simulations results show that potential fuel economy improvement is achieved relative to the conventional one which only optimized the ICE efficiency by the novel SSO of ICE strategy, and fuel consumption per voltage increment decreases a lot during the parking charge by the novel idle optimization of ICE strategy.

An Investigation into Regenerative Braking Control Strategy for Hybrid Electric Vehicle

Energy regeneration during braking is an important technique for hybrid electric vehicle (HEV) to improve their fuel economy and extend their driving range. Due to the effect of regenerative braking torque which is added by electric motor, the braking torque distribution between front and rear axles should be changed and the control logic of anti-lock braking system (ABS) ought to be adjusted according to the regenerative braking torque. This paper put forward a braking control strategy for hybrid electric vehicle; the control strategy is implemented with eight DOFs (Degree-of-Freedom) nonlinear vehicle forward simulation model which is built under the environment of Matlab/Simulink. Based on target wheel slip ratio, a fuzzy logic approach was applied to maintain the optimal target slip ratio so that best compromise between hydraulic torque and regenerative torque can be obtained for the vehicle.