Reinforcement Learning-Based Energy Management for Hybrid Power Systems:State-of-the-Art Survey,Review,and Perspectives

The new energy vehicle plays a crucial role in green transportation,and the energy management strategy of hybrid power systems is essential for ensuring energy-efficient driving.This paper presents a state-of-the-art survey and review of reinforcement learning-based energy management strategies for hybrid power systems.Additionally,it envisions the outlook for autonomous intelligent hybrid electric vehicles,with reinforcement learning as the foundational technology.First of all,to provide a macro view of historical development,the brief history of deep learning,reinforcement learning,and deep reinforcement learning is presented in the form of a timeline.Then,the comprehensive survey and review are conducted by collecting papers from mainstream academic databases.Enumerating most of the contributions based on three main directions—algorithm innovation,powertrain innovation,and environment innovation—provides an objective review of the research status.Finally,to advance the application of reinforcement learning in autonomous intelligent hybrid electric vehicles,future research plans positioned as“Alpha HEV”are envisioned,integrating Autopilot and energy-saving control.

Comparative Study of Trajectory Tracking Control for Automated Vehicles via Model Predictive Control and Robust H-infinity State Feedback Control

A comparative study of model predictive control(MPC)schemes and robust Hstate feedback control(RSC)method for trajectory tracking is proposed in this paper.The main objective of this paper is to compare MPC and RSC controllers’performance in tracking predefined trajectory under different scenarios.MPC controller is designed based on the simple longitudinal-yaw-lateral motions of a single-track vehicle with a linear tire,which is an approximation of the more realistic model of a vehicle with double-track motion with a non-linear tire mode.RSC is designed on the basis of the same method as adopted for the MPC controller to achieve a fair comparison.Then,three test cases are built in CarSim-Simulink joint platform.Specifically,the verification test is used to test the tracking accuracy of MPC and RSC controller under well road conditions.Besides,the double lane change test with low road adhesion is designed to find the maximum velocity that both controllers can carry out while guaranteeing stability.Furthermore,an extreme curve test is built where the road adhesion changes suddenly,in order to test the performance of both controllers under extreme conditions.Finally,the advantages and disadvantages of MPC and RSC under different scenarios are also discussed.

Deep Reinforcement Learning Enabled Decision-Making for Autonomous Driving at Intersections

Road intersection is one of the most complex and accident-prone traffic scenarios,so it’s challenging for autonomous vehicles(AVs)to make safe and efficient decisions at the intersections.Most of the related studies focus on the solution to a single scenario or only guarantee safety without considering driving efficiency.To address these problems,this study proposed a deep reinforcement learning enabled decision-making framework for AVs to drive through intersections automatically,safely and efficiently.The mapping relationship between traffic images and vehicle operations was obtained by an end-to-end decision-making framework established by convolutional neural networks.Traffic images collected at two timesteps were used to calculate the relative velocity between vehicles.Markov decision process was employed to model the interaction between AVs and other vehicles,and the deep Q-network algorithm was utilized to obtain the optimal driving policy regarding safety and efficiency.To verify the effectiveness of the proposed decision-making framework,the top three accident-prone crossing path crash scenarios at intersections were simulated,when different initial vehicle states were adopted for better generalization capability.The results showed that the developed method could make AVs drive safely and efficiently through intersections in all of the tested scenarios.

Study of Longitudinal-Vertical Dynamics for In-Wheel Motor-Driven Electric Vehicles

The in-wheel motor(IWM)-driven electric vehicles(EVs)attract increasing attention due to their advantages in dimensions and controllability.The majority of the current studies on IWM are carried out with the assumption of an ideal actuator,in which the coupling effects between the non-ideal IWM and vehicle are ignored.This paper uses the braking process as an example to investigate the longitudinal-vertical dynamics of IWM-driven EVs while considering the mechanical-electrical coupling effect.First,a nonlinear switched reluctance motor model is developed,and the unbalanced electric magnetic force(UEMF)induced by static and dynamic mixed eccentricity is analyzed.Then,the UEMF is decomposed into longitudinal and vertical directions and included in the longitudinal-vertical vehicle dynamics.The coupling dynamics are demonstrated under different vehicle braking scenarios;numerical simulations are carried out for various road grades,road friction,and vehicle velocities.A novel dynamics vibration absorbing system is adopted to improve the vehicle dynamics.Finally,the simulation results show that vehicle vertical dynamic performance is enhanced.

Comprehensive Analysis and Optimization of Dynamic Vibration-Absorbing Structures for Electric Vehicles Driven by In-Wheel Motors

Distributed-drive electric vehicles(EVs)replace internal combustion engine with multiple motors,and the novel configura-tion results in new dynamic-related issues.This paper studies the coupling effects between the parameters and responses of dynamic vibration-absorbing structures(DVAS)for EVs driven by in-wheel motors(IWM).Firstly,a DVAS-based quarter suspension model is developed for distributed-drive EVs,from which nine parameters and five responses are selected for the coupling effect analysis.A two-stage global sensitivity analysis is then utilized to investigate the effect of each parameter on the responses.The control of the system is then converted into a multiobjective optimization problem with the defined system parameters being the optimization variables,and three dynamic limitations regarding both motor and suspension subsystems are taken as the constraints.A particle swarm optimization approach is then used to either improve ride comfort or mitigate IWM vibration,and two optimized parameter sets for these two objects are provided at last.Simulation results provide in-depth conclusions for the coupling effects between parameters and responses,as well as a guideline on how to design system parameters for contradictory objectives.It can be concluded that either passenger comfort or motor lifespan can be reduced up to 36%and 15%by properly changing the IWM suspension system parameters.

Drivers’ EEG Responses to Different Distraction Tasks

Driver distraction has been deemed a major cause of traffic accidents.However,drivers’brain response activities to different distraction types have not been well investigated.The purpose of this study is to investigate the response of electroencephalography(EEG)activities to different distraction tasks.In the conducted simulation tests,three secondary tasks(i.e.,a clock task,a 2-back task,and a navigation task)are designed to induce different types of driver distractions.Twenty-four participants are recruited for the designed tests,and differences in drivers’brain response activities concerning distraction types are investigated.The results show that the differences in comprehensive distraction are more significant than that in single cognitive distraction.Friedman test and post hoc two-tailed Nemenyi test are conducted to further identify the differences in band activities among brain regions.The results show that the theta energy in the frontal lobe is significantly higher than that in other brain regions in distracted driving,whereas the alpha energy in the temporal lobe significantly decreases compared to other brain regions.These results provide theoretical references for the development of distraction detection systems based on EEG signals.

Correction to: Comprehensive Analysis and Optimization of Dynamic Vibration-Absorbing Structures for Electric Vehicles Driven by In-Wheel Motors

The original version of this article unfortunately contained a mistake.The last sentence of the abstract was incorrect.The corrected sentence is given below.“It can be concluded that either passenger comfort or motor lifespan can be improved up to 15%and 36%by properly changing the IWM suspension system parameters”.