Decision-Making Models Based on Meta-Reinforcement Learning for Intelligent Vehicles at Urban Intersections

Behavioral decision-making at urban intersections is one of the primary difficulties currently impeding the development of intelligent vehicle technology.The problem is that existing decision-making algorithms cannot effectively deal with complex random scenarios at urban intersections.To deal with this,a deep deterministic policy gradient(DDPG)decision-making algorithm(T-DDPG)based on a time-series Markov decision process(T-MDP)was developed,where the state was extended to collect observations from several consecutive frames.Experiments found that T-DDPG performed better in terms of convergence and generalizability in complex intersection scenarios than a traditional DDPG algorithm.Furthermore,model-agnostic meta-learning(MAML)was incorporated into the T-DDPG algorithm to improve the training method,leading to a decision algorithm(T-MAML-DDPG)based on a secondary gradient.Simulation experiments of intersection scenarios were carried out on the Gym-Carla platform to verify and compare the decision models.The results showed that T-MAML-DDPG was able to easily deal with the random states of complex intersection scenarios,which could improve traffic safety and efficiency.The above decision-making models based on meta-reinforcement learning are significant for enhancing the decision-making ability of intelligent vehicles at urban intersections.

Human-Machine Shared Lateral Control Strategy for Intelligent Vehicles Based on Human Driver Risk Perception Reliability

Intelligent vehicle(Ⅳ)technology has developed rapidly in recent years.However,achieving fully unmanned driving still presents numerous challenges,which means that human drivers will continue to play a vital role in vehicle operation for the foreseeable future.Human-machine shared driving,involving cooperation between a human driver and an automated driving system(AVS),has been widely regarded as a necessary stage for the development of IVs.Focusing onⅣdriving safety,this study proposed a human-machine shared lateral control strategy(HSLCS)based on the reliability of driver risk perception.The HSLCS starts by identifying the effective areas of driver risk perception based on eye movements.It establishes an anisotropic driving risk field,which serves as the foundation for the AVS to assess risk levels.Building upon the cumulative and diminishing effects of risk perception,the proposed approach leverages the driver's risk perception effective area and converts the risk field into a representation aligned with the driver's perspective.Subsequently,it quantifies the reliability of the driver's risk perception by using area-matching rules.Finally,based on the driver’s risk perception reliability and dif-ferences in lateral driving operation between the human driver and the AVS,the dynamic distribution of driving authority is achieved through a fuzzy rule-based system,and the human-machine shared lateral control is completed by using model predictive control.The HSLCS was tested across various scenarios on a driver-in-the-loop test platform.The results show that the HSLCS can realize the synergy and complementarity of human and machine intelligence,effectively ensuring the safety ofⅣoperation.

Ensuring Secure Platooning of Constrained Intelligent and Connected Vehicles Against Byzantine Attacks:A Distributed MPC Framework

This study investigates resilient platoon control for constrained intelligent and connected vehicles(ICVs)against F-local Byzantine attacks.We introduce a resilient distributed model-predictive platooning control framework for such ICVs.This framework seamlessly integrates the predesigned optimal control with distributed model predictive control(DMPC)optimization and introduces a unique distributed attack detector to ensure the reliability of the transmitted information among vehicles.Notably,our strategy uses previously broadcasted information and a specialized convex set,termed the“resilience set”,to identify unreliable data.This approach significantly eases graph robustness prerequisites,requiring only an(F+1)-robust graph,in contrast to the established mean sequence reduced algorithms,which require a minimum(2F+1)-robust graph.Additionally,we introduce a verification algorithm to restore trust in vehicles under minor attacks,further reducing communication network robustness.Our analysis demonstrates the recursive feasibility of the DMPC optimization.Furthermore,the proposed method achieves exceptional control performance by minimizing the discrepancies between the DMPC control inputs and predesigned platoon control inputs,while ensuring constraint compliance and cybersecurity.Simulation results verify the effectiveness of our theoretical findings.

Design and Implementation of a Battery Big Data Platform Through Intelligent Connected Electric Vehicles

The development of a battery management algorithm is highly dependent on high-quality battery operation data,especially the data in extreme conditions such as low temperatures.The data in faults are also essential for failure and safety management research.This study developed a battery big data platform to realize vehicle operation,energy interaction and data management.First,we developed an electric vehicle with vehicle navigation and position detection and designed an environmental cabin that allows the vehicle to operate autonomously.Second,charging and heating systems based on wireless energy transfer were developed and equipped on the vehicle to investigate optimal charging and heating methods of the batteries in the vehicle.Third,the data transmission network was designed,a real-time monitoring interface was developed,and the self-developed battery management system was used to measure,collect,upload,and store battery operation data in real time.Finally,experimental validation was performed on the platform.Results demonstrate the efficiency and reliability of the platform.Battery state of charge estimation is used as an example to illustrate the availability of battery operation data.

A digital twins enabled underwater intelligent internet vehicle path planning system via reinforcement learning and edge computing

The Autonomous Underwater Glider(AUG)is a kind of prevailing underwater intelligent internet vehicle and occupies a dominant position in industrial applications,in which path planning is an essential problem.Due to the complexity and variability of the ocean,accurate environment modeling and flexible path planning algorithms are pivotal challenges.The traditional models mainly utilize mathematical functions,which are not complete and reliable.Most existing path planning algorithms depend on the environment and lack flexibility.To overcome these challenges,we propose a path planning system for underwater intelligent internet vehicles.It applies digital twins and sensor data to map the real ocean environment to a virtual digital space,which provides a comprehensive and reliable environment for path simulation.We design a value-based reinforcement learning path planning algorithm and explore the optimal network structure parameters.The path simulation is controlled by a closed-loop model integrated into the terminal vehicle through edge computing.The integration of state input enriches the learning of neural networks and helps to improve generalization and flexibility.The task-related reward function promotes the rapid convergence of the training.The experimental results prove that our reinforcement learning based path planning algorithm has great flexibility and can effectively adapt to a variety of different ocean conditions.

Autonomous parking control for intelligent vehicles based on a novel algorithm

Along with the increasing number of vehicles, parking space becomes narrow gradually, safety parking puts forward higher requirements on the driver＇s driving technology. How to safely, quickly and accurately park the vehiclo to parking space right？ This paper presents an automatic parking scheme based on trajectory planning, which analyzing the mechanical model oftbe vehicle, establishing vehicle steering model and parking model, coming to the conclusion that it is the turning radius is independent of the vehicle speed at low speed. The Matlab simulation environment verifies the correctness and effectiveness of the proposed algorithm for parking. A class of the automatic parking problem of intelligent vehicles is solved.

Attacks and defences on intelligent connected vehicles:a survey

Intelligent vehicles are advancing at a fast speed with the improvement of automation and connectivity,which opens up new possibilities for different cyber-attacks,including in-vehicle attacks(e.g.,hijacking attacks)and vehicle-to-everything communicationattacks(e.g.,data theft).These problems are becoming increasingly serious with the development of 4G LTE and 5G communication technologies.Although many efforts are made to improve the resilience to cyber attacks,there are still many unsolved challenges.This paper first identifies some major security attacks on intelligent connected vehicles.Then,we investigate and summarize the available defences against these attacks and classify them into four categories:cryptography,network security,software vulnerability detection,and malware detection.Remaining challenges and future directions for preventing attacks on intelligent vehicle systems have been discussed as well.

Off-road testing scenario design and library generation for intelligent vehicles

To realize the widespread application and continuous functional development of intelligent vehicles,test and evaluation of vehicle's functionality and Safety Performance in complex off-road scenarios are fundamental.Since traditional distance-based road tests cannot meet the evolving test requirements,a method to design the function-based off-road testing scenario library for intelligent vehicles(IV)is proposed in this paper.The testing scenario library is defined as a critical set of scenarios that can be used for IV tests.First,for the complex and diverse off-road scenarios,a hierarchical,structural model of the test scenario is built.Then,the critical test scenarios are selected adaptively according to the vehicle model to be tested.Next,those parameters representing the challenging test scenarios are selected.The selected parameters need to fit the natural distribution probability of scenarios.The critical test-scenario library is built combing these parameters with the structural model.Finally,the test scenarios that are most approximate to the natural driving scenario are determined with importance sampling theory.The test-scenario library built with this method can provide more critical test scenarios,and is widely applicable despite different vehicle models.Verified by simulation in the off-road interaction scenarios,test would be accelerated significantly with this method,about 800 times faster than testing in the natural road environment.

Optimal control methods in intelligent vehicles

In recent years,intelligent vehicles(IVs)have become a hot spot in automotive industry.Key technologies of IVs range over the field of sensing,decision-making and control.Among them,control technology provides an enabling support for improving autonomous driving safety,reducing energy consumption and carbon emission.This paper focuses on some aspects of applying advanced control methodologies in IVs through several selected examples including eco-driving and MPC-based driver modelling.

A Ray Tracing and Joint Spectrum Based Clustering and Tracking Algorithm for Internet of Intelligent Vehicles

Driven by the rapid growth in information services provided by the Internet and the appearance of new multimedia applications,millimeter wave is foreseen as a key enabler towards the Internet of intelligent vehicles(IoIV)for urban traffic safety enhancement.In this regard,cluster-based channel modeling has become an important research topic in the realm of emergency communications.To fully understand the cluster-based channel model,a series of vehicle-to-infrastructure(V2I)channel simulations at 22.6 GHz are conducted by a three-dimensional ray tracing(RT)simulator.The clustering and tracking algorithm is proposed and analyzed from three aspects by the obtained simulation results.The multiple signal classification estimation spectrum is applied to restrain the influence of antenna sidelobes and identify targets at first.Based on the fundamentals,the clusters can be identified and subsequently tracked using the proposed approach.The impacts of antenna sidelobes,angle resolution of beam rotation,and non-line-of-sight propagation path on the performance of clustering and tracking are evaluated.The multi-component-level RT results are adopted as comparison benchmarks,which reflect the ground truth.This work aims to provide a full picture of the clustering characteristics for designing and analyzing emergency communication systems.

Cloud Control System Architectures,Technologies and Applications on Intelligent and Connected Vehicles:a Review

The electrification of vehicle helps to improve its operation efficiency and safety.Due to fast development of network,sensors,as well as computing technology,it becomes realizable to have vehicles driving autonomously.To achieve autonomous driving,several steps,including environment perception,path-planning,and dynamic control,need to be done.However,vehicles equipped with on-board sensors still have limitations in acquiring necessary environmental data for optimal driving decisions.Intelligent and connected vehicles(ICV)cloud control system(CCS)has been introduced as a new concept as it is a potentially synthetic solution for high level automated driving to improve safety and optimize traffic flow in intelligent transportation.This paper systematically investigated the concept of cloud control system from cloud related applications on ICVs,and cloud control system architecture design,as well as its core technologies development.Based on the analysis,the challenges and suggestions on cloud control system development have been addressed.

Big Wave of the Intelligent Connected Vehicles

Internet of things is deemed as the one of the great revolution after the age of Industrial Revolution.With the development of the communication technology,more and more entities are connected to the communication network and become one of the elements in the network.Over recent decades,in the area of intelligent transportation,pedestrian and transport infrastructure are connected to the communication network to improve the driving safety and traffic efficiency which is known as the ICV(Intelligent Connected Vehicle).This paper summarizes the global ICV progresses in the past decades and the latest activities of ICV in China,and introduces various aspects regarding the recent development of the ICV,including industry development,spectrum and standard,at the same time.

Key points of investigation and analysis on traffic accidents involving intelligent vehicles

Intellectualization is regarded as the future mainstream development trend of the automobile industry.The automation level of intelligent vehicles is relatively low so far,and the road traffic system will be in a mixed state of non-autonomous vehicles and vehicles with different levels of automation for a long time.Therefore,the road traffic system will be more complex with more diverse accidents.This paper analysed the characteristics and causal factors of intelligent vehicle accidents.Based on the problems existing in investigation,analysis and liability identification of intelligent vehicle accident,the study proposed a preliminary accident investigation framework and method,summarized the key points of accident analysis from the perspectives of technical defects,information security and passive safety,and specified the liability subjects for intelligent vehicle accidents and their corresponding legal liability.The results from this study contributed to the development of intelligent vehicle accident investigation and disposal,and provided the reference for the improvement of vehicle safety and accident prevention.

Mass estimation method for intelligent vehicles based on fusion of machine learning and vehicle dynamic model

Vehicle mass is an important parameter for motion control of intelligent vehicles,but is hard to directly measure using normal sensors.Therefore,accurate estimation of vehicle mass becomes crucial.In this paper,a vehicle mass estimation method based on fusion of machine learning and vehicle dynamic model is introduced.In machine learning method,a feedforward neural network(FFNN)is used to learn the relationship between vehicle mass and other state parameters,namely longitudinal speed and acceleration,driving or braking torque,and wheel angular speed.In dynamics-based method,recursive least square(RLS)with forgetting factor based on vehicle dynamic model is used to estimate the vehicle mass.According to the reliability of each method under different conditions,these two methods are fused using fuzzy logic.Simulation tests under New European Driving Cycle(NEDC)condition are carried out.The simulation results show that the estimation accuracy of the fusion method is around 97%,and that the fusion method performs better stability and robustness compared with each single method.

Hierarchical CNNPID Based Active Steering Control Method for Intelligent Vehicle Facing Emergency Lane-Changing

To resolve the response delay and overshoot problems of intelligent vehicles facing emergency lane-changing due to proportional-integral-differential(PID)parameter variation,an active steering control method based on Convolutional Neural Network and PID(CNNPID)algorithm is constructed.First,a steering control model based on normal distribution probability function,steady constant radius steering,and instantaneous lane-change-based active for straight and curved roads is established.Second,based on the active steering control model,a three-dimensional constraint-based fifth-order polynomial equation lane-change path is designed to address the stability problem with supersaturation and sideslip due to emergency lane changing.In addition,a hierarchical CNNPID Controller is constructed which includes two layers to avoid collisions facing emergency lane changing,namely,the lane change path tracking PID control layer and the CNN control performance optimization layer.The scaled conjugate gradient backpropagation-based forward propagation control law is designed to optimize the PID control performance based on input parameters,and the elastic backpropagation-based module is adopted for weight correction.Finally,comparison studies and simulation/real vehicle test results are presented to demonstrate the effectiveness,significance,and advantages of the proposed controller.

Cognitive Granular-Based Path Planning and Tracking for Intelligent Vehicle with Multi-Segment Bezier Curve Stitching

Unmanned vehicles are currently facing many difficulties and challenges in improving safety performance when running in complex urban road traffic environments,such as low intelligence and poor comfort perfor-mance in the driving process.The real-time performance of vehicles and the comfort requirements of passengers in path planning and tracking control of unmanned vehicles have attracted more and more attentions.In this paper,in order to improve the real-time performance of the autonomous vehicle planning module and the comfort requirements of passengers that a local granular-based path planning method and tracking control based on multi-segment Bezier curve splicing and model predictive control theory are pro-posed.Especially,the maximum trajectory curvature satisfying ride comfort is regarded as an important constraint condition,and the corresponding curvature threshold is utilized to calculate the control points of Bezier curve.By using low-order interpolation curve splicing,the planning computation is reduced,and the real-time performance of planning is improved,com-pared with one-segment curve fitting method.Furthermore,the comfort performance of the planned path is reflected intuitively by the curvature information of the path.Finally,the effectiveness of the proposed control method is verified by the co-simulation platform built by MATLAB/Simulink and Carsim.The simulation results show that the path tracking effect of multi-segment Bezier curve fitting is better than that of high-order curve planning in terms of real-time performance and comfort.

Simulation and Field Testing of Multiple Vehicles Collision Avoidance Algorithms

A global planning algorithm for intelligent vehicles is designed based on the A* algorithm, which provides intelligent vehicles with a global path towards their destinations. A distributed real-time multiple vehicle collision avoidance(MVCA)algorithm is proposed by extending the reciprocal n-body collision avoidance method. MVCA enables the intelligent vehicles to choose their destinations and control inputs independently,without needing to negotiate with each other or with the coordinator. Compared to the centralized trajectory-planning algorithm, MVCA reduces computation costs and greatly improves the robustness of the system. Because the destination of each intelligent vehicle can be regarded as private, which can be protected by MVCA, at the same time MVCA can provide a real-time trajectory planning for intelligent vehicles. Therefore,MVCA can better improve the safety of intelligent vehicles. The simulation was conducted in MATLAB, including crossroads scene simulation and circular exchange position simulation. The results show that MVCA behaves safely and reliably. The effects of latency and packet loss on MVCA are also statistically investigated through theoretically formulating broadcasting process based on one-dimensional Markov chain. The results uncover that the tolerant delay should not exceed the half of deciding cycle of trajectory planning, and shortening the sending interval could alleviate the negative effects caused by the packet loss to an extent. The cases of short delay(< 100100 ms) and low packet loss(< 5%) can bring little influence to those trajectory planning algorithms that only depend on V2 V to sense the context, but the unpredictable collision may occur if the delay and packet loss are further worsened. The MVCA was also tested by a real intelligent vehicle, the test results prove the operability of MVCA.

Development of an Intelligent Vehicle Experiment System

Existing vehicle experiment systems tend to focus on the research of vehicle dynamics by conducting performance tests on every system or some parts of the vehicle so as to improve the entire performance of the vehicle. Virtual technology is widely utilized in various vehicle test-beds. These test-beds are mainly used to simulate the driving training, conduct the research on drivers＇ behaviors, or give virtual demonstrations of the transportation environment. However, the study on the active safety of the running vehicle in the virtual environment is still insufficient. A virtual scene including roads and vehicles is developed by using the software Creator and Vega, and radars and cameras are also simulated in the scene. Based on dSPACE＇s rapid prototyping simulation and its single board DS1103, a simulation model including vehicle control signals is set up in MATLAB/Simulink, the model is then built into C code, and the system defined file（SDF） is downloaded to the DS1103 board through the experiment debug software ControlDesk and is kept running. Programming is made by mixing Visual C＋＋ 6.0, MATLAB API and Vega API. Control signals are read out by invoking library function MLIB/MTRACE of dSPACE. All the input, output, and system state values are acquired by arithmetic and are dynamically associated with the running status of the virtual vehicle. An intelligent vehicle experiment system is thus developed by virtue of program and integration. The system has not only the demonstration function, such as general driving, cruise control, active avoiding collision, but also the function of virtual experiment. Parameters of the system can be set according to needs, and the virtual test results can be analyzed and studied and used for the comparison with the existing models. The system reflects the running of the intelligent vehicle in the virtual traffic environment, at the same time, the system is a new attempt performed on the intelligent vehicle travel research and provides also a new research method for the development of intelligent vehicles.

Intelligent vehicle lateral controller design based on genetic algorithmand T-S fuzzy-neural network

Non-linearity and parameter time-variety are inherent properties of lateral motions of a vehicle. How to effectively control intelligent vehicle (IV) lateral motions is a challenging task. Controller design can be regarded as a process of searching optimal structure from controller structure space and searching optimal parameters from parameter space. Based on this view, an intelligent vehicle lateral motions controller was designed. The controller structure was constructed by T-S fuzzy-neural network (FNN). Its parameters were searched and selected with genetic algorithm (GA). The simulation results indicate that the controller designed has strong robustness, high precision and good ride quality, and it can effectively resolve IV lateral motion non-linearity and time-variant parameters problem.

Target Path Tracking Method of Intelligent Vehicle Based on Competitive Cooperative Game

To improve intelligent vehicle drive performance and avoid vehicle side-slip during target path tracking,a linearized four-wheel vehicle model is adopted as a predictive control model,and an intelligent ve-hicle target path tracking method based on a competitive cooperative game is proposed.The design variables are divided into different strategic spaces owned by each player by calculating the affecting factors of the design variables with objective functions and fuzzy clustering.Based on the competitive cooperative game model,each game player takes its payoff as a mono-objective to optimize its own strategic space and obtain the best strategy to deal with others.The best strategies were combined into the game strategy set.Considering the front wheel angle and side slip angle increment constraint,tire side-slip angle,and tire side slip deflection dynamics,it took the path tracking state model was used as the objective,function and the calculation was validated by competitive cooperative game theory.The results demonstrated the effectiveness of the proposed algorithm.The experimental results show that this method can track an intelligent vehicle quickly and steadily and has good real-time per-formance.