Decision-Making Models Based on Meta-Reinforcement Learning for Intelligent Vehicles at Urban Intersections

Behavioral decision-making at urban intersections is one of the primary difficulties currently impeding the development of intelligent vehicle technology.The problem is that existing decision-making algorithms cannot effectively deal with complex random scenarios at urban intersections.To deal with this,a deep deterministic policy gradient(DDPG)decision-making algorithm(T-DDPG)based on a time-series Markov decision process(T-MDP)was developed,where the state was extended to collect observations from several consecutive frames.Experiments found that T-DDPG performed better in terms of convergence and generalizability in complex intersection scenarios than a traditional DDPG algorithm.Furthermore,model-agnostic meta-learning(MAML)was incorporated into the T-DDPG algorithm to improve the training method,leading to a decision algorithm(T-MAML-DDPG)based on a secondary gradient.Simulation experiments of intersection scenarios were carried out on the Gym-Carla platform to verify and compare the decision models.The results showed that T-MAML-DDPG was able to easily deal with the random states of complex intersection scenarios,which could improve traffic safety and efficiency.The above decision-making models based on meta-reinforcement learning are significant for enhancing the decision-making ability of intelligent vehicles at urban intersections.

Ensuring Secure Platooning of Constrained Intelligent and Connected Vehicles Against Byzantine Attacks:A Distributed MPC Framework

This study investigates resilient platoon control for constrained intelligent and connected vehicles(ICVs)against F-local Byzantine attacks.We introduce a resilient distributed model-predictive platooning control framework for such ICVs.This framework seamlessly integrates the predesigned optimal control with distributed model predictive control(DMPC)optimization and introduces a unique distributed attack detector to ensure the reliability of the transmitted information among vehicles.Notably,our strategy uses previously broadcasted information and a specialized convex set,termed the“resilience set”,to identify unreliable data.This approach significantly eases graph robustness prerequisites,requiring only an(F+1)-robust graph,in contrast to the established mean sequence reduced algorithms,which require a minimum(2F+1)-robust graph.Additionally,we introduce a verification algorithm to restore trust in vehicles under minor attacks,further reducing communication network robustness.Our analysis demonstrates the recursive feasibility of the DMPC optimization.Furthermore,the proposed method achieves exceptional control performance by minimizing the discrepancies between the DMPC control inputs and predesigned platoon control inputs,while ensuring constraint compliance and cybersecurity.Simulation results verify the effectiveness of our theoretical findings.

Human-Machine Shared Lateral Control Strategy for Intelligent Vehicles Based on Human Driver Risk Perception Reliability

Intelligent vehicle(Ⅳ)technology has developed rapidly in recent years.However,achieving fully unmanned driving still presents numerous challenges,which means that human drivers will continue to play a vital role in vehicle operation for the foreseeable future.Human-machine shared driving,involving cooperation between a human driver and an automated driving system(AVS),has been widely regarded as a necessary stage for the development of IVs.Focusing onⅣdriving safety,this study proposed a human-machine shared lateral control strategy(HSLCS)based on the reliability of driver risk perception.The HSLCS starts by identifying the effective areas of driver risk perception based on eye movements.It establishes an anisotropic driving risk field,which serves as the foundation for the AVS to assess risk levels.Building upon the cumulative and diminishing effects of risk perception,the proposed approach leverages the driver's risk perception effective area and converts the risk field into a representation aligned with the driver's perspective.Subsequently,it quantifies the reliability of the driver's risk perception by using area-matching rules.Finally,based on the driver’s risk perception reliability and dif-ferences in lateral driving operation between the human driver and the AVS,the dynamic distribution of driving authority is achieved through a fuzzy rule-based system,and the human-machine shared lateral control is completed by using model predictive control.The HSLCS was tested across various scenarios on a driver-in-the-loop test platform.The results show that the HSLCS can realize the synergy and complementarity of human and machine intelligence,effectively ensuring the safety ofⅣoperation.

Heterogeneous Task Allocation Model and Algorithm for Intelligent Connected Vehicles

With the development of vehicles towards intelligence and connectivity,vehicular data is diversifying and growing dramatically.A task allocation model and algorithm for heterogeneous Intelligent Connected Vehicle(ICV)applications are proposed for the dispersed computing network composed of heterogeneous task vehicles and Network Computing Points(NCPs).Considering the amount of task data and the idle resources of NCPs,a computing resource scheduling model for NCPs is established.Taking the heterogeneous task execution delay threshold as a constraint,the optimization problem is described as the problem of maximizing the utilization of computing resources by NCPs.The proposed problem is proven to be NP-hard by using the method of reduction to a 0-1 knapsack problem.A many-to-many matching algorithm based on resource preferences is proposed.The algorithm first establishes the mutual preference lists based on the adaptability of the task requirements and the resources provided by NCPs.This enables the filtering out of un-schedulable NCPs in the initial stage of matching,reducing the solution space dimension.To solve the matching problem between ICVs and NCPs,a new manyto-many matching algorithm is proposed to obtain a unique and stable optimal matching result.The simulation results demonstrate that the proposed scheme can improve the resource utilization of NCPs by an average of 9.6%compared to the reference scheme,and the total performance can be improved by up to 15.9%.

Design and Implementation of a Battery Big Data Platform Through Intelligent Connected Electric Vehicles

The development of a battery management algorithm is highly dependent on high-quality battery operation data,especially the data in extreme conditions such as low temperatures.The data in faults are also essential for failure and safety management research.This study developed a battery big data platform to realize vehicle operation,energy interaction and data management.First,we developed an electric vehicle with vehicle navigation and position detection and designed an environmental cabin that allows the vehicle to operate autonomously.Second,charging and heating systems based on wireless energy transfer were developed and equipped on the vehicle to investigate optimal charging and heating methods of the batteries in the vehicle.Third,the data transmission network was designed,a real-time monitoring interface was developed,and the self-developed battery management system was used to measure,collect,upload,and store battery operation data in real time.Finally,experimental validation was performed on the platform.Results demonstrate the efficiency and reliability of the platform.Battery state of charge estimation is used as an example to illustrate the availability of battery operation data.

A digital twins enabled underwater intelligent internet vehicle path planning system via reinforcement learning and edge computing

The Autonomous Underwater Glider(AUG)is a kind of prevailing underwater intelligent internet vehicle and occupies a dominant position in industrial applications,in which path planning is an essential problem.Due to the complexity and variability of the ocean,accurate environment modeling and flexible path planning algorithms are pivotal challenges.The traditional models mainly utilize mathematical functions,which are not complete and reliable.Most existing path planning algorithms depend on the environment and lack flexibility.To overcome these challenges,we propose a path planning system for underwater intelligent internet vehicles.It applies digital twins and sensor data to map the real ocean environment to a virtual digital space,which provides a comprehensive and reliable environment for path simulation.We design a value-based reinforcement learning path planning algorithm and explore the optimal network structure parameters.The path simulation is controlled by a closed-loop model integrated into the terminal vehicle through edge computing.The integration of state input enriches the learning of neural networks and helps to improve generalization and flexibility.The task-related reward function promotes the rapid convergence of the training.The experimental results prove that our reinforcement learning based path planning algorithm has great flexibility and can effectively adapt to a variety of different ocean conditions.

Development status and prospect of cold chain logistics of fruits,vegetables,and agricultural products based on intelligent technology

The cold chain in the production area of fruits and vegetables is the primary link to reduce product loss and improve product quality,but it is also a weak link.With the application of big data technology in cold chain logistics,intelligent devices,and technologies have become important carriers for improving the efficiency of cold chain logistics in fruit and vegetable production areas,extending the shelf life of fruits and vegetables,and reducing fruit and vegetable losses.They have many advantages in fruit and vegetable pre-cooling,sorting and packaging,testing,warehousing,transportation,and other aspects.This article summarizes the rapidly developing and widely used intelligent technologies at home and abroad in recent years,including automated guided vehicle intelligent handling based on electromagnetic or optical technology,intelligent sorting based on sensors,electronic optics,and other technologies,intelligent detection based on computer vision technology,intelligent transportation based on perspective imaging technology,etc.It analyses and studies the innovative research and achievements of various scholars in applying intelligent technology in fruit and vegetable cold chain storage,sorting,detection,transportation,and other links,and improves the efficiency of fruit and vegetable cold chain logistics.However,applying intelligent technology in fruit and vegetable cold chain logistics also faces many problems.The challenges of high cost,difficulty in technological integration,and talent shortages have limited the development of intelligent technology in the field of fruit and vegetable cold chains.To solve the current problems,it is proposed that costs be controlled through independent research and development,technological innovation,and other means to lower the entry threshold for small enterprises.Strengthen integrating intelligent technology and cold chain logistics systems to improve data security and system compatibility.At the same time,the government should introduce relevant policies,provide necessary financial support,and establish talent training mechanisms.Accelerate the development and improvement of intelligent technology standards in the field of cold chain logistics.Through technological innovation,cost control,talent cultivation,and policy guidance,we aim to promote the upgrading of the agricultural industry and provide ideas for improving the quality and efficiency of fruit and vegetable cold chain logistics.

Attacks and defences on intelligent connected vehicles:a survey

Intelligent vehicles are advancing at a fast speed with the improvement of automation and connectivity,which opens up new possibilities for different cyber-attacks,including in-vehicle attacks(e.g.,hijacking attacks)and vehicle-to-everything communicationattacks(e.g.,data theft).These problems are becoming increasingly serious with the development of 4G LTE and 5G communication technologies.Although many efforts are made to improve the resilience to cyber attacks,there are still many unsolved challenges.This paper first identifies some major security attacks on intelligent connected vehicles.Then,we investigate and summarize the available defences against these attacks and classify them into four categories:cryptography,network security,software vulnerability detection,and malware detection.Remaining challenges and future directions for preventing attacks on intelligent vehicle systems have been discussed as well.

Autonomous parking control for intelligent vehicles based on a novel algorithm

Along with the increasing number of vehicles, parking space becomes narrow gradually, safety parking puts forward higher requirements on the driver＇s driving technology. How to safely, quickly and accurately park the vehiclo to parking space right？ This paper presents an automatic parking scheme based on trajectory planning, which analyzing the mechanical model oftbe vehicle, establishing vehicle steering model and parking model, coming to the conclusion that it is the turning radius is independent of the vehicle speed at low speed. The Matlab simulation environment verifies the correctness and effectiveness of the proposed algorithm for parking. A class of the automatic parking problem of intelligent vehicles is solved.

Off-road testing scenario design and library generation for intelligent vehicles

To realize the widespread application and continuous functional development of intelligent vehicles,test and evaluation of vehicle's functionality and Safety Performance in complex off-road scenarios are fundamental.Since traditional distance-based road tests cannot meet the evolving test requirements,a method to design the function-based off-road testing scenario library for intelligent vehicles(IV)is proposed in this paper.The testing scenario library is defined as a critical set of scenarios that can be used for IV tests.First,for the complex and diverse off-road scenarios,a hierarchical,structural model of the test scenario is built.Then,the critical test scenarios are selected adaptively according to the vehicle model to be tested.Next,those parameters representing the challenging test scenarios are selected.The selected parameters need to fit the natural distribution probability of scenarios.The critical test-scenario library is built combing these parameters with the structural model.Finally,the test scenarios that are most approximate to the natural driving scenario are determined with importance sampling theory.The test-scenario library built with this method can provide more critical test scenarios,and is widely applicable despite different vehicle models.Verified by simulation in the off-road interaction scenarios,test would be accelerated significantly with this method,about 800 times faster than testing in the natural road environment.

Optimal control methods in intelligent vehicles

In recent years,intelligent vehicles(IVs)have become a hot spot in automotive industry.Key technologies of IVs range over the field of sensing,decision-making and control.Among them,control technology provides an enabling support for improving autonomous driving safety,reducing energy consumption and carbon emission.This paper focuses on some aspects of applying advanced control methodologies in IVs through several selected examples including eco-driving and MPC-based driver modelling.

A Ray Tracing and Joint Spectrum Based Clustering and Tracking Algorithm for Internet of Intelligent Vehicles

Driven by the rapid growth in information services provided by the Internet and the appearance of new multimedia applications,millimeter wave is foreseen as a key enabler towards the Internet of intelligent vehicles(IoIV)for urban traffic safety enhancement.In this regard,cluster-based channel modeling has become an important research topic in the realm of emergency communications.To fully understand the cluster-based channel model,a series of vehicle-to-infrastructure(V2I)channel simulations at 22.6 GHz are conducted by a three-dimensional ray tracing(RT)simulator.The clustering and tracking algorithm is proposed and analyzed from three aspects by the obtained simulation results.The multiple signal classification estimation spectrum is applied to restrain the influence of antenna sidelobes and identify targets at first.Based on the fundamentals,the clusters can be identified and subsequently tracked using the proposed approach.The impacts of antenna sidelobes,angle resolution of beam rotation,and non-line-of-sight propagation path on the performance of clustering and tracking are evaluated.The multi-component-level RT results are adopted as comparison benchmarks,which reflect the ground truth.This work aims to provide a full picture of the clustering characteristics for designing and analyzing emergency communication systems.

Cloud Control System Architectures,Technologies and Applications on Intelligent and Connected Vehicles:a Review

The electrification of vehicle helps to improve its operation efficiency and safety.Due to fast development of network,sensors,as well as computing technology,it becomes realizable to have vehicles driving autonomously.To achieve autonomous driving,several steps,including environment perception,path-planning,and dynamic control,need to be done.However,vehicles equipped with on-board sensors still have limitations in acquiring necessary environmental data for optimal driving decisions.Intelligent and connected vehicles(ICV)cloud control system(CCS)has been introduced as a new concept as it is a potentially synthetic solution for high level automated driving to improve safety and optimize traffic flow in intelligent transportation.This paper systematically investigated the concept of cloud control system from cloud related applications on ICVs,and cloud control system architecture design,as well as its core technologies development.Based on the analysis,the challenges and suggestions on cloud control system development have been addressed.

Big Wave of the Intelligent Connected Vehicles

Internet of things is deemed as the one of the great revolution after the age of Industrial Revolution.With the development of the communication technology,more and more entities are connected to the communication network and become one of the elements in the network.Over recent decades,in the area of intelligent transportation,pedestrian and transport infrastructure are connected to the communication network to improve the driving safety and traffic efficiency which is known as the ICV(Intelligent Connected Vehicle).This paper summarizes the global ICV progresses in the past decades and the latest activities of ICV in China,and introduces various aspects regarding the recent development of the ICV,including industry development,spectrum and standard,at the same time.

Key points of investigation and analysis on traffic accidents involving intelligent vehicles

Intellectualization is regarded as the future mainstream development trend of the automobile industry.The automation level of intelligent vehicles is relatively low so far,and the road traffic system will be in a mixed state of non-autonomous vehicles and vehicles with different levels of automation for a long time.Therefore,the road traffic system will be more complex with more diverse accidents.This paper analysed the characteristics and causal factors of intelligent vehicle accidents.Based on the problems existing in investigation,analysis and liability identification of intelligent vehicle accident,the study proposed a preliminary accident investigation framework and method,summarized the key points of accident analysis from the perspectives of technical defects,information security and passive safety,and specified the liability subjects for intelligent vehicle accidents and their corresponding legal liability.The results from this study contributed to the development of intelligent vehicle accident investigation and disposal,and provided the reference for the improvement of vehicle safety and accident prevention.

Cognitive Granular-Based Path Planning and Tracking for Intelligent Vehicle with Multi-Segment Bezier Curve Stitching

Unmanned vehicles are currently facing many difficulties and challenges in improving safety performance when running in complex urban road traffic environments,such as low intelligence and poor comfort perfor-mance in the driving process.The real-time performance of vehicles and the comfort requirements of passengers in path planning and tracking control of unmanned vehicles have attracted more and more attentions.In this paper,in order to improve the real-time performance of the autonomous vehicle planning module and the comfort requirements of passengers that a local granular-based path planning method and tracking control based on multi-segment Bezier curve splicing and model predictive control theory are pro-posed.Especially,the maximum trajectory curvature satisfying ride comfort is regarded as an important constraint condition,and the corresponding curvature threshold is utilized to calculate the control points of Bezier curve.By using low-order interpolation curve splicing,the planning computation is reduced,and the real-time performance of planning is improved,com-pared with one-segment curve fitting method.Furthermore,the comfort performance of the planned path is reflected intuitively by the curvature information of the path.Finally,the effectiveness of the proposed control method is verified by the co-simulation platform built by MATLAB/Simulink and Carsim.The simulation results show that the path tracking effect of multi-segment Bezier curve fitting is better than that of high-order curve planning in terms of real-time performance and comfort.

Mass estimation method for intelligent vehicles based on fusion of machine learning and vehicle dynamic model

Vehicle mass is an important parameter for motion control of intelligent vehicles,but is hard to directly measure using normal sensors.Therefore,accurate estimation of vehicle mass becomes crucial.In this paper,a vehicle mass estimation method based on fusion of machine learning and vehicle dynamic model is introduced.In machine learning method,a feedforward neural network(FFNN)is used to learn the relationship between vehicle mass and other state parameters,namely longitudinal speed and acceleration,driving or braking torque,and wheel angular speed.In dynamics-based method,recursive least square(RLS)with forgetting factor based on vehicle dynamic model is used to estimate the vehicle mass.According to the reliability of each method under different conditions,these two methods are fused using fuzzy logic.Simulation tests under New European Driving Cycle(NEDC)condition are carried out.The simulation results show that the estimation accuracy of the fusion method is around 97%,and that the fusion method performs better stability and robustness compared with each single method.

Hierarchical CNNPID Based Active Steering Control Method for Intelligent Vehicle Facing Emergency Lane-Changing

To resolve the response delay and overshoot problems of intelligent vehicles facing emergency lane-changing due to proportional-integral-differential(PID)parameter variation,an active steering control method based on Convolutional Neural Network and PID(CNNPID)algorithm is constructed.First,a steering control model based on normal distribution probability function,steady constant radius steering,and instantaneous lane-change-based active for straight and curved roads is established.Second,based on the active steering control model,a three-dimensional constraint-based fifth-order polynomial equation lane-change path is designed to address the stability problem with supersaturation and sideslip due to emergency lane changing.In addition,a hierarchical CNNPID Controller is constructed which includes two layers to avoid collisions facing emergency lane changing,namely,the lane change path tracking PID control layer and the CNN control performance optimization layer.The scaled conjugate gradient backpropagation-based forward propagation control law is designed to optimize the PID control performance based on input parameters,and the elastic backpropagation-based module is adopted for weight correction.Finally,comparison studies and simulation/real vehicle test results are presented to demonstrate the effectiveness,significance,and advantages of the proposed controller.

Simulation and Field Testing of Multiple Vehicles Collision Avoidance Algorithms

A global planning algorithm for intelligent vehicles is designed based on the A* algorithm, which provides intelligent vehicles with a global path towards their destinations. A distributed real-time multiple vehicle collision avoidance(MVCA)algorithm is proposed by extending the reciprocal n-body collision avoidance method. MVCA enables the intelligent vehicles to choose their destinations and control inputs independently,without needing to negotiate with each other or with the coordinator. Compared to the centralized trajectory-planning algorithm, MVCA reduces computation costs and greatly improves the robustness of the system. Because the destination of each intelligent vehicle can be regarded as private, which can be protected by MVCA, at the same time MVCA can provide a real-time trajectory planning for intelligent vehicles. Therefore,MVCA can better improve the safety of intelligent vehicles. The simulation was conducted in MATLAB, including crossroads scene simulation and circular exchange position simulation. The results show that MVCA behaves safely and reliably. The effects of latency and packet loss on MVCA are also statistically investigated through theoretically formulating broadcasting process based on one-dimensional Markov chain. The results uncover that the tolerant delay should not exceed the half of deciding cycle of trajectory planning, and shortening the sending interval could alleviate the negative effects caused by the packet loss to an extent. The cases of short delay(< 100100 ms) and low packet loss(< 5%) can bring little influence to those trajectory planning algorithms that only depend on V2 V to sense the context, but the unpredictable collision may occur if the delay and packet loss are further worsened. The MVCA was also tested by a real intelligent vehicle, the test results prove the operability of MVCA.

Traffic light detection and recognition in intersections based on intelligent vehicle

To ensure revulsive driving of intelligent vehicles at intersections, a method is presented to detect and recognize the traffic lights. First, the stabling siding at intersections is detected by applying Hough transformation. Then, the colors of traffic lights are detected with color space transformation. Finally, self-associative memory is used to recognize the countdown characters of the traffic lights. Test results at 20 real intersections show that the ratio of correct stabling siding recognition reaches up to 90%;and the ratios of recognition of traffic lights and divided characters are 85% and 97%, respectively. The research proves that the method is efficient for the detection of stabling siding and is robust enough to recognize the characters from images with noise and broken edges.