Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

Human-Like Decision-Making of Autonomous Vehicles in Dynamic Traffic Scenarios

With the maturation of autonomous driving technology, the use of autonomous vehicles in a socially acceptable manner has become a growing demand of the public. Human-like autonomous driving is expected due to the impact of the differences between autonomous vehicles and human drivers on safety.Although human-like decision-making has become a research hotspot, a unified theory has not yet been formed, and there are significant differences in the implementation and performance of existing methods. This paper provides a comprehensive overview of human-like decision-making for autonomous vehicles. The following issues are discussed: 1) The intelligence level of most autonomous driving decision-making algorithms;2) The driving datasets and simulation platforms for testing and verifying human-like decision-making;3) The evaluation metrics of human-likeness;personalized driving;the application of decisionmaking in real traffic scenarios;and 4) The potential research direction of human-like driving. These research results are significant for creating interpretable human-like driving models and applying them in dynamic traffic scenarios. In the future, the combination of intuitive logical reasoning and hierarchical structure will be an important topic for further research. It is expected to meet the needs of human-like driving.

Modeling and TOPSIS-GRA Algorithm for Autonomous Driving Decision-Making Under 5G-V2X Infrastructure

This paper is to explore the problems of intelligent connected vehicles(ICVs)autonomous driving decision-making under a 5G-V2X structured road environment.Through literature review and interviews with autonomous driving practitioners,this paper firstly puts forward a logical framework for designing a cerebrum-like autonomous driving system.Secondly,situated on this framework,it builds a hierarchical finite state machine(HFSM)model as well as a TOPSIS-GRA algorithm for making ICV autonomous driving decisions by employing a data fusion approach between the entropy weight method(EWM)and analytic hierarchy process method(AHP)and by employing a model fusion approach between the technique for order preference by similarity to an ideal solution(TOPSIS)and grey relational analysis(GRA).The HFSM model is composed of two layers:the global FSM model and the local FSM model.The decision of the former acts as partial input information of the latter and the result of the latter is sent forward to the local pathplanning module,meanwhile pulsating feedback to the former as real-time refresh data.To identify different traffic scenarios in a cerebrum-like way,the global FSM model is designed as 7 driving behavior states and 17 driving characteristic events,and the local FSM model is designed as 16 states and 8 characteristic events.In respect to designing a cerebrum-like algorithm for state transition,this paper firstly fuses AHP weight and EWM weight at their output layer to generate a synthetic weight coefficient for each characteristic event;then,it further fuses TOPSIS method and GRA method at the model building layer to obtain the implementable order of state transition.To verify the feasibility,reliability,and safety of theHFSMmodel aswell as its TOPSISGRA state transition algorithm,this paper elaborates on a series of simulative experiments conducted on the PreScan8.50 platform.The results display that the accuracy of obstacle detection gets 98%,lane line prediction is beyond 70 m,the speed of collision avoidance is higher than 45 km/h,the distance of collision avoidance is less than 5 m,path planning time for obstacle avoidance is averagely less than 50 ms,and brake deceleration is controlled under 6 m/s2.These technical indexes support that the driving states set and characteristic events set for the HFSM model as well as its TOPSIS-GRA algorithm may bring about cerebrum-like decision-making effectiveness for ICV autonomous driving under 5G-V2X intelligent road infrastructure.

UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

AUTONOMOUS AGENT FRAMEWORK AND ITS DECISION-MAKING

Autonomy, a key property associated with the agent, is an important topic in the current research of the agent theory. Although no definition of the agent autonomy is universally accepted, an important aspect of the agent autonomy is the decision-making capability of the agents. This paper investigates the autonomy of the agent, presents a framework for autonomous agent and discusses its decision-making process. Started with introducing a language for representing autonomous agent, a framework is proposed for modeling autonomous agent based on a BDI model and the situation calculus. Finally, a kind of decision-making process of the autonomous agent is presented.

Planning and Decision-making for Connected Autonomous Vehicles at Road Intersections:A Review

Planning and decision-making technology at intersections is a comprehensive research problem in intelligent transportation systems due to the uncertainties caused by a variety of traffic participants.As wireless communication advances,vehicle infrastructure integrated algorithms designed for intersection planning and decision-making have received increasing attention.In this paper,the recent studies on the planning and decision-making technologies at intersections are primarily overviewed.The general planning and decision-making approaches are presented,which include graph-based approach,prediction base approach,optimization-based approach and machine learning based approach.Since connected autonomous vehicles(CAVs)is the future direction for the automated driving area,we summarized the evolving planning and decision-making methods based on vehicle infrastructure cooperative technologies.Both four-way signalized and unsignalized intersection(s)are investigated under purely automated driving traffic and mixed traffic.The study benefit from current strategies,protocols,and simulation tools to help researchers identify the presented approaches’challenges and determine the research gaps,and several remaining possible research problems that need to be solved in the future.

Probabilistic Lane-Change Decision-Making and Planning for Autonomous Heavy Vehicles

To improve the safety and driving stability of the autonomous heavy truck, it is necessary to consider the differences of driving behavior and drivable trajectories between the heavy trucks and passenger cars. This study proposes a probabilistic decision-making and trajectory planning framework for the autonomous heavy trucks. Firstly, the driving decision process is divided into intention generation and feasibility evaluations, which are realized using the utility theory and risk assessment, respectively. Subsequently the driving decision is made and sent to the trajectory planning module. In order to reflect the greater risks of the truck to other surrounding vehicles, the aggressiveness index(AI) is proposed and quantified to infer the asymmetrical risk level of lane-change maneuver. In the planning stage, the lateral and roll dynamics stability domains are developed as the constraints to exclude the candidate trajectories that would cause vehicle instability. Finally, the simulation results are compared between the proposed model and the artificial potential filed model in the scenarios extracted from the naturalistic driving data. It is shown that the proposed framework can provide the human-like lane-change decisions and truck-friendly trajectories, and performs well in dynamic driving environments.

Driving decision-making analysis of car-following for autonomous vehicle under complex urban environment

The decision-making under complex urban environment become one of the key issues that restricts the rapid development of the autonomous vehicles. The difficulty in making timely and accurate decisions like human beings under highly dynamic traffic environment is a major challenge for autonomous driving. Car-following has been regarded as the simplest but essential driving behavior among driving tasks and has received extensive attention from researchers around the world. This work addresses this problem and proposes a novel method RSAN(rough-set artificial neural network) to learn the decisions from excellent human drivers. A virtual urban traffic environment was built by Pre Scan and driving simulation was conducted to obtain a broad set of relevant data such as experienced drivers' behavior data and surrounding vehicles' motion data. Then, rough set was used to preprocess these data to extract the key influential factors on decision and reduce the impact of uncertain data and noise data. And the car-following decision was learned by neural network in which key factor was the input and acceleration was the output. The result shows the better convergence speed and the better decision accuracy of RSAN than ANN. Findings of this work contributes to the empirical understanding of driver's decision-making process and it provides a theoretical basis for the study of car-following decision-making under complex and dynamic environment.

The Impact of Online Courses towards Students in Autonomous Learning Based on New Media Technologies

In the 21st century,the rapid development of online technology has dramatically transformed people’s way of lives.The emergence of high-tech products has also boosted modern education to embrace informationization and virtualization.With the promotion and development of online courses,autonomous learning is now emerging among students in colleges and universities.If they want to learn relevant professional knowledge,they could use networking and information technology with relevant devices.This learning method could not only impact traditional education but also facilitate students to explore new ways to learn autonomously.This paper is to discuss the impact of online courses towards students in autonomous learning by analyzing its current learning situation,the feature of this new form and its effects towards students.

Metacognitive Ability and its Relationship with CET4 Score——A Study on Science Students under Online College English Autonomous Learning

By using 162 third-year science students from the Independent College in Shandong University of Science and Tech nology,this paper investigated the relationship between their metacognitive ability and their CET4 score.The results indicated that their metacognitive ability,and the three subcategories have positive significant correlations with the students'CET4 score.

Research on the Online College English Teaching Mode from the Perspective of Autonomous Learning

In order to explore the online college English teaching mode from the perspective of autonomous learning and facilitate college students to better learn English independently as well as to master their English application skills,this article expounds the significance of online English teaching through theoretical analysis.At the same time,it expounds how to realize online English teaching in the perspective of autonomous learning,so as to improve the quality of English teaching and students’English skills.

Exploring the Attractiveness of Companies Branding D&I to Potential Employees in Online Recruitment

Concrete action on diversity and inclusion(D&I)is critical to companies’value creation and brand.Companies that intentionally fail to attract and retain a diverse talent pool will struggle to innovate,compete,and prosper.The advantages of a diverse and inclusive workplace include the reduction of homogenous thinking and an increase in novel ideas and perspectives due to the wealth of talent from diverse backgrounds.Previous studies have focused on the benefits of corporate D&I from a corporate perspective;however,there is scant research on the impact of corporate D&I from employees’perspectives.Resultingly,this study,through an online questionnaire,proposes to examine the impact of employer brand D&I on potential employees in online recruitment channels from the perspective of potential employees.The study results indicate that most Chinese companies still exhibit low levels of D&I during recruitment.This study also included an empirical analysis of 267 valid samples,which showed that:(1)the D&I of employer brand perceptions positively impact job search intentions,behaviour,and decision making.(2)Companies’D&I has the most significant influence on job seekers’decision making,followed by behavioural influence and cognitive influence.(3)The type of job seeker plays a moderating role in the relationship between employer brand perception and job search intention,behaviour,and decision making.(4)Online recruitment’s increasing popularity amplifies employer branding’s effect.Based on the findings,this study suggests how companies can use D&I branding strategies to attract quality talent and increase job seekers’job search intentions.

Cooperative Decision-Making for Multiple UAVs Autonomous Confrontation

This paper presents a rule-based framework for addressing decision-making problems within the context of the\UI-STRIVE"Competition.First,two distinct autonomous confrontation scenarios are described:autonomous air combat and cooperative interception.Second,a State-Event-Condition-Action(SECA)decision-making framework is developed,which integrates thefinite state machine and event-condition-action frameworks.This framework provides three products to describe rules,i.e.the SECA model,the SECA state chart,and the SECA rule description.Third,the situation assessment and target assignment during autonomous air combat are investigated,and the mathematical models are established.Finally,the decisionmaking model's rationality and feasibility are verified through data simulation and analysis.

An efficient and impartial online algorithm for kidney assignment network

An online algorithm balancing the efficiency and equity principles is proposed for the kidney resource assignment when only the current patient and resource information is known to the assignment network. In the algorithm, the assignment is made according to the priority, which is calculated according to the efficiency principle and the equity principle. The efficiency principle is concerned with the post-transplantation immunity spending caused by the possible post-operation immunity rejection and patient’s mental depression due to the HLA mismatch. The equity principle is concerned with three other factors, namely the treatment spending incurred starting from the day of registering with the kidney assignment network, the post-operation immunity spending and the negative effects of waiting for kidney resources on the clinical efficiency. The competitive analysis conducted through computer simulation indicates that the efficiency competitive ratio is between 6.29 and 10.43 and the equity competitive ratio is between 1.31 and 5.21, demonstrating that the online algorithm is of great significance in application.

Experimental Tests of Autonomous Ground Vehicles with Preview

This paper describes the design and experimental tests of a path planning and reference tracking algorithm for autonomous ground vehicles. The ground vehicles under consideration are equipped with forward looking sensors that provide a preview capability over a certain horizon. A two-level control framework is proposed for real-time implementation of the model predictive control （MPC） algorithm, where the high-level performs on-line optimization to generate the best possible local reference respect to various constraints and the low-level commands the vehicle to follow realistic trajectories generated by the high-level controller. The proposed control scheme is implemented on an indoor testbed through networks with satisfactory performance.

Business Plan for Autonomous Delivery Robot

This paper introduces an autonomous robot (AR) cart to execute the last mile delivery task. We use navigation and intelligent avoidance algorithms to plan the path of the automatic robot. When AR encounters a new unrecognizable terrain, it will give control to the customer who can control the AR on its mobile app and navigate to the specified destination. We have initially designed an autonomous delivery robot with the cost of 2774 dollars.

Eighteen Excellent Masterpieces Having Online Blooming in the World

From February 4 to February 26,2021 "Happy Chinese New Year"online eve nt was held in the cloud.Headquarters of China Cultural Centers and Tourism Offices(under preparation)together with the institutions of the Ministry of Cultural and Tourism,the departments of culture and tourism of the provinces,municipalities and autonomous regions elaborately designed and produced a series of themed online cultural and tourist products rich in content and form.

An Overview of Autonomous Learning and Reflections on EFL Teaching

This paper mainly focuses on the development of autonomous learning worldwide and its reflections on English teachingin China.Autonomous learning,which is beneficial to one’s lifelong journey,is the ability to take responsibility for one’s learning.The final goal of education is to cultivate students with autonomous learning skills.As for EFL teachers,three steps are as followed.Firstly,EFL teachers need to be trained to learn autonomously.Secondly,EFL teachers should construct some online autonomouslearning platforms.Thirdly,EFL teachers need to motivate learners to listen and watch more about English to arouse students’ in-terests in the target language.

线上线下融合背景下高职学生自主学习模式研究

随着互联网和信息技术的快速发展,线上线下融合的教学模式在高职教育中得到了广泛应用,而自主学习可以理解为学习方法和学习态度的表现形式,不仅能够提升学生的学习效果,还可以不断增强学生的学习能力。在线上线下融合背景下,高职学生在自主学习模式方面表现出较高的自觉和参与度,自主学习环境的构建和自主学习资源的选择和开发对于学生的自主学习能力的提高起到了积极的作用。基于此,针对线上线下融合背景下高职学生自主学习模式进行分析,并提出了一些改进和优化的建议,包括加强学生自主学习能力培养、提供更多的指导和支持、加强线上线下教学环节的融合等。这些建议有助于提高高职学生的学习效果和学习质量,为高职教育的发展和改革提供参考。