Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

Human-Like Decision-Making of Autonomous Vehicles in Dynamic Traffic Scenarios

With the maturation of autonomous driving technology, the use of autonomous vehicles in a socially acceptable manner has become a growing demand of the public. Human-like autonomous driving is expected due to the impact of the differences between autonomous vehicles and human drivers on safety.Although human-like decision-making has become a research hotspot, a unified theory has not yet been formed, and there are significant differences in the implementation and performance of existing methods. This paper provides a comprehensive overview of human-like decision-making for autonomous vehicles. The following issues are discussed: 1) The intelligence level of most autonomous driving decision-making algorithms;2) The driving datasets and simulation platforms for testing and verifying human-like decision-making;3) The evaluation metrics of human-likeness;personalized driving;the application of decisionmaking in real traffic scenarios;and 4) The potential research direction of human-like driving. These research results are significant for creating interpretable human-like driving models and applying them in dynamic traffic scenarios. In the future, the combination of intuitive logical reasoning and hierarchical structure will be an important topic for further research. It is expected to meet the needs of human-like driving.

Modeling and TOPSIS-GRA Algorithm for Autonomous Driving Decision-Making Under 5G-V2X Infrastructure

This paper is to explore the problems of intelligent connected vehicles(ICVs)autonomous driving decision-making under a 5G-V2X structured road environment.Through literature review and interviews with autonomous driving practitioners,this paper firstly puts forward a logical framework for designing a cerebrum-like autonomous driving system.Secondly,situated on this framework,it builds a hierarchical finite state machine(HFSM)model as well as a TOPSIS-GRA algorithm for making ICV autonomous driving decisions by employing a data fusion approach between the entropy weight method(EWM)and analytic hierarchy process method(AHP)and by employing a model fusion approach between the technique for order preference by similarity to an ideal solution(TOPSIS)and grey relational analysis(GRA).The HFSM model is composed of two layers:the global FSM model and the local FSM model.The decision of the former acts as partial input information of the latter and the result of the latter is sent forward to the local pathplanning module,meanwhile pulsating feedback to the former as real-time refresh data.To identify different traffic scenarios in a cerebrum-like way,the global FSM model is designed as 7 driving behavior states and 17 driving characteristic events,and the local FSM model is designed as 16 states and 8 characteristic events.In respect to designing a cerebrum-like algorithm for state transition,this paper firstly fuses AHP weight and EWM weight at their output layer to generate a synthetic weight coefficient for each characteristic event;then,it further fuses TOPSIS method and GRA method at the model building layer to obtain the implementable order of state transition.To verify the feasibility,reliability,and safety of theHFSMmodel aswell as its TOPSISGRA state transition algorithm,this paper elaborates on a series of simulative experiments conducted on the PreScan8.50 platform.The results display that the accuracy of obstacle detection gets 98%,lane line prediction is beyond 70 m,the speed of collision avoidance is higher than 45 km/h,the distance of collision avoidance is less than 5 m,path planning time for obstacle avoidance is averagely less than 50 ms,and brake deceleration is controlled under 6 m/s2.These technical indexes support that the driving states set and characteristic events set for the HFSM model as well as its TOPSIS-GRA algorithm may bring about cerebrum-like decision-making effectiveness for ICV autonomous driving under 5G-V2X intelligent road infrastructure.

Research on Maneuver Decision-Making of Multi-Agent Adversarial Game in a Random Interference Environment

The strategy evolution process of game players is highly uncertain due to random emergent situations and other external disturbances.This paper investigates the issue of strategy interaction and behavioral decision-making among game players in simulated confrontation scenarios within a random interference environment.It considers the possible risks that random disturbances may pose to the autonomous decision-making of game players,as well as the impact of participants’manipulative behaviors on the state changes of the players.A nonlinear mathematical model is established to describe the strategy decision-making process of the participants in this scenario.Subsequently,the strategy selection interaction relationship,strategy evolution stability,and dynamic decision-making process of the game players are investigated and verified by simulation experiments.The results show that maneuver-related parameters and random environmental interference factors have different effects on the selection and evolutionary speed of the agent’s strategies.Especially in a highly uncertain environment,even small information asymmetry or miscalculation may have a significant impact on decision-making.This also confirms the feasibility and effectiveness of the method proposed in the paper,which can better explain the behavioral decision-making process of the agent in the interaction process.This study provides feasibility analysis ideas and theoretical references for improving multi-agent interactive decision-making and the interpretability of the game system model.

UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

AUTONOMOUS AGENT FRAMEWORK AND ITS DECISION-MAKING

Autonomy, a key property associated with the agent, is an important topic in the current research of the agent theory. Although no definition of the agent autonomy is universally accepted, an important aspect of the agent autonomy is the decision-making capability of the agents. This paper investigates the autonomy of the agent, presents a framework for autonomous agent and discusses its decision-making process. Started with introducing a language for representing autonomous agent, a framework is proposed for modeling autonomous agent based on a BDI model and the situation calculus. Finally, a kind of decision-making process of the autonomous agent is presented.

Decision-Making and Management of Self-Care in Persons with Traumatic Spinal Cord Injuries: A Preliminary Study

Patients and physicians understand the importance of self-care following spinal cord injury (SCI), yet many individuals with SCI do not adhere to recommended self-care activities despite logistical supports. Neurobehavioral determinants of SCI self-care behavior, such as impulsivity, are not widely studied, yet understanding them could inform efforts to improve SCI self-care. We explored associations between impulsivity and self-care in an observational study of 35 US adults age 18 - 50 who had traumatic SCI with paraplegia at least six months before assessment. The primary outcome measure was self-reported self-care. In LASSO regression models that included all neurobehavioral measures and demographics as predictors of self-care, dispositional measures of greater impulsivity (negative urgency, lack of premeditation, lack of perseverance), and reduced mindfulness were associated with reduced self-care. Outcome (magnitude) sensitivity, a latent decision-making parameter derived from computationally modeling successive choices in a gambling task, was also associated with self-care behavior. These results are preliminary;more research is needed to demonstrate the utility of these findings in clinical settings. Information about associations between impulsivity and poor self-care in people with SCI could guide the development of interventions to improve SCI self-care and help patients with elevated risks related to self-care and secondary health conditions.

Behavioral Decision-Making of Key Stakeholders in Public-Private Partnerships:A Hybrid Method and Benefit Distribution Study

Public-private partnerships(PPPs)have been used by governments around the world to procure and construct infrastructural amenities.It relies on private sector expertise and funding to achieve this lofty objective.However,given the uncertainties of project management,transparency,accountability,and expropriation,this phenomenon has gained tremendous attention in recent years due to the important role it plays in curbing infrastructural deficits globally.Interestingly,the reasonable benefit distribution scheme in a PPP project is related to the behavior decisionmaking of the government and social capital,aswell as the performance of the project.In this paper,the government and social capital which are the key stakeholders of PPP projects were selected as the research objects.Based on the fuzzy expected value model and game theory,a hybrid method was adopted in this research taking into account the different risk preferences of both public entities and private parties under the fuzzy demand environment.To alleviate the problem of insufficient utilization of social capital in a PPP project,this paper seeks to grasp the relationship that exists between the benefit distribution of stakeholders,their behavioral decision-making,and project performance,given that they impact the performance of both public entities and private parties,as well as assist in maximizing the overall utility of the project.Furthermore,four game models were constructed in this study,while the expected value and opportunity-constrained programming model for optimal decision-making were derived using alternate perspectives of both centralized decision-making and decentralized decision-making.Afterward,the optimal behavioral decision-making of public entities and private parties in four scenarios was discussed and thereafter compared,which led to an ensuing discussion on the benefit distribution system under centralized decision-making.Lastly,based on an example case,the influence of different confidence levels,price,and fuzzy uncertainties of PPP projects on the equilibrium strategy results of both parties were discussed,giving credence to the effectiveness of the hybrid method.The results indicate that adjusting different confidence levels yields different equilibriumpoints,and therefore signposts that social capital has a fair perception of opportunities,as well as identifies reciprocal preferences.Nevertheless,we find that an increase in the cost coefficient of the government and social capital does not inhibit the effort of both parties.Our results also indicate that a reasonable benefit distribution of PPP projects can assist them in realizing optimum Pareto improvements over time.The results provide us with very useful strategies and recommendations to improve the overall performance of PPP projects in China.

Planning and Decision-making for Connected Autonomous Vehicles at Road Intersections:A Review

Planning and decision-making technology at intersections is a comprehensive research problem in intelligent transportation systems due to the uncertainties caused by a variety of traffic participants.As wireless communication advances,vehicle infrastructure integrated algorithms designed for intersection planning and decision-making have received increasing attention.In this paper,the recent studies on the planning and decision-making technologies at intersections are primarily overviewed.The general planning and decision-making approaches are presented,which include graph-based approach,prediction base approach,optimization-based approach and machine learning based approach.Since connected autonomous vehicles(CAVs)is the future direction for the automated driving area,we summarized the evolving planning and decision-making methods based on vehicle infrastructure cooperative technologies.Both four-way signalized and unsignalized intersection(s)are investigated under purely automated driving traffic and mixed traffic.The study benefit from current strategies,protocols,and simulation tools to help researchers identify the presented approaches’challenges and determine the research gaps,and several remaining possible research problems that need to be solved in the future.

Probabilistic Lane-Change Decision-Making and Planning for Autonomous Heavy Vehicles

To improve the safety and driving stability of the autonomous heavy truck, it is necessary to consider the differences of driving behavior and drivable trajectories between the heavy trucks and passenger cars. This study proposes a probabilistic decision-making and trajectory planning framework for the autonomous heavy trucks. Firstly, the driving decision process is divided into intention generation and feasibility evaluations, which are realized using the utility theory and risk assessment, respectively. Subsequently the driving decision is made and sent to the trajectory planning module. In order to reflect the greater risks of the truck to other surrounding vehicles, the aggressiveness index(AI) is proposed and quantified to infer the asymmetrical risk level of lane-change maneuver. In the planning stage, the lateral and roll dynamics stability domains are developed as the constraints to exclude the candidate trajectories that would cause vehicle instability. Finally, the simulation results are compared between the proposed model and the artificial potential filed model in the scenarios extracted from the naturalistic driving data. It is shown that the proposed framework can provide the human-like lane-change decisions and truck-friendly trajectories, and performs well in dynamic driving environments.

Driving decision-making analysis of car-following for autonomous vehicle under complex urban environment

The decision-making under complex urban environment become one of the key issues that restricts the rapid development of the autonomous vehicles. The difficulty in making timely and accurate decisions like human beings under highly dynamic traffic environment is a major challenge for autonomous driving. Car-following has been regarded as the simplest but essential driving behavior among driving tasks and has received extensive attention from researchers around the world. This work addresses this problem and proposes a novel method RSAN(rough-set artificial neural network) to learn the decisions from excellent human drivers. A virtual urban traffic environment was built by Pre Scan and driving simulation was conducted to obtain a broad set of relevant data such as experienced drivers' behavior data and surrounding vehicles' motion data. Then, rough set was used to preprocess these data to extract the key influential factors on decision and reduce the impact of uncertain data and noise data. And the car-following decision was learned by neural network in which key factor was the input and acceleration was the output. The result shows the better convergence speed and the better decision accuracy of RSAN than ANN. Findings of this work contributes to the empirical understanding of driver's decision-making process and it provides a theoretical basis for the study of car-following decision-making under complex and dynamic environment.

Reinforcement Learning Behavioral Control for Nonlinear Autonomous System

Behavior-based autonomous systems rely on human intelligence to resolve multi-mission conflicts by designing mission priority rules and nonlinear controllers.In this work,a novel twolayer reinforcement learning behavioral control(RLBC)method is proposed to reduce such dependence by trial-and-error learning.Specifically,in the upper layer,a reinforcement learning mission supervisor(RLMS)is designed to learn the optimal mission priority.Compared with existing mission supervisors,the RLMS improves the dynamic performance of mission priority adjustment by maximizing cumulative rewards and reducing hardware storage demand when using neural networks.In the lower layer,a reinforcement learning controller(RLC)is designed to learn the optimal control policy.Compared with existing behavioral controllers,the RLC reduces the control cost of mission priority adjustment by balancing control performance and consumption.All error signals are proved to be semi-globally uniformly ultimately bounded(SGUUB).Simulation results show that the number of mission priority adjustment and the control cost are significantly reduced compared to some existing mission supervisors and behavioral controllers,respectively.

On the role of impulsivity and decision-making in suicidal behavior

Suicide risk constitutes a complex set of interacting demographic, clinical, psychobiological and environmental variables. Impulsivity is a long-known risk factor for suicide attempts. However, research based on clearer conceptual refinement in this area is imperative. One emerging field of study is that of decision-making. Impulsivity involves a failure of higher-order control, including decision-making. Using standardized operational definitions that take into consideration relevant aspects of impulsivity, including state- and trait-components and a deeper understanding of the process of decision-making in the suicidal mind, we may come a step closer to understanding suicidality and winning the fight in this scourge of human suffering.

Summarizing vehicle driving decision-making methods on vulnerable road user collision avoidance

With the development of intelligent vehicles and autonomous driving technology,the safety of vulnerable road user(VRU)in traffic has been more guaranteed,and many research achievements have been made in the key area of collision avoidance decision-making methods.In this paper,the knowledge mapping method is used to mine the available literature in depth,and it is found that the research focus has shifted from the traditional accident cause analysis to emerging deep learning and virtual reality technology.This paper summarizes research on the three core dimensions of environmental perception,behavior cognition and collision avoidance decision-making in intelligent vehicle systems.In terms of perception,accurate identification of pedestrians and cyclists in complex environments is a major demand for VRU perception;in terms of behavior cognition,the coupling of VRU intention identification and motion trajectory prediction and other multiple factors needs further research;in terms of decision-making,the intention identification and trajectory prediction of collision objects are not included in the risk assessment model,and there is a lack of exploration specifically for cyclists'collision risk.On this basis,this paper provides guidance for the improvement of traffic safety of contemporary VRU under the conditions of intelligent and connected transportation.

The Intellectualized Architecture of the Autonomous Micro- Mobile Robot Based- Behavior

Given the difficulty in hand coding task schemes, an intellectualized architecture of the autonomous micro mobile robot based behavior for fault repair was presented. Integrating the reinforcement learning and the group behavior evolution simulating the human's learning and evolution, the autonomous micro mobile robot will automatically generate the suited actions satisfied the environment. However, the designer only devises some basic behaviors, which decreases the workload of the designer and cognitive deficiency of the robot to the environment. The results of simulation have shown that the architecture endows micro robot with the ability of learning, adaptation and robustness, also with the ability of accomplishing the given task.

Cooperative Decision-Making for Multiple UAVs Autonomous Confrontation

This paper presents a rule-based framework for addressing decision-making problems within the context of the\UI-STRIVE"Competition.First,two distinct autonomous confrontation scenarios are described:autonomous air combat and cooperative interception.Second,a State-Event-Condition-Action(SECA)decision-making framework is developed,which integrates thefinite state machine and event-condition-action frameworks.This framework provides three products to describe rules,i.e.the SECA model,the SECA state chart,and the SECA rule description.Third,the situation assessment and target assignment during autonomous air combat are investigated,and the mathematical models are established.Finally,the decisionmaking model's rationality and feasibility are verified through data simulation and analysis.

Tourist behavior patterns in the ecotourism destination in the west of China: A case study of Gannan Tibetan Autonomous Prefecture

This study takes Gannan Tibetan Autonomous Prefecture as the place of case study and tourists as research objects. From the perspectives of geographical distribution of source tourist markets, Tourist activity behavioral and spatial patterns of Tourists, this study looks into the geographical structure of the source tourists and spatial patterns by geography. The analysis of 341 questionnaires on tourists shows that:(1) The tourism cycle of Gannan is in the development phase, competing with adjacent Aba, and greatly impacted by the substitution effect and shadow effect of Aba.(2) The spatial distribution of tourist sources is concentrated, indicating that Gannan is a regional tourism destination. The temporal distance of tourists is mainly concentrated within the 6-hour traffi c circle.(3) Gannan Tibetan Autonomous Prefecture has already become the composite tourist destination dominated by leisure vacation. The minority folkcustom and special landscape are the most attractive tourism resources. Due to the impact of man-land harmonious lifestyle in the tourist areas, the environmental attitude of tourists is improved, and the transportation and shopping are the most vulnerable links in tourism service in Gannan Tibetan Autonomous Prefecture.(4) The spatial behavior of tourists in Gannan is mainly of single-destination style(52%), Transit leg and circle tour style(7%) as well as circle tour style(41%). The spatial distribution of tourist fl ow in Gannan shows a signifi cant feature "more in the north, less in the south and dependent on National Road". Tourism resources, transport facilities, regional competition and lack of route connecting different ecological units are important causes of the spatial distribution of self-help tourists.

A Structural Analysis of Senior High School Students’English Achievement Goal Orientation,Learning Anxiety,and Autonomous Learning Behavior

The present study aimed to investigate senior high school students to explore the relationships among their English achievement goal orientations,learning anxiety,and autonomous learning behavior.748 first-year senior high school students in Guizhou Province,China were selected as participants.A comprehensive questionnaire measuring the above variables was designed to collect the data.The Structural Equation Modeling(SEM)was used to analyze the data.The results showed that the model had good fit to the sample.The students’mastery goals and performance-approach goals positively contributed to their autonomous learning behavior,whereas their performance-avoidance goals were negatively associated with their autonomous learning behavior.The students’mastery goals effectively reduced their learning anxiety,but their performance-approach goals and performance-avoidance goals engendered learning anxiety.The students’learning anxiety and their autonomous learning behavior were negatively correlated.

Factors That Promote Autonomous and Controlled Motivation in Self-management Behavior of Hemodialysis Patients

This study examined the differences and primary factors from the impact of autonomous motivation and controlled motivation on the self-management behavior of hemodialysis patients.Anonymous,self-describing questionnaires were used for research on nine different dialysis facilities of 413 people who regularly visit.From using the primary factor results of multiple regression analysis,that took autonomous motivation and controlled motivation as the dependent variable,a path diagram was created that led to each motivation.The acknowledgement of autonomy support facilitated whether it was autonomous motivation or controlled motivation(The standardized coefficient was 0.385,0.346,p<0.0001).Positive evaluation coping skills were a primary factor that promoted autonomous motivation,while trait anxiety,disorders of social activities,and lack of motivation were primary factors that promoted controlled motivation.In order to raise the autonomous motivation to promote self-management behavior in patients with hemodialysis treatment,situations that easily cause amotivation and anxiety,as well as tendencies for depression should be assessed.Also the encouragement to attain positive evaluation coping skills to support patient autonomy appears to be effective.