Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

AUTONOMOUS AGENT FRAMEWORK AND ITS DECISION-MAKING

Autonomy, a key property associated with the agent, is an important topic in the current research of the agent theory. Although no definition of the agent autonomy is universally accepted, an important aspect of the agent autonomy is the decision-making capability of the agents. This paper investigates the autonomy of the agent, presents a framework for autonomous agent and discusses its decision-making process. Started with introducing a language for representing autonomous agent, a framework is proposed for modeling autonomous agent based on a BDI model and the situation calculus. Finally, a kind of decision-making process of the autonomous agent is presented.

A Synergistic Multi-Attribute Decision-Making Method for Educational Institutions Evaluation Using Similarity Measures of Possibility Pythagorean Fuzzy Hypersoft Sets

Due to the numerous variables to take into account as well as the inherent ambiguity and uncertainty,evaluating educational institutions can be difficult.The concept of a possibility Pythagorean fuzzy hypersoft set(pPyFHSS)is more flexible in this regard than other theoretical fuzzy set-like models,even though some attempts have been made in the literature to address such uncertainties.This study investigates the elementary notions of pPyFHSS including its set-theoretic operations union,intersection,complement,OR-and AND-operations.Some results related to these operations are also modified for pPyFHSS.Additionally,the similarity measures between pPyFHSSs are formulated with the assistance of numerical examples and results.Lastly,an intelligent decision-assisted mechanism is developed with the proposal of a robust algorithm based on similarity measures for solving multi-attribute decision-making(MADM)problems.A case study that helps the decision-makers assess the best educational institution is discussed to validate the suggested system.The algorithmic results are compared with the most pertinent model to evaluate the adaptability of pPyFHSS,as it generalizes the classical possibility fuzzy set-like theoretical models.Similarly,while considering significant evaluating factors,the flexibility of pPyFHSS is observed through structural comparison.

Medical Diagnosis Based on Multi-Attribute Group Decision-Making Using Extension Fuzzy Sets,Aggregation Operators and Basic Uncertainty Information Granule

Accurate medical diagnosis,which involves identifying diseases based on patient symptoms,is often hindered by uncertainties in data interpretation and retrieval.Advanced fuzzy set theories have emerged as effective tools to address these challenges.In this paper,new mathematical approaches for handling uncertainty in medical diagnosis are introduced using q-rung orthopair fuzzy sets(q-ROFS)and interval-valued q-rung orthopair fuzzy sets(IVq-ROFS).Three aggregation operators are proposed in our methodologies:the q-ROF weighted averaging(q-ROFWA),the q-ROF weighted geometric(q-ROFWG),and the q-ROF weighted neutrality averaging(qROFWNA),which enhance decision-making under uncertainty.These operators are paired with ranking methods such as the similarity measure,score function,and inverse score function to improve the accuracy of disease identification.Additionally,the impact of varying q-rung values is explored through a sensitivity analysis,extending the analysis beyond the typical maximum value of 3.The Basic Uncertain Information(BUI)method is employed to simulate expert opinions,and aggregation operators are used to combine these opinions in a group decisionmaking context.Our results provide a comprehensive comparison of methodologies,highlighting their strengths and limitations in diagnosing diseases based on uncertain patient data.

Autonomous sortie scheduling for carrier aircraft fleet under towing mode

Safe and efficient sortie scheduling on the confined flight deck is crucial for maintaining high combat effectiveness of the aircraft carrier.The primary difficulty exactly lies in the spatiotemporal coordination,i.e.,allocation of limited supporting resources and collision-avoidance between heterogeneous dispatch entities.In this paper,the problem is investigated in the perspective of hybrid flow-shop scheduling problem by synthesizing the precedence,space and resource constraints.Specifically,eight processing procedures are abstracted,where tractors,preparing spots,catapults,and launching are virtualized as machines.By analyzing the constraints in sortie scheduling,a mixed-integer planning model is constructed.In particular,the constraint on preparing spot occupancy is improved to further enhance the sortie efficiency.The basic trajectory library for each dispatch entity is generated and a delayed strategy is integrated to address the collision-avoidance issue.To efficiently solve the formulated HFSP,which is essentially a combinatorial problem with tightly coupled constraints,a chaos-initialized genetic algorithm is developed.The solution framework is validated by the simulation environment referring to the Fort-class carrier,exhibiting higher sortie efficiency when compared to existing strategies.And animation of the simulation results is available at www.bilibili.com/video/BV14t421A7Tt/.The study presents a promising supporting technique for autonomous flight deck operation in the foreseeable future,and can be easily extended to other supporting scenarios,e.g.,ammunition delivery and aircraft maintenance.

Continuum of glucose and bone metabolism impairment across autonomous cortisol secretion: A cross-sectional study

BACKGROUND Autonomous cortisol secretion(ACS)is linked to a higher prevalence of metabolic abnormalities and an increased risk of major adverse cardiovascular events.AIM To evaluate glucose and bone metabolism in patients with ACS using a continuous glucose monitoring system(CGMS)and dual-energy X-ray absorptiometry(DXA).METHODS Patients diagnosed with ACS,including Cushing syndrome,mild ACS(MACS),and nonfunctional adrenal incidentaloma(NFAI),were recruited for this study.Glucose variability and glycemic status were assessed using CGMS.Regional bone mineral content(BMC),bone mineral density(BMD),and bone area(BA)were evaluated using DXA.CGMS-and DXA-derived parameters were compared across the subgroups of ACS.Correlation analysis was performed to examine relationships between varying degrees of cortisol secretion,measured by cortisol after 1 mg overnight dexamethasone suppression test(DST)or 24-hour urine free cortisol(24h UFC),and CGMS-or DXA-derived parameters.RESULTS A total of 64 patients with ACS were included in this study:19 with Cushing syndrome,11 with MACS,and 34 with NFAI.Glucose variability,time above range(TAR),and time in range(TIR)along with specific areal BMC,BMD,and BA,differed significantly between groups of Cushing syndrome and NFAI.A significant positive correlation was observed between glucose variability or TAR and cortisol after 1 mg overnight DST or 24h UFC.By contrast,TIR,along with regional BMC,BMD,and BA,were negatively correlated with varying degrees of cortisol secretion.CONCLUSION Glucose and bone metabolism impairments are on a continuum alteration from NFAI to MACS and Cushing syndrome.Prompt attention should be given to these patients with ACS,especially those with mild hormone secretion.Parameters of glucose variability and glycemic status along with bone condition in regions rich in cancellous bone will provide valuable information.

Planning and Decision-making for Connected Autonomous Vehicles at Road Intersections:A Review

Planning and decision-making technology at intersections is a comprehensive research problem in intelligent transportation systems due to the uncertainties caused by a variety of traffic participants.As wireless communication advances,vehicle infrastructure integrated algorithms designed for intersection planning and decision-making have received increasing attention.In this paper,the recent studies on the planning and decision-making technologies at intersections are primarily overviewed.The general planning and decision-making approaches are presented,which include graph-based approach,prediction base approach,optimization-based approach and machine learning based approach.Since connected autonomous vehicles(CAVs)is the future direction for the automated driving area,we summarized the evolving planning and decision-making methods based on vehicle infrastructure cooperative technologies.Both four-way signalized and unsignalized intersection(s)are investigated under purely automated driving traffic and mixed traffic.The study benefit from current strategies,protocols,and simulation tools to help researchers identify the presented approaches’challenges and determine the research gaps,and several remaining possible research problems that need to be solved in the future.

Probabilistic Lane-Change Decision-Making and Planning for Autonomous Heavy Vehicles

To improve the safety and driving stability of the autonomous heavy truck, it is necessary to consider the differences of driving behavior and drivable trajectories between the heavy trucks and passenger cars. This study proposes a probabilistic decision-making and trajectory planning framework for the autonomous heavy trucks. Firstly, the driving decision process is divided into intention generation and feasibility evaluations, which are realized using the utility theory and risk assessment, respectively. Subsequently the driving decision is made and sent to the trajectory planning module. In order to reflect the greater risks of the truck to other surrounding vehicles, the aggressiveness index(AI) is proposed and quantified to infer the asymmetrical risk level of lane-change maneuver. In the planning stage, the lateral and roll dynamics stability domains are developed as the constraints to exclude the candidate trajectories that would cause vehicle instability. Finally, the simulation results are compared between the proposed model and the artificial potential filed model in the scenarios extracted from the naturalistic driving data. It is shown that the proposed framework can provide the human-like lane-change decisions and truck-friendly trajectories, and performs well in dynamic driving environments.

Driving decision-making analysis of car-following for autonomous vehicle under complex urban environment

The decision-making under complex urban environment become one of the key issues that restricts the rapid development of the autonomous vehicles. The difficulty in making timely and accurate decisions like human beings under highly dynamic traffic environment is a major challenge for autonomous driving. Car-following has been regarded as the simplest but essential driving behavior among driving tasks and has received extensive attention from researchers around the world. This work addresses this problem and proposes a novel method RSAN(rough-set artificial neural network) to learn the decisions from excellent human drivers. A virtual urban traffic environment was built by Pre Scan and driving simulation was conducted to obtain a broad set of relevant data such as experienced drivers' behavior data and surrounding vehicles' motion data. Then, rough set was used to preprocess these data to extract the key influential factors on decision and reduce the impact of uncertain data and noise data. And the car-following decision was learned by neural network in which key factor was the input and acceleration was the output. The result shows the better convergence speed and the better decision accuracy of RSAN than ANN. Findings of this work contributes to the empirical understanding of driver's decision-making process and it provides a theoretical basis for the study of car-following decision-making under complex and dynamic environment.

Human-Like Decision-Making of Autonomous Vehicles in Dynamic Traffic Scenarios

With the maturation of autonomous driving technology, the use of autonomous vehicles in a socially acceptable manner has become a growing demand of the public. Human-like autonomous driving is expected due to the impact of the differences between autonomous vehicles and human drivers on safety.Although human-like decision-making has become a research hotspot, a unified theory has not yet been formed, and there are significant differences in the implementation and performance of existing methods. This paper provides a comprehensive overview of human-like decision-making for autonomous vehicles. The following issues are discussed: 1) The intelligence level of most autonomous driving decision-making algorithms;2) The driving datasets and simulation platforms for testing and verifying human-like decision-making;3) The evaluation metrics of human-likeness;personalized driving;the application of decisionmaking in real traffic scenarios;and 4) The potential research direction of human-like driving. These research results are significant for creating interpretable human-like driving models and applying them in dynamic traffic scenarios. In the future, the combination of intuitive logical reasoning and hierarchical structure will be an important topic for further research. It is expected to meet the needs of human-like driving.

Modeling and TOPSIS-GRA Algorithm for Autonomous Driving Decision-Making Under 5G-V2X Infrastructure

This paper is to explore the problems of intelligent connected vehicles(ICVs)autonomous driving decision-making under a 5G-V2X structured road environment.Through literature review and interviews with autonomous driving practitioners,this paper firstly puts forward a logical framework for designing a cerebrum-like autonomous driving system.Secondly,situated on this framework,it builds a hierarchical finite state machine(HFSM)model as well as a TOPSIS-GRA algorithm for making ICV autonomous driving decisions by employing a data fusion approach between the entropy weight method(EWM)and analytic hierarchy process method(AHP)and by employing a model fusion approach between the technique for order preference by similarity to an ideal solution(TOPSIS)and grey relational analysis(GRA).The HFSM model is composed of two layers:the global FSM model and the local FSM model.The decision of the former acts as partial input information of the latter and the result of the latter is sent forward to the local pathplanning module,meanwhile pulsating feedback to the former as real-time refresh data.To identify different traffic scenarios in a cerebrum-like way,the global FSM model is designed as 7 driving behavior states and 17 driving characteristic events,and the local FSM model is designed as 16 states and 8 characteristic events.In respect to designing a cerebrum-like algorithm for state transition,this paper firstly fuses AHP weight and EWM weight at their output layer to generate a synthetic weight coefficient for each characteristic event;then,it further fuses TOPSIS method and GRA method at the model building layer to obtain the implementable order of state transition.To verify the feasibility,reliability,and safety of theHFSMmodel aswell as its TOPSISGRA state transition algorithm,this paper elaborates on a series of simulative experiments conducted on the PreScan8.50 platform.The results display that the accuracy of obstacle detection gets 98%,lane line prediction is beyond 70 m,the speed of collision avoidance is higher than 45 km/h,the distance of collision avoidance is less than 5 m,path planning time for obstacle avoidance is averagely less than 50 ms,and brake deceleration is controlled under 6 m/s2.These technical indexes support that the driving states set and characteristic events set for the HFSM model as well as its TOPSIS-GRA algorithm may bring about cerebrum-like decision-making effectiveness for ICV autonomous driving under 5G-V2X intelligent road infrastructure.

Conditions for autonomous choice： a qualitative study of older adults＇ experience of decision-making in TAVR

Background Patient autonomy is a leading principle in bioethics and a basis for shared decision making. This study explores conditions for an autonomous choice experienced by older adults who recently underwent trans-catheter aortic valve replacement （TAVR）. Methods Qualitative study entailing semi-structured interviews of a purposive sample often older （range 73-89, median 83.5 years） adults after TAVR （median 23 days）. The study setting was a cardiac department at a university hospital performing TAVR since 2010. Analysis was by systematic text condensation. Results Even when choice seemed hard or absent, TAVR-patients deliberately took the chance offered them by processing risk assessment, ambivalence and fate. They regarded declining the treatment to be worse than accepting the risk related to the procedure. The experience of being thoroughly advised by their physician formed the basis of an autonomous trust. The trust they felt for the physicians＇ recommendations mitigated ambivalence about the procedure and risks. TAVR patients expressed feelings consistent with self-empowerment and claimed that it had to be their decision. Even so, choosing the intervention as an obligation to their family or passively accepting it was also reported. Conclusions Older TAVR patients＇ experience of an autonomous decision may encompass frank tradeoff; deliberate physician dependency as well as a resilient self-view. Physicians should be especially aware of how older adults＇ subtle cognitive declines and inclinations to preserve their identities which can influence their medical decision making when obtaining in- formed consent. Cardiologists and other providers may also use these insights to develop new strategies that better respond to such inherent complexities.

Situational continuity-based air combat autonomous maneuvering decision-making

In order to improve the performance of UAV's autonomous maneuvering decision-making,this paper proposes a decision-making method based on situational continuity.The algorithm in this paper designs a situation evaluation function with strong guidance,then trains the Long Short-Term Memory(LSTM)under the framework of Deep Q Network(DQN)for air combat maneuvering decision-making.Considering the continuity between adjacent situations,the method takes multiple consecutive situations as one input of the neural network.To reflect the difference between adjacent situations,the method takes the difference of situation evaluation value as the reward of reinforcement learning.In different scenarios,the algorithm proposed in this paper is compared with the algorithm based on the Fully Neural Network(FNN)and the algorithm based on statistical principles respectively.The results show that,compared with the FNN algorithm,the algorithm proposed in this paper is more accurate and forwardlooking.Compared with the algorithm based on the statistical principles,the decision-making of the algorithm proposed in this paper is more efficient and its real-time performance is better.

A Survey on an Emerging Safety Challenge for Autonomous Vehicles:Safety of the Intended Functionality

As the complexity of autonomous vehicles(AVs)continues to increase and artificial intelligence algorithms are becoming increasingly ubiquitous,a novel safety concern known as the safety of the intended functionality(SOTIF)has emerged,presenting significant challenges to the widespread deployment of AVs.SOTIF focuses on issues arising from the functional insufficiencies of the AVs’intended functionality or its implementation,apart from conventional safety considerations.From the systems engineering standpoint,this study offers a comprehensive exploration of the SOTIF landscape by reviewing academic research,practical activities,challenges,and perspectives across the development,verification,validation,and operation phases.Academic research encompasses system-level SOTIF studies and algorithm-related SOTIF issues and solutions.Moreover,it encapsulates practical SOTIF activities undertaken by corporations,government entities,and academic institutions spanning international and Chinese contexts,focusing on the overarching methodologies and practices in different phases.Finally,the paper presents future challenges and outlook pertaining to the development,verification,validation,and operation phases,motivating stakeholders to address the remaining obstacles and challenges.

Attention Markets of Blockchain-Based Decentralized Autonomous Organizations

The attention is a scarce resource in decentralized autonomous organizations(DAOs),as their self-governance relies heavily on the attention-intensive decision-making process of“proposal and voting”.To prevent the negative effects of pro-posers’attention-capturing strategies that contribute to the“tragedy of the commons”and ensure an efficient distribution of attention among multiple proposals,it is necessary to establish a market-driven allocation scheme for DAOs’attention.First,the Harberger tax-based attention markets are designed to facilitate its allocation via continuous and automated trading,where the individualized Harberger tax rate(HTR)determined by the pro-posers’reputation is adopted.Then,the Stackelberg game model is formulated in these markets,casting attention to owners in the role of leaders and other competitive proposers as followers.Its equilibrium trading strategies are also discussed to unravel the intricate dynamics of attention pricing.Moreover,utilizing the single-round Stackelberg game as an illustrative example,the existence of Nash equilibrium trading strategies is demonstrated.Finally,the impact of individualized HTR on trading strategies is investigated,and results suggest that it has a negative correlation with leaders’self-accessed prices and ownership duration,but its effect on their revenues varies under different conditions.This study is expected to provide valuable insights into leveraging attention resources to improve DAOs’governance and decision-making process.

Path-Following Control With Obstacle Avoidance of Autonomous Surface Vehicles Subject to Actuator Faults

This paper investigates the path-following control problem with obstacle avoidance of autonomous surface vehicles in the presence of actuator faults,uncertainty and external disturbances.Autonomous surface vehicles inevitably suffer from actuator faults in complex sea environments,which may cause existing obstacle avoidance strategies to fail.To reduce the influence of actuator faults,an improved artificial potential function is constructed by introducing the lower bound of actuator efficiency factors.The nonlinear state observer,which only depends on measurable position information of the autonomous surface vehicle,is used to address uncertainties and external disturbances.By using a backstepping technique and adaptive mechanism,a path-following control strategy with obstacle avoidance and fault tolerance is designed which can ensure that the tracking errors converge to a small neighborhood of zero.Compared with existing results,the proposed control strategy has the capability of obstacle avoidance and fault tolerance simultaneously.Finally,the comparison results through simulations are given to verify the effectiveness of the proposed method.

UAV maneuvering decision-making algorithm based on deep reinforcement learning under the guidance of expert experience

Autonomous umanned aerial vehicle(UAV) manipulation is necessary for the defense department to execute tactical missions given by commanders in the future unmanned battlefield. A large amount of research has been devoted to improving the autonomous decision-making ability of UAV in an interactive environment, where finding the optimal maneuvering decisionmaking policy became one of the key issues for enabling the intelligence of UAV. In this paper, we propose a maneuvering decision-making algorithm for autonomous air-delivery based on deep reinforcement learning under the guidance of expert experience. Specifically, we refine the guidance towards area and guidance towards specific point tasks for the air-delivery process based on the traditional air-to-surface fire control methods.Moreover, we construct the UAV maneuvering decision-making model based on Markov decision processes(MDPs). Specifically, we present a reward shaping method for the guidance towards area and guidance towards specific point tasks using potential-based function and expert-guided advice. The proposed algorithm could accelerate the convergence of the maneuvering decision-making policy and increase the stability of the policy in terms of the output during the later stage of training process. The effectiveness of the proposed maneuvering decision-making policy is illustrated by the curves of training parameters and extensive experimental results for testing the trained policy.

Orientation and Decision-Making for Soccer Based on Sports Analytics and AI:A Systematic Review

Due to ever-growing soccer data collection approaches and progressing artificial intelligence(AI) methods, soccer analysis, evaluation, and decision-making have received increasing interest from not only the professional sports analytics realm but also the academic AI research community. AI brings gamechanging approaches for soccer analytics where soccer has been a typical benchmark for AI research. The combination has been an emerging topic. In this paper, soccer match analytics are taken as a complete observation-orientation-decision-action(OODA) loop.In addition, as in AI frameworks such as that for reinforcement learning, interacting with a virtual environment enables an evolving model. Therefore, both soccer analytics in the real world and virtual domains are discussed. With the intersection of the OODA loop and the real-virtual domains, available soccer data, including event and tracking data, and diverse orientation and decisionmaking models for both real-world and virtual soccer matches are comprehensively reviewed. Finally, some promising directions in this interdisciplinary area are pointed out. It is claimed that paradigms for both professional sports analytics and AI research could be combined. Moreover, it is quite promising to bridge the gap between the real and virtual domains for soccer match analysis and decision-making.

Ethical Decision-Making Framework Based on Incremental ILP Considering Conflicts

Humans are experiencing the inclusion of artificial agents in their lives,such as unmanned vehicles,service robots,voice assistants,and intelligent medical care.If the artificial agents cannot align with social values or make ethical decisions,they may not meet the expectations of humans.Traditionally,an ethical decision-making framework is constructed by rule-based or statistical approaches.In this paper,we propose an ethical decision-making framework based on incremental ILP(Inductive Logic Programming),which can overcome the brittleness of rule-based approaches and little interpretability of statistical approaches.As the current incremental ILP makes it difficult to solve conflicts,we propose a novel ethical decision-making framework considering conflicts in this paper,which adopts our proposed incremental ILP system.The framework consists of two processes:the learning process and the deduction process.The first process records bottom clauses with their score functions and learns rules guided by the entailment and the score function.The second process obtains an ethical decision based on the rules.In an ethical scenario about chatbots for teenagers’mental health,we verify that our framework can learn ethical rules and make ethical decisions.Besides,we extract incremental ILP from the framework and compare it with the state-of-the-art ILP systems based on ASP(Answer Set Programming)focusing on conflict resolution.The results of comparisons show that our proposed system can generate better-quality rules than most other systems.