Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Attention Markets of Blockchain-Based Decentralized Autonomous Organizations

The attention is a scarce resource in decentralized autonomous organizations(DAOs),as their self-governance relies heavily on the attention-intensive decision-making process of“proposal and voting”.To prevent the negative effects of pro-posers’attention-capturing strategies that contribute to the“tragedy of the commons”and ensure an efficient distribution of attention among multiple proposals,it is necessary to establish a market-driven allocation scheme for DAOs’attention.First,the Harberger tax-based attention markets are designed to facilitate its allocation via continuous and automated trading,where the individualized Harberger tax rate(HTR)determined by the pro-posers’reputation is adopted.Then,the Stackelberg game model is formulated in these markets,casting attention to owners in the role of leaders and other competitive proposers as followers.Its equilibrium trading strategies are also discussed to unravel the intricate dynamics of attention pricing.Moreover,utilizing the single-round Stackelberg game as an illustrative example,the existence of Nash equilibrium trading strategies is demonstrated.Finally,the impact of individualized HTR on trading strategies is investigated,and results suggest that it has a negative correlation with leaders’self-accessed prices and ownership duration,but its effect on their revenues varies under different conditions.This study is expected to provide valuable insights into leveraging attention resources to improve DAOs’governance and decision-making process.

Path-Following Control With Obstacle Avoidance of Autonomous Surface Vehicles Subject to Actuator Faults

This paper investigates the path-following control problem with obstacle avoidance of autonomous surface vehicles in the presence of actuator faults,uncertainty and external disturbances.Autonomous surface vehicles inevitably suffer from actuator faults in complex sea environments,which may cause existing obstacle avoidance strategies to fail.To reduce the influence of actuator faults,an improved artificial potential function is constructed by introducing the lower bound of actuator efficiency factors.The nonlinear state observer,which only depends on measurable position information of the autonomous surface vehicle,is used to address uncertainties and external disturbances.By using a backstepping technique and adaptive mechanism,a path-following control strategy with obstacle avoidance and fault tolerance is designed which can ensure that the tracking errors converge to a small neighborhood of zero.Compared with existing results,the proposed control strategy has the capability of obstacle avoidance and fault tolerance simultaneously.Finally,the comparison results through simulations are given to verify the effectiveness of the proposed method.

Dynamic Routing of Multiple QoS-Required Flows in Cloud-Edge Autonomous Multi-Domain Data Center Networks

The 6th generation mobile networks(6G)network is a kind of multi-network interconnection and multi-scenario coexistence network,where multiple network domains break the original fixed boundaries to form connections and convergence.In this paper,with the optimization objective of maximizing network utility while ensuring flows performance-centric weighted fairness,this paper designs a reinforcement learning-based cloud-edge autonomous multi-domain data center network architecture that achieves single-domain autonomy and multi-domain collaboration.Due to the conflict between the utility of different flows,the bandwidth fairness allocation problem for various types of flows is formulated by considering different defined reward functions.Regarding the tradeoff between fairness and utility,this paper deals with the corresponding reward functions for the cases where the flows undergo abrupt changes and smooth changes in the flows.In addition,to accommodate the Quality of Service(QoS)requirements for multiple types of flows,this paper proposes a multi-domain autonomous routing algorithm called LSTM+MADDPG.Introducing a Long Short-Term Memory(LSTM)layer in the actor and critic networks,more information about temporal continuity is added,further enhancing the adaptive ability changes in the dynamic network environment.The LSTM+MADDPG algorithm is compared with the latest reinforcement learning algorithm by conducting experiments on real network topology and traffic traces,and the experimental results show that LSTM+MADDPG improves the delay convergence speed by 14.6%and delays the start moment of packet loss by 18.2%compared with other algorithms.

Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

RRT Autonomous Detection Algorithm Based on Multiple Pilot Point Bias Strategy and Karto SLAM Algorithm

A Rapid-exploration Random Tree(RRT)autonomous detection algorithm based on the multi-guide-node deflection strategy and Karto Simultaneous Localization and Mapping(SLAM)algorithm was proposed to solve the problems of low efficiency of detecting frontier boundary points and drift distortion in the process of map building in the traditional RRT algorithm in the autonomous detection strategy of mobile robot.Firstly,an RRT global frontier boundary point detection algorithm based on the multi-guide-node deflection strategy was put forward,which introduces the reference value of guide nodes’deflection probability into the random sampling function so that the global search tree can detect frontier boundary points towards the guide nodes according to random probability.After that,a new autonomous detection algorithm for mobile robots was proposed by combining the graph optimization-based Karto SLAM algorithm with the previously improved RRT algorithm.The algorithm simulation platform based on the Gazebo platform was built.The simulation results show that compared with the traditional RRT algorithm,the proposed RRT autonomous detection algorithm can effectively reduce the time of autonomous detection,plan the length of detection trajectory under the condition of high average detection coverage,and complete the task of autonomous detection mapping more efficiently.Finally,with the help of the ROS-based mobile robot experimental platform,the performance of the proposed algorithm was verified in the real environment of different obstacles.The experimental results show that in the actual environment of simple and complex obstacles,the proposed RRT autonomous detection algorithm was superior to the traditional RRT autonomous detection algorithm in the time of detection,length of detection trajectory,and average coverage,thus improving the efficiency and accuracy of autonomous detection.

General Optimal Trajectory Planning:Enabling Autonomous Vehicles with the Principle of Least Action

This study presents a general optimal trajectory planning(GOTP)framework for autonomous vehicles(AVs)that can effectively avoid obstacles and guide AVs to complete driving tasks safely and efficiently.Firstly,we employ the fifth-order Bezier curve to generate and smooth the reference path along the road centerline.Cartesian coordinates are then transformed to achieve the curvature continuity of the generated curve.Considering the road constraints and vehicle dynamics,limited polynomial candidate trajectories are generated and smoothed in a curvilinear coordinate system.Furthermore,in selecting the optimal trajectory,we develop a unified and auto-tune objective function based on the principle of least action by employing AVs to simulate drivers’behavior and summarizing their manipulation characteristics of“seeking benefits and avoiding losses.”Finally,by integrating the idea of receding-horizon optimization,the proposed framework is achieved by considering dynamic multi-performance objectives and selecting trajectories that satisfy feasibility,optimality,and adaptability.Extensive simulations and experiments are performed,and the results demonstrate the framework’s feasibility and effectiveness,which avoids both dynamic and static obstacles and applies to various scenarios with multi-source interactive traffic participants.Moreover,we prove that the proposed method can guarantee real-time planning and safety requirements compared to drivers’manipulation.

A Survey on an Emerging Safety Challenge for Autonomous Vehicles:Safety of the Intended Functionality

As the complexity of autonomous vehicles(AVs)continues to increase and artificial intelligence algorithms are becoming increasingly ubiquitous,a novel safety concern known as the safety of the intended functionality(SOTIF)has emerged,presenting significant challenges to the widespread deployment of AVs.SOTIF focuses on issues arising from the functional insufficiencies of the AVs’intended functionality or its implementation,apart from conventional safety considerations.From the systems engineering standpoint,this study offers a comprehensive exploration of the SOTIF landscape by reviewing academic research,practical activities,challenges,and perspectives across the development,verification,validation,and operation phases.Academic research encompasses system-level SOTIF studies and algorithm-related SOTIF issues and solutions.Moreover,it encapsulates practical SOTIF activities undertaken by corporations,government entities,and academic institutions spanning international and Chinese contexts,focusing on the overarching methodologies and practices in different phases.Finally,the paper presents future challenges and outlook pertaining to the development,verification,validation,and operation phases,motivating stakeholders to address the remaining obstacles and challenges.

Perception Enhanced Deep Deterministic Policy Gradient for Autonomous Driving in Complex Scenarios

Autonomous driving has witnessed rapid advancement;however,ensuring safe and efficient driving in intricate scenarios remains a critical challenge.In particular,traffic roundabouts bring a set of challenges to autonomous driving due to the unpredictable entry and exit of vehicles,susceptibility to traffic flow bottlenecks,and imperfect data in perceiving environmental information,rendering them a vital issue in the practical application of autonomous driving.To address the traffic challenges,this work focused on complex roundabouts with multi-lane and proposed a Perception EnhancedDeepDeterministic Policy Gradient(PE-DDPG)for AutonomousDriving in the Roundabouts.Specifically,themodel incorporates an enhanced variational autoencoder featuring an integrated spatial attention mechanism alongside the Deep Deterministic Policy Gradient framework,enhancing the vehicle’s capability to comprehend complex roundabout environments and make decisions.Furthermore,the PE-DDPG model combines a dynamic path optimization strategy for roundabout scenarios,effectively mitigating traffic bottlenecks and augmenting throughput efficiency.Extensive experiments were conducted with the collaborative simulation platform of CARLA and SUMO,and the experimental results show that the proposed PE-DDPG outperforms the baseline methods in terms of the convergence capacity of the training process,the smoothness of driving and the traffic efficiency with diverse traffic flow patterns and penetration rates of autonomous vehicles(AVs).Generally,the proposed PE-DDPGmodel could be employed for autonomous driving in complex scenarios with imperfect data.

Evaluation of an Autonomous Vehicle User Interface for Sensory Impaired Users

Autonomous vehicles (AVs) hold immense promises in revolutionizing transportation, and their potential benefits extend to individuals with impairments, particularly those with vision and hearing impairments. However, the accommodation of these individuals in AVs requires developing advanced user interfaces. This paper describes an explorative study of a multimodal user interface for autonomous vehicles, specifically developed for passengers with sensory (vision and/or hearing) impairments. In a driving simulator, 32 volunteers with simulated sensory impairments, were exposed to multiple drives in an autonomous vehicle while freely interacting with standard and inclusive variants of the infotainment and navigation system interface. The two user interfaces differed in graphical layout and voice messages, which adopted inclusive design principles for the inclusive variant. Questionnaires and structured interviews were conducted to collect participants’ impressions. The data analysis reports positive user experiences, but also identifies technical challenges. Verified guidelines are provided for further development of inclusive user interface solutions.

Probabilistic Global Maximum Power Point Tracking Algorithm for Continuously Varying Partial Shading Conditions on Autonomous PV Systems

A photovoltaic (PV) string with multiple modules with bypass diodes frequently deployed on a variety of autonomous PV systems may present multiple power peaks under uneven shading. For optimal solar harvesting, there is a need for a control schema to force the PV string to operate at global maximum power point (GMPP). While a lot of tracking methods have been proposed in the literature, they are usually complex and do not fully take advantage of the available characteristics of the PV array. This work highlights how the voltage at operating point and the forward voltage of the bypass diode are considered to design a global maximum power point tracking (GMPPT) algorithm with a very limited global search phase called Fast GMPPT. This algorithm successfully tracks GMPP between 94% and 98% of the time under a theoretical evaluation. It is then compared against Perturb and Observe, Deterministic Particle Swarm Optimization, and Grey Wolf Optimization under a sequence of irradiance steps as well as a power-over-voltage characteristics profile that mimics the electrical characteristics of a PV string under varying partial shading conditions. Overall, the simulation with the sequence of irradiance steps shows that while Fast GMPPT does not have the best convergence time, it has an excellent convergence rate as well as causes the least amount of power loss during the global search phase. Experimental test under varying partial shading conditions shows that while the GMPPT proposal is simple and lightweight, it is very performant under a wide range of dynamically varying partial shading conditions and boasts the best energy efficiency (94.74%) out of the 4 tested algorithms.

Enhancing Autonomy Capability in Regional Power Grids:A Strategic Planning Approach with Multiple Autonomous Evaluation Indexes

After the integration of large-scale DistributedGeneration(DG)into the distribution network,the randomness and volatility of its output result in a reduction of spatiotemporal alignment between power generation and demand in the distribution network,exacerbating the phenomenon of wind and solar power wastage.As a novel power system model,the fundamental concept of Regional Autonomous Power Grids(RAPGs)is to achieve localized management and energy autonomy,thereby facilitating the effective consumption of DGs.Therefore,this paper proposes a distributed resource planning strategy that enhances the autonomy capabilities of regional power grids by considering multiple evaluation indexes for autonomy.First,a regional Energy Storage(ES)configuration strategy is proposed.This strategy can select a suitable reference value for the upper limit of ES configuration based on the regional load andDGoutput to maximize the elimination of source load deviations in the region as the upper limit constraint of ES capacity.Then,a control strategy for regional ES is proposed,the charging and discharging reference line of ES is set,and multiple autonomy and economic indexes are used as objective functions to select different proportions of ES to control the distributed resources of the regional power grid and establish evaluation indexes of the internal regional generation and load power ratio,the proportion of power supply matching hours,new energy consumption rate and tie line power imbalance outside the region to evaluate changes in the regional autonomy capabilities.The final simulation results showthat in the real regional grid example,the planning method in the planning year in the region of the overall power supply matching hour ratio and new energy consumption rate increased by 3.9%and 4.8%on average,and the power imbalance of the tie line decreased by 7.8%on average.The proposed planning approach enables the maximization of regional autonomy while effectively smoothing the fluctuation of power exchange between the regional grid and the higher-level grid.This presents a rational and effective planning solution for the regional grid,facilitating the coordinated development between the region and the distribution network.

Efficient Vision Transformers for Autonomous Off-Road Perception Systems

The development of autonomous vehicles has become one of the greatest research endeavors in recent years. These vehicles rely on many complex systems working in tandem to make decisions. For practical use and safety reasons, these systems must not only be accurate, but also quickly detect changes in the surrounding environment. In autonomous vehicle research, the environment perception system is one of the key components of development. Environment perception systems allow the vehicle to understand its surroundings. This is done by using cameras, light detection and ranging (LiDAR), with other sensor systems and modalities. Deep learning computer vision algorithms have been shown to be the strongest tool for translating camera data into accurate and safe traversability decisions regarding the environment surrounding a vehicle. In order for a vehicle to safely traverse an area in real time, these computer vision algorithms must be accurate and have low latency. While much research has studied autonomous driving for traversing well-structured urban environments, limited research exists evaluating perception system improvements in off-road settings. This research aims to investigate the adaptability of several existing deep-learning architectures for semantic segmentation in off-road environments. Previous studies of two Convolutional Neural Network (CNN) architectures are included for comparison with new evaluation of Vision Transformer (ViT) architectures for semantic segmentation. Our results demonstrate viability of ViT architectures for off-road perception systems, having a strong segmentation accuracy, lower inference speed and memory footprint compared to previous results with CNN architectures.

Ethnic Customs and Rural Tourism Development in Xinbin Manchu Autonomous County of Fushun City

The development of ethnic minority tourism is currently a hot topic in domestic tourism development.As an important component of Chinese civilization,the Manchu people have created brilliant culture in the long river of historical development.As the hometown of the Manchu people,Fushun has unique folk cultural tourism resources and a strong ethnic flavor.Nowadays,under the promotion of the rural revitalization strategy,the construction of new rural areas is constantly developing,and rural tourism is gradually becoming a new industry.Therefore,in the context of the increasingly prosperous rural tourism industry,it has become increasingly important to combine the ethnic customs of Manchu culture with rural tourism.Taking the ethnic customs and integrated development of rural tourism in Xinbin Manchu Autonomous County of Fushun City,Liaoning Province as the research object,this paper mainly sorts out the current situation and characteristics of rural tourism development in the region,systematically explores the problems in development and how to further optimize development,and proposes new models suitable for the development of folk tourism in Xinbin of Fushun,in order to achieve maximum economic and social benefits and provide a reference for promoting the development of tourism in the region.

Results of the virtual simulation teaching method on autonomous learning competencies of undergraduate nursing students

Background:With the continuous development of information technology,most universities use mobile teaching platforms for classroom teaching.With the help of the national virtual simulation experimental teaching project sharing platform,students can enhance self-directed learning through the virtual simulation operations of the project.Purpose:To explore the application of virtual simulation experiment in teaching the fundamentals of nursing practice based on the Platform of the National Virtual Simulation Experiment Teaching Project during the COVID-19 pandemic analyze the impact of this teaching method on the autonomous learning ability of undergraduate nursing students.Methods:Convenience sampling was used to select 121 nursing undergraduates from Y University’s School of Nursing;the online teaching of fundamentals of nursing practice was conducted to the students.After taking the course,questionnaires were distributed to the undergraduate nursing students to collect their perceptions regarding the use of the virtual simulation experiment platform and autonomous learning competencies.Results:Most students expressed their preference for the virtual simulation teaching platform,and their satisfaction with the project evaluation was high 83.05%.They hoped to promote the application in future experimental teaching.Undergraduate nursing students believed that the virtual simulation teaching platform was conducive to cultivating clinical thinking ability,could stimulate learning interest,enhanced autonomous learning competencies.Conclusion:During the pandemic,the virtual simulation teaching platform for a lecture on in nursing education has achieved good results in both the aspects of teaching and student learning.Teachers efficiently used their training time and reduced their teaching burden.Moreover,the laboratory cost was also reduced.For undergraduate nursing students,the system was conducive to cultivating clinical thinking ability,stimulating their interest in learning,enhancing their learning and comprehension abilities and learning initiative.

A Combined Reinforcement Learning and Model Predictive Control for Car-Following Maneuver of Autonomous Vehicles

Model predictive control is widely used in the design of autonomous driving algorithms.However,its parameters are sensitive to dynamically varying driving conditions,making it difficult to be implemented into practice.As a result,this study presents a self-learning algorithm based on reinforcement learning to tune a model predictive controller.Specifically,the proposed algorithm is used to extract features of dynamic traffic scenes and adjust the weight coefficients of the model predictive controller.In this method,a risk threshold model is proposed to classify the risk level of the scenes based on the scene features,and aid in the design of the reinforcement learning reward function and ultimately improve the adaptability of the model predictive controller to real-world scenarios.The proposed algorithm is compared to a pure model predictive controller in car-following case.According to the results,the proposed method enables autonomous vehicles to adjust the priority of performance indices reasonably in different scenarios according to risk variations,showing a good scenario adaptability with safety guaranteed.

Cooperative Target Tracking of Multiple Autonomous Surface Vehicles Under Switching Interaction Topologies

This paper is concerned with the cooperative target tracking of multiple autonomous surface vehicles(ASVs)under switching interaction topologies.For the target to be tracked,only its position can be measured/received by some of the ASVs,and its velocity is unavailable to all the ASVs.A distributed extended state observer taking into consideration switching topologies is designed to integrally estimate unknown target dynamics and neighboring ASVs'dynamics.Accordingly,a novel kinematic controller is designed,which takes full advantage of known information and avoids the approximation of some virtual control vectors.Moreover,a disturbance observer is presented to estimate unknown time-varying environmental disturbance.Furthermore,a distributed dynamic controller is designed to regulate the involved ASVs to cooperatively track the target.It enables each ASV to adjust its forces and moments according to the received information from its neighbors.The effectiveness of the derived results is demonstrated through cooperative target tracking performance analysis for a tracking system composed of five interacting ASVs.

Special Section on Autonomous Decision and Cooperative Control of UAV Swarms

UAV swarm intelligence technology is a research hotspot and has broad application prospects.Autonomous decisionmaking and cooperative control are the key technologies to realize swarm intelligence.Autonomous decision-making is an important embodiment of UAV swarm intelligence.Cooperative control is the key factor to determine the success or failure of swarm tasks.The purpose of this special section is to discuss the technical system of swarm autonomous decision-making and the theory and method of collaborative control.The special section publishes both review articles and original contributions on the theory and method of autonomous decision-making and collaborative control for unmanned system swarms.The topics of interest include development overviews,mathematical modeling,theories and algorithms,as well as simulation and test technologies of autonomous decision making,cooperative control and game for unmanned system swarms.

MPC-based Motion Planning and Control Enables Smarter and Safer Autonomous Marine Vehicles:Perspectives and a Tutorial Survey

Autonomous marine vehicles(AMVs)have received considerable attention in the past few decades,mainly because they play essential roles in broad marine applications such as environmental monitoring and resource exploration.Recent advances in the field of communication technologies,perception capability,computational power and advanced optimization algorithms have stimulated new interest in the development of AMVs.In order to deploy the constrained AMVs in the complex dynamic maritime environment,it is crucial to enhance the guidance and control capabilities through effective and practical planning,and control algorithms.Model predictive control(MPC)has been exceptionally successful in different fields due to its ability to systematically handle constraints while optimizing control performance.This paper aims to provide a review of recent progress in the context of motion planning and control for AMVs from the perceptive of MPC.Finally,future research trends and directions in this substantial research area of AMVs are highlighted.

Role-based Bayesian decision framework for autonomous unmanned systems

In the process of performing a task,autonomous unmanned systems face the problem of scene changing,which requires the ability of real-time decision-making under dynamically changing scenes.Therefore,taking the unmanned system coordinative region control operation as an example,this paper combines knowledge representation with probabilistic decisionmaking and proposes a role-based Bayesian decision model for autonomous unmanned systems that integrates scene cognition and individual preferences.Firstly,according to utility value decision theory,the role-based utility value decision model is proposed to realize task coordination according to the preference of the role that individual is assigned.Then,multi-entity Bayesian network is introduced for situation assessment,by which scenes and their uncertainty related to the operation are semantically described,so that the unmanned systems can conduct situation awareness in a set of scenes with uncertainty.Finally,the effectiveness of the proposed method is verified in a virtual task scenario.This research has important reference value for realizing scene cognition,improving cooperative decision-making ability under dynamic scenes,and achieving swarm level autonomy of unmanned systems.