Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Attention Markets of Blockchain-Based Decentralized Autonomous Organizations

The attention is a scarce resource in decentralized autonomous organizations(DAOs),as their self-governance relies heavily on the attention-intensive decision-making process of“proposal and voting”.To prevent the negative effects of pro-posers’attention-capturing strategies that contribute to the“tragedy of the commons”and ensure an efficient distribution of attention among multiple proposals,it is necessary to establish a market-driven allocation scheme for DAOs’attention.First,the Harberger tax-based attention markets are designed to facilitate its allocation via continuous and automated trading,where the individualized Harberger tax rate(HTR)determined by the pro-posers’reputation is adopted.Then,the Stackelberg game model is formulated in these markets,casting attention to owners in the role of leaders and other competitive proposers as followers.Its equilibrium trading strategies are also discussed to unravel the intricate dynamics of attention pricing.Moreover,utilizing the single-round Stackelberg game as an illustrative example,the existence of Nash equilibrium trading strategies is demonstrated.Finally,the impact of individualized HTR on trading strategies is investigated,and results suggest that it has a negative correlation with leaders’self-accessed prices and ownership duration,but its effect on their revenues varies under different conditions.This study is expected to provide valuable insights into leveraging attention resources to improve DAOs’governance and decision-making process.

Path-Following Control With Obstacle Avoidance of Autonomous Surface Vehicles Subject to Actuator Faults

This paper investigates the path-following control problem with obstacle avoidance of autonomous surface vehicles in the presence of actuator faults,uncertainty and external disturbances.Autonomous surface vehicles inevitably suffer from actuator faults in complex sea environments,which may cause existing obstacle avoidance strategies to fail.To reduce the influence of actuator faults,an improved artificial potential function is constructed by introducing the lower bound of actuator efficiency factors.The nonlinear state observer,which only depends on measurable position information of the autonomous surface vehicle,is used to address uncertainties and external disturbances.By using a backstepping technique and adaptive mechanism,a path-following control strategy with obstacle avoidance and fault tolerance is designed which can ensure that the tracking errors converge to a small neighborhood of zero.Compared with existing results,the proposed control strategy has the capability of obstacle avoidance and fault tolerance simultaneously.Finally,the comparison results through simulations are given to verify the effectiveness of the proposed method.

Automatic fovea detection and choroid segmentation for choroidal thickness assessment in optical coherence tomography

AIM:To develop an automated model for subfoveal choroidal thickness(SFCT)detection in optical coherence tomography(OCT)images,addressing manual fovea location and choroidal contour challenges.METHODS:Two procedures were proposed:defining the fovea and segmenting the choroid.Fovea localization from B-scan OCT image sequence with three-dimensional reconstruction(LocBscan-3D)predicted fovea location using central foveal depression features,and fovea localization from two-dimensional en-face OCT(LocEN-2D)used a mask region-based convolutional neural network(Mask R-CNN)model for optic disc detection,and determined the fovea location based on optic disc relative position.Choroid segmentation also employed Mask R-CNN.RESULTS:For 53 eyes in 28 healthy subjects,LocBscan-3D’s mean difference between manual and predicted fovea locations was 170.0μm,LocEN-2D yielded 675.9μm.LocEN-2D performed better in non-high myopia group(P=0.02).SFCT measurements from Mask R-CNN aligned with manual values.CONCLUSION:Our models accurately predict SFCT in OCT images.LocBscan-3D excels in precise fovea localization even with high myopia.LocEN-2D shows high detection rates but lower accuracy especially in the high myopia group.Combining both models offers a robust SFCT assessment approach,promising efficiency and accuracy for large-scale studies and clinical use.

A Retrodirective Array Enabled by CMOS Chips for Two-Way Wireless Communication with Automatic Beam Tracking

This article proposes and demonstrates a retrodirective array(RDA)for two-way wireless communication with automatic beam tracking.The proposed RDA is enabled by specifically designed chips made using a domestic complementary metal-oxide semiconductor(CMOS)process.The highly integrated CMOS chip includes a receiving(Rx)chain,a transmitting(Tx)chain,and a unique tracking phaselocked loop(PLL)for the crucial conjugated phase recovery in the RDA.This article also proposes a method to reduce the beam pointing error(BPE)in a conventional RDA.To validate the above ideas simply yet without loss of generality,a 2.4 GHz RDA is demonstrated through two-way communication links between the Rx and Tx chains,and an on-chip quadrature coupler is designed to achieve a nonretrodirective signal suppression of 23 dBc.The experimental results demonstrate that the proposed RDA,which incorporates domestically manufactured low-cost 0.18 lm CMOS chips,is capable of automatically tracking beams covering±40with a reduced BPE.Each CMOS chip in the RDA has a compact size of 4.62 mm^(2) and a low power consumption of 0.15 W.To the best of the authors’knowledge,this is the first research to demonstrate an RDA with a fully customized CMOS chip for wireless communication with automatic beam tracking。

Dynamic Routing of Multiple QoS-Required Flows in Cloud-Edge Autonomous Multi-Domain Data Center Networks

The 6th generation mobile networks(6G)network is a kind of multi-network interconnection and multi-scenario coexistence network,where multiple network domains break the original fixed boundaries to form connections and convergence.In this paper,with the optimization objective of maximizing network utility while ensuring flows performance-centric weighted fairness,this paper designs a reinforcement learning-based cloud-edge autonomous multi-domain data center network architecture that achieves single-domain autonomy and multi-domain collaboration.Due to the conflict between the utility of different flows,the bandwidth fairness allocation problem for various types of flows is formulated by considering different defined reward functions.Regarding the tradeoff between fairness and utility,this paper deals with the corresponding reward functions for the cases where the flows undergo abrupt changes and smooth changes in the flows.In addition,to accommodate the Quality of Service(QoS)requirements for multiple types of flows,this paper proposes a multi-domain autonomous routing algorithm called LSTM+MADDPG.Introducing a Long Short-Term Memory(LSTM)layer in the actor and critic networks,more information about temporal continuity is added,further enhancing the adaptive ability changes in the dynamic network environment.The LSTM+MADDPG algorithm is compared with the latest reinforcement learning algorithm by conducting experiments on real network topology and traffic traces,and the experimental results show that LSTM+MADDPG improves the delay convergence speed by 14.6%and delays the start moment of packet loss by 18.2%compared with other algorithms.

Research on Automatic Test System of Engine Blade Natural Frequency

Blades are one of the important components on aircraft engines.If they break due to vibration failure,the normal operation of the entire engine will be offected.Therefore,it is necessary to measure their natural frequency before installing them on the engine to avoid resonance.At present,most blade vibration testing systems require manual operation by operators,which has high requirements for operators and the testing process is also very cumbersome.Therefore,the testing efficiency is low and cannot meet the needs of efficient testing.To solve the current problems of low testing efficiency and high operational requirements,a high-precision and high-efficiency automatic test system is designed.The testing accuracy of this system can reach ±1%,and the testing efficiency is improved by 37% compared to manual testing.Firstly,the influence of compression force and vibration exciter position on natural frequency test is analyzed by amplitude-frequency curve,so as to calibrate servo cylinder and fourdimensional motion platform.Secondly,the sine wave signal is used as the excitation to sweep the blade linearly,and the natural frequency is determined by the amplitude peak in the frequency domain.Finally,the accuracy experiment and efficiency experiment are carried out on the developed test system,whose results verify its high efficiency and high precision.

Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

Research on a Monte Carlo global variance reduction method based on an automatic importance sampling method

Global variance reduction is a bottleneck in Monte Carlo shielding calculations.The global variance reduction problem requires that the statistical error of the entire space is uniform.This study proposed a grid-AIS method for the global variance reduction problem based on the AIS method,which was implemented in the Monte Carlo program MCShield.The proposed method was validated using the VENUS-Ⅲ international benchmark problem and a self-shielding calculation example.The results from the VENUS-Ⅲ benchmark problem showed that the grid-AIS method achieved a significant reduction in the variance of the statistical errors of the MESH grids,decreasing from 1.08×10^(-2) to 3.84×10^(-3),representing a 64.00% reduction.This demonstrates that the grid-AIS method is effective in addressing global issues.The results of the selfshielding calculation demonstrate that the grid-AIS method produced accurate computational results.Moreover,the grid-AIS method exhibited a computational efficiency approximately one order of magnitude higher than that of the AIS method and approximately two orders of magnitude higher than that of the conventional Monte Carlo method.

General Optimal Trajectory Planning:Enabling Autonomous Vehicles with the Principle of Least Action

This study presents a general optimal trajectory planning(GOTP)framework for autonomous vehicles(AVs)that can effectively avoid obstacles and guide AVs to complete driving tasks safely and efficiently.Firstly,we employ the fifth-order Bezier curve to generate and smooth the reference path along the road centerline.Cartesian coordinates are then transformed to achieve the curvature continuity of the generated curve.Considering the road constraints and vehicle dynamics,limited polynomial candidate trajectories are generated and smoothed in a curvilinear coordinate system.Furthermore,in selecting the optimal trajectory,we develop a unified and auto-tune objective function based on the principle of least action by employing AVs to simulate drivers’behavior and summarizing their manipulation characteristics of“seeking benefits and avoiding losses.”Finally,by integrating the idea of receding-horizon optimization,the proposed framework is achieved by considering dynamic multi-performance objectives and selecting trajectories that satisfy feasibility,optimality,and adaptability.Extensive simulations and experiments are performed,and the results demonstrate the framework’s feasibility and effectiveness,which avoids both dynamic and static obstacles and applies to various scenarios with multi-source interactive traffic participants.Moreover,we prove that the proposed method can guarantee real-time planning and safety requirements compared to drivers’manipulation.

RRT Autonomous Detection Algorithm Based on Multiple Pilot Point Bias Strategy and Karto SLAM Algorithm

A Rapid-exploration Random Tree(RRT)autonomous detection algorithm based on the multi-guide-node deflection strategy and Karto Simultaneous Localization and Mapping(SLAM)algorithm was proposed to solve the problems of low efficiency of detecting frontier boundary points and drift distortion in the process of map building in the traditional RRT algorithm in the autonomous detection strategy of mobile robot.Firstly,an RRT global frontier boundary point detection algorithm based on the multi-guide-node deflection strategy was put forward,which introduces the reference value of guide nodes’deflection probability into the random sampling function so that the global search tree can detect frontier boundary points towards the guide nodes according to random probability.After that,a new autonomous detection algorithm for mobile robots was proposed by combining the graph optimization-based Karto SLAM algorithm with the previously improved RRT algorithm.The algorithm simulation platform based on the Gazebo platform was built.The simulation results show that compared with the traditional RRT algorithm,the proposed RRT autonomous detection algorithm can effectively reduce the time of autonomous detection,plan the length of detection trajectory under the condition of high average detection coverage,and complete the task of autonomous detection mapping more efficiently.Finally,with the help of the ROS-based mobile robot experimental platform,the performance of the proposed algorithm was verified in the real environment of different obstacles.The experimental results show that in the actual environment of simple and complex obstacles,the proposed RRT autonomous detection algorithm was superior to the traditional RRT autonomous detection algorithm in the time of detection,length of detection trajectory,and average coverage,thus improving the efficiency and accuracy of autonomous detection.

A Survey on an Emerging Safety Challenge for Autonomous Vehicles:Safety of the Intended Functionality

As the complexity of autonomous vehicles(AVs)continues to increase and artificial intelligence algorithms are becoming increasingly ubiquitous,a novel safety concern known as the safety of the intended functionality(SOTIF)has emerged,presenting significant challenges to the widespread deployment of AVs.SOTIF focuses on issues arising from the functional insufficiencies of the AVs’intended functionality or its implementation,apart from conventional safety considerations.From the systems engineering standpoint,this study offers a comprehensive exploration of the SOTIF landscape by reviewing academic research,practical activities,challenges,and perspectives across the development,verification,validation,and operation phases.Academic research encompasses system-level SOTIF studies and algorithm-related SOTIF issues and solutions.Moreover,it encapsulates practical SOTIF activities undertaken by corporations,government entities,and academic institutions spanning international and Chinese contexts,focusing on the overarching methodologies and practices in different phases.Finally,the paper presents future challenges and outlook pertaining to the development,verification,validation,and operation phases,motivating stakeholders to address the remaining obstacles and challenges.

Perception Enhanced Deep Deterministic Policy Gradient for Autonomous Driving in Complex Scenarios

Autonomous driving has witnessed rapid advancement;however,ensuring safe and efficient driving in intricate scenarios remains a critical challenge.In particular,traffic roundabouts bring a set of challenges to autonomous driving due to the unpredictable entry and exit of vehicles,susceptibility to traffic flow bottlenecks,and imperfect data in perceiving environmental information,rendering them a vital issue in the practical application of autonomous driving.To address the traffic challenges,this work focused on complex roundabouts with multi-lane and proposed a Perception EnhancedDeepDeterministic Policy Gradient(PE-DDPG)for AutonomousDriving in the Roundabouts.Specifically,themodel incorporates an enhanced variational autoencoder featuring an integrated spatial attention mechanism alongside the Deep Deterministic Policy Gradient framework,enhancing the vehicle’s capability to comprehend complex roundabout environments and make decisions.Furthermore,the PE-DDPG model combines a dynamic path optimization strategy for roundabout scenarios,effectively mitigating traffic bottlenecks and augmenting throughput efficiency.Extensive experiments were conducted with the collaborative simulation platform of CARLA and SUMO,and the experimental results show that the proposed PE-DDPG outperforms the baseline methods in terms of the convergence capacity of the training process,the smoothness of driving and the traffic efficiency with diverse traffic flow patterns and penetration rates of autonomous vehicles(AVs).Generally,the proposed PE-DDPGmodel could be employed for autonomous driving in complex scenarios with imperfect data.

Automatic Miscalibration Detection and Correction of LiDAR and Camera Using Motion Cues

This paper aims to develop an automatic miscalibration detection and correction framework to maintain accurate calibration of LiDAR and camera for autonomous vehicle after the sensor drift.First,a monitoring algorithm that can continuously detect the miscalibration in each frame is designed,leveraging the rotational motion each individual sensor observes.Then,as sensor drift occurs,the projection constraints between visual feature points and LiDAR 3-D points are used to compute the scaled camera motion,which is further utilized to align the drifted LiDAR scan with the camera image.Finally,the proposed method is sufficiently compared with two representative approaches in the online experiments with varying levels of random drift,then the method is further extended to the offline calibration experiment and is demonstrated by a comparison with two existing benchmark methods.

Design of Fully Automatic Specification Selection System for Resistance Welding Equipment

A system for fully automatic selection of welding specifications in resistance welding equipment has been developed to address the problem of workers frequently choosing the wrong specifications during manual welding of multiple parts on a single machine in automobile factories. The system incorporates an automatic recognition system for different workpiece materials using the added machine fixture,visual detection system for nuts and bolts,and secondary graphical confirmation to ensure the correctness of specification calling. This system achieves reliable,fully automatic selection of welding specifications in resistance welding equipment and has shown significant effects in improving welding quality for massproduced workpieces,while solving the problem of specification calling errors that can occur with traditional methods involving process charts and code adjustments. This system is particularly suitable for promoting applications in manual welding of multiple parts on a single machine in automobile factories,ensuring correct specification calling and welding quality.

Automatic modulation recognition of radiation source signals based on two-dimensional data matrix and improved residual neural network

Automatic modulation recognition(AMR)of radiation source signals is a research focus in the field of cognitive radio.However,the AMR of radiation source signals at low SNRs still faces a great challenge.Therefore,the AMR method of radiation source signals based on two-dimensional data matrix and improved residual neural network is proposed in this paper.First,the time series of the radiation source signals are reconstructed into two-dimensional data matrix,which greatly simplifies the signal preprocessing process.Second,the depthwise convolution and large-size convolutional kernels based residual neural network(DLRNet)is proposed to improve the feature extraction capability of the AMR model.Finally,the model performs feature extraction and classification on the two-dimensional data matrix to obtain the recognition vector that represents the signal modulation type.Theoretical analysis and simulation results show that the AMR method based on two-dimensional data matrix and improved residual network can significantly improve the accuracy of the AMR method.The recognition accuracy of the proposed method maintains a high level greater than 90% even at -14 dB SNR.

Bearing mechanism of roof and rib support structure in automatically formed roadway and its support design method

Non-pillar mining technology with automatically formed roadway is a new mining method without coal pillar reservation and roadway excavation.The stability control of automatically formed roadway is the key to the successful application of the new method.In order to realize the stability control of the roadway surrounding rock,the mechanical model of the roof and rib support structure is established,and the influence mechanism of the automatically formed roadway parameters on the compound force is revealed.On this basis,the roof and rib support structure technology of confined lightweight concrete is proposed,and its mechanical tests under different eccentricity are carried out.The results show that the bearing capacity of confined lightweight concrete specimens is basically the same as that of ordinary confined concrete specimens.The bearing capacity of confined lightweight concrete specimens under different eccentricities is 1.95 times higher than those of U-shaped steel specimens.By comparing the test results with the theoretical calculated results of the confined concrete,the calculation method of the bearing capacity for the confined lightweight concrete structure is selected.The design method of confined lightweight concrete support structure is established,and is successfully applied in the extra-large mine,Ningtiaota Coal Mine,China.

Probabilistic Global Maximum Power Point Tracking Algorithm for Continuously Varying Partial Shading Conditions on Autonomous PV Systems

A photovoltaic (PV) string with multiple modules with bypass diodes frequently deployed on a variety of autonomous PV systems may present multiple power peaks under uneven shading. For optimal solar harvesting, there is a need for a control schema to force the PV string to operate at global maximum power point (GMPP). While a lot of tracking methods have been proposed in the literature, they are usually complex and do not fully take advantage of the available characteristics of the PV array. This work highlights how the voltage at operating point and the forward voltage of the bypass diode are considered to design a global maximum power point tracking (GMPPT) algorithm with a very limited global search phase called Fast GMPPT. This algorithm successfully tracks GMPP between 94% and 98% of the time under a theoretical evaluation. It is then compared against Perturb and Observe, Deterministic Particle Swarm Optimization, and Grey Wolf Optimization under a sequence of irradiance steps as well as a power-over-voltage characteristics profile that mimics the electrical characteristics of a PV string under varying partial shading conditions. Overall, the simulation with the sequence of irradiance steps shows that while Fast GMPPT does not have the best convergence time, it has an excellent convergence rate as well as causes the least amount of power loss during the global search phase. Experimental test under varying partial shading conditions shows that while the GMPPT proposal is simple and lightweight, it is very performant under a wide range of dynamically varying partial shading conditions and boasts the best energy efficiency (94.74%) out of the 4 tested algorithms.

A Few-Shot Learning-Based Automatic Modulation Classification Method for Internet of Things

Due to the limited computational capability and the diversity of the Internet of Things devices working in different environment,we consider fewshot learning-based automatic modulation classification(AMC)to improve its reliability.A data enhancement module(DEM)is designed by a convolutional layer to supplement frequency-domain information as well as providing nonlinear mapping that is beneficial for AMC.Multimodal network is designed to have multiple residual blocks,where each residual block has multiple convolutional kernels of different sizes for diverse feature extraction.Moreover,a deep supervised loss function is designed to supervise all parts of the network including the hidden layers and the DEM.Since different model may output different results,cooperative classifier is designed to avoid the randomness of single model and improve the reliability.Simulation results show that this few-shot learning-based AMC method can significantly improve the AMC accuracy compared to the existing methods.

Enhancing Autonomy Capability in Regional Power Grids:A Strategic Planning Approach withMultiple Autonomous Evaluation Indexes

After the integration of large-scale DistributedGeneration(DG)into the distribution network,the randomness and volatility of its output result in a reduction of spatiotemporal alignment between power generation and demand in the distribution network,exacerbating the phenomenon of wind and solar power wastage.As a novel power system model,the fundamental concept of Regional Autonomous Power Grids(RAPGs)is to achieve localized management and energy autonomy,thereby facilitating the effective consumption of DGs.Therefore,this paper proposes a distributed resource planning strategy that enhances the autonomy capabilities of regional power grids by considering multiple evaluation indexes for autonomy.First,a regional Energy Storage(ES)configuration strategy is proposed.This strategy can select a suitable reference value for the upper limit of ES configuration based on the regional load andDGoutput to maximize the elimination of source load deviations in the region as the upper limit constraint of ES capacity.Then,a control strategy for regional ES is proposed,the charging and discharging reference line of ES is set,and multiple autonomy and economic indexes are used as objective functions to select different proportions of ES to control the distributed resources of the regional power grid and establish evaluation indexes of the internal regional generation and load power ratio,the proportion of power supply matching hours,new energy consumption rate and tie line power imbalance outside the region to evaluate changes in the regional autonomy capabilities.The final simulation results showthat in the real regional grid example,the planning method in the planning year in the region of the overall power supply matching hour ratio and new energy consumption rate increased by 3.9%and 4.8%on average,and the power imbalance of the tie line decreased by 7.8%on average.The proposed planning approach enables the maximization of regional autonomy while effectively smoothing the fluctuation of power exchange between the regional grid and the higher-level grid.This presents a rational and effective planning solution for the regional grid,facilitating the coordinated development between the region and the distribution network.