Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

用Velocity Verlet积分器改进HMC抽样方法

Hamilton Monte Carlo (HMC)方法是一种常用的快速抽样方法.在对哈密顿方程进行抽样时,HMC方法使用Leapfrog积分器,这可能造成方程的位置及动量的迭代值在时间上不同步,其产生的误差会降低抽样效率及抽样结果的稳定性.为此,本文提出了IHMC(Improved HMC)方法,该方法用Velocity Verlet积分器替代Leapfrog积分器,每次迭代时都计算两变量在同一时刻的值.为验证方法的效果,本文进行了两个实验,一个是将该方法应用于非对称随机波动率模型(RASV模型)的参数估计,另一个是将方法应用于方差伽马分布的抽样,结果显示:IHMC方法比HMC方法的效率更高、结果更稳定.

ST-LSTM-SA:A New Ocean Sound Velocity Field Prediction Model Based on Deep Learning

The scarcity of in-situ ocean observations poses a challenge for real-time information acquisition in the ocean.Among the crucial hydroacoustic environmental parameters,ocean sound velocity exhibits significant spatial and temporal variability and it is highly relevant to oceanic research.In this study,we propose a new data-driven approach,leveraging deep learning techniques,for the prediction of sound velocity fields(SVFs).Our novel spatiotemporal prediction model,STLSTM-SA,combines Spatiotemporal Long Short-Term Memory(ST-LSTM) with a self-attention mechanism to enable accurate and real-time prediction of SVFs.To circumvent the limited amount of observational data,we employ transfer learning by first training the model using reanalysis datasets,followed by fine-tuning it using in-situ analysis data to obtain the final prediction model.By utilizing the historical 12-month SVFs as input,our model predicts the SVFs for the subsequent three months.We compare the performance of five models:Artificial Neural Networks(ANN),Long ShortTerm Memory(LSTM),Convolutional LSTM(ConvLSTM),ST-LSTM,and our proposed ST-LSTM-SA model in a test experiment spanning 2019 to 2022.Our results demonstrate that the ST-LSTM-SA model significantly improves the prediction accuracy and stability of sound velocity in both temporal and spatial dimensions.The ST-LSTM-SA model not only accurately predicts the ocean sound velocity field(SVF),but also provides valuable insights for spatiotemporal prediction of other oceanic environmental variables.

In vivo label-free measurement of blood flow velocity symmetry based on dual line scanning third-harmonic generation microscopy excited at the 1700 nm window

Measurement of bloodflow velocity is key to understanding physiology and pathology in vivo.While most measurements are performed at the middle of the blood vessel,little research has been done on characterizing the instantaneous bloodflow velocity distribution.This is mainly due to the lack of measurement technology with high spatial and temporal resolution.Here,we tackle this problem with our recently developed dual-wavelength line-scan third-harmonic generation(THG)imaging technology.Simultaneous acquisition of dual-wavelength THG line-scanning signals enables measurement of bloodflow velocities at two radially symmetric positions in both venules and arterioles in mouse brain in vivo.Our results clearly show that the instantaneous bloodflow velocity is not symmetric under general conditions.

Flood Velocity Prediction Using Deep Learning Approach

Floods are one of the most serious natural disasters that can cause huge societal and economic losses.Extensive research has been conducted on topics like flood monitoring,prediction,and loss estimation.In these research fields,flood velocity plays a crucial role and is an important factor that influences the reliability of the outcomes.Traditional methods rely on physical models for flood simulation and prediction and could generate accurate results but often take a long time.Deep learning technology has recently shown significant potential in the same field,especially in terms of efficiency,helping to overcome the time-consuming associated with traditional methods.This study explores the potential of deep learning models in predicting flood velocity.More specifically,we use a Multi-Layer Perceptron(MLP)model,a specific type of Artificial Neural Networks(ANNs),to predict the velocity in the test area of the Lundesokna River in Norway with diverse terrain conditions.Geographic data and flood velocity simulated based on the physical hydraulic model are used in the study for the pre-training,optimization,and testing of the MLP model.Our experiment indicates that the MLP model has the potential to predict flood velocity in diverse terrain conditions of the river with acceptable accuracy against simulated velocity results but with a significant decrease in training time and testing time.Meanwhile,we discuss the limitations for the improvement in future work.

Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

Orientation and Decision-Making for Soccer Based on Sports Analytics and AI:A Systematic Review

Due to ever-growing soccer data collection approaches and progressing artificial intelligence(AI) methods, soccer analysis, evaluation, and decision-making have received increasing interest from not only the professional sports analytics realm but also the academic AI research community. AI brings gamechanging approaches for soccer analytics where soccer has been a typical benchmark for AI research. The combination has been an emerging topic. In this paper, soccer match analytics are taken as a complete observation-orientation-decision-action(OODA) loop.In addition, as in AI frameworks such as that for reinforcement learning, interacting with a virtual environment enables an evolving model. Therefore, both soccer analytics in the real world and virtual domains are discussed. With the intersection of the OODA loop and the real-virtual domains, available soccer data, including event and tracking data, and diverse orientation and decisionmaking models for both real-world and virtual soccer matches are comprehensively reviewed. Finally, some promising directions in this interdisciplinary area are pointed out. It is claimed that paradigms for both professional sports analytics and AI research could be combined. Moreover, it is quite promising to bridge the gap between the real and virtual domains for soccer match analysis and decision-making.

Ethical Decision-Making Framework Based on Incremental ILP Considering Conflicts

Humans are experiencing the inclusion of artificial agents in their lives,such as unmanned vehicles,service robots,voice assistants,and intelligent medical care.If the artificial agents cannot align with social values or make ethical decisions,they may not meet the expectations of humans.Traditionally,an ethical decision-making framework is constructed by rule-based or statistical approaches.In this paper,we propose an ethical decision-making framework based on incremental ILP(Inductive Logic Programming),which can overcome the brittleness of rule-based approaches and little interpretability of statistical approaches.As the current incremental ILP makes it difficult to solve conflicts,we propose a novel ethical decision-making framework considering conflicts in this paper,which adopts our proposed incremental ILP system.The framework consists of two processes:the learning process and the deduction process.The first process records bottom clauses with their score functions and learns rules guided by the entailment and the score function.The second process obtains an ethical decision based on the rules.In an ethical scenario about chatbots for teenagers’mental health,we verify that our framework can learn ethical rules and make ethical decisions.Besides,we extract incremental ILP from the framework and compare it with the state-of-the-art ILP systems based on ASP(Answer Set Programming)focusing on conflict resolution.The results of comparisons show that our proposed system can generate better-quality rules than most other systems.

Offline Reinforcement Learning with Constrained Hybrid Action Implicit Representation Towards Wargaming Decision-Making

Reinforcement Learning(RL)has emerged as a promising data-driven solution for wargaming decision-making.However,two domain challenges still exist:(1)dealing with discrete-continuous hybrid wargaming control and(2)accelerating RL deployment with rich offline data.Existing RL methods fail to handle these two issues simultaneously,thereby we propose a novel offline RL method targeting hybrid action space.A new constrained action representation technique is developed to build a bidirectional mapping between the original hybrid action space and a latent space in a semantically consistent way.This allows learning a continuous latent policy with offline RL with better exploration feasibility and scalability and reconstructing it back to a needed hybrid policy.Critically,a novel offline RL optimization objective with adaptively adjusted constraints is designed to balance the alleviation and generalization of out-of-distribution actions.Our method demonstrates superior performance and generality across different tasks,particularly in typical realistic wargaming scenarios.

Origin of tradeoff between movement velocity and attachment duration of kinesin motor on a microtubule

Kinesin-1 motor protein is a homodimer containing two identical motor domains connected by a common long coiledcoil stalk via two flexible neck linkers. The motor can step on a microtubule with a velocity of about 1 μm·s-1and an attachment duration of about 1 s under physiological conditions. The available experimental data indicate a tradeoff between velocity and attachment duration under various experimental conditions, such as variation of the solution temperature,variation of the strain between the two motor domains, and so on. However, the underlying mechanism of the tradeoff is unknown. Here, the mechanism is explained by a theoretical study of the dynamics of the motor under various experimental conditions, reproducing quantitatively the available experimental data and providing additional predictions. How the various experimental conditions lead to different decreasing rates of attachment duration versus velocity is also explained.

Stroke Risk Assessment Decision-Making Using a Machine Learning Model:Logistic-AdaBoost

Stroke is a chronic cerebrovascular disease that carries a high risk.Stroke risk assessment is of great significance in preventing,reversing and reducing the spread and the health hazards caused by stroke.Aiming to objectively predict and identify strokes,this paper proposes a new stroke risk assessment decision-making model named Logistic-AdaBoost(Logistic-AB)based on machine learning.First,the categorical boosting(CatBoost)method is used to perform feature selection for all features of stroke,and 8 main features are selected to form a new index evaluation system to predict the risk of stroke.Second,the borderline synthetic minority oversampling technique(SMOTE)algorithm is applied to transform the unbalanced stroke dataset into a balanced dataset.Finally,the stroke risk assessment decision-makingmodel Logistic-AB is constructed,and the overall prediction performance of this new model is evaluated by comparing it with ten other similar models.The comparison results show that the new model proposed in this paper performs better than the two single algorithms(logistic regression and AdaBoost)on the four indicators of recall,precision,F1 score,and accuracy,and the overall performance of the proposed model is better than that of common machine learning algorithms.The Logistic-AB model presented in this paper can more accurately predict patients’stroke risk.

Research on Maneuver Decision-Making of Multi-Agent Adversarial Game in a Random Interference Environment

The strategy evolution process of game players is highly uncertain due to random emergent situations and other external disturbances.This paper investigates the issue of strategy interaction and behavioral decision-making among game players in simulated confrontation scenarios within a random interference environment.It considers the possible risks that random disturbances may pose to the autonomous decision-making of game players,as well as the impact of participants’manipulative behaviors on the state changes of the players.A nonlinear mathematical model is established to describe the strategy decision-making process of the participants in this scenario.Subsequently,the strategy selection interaction relationship,strategy evolution stability,and dynamic decision-making process of the game players are investigated and verified by simulation experiments.The results show that maneuver-related parameters and random environmental interference factors have different effects on the selection and evolutionary speed of the agent’s strategies.Especially in a highly uncertain environment,even small information asymmetry or miscalculation may have a significant impact on decision-making.This also confirms the feasibility and effectiveness of the method proposed in the paper,which can better explain the behavioral decision-making process of the agent in the interaction process.This study provides feasibility analysis ideas and theoretical references for improving multi-agent interactive decision-making and the interpretability of the game system model.

Probing the Galactic Halo with RR Lyrae Stars.VI.The Radial Velocity Curve Templates of RRc Stars

We present radial velocity(RV)curve templates of RR Lyrae first-overtone(RRc)stars constructed with the Mg I b triplet and Hαlines using time-domain Medium-Resolution Survey spectra of seven RRc stars from Large Sky Area Multi-Object Fiber Spectroscopic Telescope(LAMOST)Data Release 9.Additionally,we derive the relation between the stellar RV curve amplitudes and g-band light curve amplitudes from Zwicky Transient Facility(ZTF)public survey.For those RRc stars without ZTF g-band light curves,we provide the conversions from the light curve amplitudes in ZTF r-and i-bands,Gaia G-band,and V-band from the All-Sky Automated Survey for Supernovae to those in ZTF g-band.We validate our RV curve templates using the RRc star SV Scl and find the uncertainties of systemic RV are less than 2.11 km s~(-1)and 6.08 km s~(-1)based on the Mg I b triplet and Hαlines,respectively.We calculate the systemic RVs of 30 RRc stars using the RV curve templates constructed with the Mg I b triplet and Hαlines and find the systemic RVs are comparable with each other.This RV curve template will be particularly useful for obtaining the systemic RV of RRc using the LAMOST spectroscopy.

Automatic velocity picking based on optimal key points tracking algorithm

Picking velocities from semblances manually is laborious and necessitates experience. Although various methods for automatic velocity picking have been developed, there remains a challenge in efficiently incorporating information from nearby gathers to ensure picked velocity aligns with seismic horizons while also improving picking accuracy. The conventional method of velocity picking from a semblance volume is computationally demanding, highlighting a need for a more efficient strategy. In this study, we introduce a novel method for automatic velocity picking based on multi-object tracking. This dynamic tracking process across different semblance panels can integrate information from nearby gathers effectively while maintaining computational efficiency. First, we employ accelerated density clustering on the velocity spectrum to discern cluster centers without the requirement for prior knowledge regarding the number of clusters. These cluster centers embody the maximum likelihood velocities of the main subsurface structures. Second, our proposed method tracks key points within the semblance volume. Kalman filter is adopted to adjust the tracking process, followed by interpolation on these tracked points to construct the final velocity model. Our synthetic data example demonstrates that our proposed algorithm can effectively rectify the picking errors of the clustering algorithm. We further compare the performances of the clustering method(CM), the proposed tracking method(TM), and the variational method(VM) on a field dataset from the Gulf of Mexico. The results attest that our method offers superior accuracy than CM, achieves comparable accuracy with VM, and benefits from a reduced computational cost.

Determination of the best materials for development and designing product using a multi-criteria decision-making

Purpose–Material selection,driven by wide and often conflicting objectives,is an important,sometimes difficult problem in material engineering.In this context,multi-criteria decision-making(MCDM)methodologies are effective.An approach of MCDM is needed to cater to criteria of material assortment simultaneously.More firms are now concerned about increasing their productivity using mathematical tools.To occupy a gap in the previous literature this research recommends an integrated MCDM and mathematical Bi-objective model for the selection of material.In addition,by using the Technique for Order Preference by Similarity to Ideal Solution(TOPSIS),the inherent ambiguities of decision-makers in paired evaluations are considered in this research.It goes on to construct a mathematical bi-objective model for determining the best item to purchase.Design/methodology/approach–The entropy perspective is implemented in this paper to evaluate the weight parameters,while the TOPSIS technique is used to determine the best and worst intermediate pipe materials for automotive exhaust system.The intermediate pipes are used to join the components of the exhaust systems.The materials usually used to manufacture intermediate pipe are SUS 436LM,SUS 430,SUS 304,SUS 436L,SUH 409 L,SUS 441 L and SUS 439L.These seven materials are evaluated based on tensile strength(TS),hardness(H),elongation(E),yield strength(YS)and cost(C).A hybrid methodology combining entropy-based criteria weighting,with the TOPSIS for alternative ranking,is pursued to identify the optimal design material for an engineered application in this paper.This study aims to help while filling the information gap in selecting the most suitable material for use in the exhaust intermediate pipes.After that,the authors searched for and considered eight materials and evaluated them on the following five criteria:(1)TS,(2)YS,(3)H,(4)E and(5)C.The first two criteria have been chosen because they can have a lot of influence on the behavior of the exhaust intermediate pipes,on their performance and on the cost.In this structure,the weights of the criteria are calculated objectively through the entropy method in order to have an unbiased assessment.This essentially measures the quantity of information each criterion contribution,indicating the relative importance of these criteria better.Subsequently,the materials were ranked using the TOPSIS method in terms of their relative performance by measuring each material from an ideal solution to determine the best alternative.The results show that SUS 309,SUS 432L and SUS 436 LM are the first three materials that the exhaust intermediate pipe optimal design should consider.Findings–The material matrix of the decision presented in Table 3 was normalized through Equation 5,as shown in Table 5,and the matrix was multiplied with weighting criteriaß_j.The obtained weighted normalized matrix V_ij is presented in Table 6.However,the ideal,worst and best value was ascertained by employing Equation 7.This study is based on the selection of material for the development of intermediate pipe using MCDM,and it involves four basic stages,i.e.method of translation criteria,screening process,method of ranking and search for methods.The selection was done through the TOPSIS method,and the criteria weight was obtained by the entropy method.The result showed that the top three materials are SUS 309,SUS 432L and SUS 436 LM,respectively.For the future work,it is suggested to select more alternatives and criteria.The comparison can also be done by using different MCDM techniques like and Choice Expressing Reality(ELECTRE),Decision-Making Trial and Evaluation Laboratory(DEMATEL)and Preference Ranking Organization Method for Enrichment Evaluation(PROMETHEE).Originality/value–The results provide important conclusions for material selection in this targeted application,verifying the employment of mutual entropy-TOPSIS methodology for a series of difficult engineering decisions in material engineering concepts that combine superior capacity with better performance as well as cost-efficiency in various engineering design.

Velocity equation for grenades while impacting on dry sand media

This paper deals with the collision of sphere shape grenades with sand media.The central issue of the article is the establishing of an empirical velocity equation of the grenade while impacting on sand that is used to solve motion equations of the mechanical mechanism inside the impact grenade fuze.The paper focuses on impact velocities that are lower than 5 m s^(-1).An experiment was conducted to study the velocity of the grenade while impacting on dry sand.A high-speed camera video was used to capture the grenade positions.The grenade velocity in the impact process was generated from these video data.Some types of fitting curves are used to regress the velocity equation of the grenade while interacting with the sand media and the best-fitting model is chosen.The result shows the regression curve has a high correlation with the experiment data for grenade velocities below 5 m s^(-1).The received regression equation is useful for analyzing the working ability of the inertial mechanism inside the impact grenade or analyzing and choosing the appropriate parameters of each part in the inertial mechanism to meet the required characteristics of the mechanism.

Swimming velocity of spherical squirmers in a square tube at finite fluid inertia

The three-dimensional lattice Boltzmann method(LBM)is used to simulate the motion of a spherical squirmer in a square tube,and the steady motion velocity of a squirmer with different Reynolds numbers(Re,ranging from 0.1 to 2)and swimming types is investigated and analyzed to better understand the swimming characteristics of microorganisms in different environments.First,as the Reynolds number increases,the effect of the inertial forces becomes significant,disrupting the squirmer's ability to maintain its theoretical velocity.Specifically,as the Reynolds number increases,the structure of the flow field around the squirmer changes,affecting its velocity of motion.Notably,the swimming velocity of the squirmer exhibits a quadratic relationship with the type of swimming and the Reynolds number.Second,the narrow tube exerts a significant inhibitory effect on the squirmer motion.In addition,although chirality does not directly affect the swimming velocity of the squirmer,it can indirectly affect the velocity by changing its motion mode.

Structural similarity of lithospheric velocity models of Chinese mainland

Existing lithospheric velocity models exhibit similar structures typically associated with the first-order tectonic features,with dissimilarities due to different data and methods used in model generation.The quantification of model structural similarity can help in interpreting the geophysical properties of Earth's interior and establishing unified models crucial in natural hazard assessment and resource exploration.Here we employ the complex wavelet structural similarity index measure(CW-SSIM)active in computer image processing to analyze the structural similarity of four lithospheric velocity models of Chinese mainland published in the past decade.We take advantage of this method in its multiscale definition and insensitivity to slight geometrical distortion like translation and scaling,which is particularly crucial in the structural similarity analysis of velocity models accounting for uncertainty and resolution.Our results show that the CW-SSIM values vary in different model pairs,horizontal locations,and depths.While variations in the inter-model CW-SSIM are partly owing to different databases in the model generation,the difference of tomography methods may significantly impact the similar structural features of models,such as the low similarities between the full-wave based FWEA18 and other three models in northeastern China.We finally suggest potential solutions for the next generation of tomographic modeling in different areas according to corresponding structural similarities of existing models.

The Spherical q-Linear Diophantine Fuzzy Multiple-Criteria Group Decision-Making Based on Differential Measure

Spherical q-linearDiophantine fuzzy sets(Sq-LDFSs)provedmore effective for handling uncertainty and vagueness in multi-criteria decision-making(MADM).It does not only cover the data in two variable parameters but is also beneficial for three parametric data.By Pythagorean fuzzy sets,the difference is calculated only between two parameters(membership and non-membership).According to human thoughts,fuzzy data can be found in three parameters(membership uncertainty,and non-membership).So,to make a compromise decision,comparing Sq-LDFSs is essential.Existing measures of different fuzzy sets do,however,can have several flaws that can lead to counterintuitive results.For instance,they treat any increase or decrease in the membership degree as the same as the non-membership degree because the uncertainty does not change,even though each parameter has a different implication.In the Sq-LDFSs comparison,this research develops the differentialmeasure(DFM).Themain goal of the DFM is to cover the unfair arguments that come from treating different types of FSs opposing criteria equally.Due to their relative positions in the attribute space and the similarity of their membership and non-membership degrees,two Sq-LDFSs formthis preference connectionwhen the uncertainty remains same in both sets.According to the degree of superiority or inferiority,two Sq-LDFSs are shown as identical,equivalent,superior,or inferior over one another.The suggested DFM’s fundamental characteristics are provided.Based on the newly developed DFM,a unique approach tomultiple criterion group decision-making is offered.Our suggestedmethod verifies the novel way of calculating the expert weights for Sq-LDFSS as in PFSs.Our proposed technique in three parameters is applied to evaluate solid-state drives and choose the optimum photovoltaic cell in two applications by taking uncertainty parameter zero.The method’s applicability and validity shown by the findings are contrasted with those obtained using various other existing approaches.To assess its stability and usefulness,a sensitivity analysis is done.

Natural vibration and critical velocity of translating Timoshenko beam with non-homogeneous boundaries

In most practical engineering applications,the translating belt wraps around two fixed wheels.The boundary conditions of the dynamic model are typically specified as simply supported or fixed boundaries.In this paper,non-homogeneous boundaries are introduced by the support wheels.Utilizing the translating belt as the mechanical prototype,the vibration characteristics of translating Timoshenko beam models with nonhomogeneous boundaries are investigated for the first time.The governing equations of Timoshenko beam are deduced by employing the generalized Hamilton's principle.The effects of parameters such as the radius of wheel and the length of belt on vibration characteristics including the equilibrium deformations,critical velocities,natural frequencies,and modes,are numerically calculated and analyzed.The numerical results indicate that the beam experiences deformation characterized by varying curvatures near the wheels.The radii of the wheels play a pivotal role in determining the change in trend of the relative difference between two beam models.Comparing the results unearths that the relative difference in equilibrium deformations between the two beam models is more pronounced with smaller-sized wheels.When the two wheels are of equal size,the critical velocities of both beam models reach their respective minima.In addition,the relative difference in natural frequencies between the two beam models exhibits nonlinear variation and can easily exceed 50%.Furthermore,as the axial velocities increase,the impact of non-homogeneous boundaries on modal shape of translating beam becomes more significant.Although dealing with non-homogeneous boundaries is challenging,beam models with non-homogeneous boundaries are more sensitive to parameters,and the differences between the two types of beams undergo some interesting variations under the influence of non-homogeneous boundaries.