Machine learning applications in stroke medicine:advancements,challenges,and future prospectives

Stroke is a leading cause of disability and mortality worldwide,necessitating the development of advanced technologies to improve its diagnosis,treatment,and patient outcomes.In recent years,machine learning techniques have emerged as promising tools in stroke medicine,enabling efficient analysis of large-scale datasets and facilitating personalized and precision medicine approaches.This abstract provides a comprehensive overview of machine learning’s applications,challenges,and future directions in stroke medicine.Recently introduced machine learning algorithms have been extensively employed in all the fields of stroke medicine.Machine learning models have demonstrated remarkable accuracy in imaging analysis,diagnosing stroke subtypes,risk stratifications,guiding medical treatment,and predicting patient prognosis.Despite the tremendous potential of machine learning in stroke medicine,several challenges must be addressed.These include the need for standardized and interoperable data collection,robust model validation and generalization,and the ethical considerations surrounding privacy and bias.In addition,integrating machine learning models into clinical workflows and establishing regulatory frameworks are critical for ensuring their widespread adoption and impact in routine stroke care.Machine learning promises to revolutionize stroke medicine by enabling precise diagnosis,tailored treatment selection,and improved prognostication.Continued research and collaboration among clinicians,researchers,and technologists are essential for overcoming challenges and realizing the full potential of machine learning in stroke care,ultimately leading to enhanced patient outcomes and quality of life.This review aims to summarize all the current implications of machine learning in stroke diagnosis,treatment,and prognostic evaluation.At the same time,another purpose of this paper is to explore all the future perspectives these techniques can provide in combating this disabling disease.

Evolutionary Decision-Making and Planning for Autonomous Driving Based on Safe and Rational Exploration and Exploitation

Decision-making and motion planning are extremely important in autonomous driving to ensure safe driving in a real-world environment.This study proposes an online evolutionary decision-making and motion planning framework for autonomous driving based on a hybrid data-and model-driven method.First,a data-driven decision-making module based on deep reinforcement learning(DRL)is developed to pursue a rational driving performance as much as possible.Then,model predictive control(MPC)is employed to execute both longitudinal and lateral motion planning tasks.Multiple constraints are defined according to the vehicle’s physical limit to meet the driving task requirements.Finally,two principles of safety and rationality for the self-evolution of autonomous driving are proposed.A motion envelope is established and embedded into a rational exploration and exploitation scheme,which filters out unreasonable experiences by masking unsafe actions so as to collect high-quality training data for the DRL agent.Experiments with a high-fidelity vehicle model and MATLAB/Simulink co-simulation environment are conducted,and the results show that the proposed online-evolution framework is able to generate safer,more rational,and more efficient driving action in a real-world environment.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

UAV-Assisted Dynamic Avatar Task Migration for Vehicular Metaverse Services: A Multi-Agent Deep Reinforcement Learning Approach

Avatars, as promising digital representations and service assistants of users in Metaverses, can enable drivers and passengers to immerse themselves in 3D virtual services and spaces of UAV-assisted vehicular Metaverses. However, avatar tasks include a multitude of human-to-avatar and avatar-to-avatar interactive applications, e.g., augmented reality navigation,which consumes intensive computing resources. It is inefficient and impractical for vehicles to process avatar tasks locally. Fortunately, migrating avatar tasks to the nearest roadside units(RSU)or unmanned aerial vehicles(UAV) for execution is a promising solution to decrease computation overhead and reduce task processing latency, while the high mobility of vehicles brings challenges for vehicles to independently perform avatar migration decisions depending on current and future vehicle status. To address these challenges, in this paper, we propose a novel avatar task migration system based on multi-agent deep reinforcement learning(MADRL) to execute immersive vehicular avatar tasks dynamically. Specifically, we first formulate the problem of avatar task migration from vehicles to RSUs/UAVs as a partially observable Markov decision process that can be solved by MADRL algorithms. We then design the multi-agent proximal policy optimization(MAPPO) approach as the MADRL algorithm for the avatar task migration problem. To overcome slow convergence resulting from the curse of dimensionality and non-stationary issues caused by shared parameters in MAPPO, we further propose a transformer-based MAPPO approach via sequential decision-making models for the efficient representation of relationships among agents. Finally, to motivate terrestrial or non-terrestrial edge servers(e.g., RSUs or UAVs) to share computation resources and ensure traceability of the sharing records, we apply smart contracts and blockchain technologies to achieve secure sharing management. Numerical results demonstrate that the proposed approach outperforms the MAPPO approach by around 2% and effectively reduces approximately 20% of the latency of avatar task execution in UAV-assisted vehicular Metaverses.

Cognitive interference decision method for air defense missile fuze based on reinforcement learning

To solve the problem of the low interference success rate of air defense missile radio fuzes due to the unified interference form of the traditional fuze interference system,an interference decision method based Q-learning algorithm is proposed.First,dividing the distance between the missile and the target into multiple states to increase the quantity of state spaces.Second,a multidimensional motion space is utilized,and the search range of which changes with the distance of the projectile,to select parameters and minimize the amount of ineffective interference parameters.The interference effect is determined by detecting whether the fuze signal disappears.Finally,a weighted reward function is used to determine the reward value based on the range state,output power,and parameter quantity information of the interference form.The effectiveness of the proposed method in selecting the range of motion space parameters and designing the discrimination degree of the reward function has been verified through offline experiments involving full-range missile rendezvous.The optimal interference form for each distance state has been obtained.Compared with the single-interference decision method,the proposed decision method can effectively improve the success rate of interference.

A Comparative Study on the Post-Buckling Behavior of Reinforced Thermoplastic Pipes(RTPs)Under External Pressure Considering Progressive Failure

The collapse pressure is a key parameter when RTPs are applied in harsh deep-water environments.To investigate the collapse of RTPs,numerical simulations and hydrostatic pressure tests are conducted.For the numerical simulations,the eigenvalue analysis and Riks analysis are combined,in which the Hashin failure criterion and fracture energy stiffness degradation model are used to simulate the progressive failure of composites,and the“infinite”boundary conditions are applied to eliminate the boundary effects.As for the hydrostatic pressure tests,RTP specimens were placed in a hydrostatic chamber after filled with water.It has been observed that the cross-section of the middle part collapses when it reaches the maximum pressure.The collapse pressure obtained from the numerical simulations agrees well with that in the experiment.Meanwhile,the applicability of NASA SP-8007 formula on the collapse pressure prediction was also discussed.It has a relatively greater difference because of the ignorance of the progressive failure of composites.For the parametric study,it is found that RTPs have much higher first-ply-failure pressure when the winding angles are between 50°and 70°.Besides,the effect of debonding and initial ovality,and the contribution of the liner and coating are also discussed.

Strong ground motion characteristics observed in the February 6,2023 M_(W)7.7 Türkiye earthquake

Türkiye is located in a seismically active region,where the Anatolian,African,and Arabian tectonic plates converge.High seismic hazards cause the region to be struck repeatedly by major earthquakes.On February 06,2023,a devastating M_(W)7.7 earthquake struck Türkiye at 01:17 am local time(01:17 UTC).In this regard,near and far-field ground motion data within the distance of 120 km are compiled and later characterized to identify the key ground motion intensity measures.Additionally,the vertical components of ground motions were examined to capture the complete three-dimensional nature of the seismic event.Moreover,the effect of Pulse-Like(PL)and Non-Pulse-Like(NPL)ground motion on a representative RC frame structure built as per the Türkiye code was investigated.The results indicate that PL behavior was observed in both horizontal and vertical components of ground motions and PL behavior were noted both near the epicenter and at higher distances from the epicenter.Moreover,the ratio of the peak vertical acceleration to peak horizontal acceleration at certain stations was found to be close to 1.Finally,the non-linear time history analysis of the representative reinforced concrete frame structure for ground motions recorded at stations located equidistant from the epicenter,indicated that PL ground motions led to more significant damage compared to NPL ground motions.

Development and application of novel high‐efficiency composite ultrafine cement grouts for roadway in fractured surrounding rocks

The fractured surrounding rocks of roadways pose major challenges to safe mining.Grouting has often been used to reinforce the surrounding rocks to mitigate the safety risks associated with fractured rocks.The aim of this study is to develop highly efficient composite ultrafine cement(CUC)grouts to reinforce the roadway in fractured surrounding rocks.The materials used are ultrafine cement(UC),ultrafine fly ash(UF),ultrafine slag(US),and additives(superplasticizer[SUP],aluminate ultrafine expansion agent[AUA],gypsum,and retarder).The fluidity,bleeding,shrinkage,setting time,chemical composition,microstructure,degree of hydration,and mechanical property of grouting materials were evaluated in this study.Also,a suitable and effective CUC grout mixture was used to reinforce the roadway in the fractured surrounding rock.The results have shown that the addition of UF and US reduces the plastic viscosity of CUC,and the best fluidity can be obtained by adding 40%UF and 10%US.Since UC and UF particles are small,the pozzolanic effect of UF promotes the hydration reaction,which is conductive to the stability of CUC grouts.In addition,fine particles of UC,UF,and US can effectively fill the pores,while the volumetric expansion of AUA and gypsum decreases the pores and thus affects the microstructure of the solidified grout.The compressive test results have shown that the addition of specific amounts of UF and US can ameliorate the mechanical properties of CUC grouts.Finally,the CUC22‐8 grout was used to reinforce the No.20322 belt roadway.The results of numerical simulation and field monitoring have indicated that grouting can efficaciously reinforce the surrounding rock of the roadway.In this research,high‐performance CUC grouts were developed for surrounding rock reinforcement of underground engineering by utilizing UC and some additives.

Microstructural characterization,tribological and corrosion behavior of AA7075-TiC composites

Aluminum alloys are the potential materials in the automobile and aerospace sectors due to their lower density,easy forming and excellent corrosion resistance.The demand of high strength-to-weight ratio materials in structural applications needs the engineering industries to seek aluminum alloy with new versions of hard and brittle ceramic particles.The microstructure,hardness,wear and corrosion behaviors of AA7075 composites with 2.5wt.%and 5wt.%TiC particles were studied.Microscopic analysis is evident that the transformation of the strong dendritic morphology to non-dendritic morphology on the incorporation of TiC into AA7075.Furthermore,the precipitation of the second-phase compounds such as Al_(2)CuMg,Al_(2)Cu andFe-rich Al_6(Cu,Fe)/Al_(7)Cu_(2)Fe)is promoted by TiC particles at inter-and intra-dendritic regions.Accordingly,the hardness of composites is improved by grain boundary strengthening and particulate strengthening mechanisms.Both coefficient of friction and wear rate have an inverse relation with TiC concentration.The base alloy without TiC shows adhesive-type wear-induced deformation due to the formation of an oxide film,while composite samples exhibit a mechanically mixed layer and abrasive-type wear behavior.Composite samples shows a higher corrosion rate due to the presence of numerous precipitates which promote pitting corrosion.

Multi-UAV cooperative maneuver decision-making for pursuitevasion using improved MADRL

Aiming at the problem of multi-UAV pursuit-evasion confrontation, a UAV cooperative maneuver method based on an improved multi-agent deep reinforcement learning(MADRL) is proposed. In this method, an improved Comm Net network based on a communication mechanism is introduced into a deep reinforcement learning algorithm to solve the multi-agent problem. A layer of gated recurrent unit(GRU) is added to the actor-network structure to remember historical environmental states. Subsequently,another GRU is designed as a communication channel in the Comm Net core network layer to refine communication information between UAVs. Finally, the simulation results of the algorithm in two sets of scenarios are given, and the results show that the method has good effectiveness and applicability.

Toward Trustworthy Decision-Making for Autonomous Vehicles:A Robust Reinforcement Learning Approach with Safety Guarantees

While autonomous vehicles are vital components of intelligent transportation systems,ensuring the trustworthiness of decision-making remains a substantial challenge in realizing autonomous driving.Therefore,we present a novel robust reinforcement learning approach with safety guarantees to attain trustworthy decision-making for autonomous vehicles.The proposed technique ensures decision trustworthiness in terms of policy robustness and collision safety.Specifically,an adversary model is learned online to simulate the worst-case uncertainty by approximating the optimal adversarial perturbations on the observed states and environmental dynamics.In addition,an adversarial robust actor-critic algorithm is developed to enable the agent to learn robust policies against perturbations in observations and dynamics.Moreover,we devise a safety mask to guarantee the collision safety of the autonomous driving agent during both the training and testing processes using an interpretable knowledge model known as the Responsibility-Sensitive Safety Model.Finally,the proposed approach is evaluated through both simulations and experiments.These results indicate that the autonomous driving agent can make trustworthy decisions and drastically reduce the number of collisions through robust safety policies.

A deep reinforcement learning approach to gasoline blending real-time optimization under uncertainty

The gasoline inline blending process has widely used real-time optimization techniques to achieve optimization objectives,such as minimizing the cost of production.However,the effectiveness of real-time optimization in gasoline blending relies on accurate blending models and is challenged by stochastic disturbances.Thus,we propose a real-time optimization algorithm based on the soft actor-critic(SAC)deep reinforcement learning strategy to optimize gasoline blending without relying on a single blending model and to be robust against disturbances.Our approach constructs the environment using nonlinear blending models and feedstocks with disturbances.The algorithm incorporates the Lagrange multiplier and path constraints in reward design to manage sparse product constraints.Carefully abstracted states facilitate algorithm convergence,and the normalized action vector in each optimization period allows the agent to generalize to some extent across different target production scenarios.Through these well-designed components,the algorithm based on the SAC outperforms real-time optimization methods based on either nonlinear or linear programming.It even demonstrates comparable performance with the time-horizon based real-time optimization method,which requires knowledge of uncertainty models,confirming its capability to handle uncertainty without accurate models.Our simulation illustrates a promising approach to free real-time optimization of the gasoline blending process from uncertainty models that are difficult to acquire in practice.

Optimization Mechanism of Mechanical Properties of Basalt Fiber-Epoxy Resin Composites by Interfacially Enriched Distribution of Nano-Starch Crystals

Fibre reinforced polymer composites have become a new generation of structural materials due to their unique advantages such as high specific strength,designability,good dimensional stability and ease of large-area monolithic forming.However,the problem of interfacial bonding between the resin matrix and the fibres limits the direct use of reinforcing fibres and has become a central difficulty in the development of basalt fibre-epoxy composites.This paper proposes a solution for enhancing the strength of the fibre-resin interface using maize starch nanocrystals,which are highly yield and eco-friendly.Firstly,in this paper,corn starch nanocrystals(SNC)were prepared by hydrolysis,and were deposited on the surface of basalt fibers by electrostatic adsorption.After that,in order to maximize the modification effect of nano-starch crystals on the interface,the basalt fiber-epoxy resin composite samples were prepared by mixing in a pressureless molding method.The test results shown that the addition of basalt fibers alone led to a reduction in the strength of the sample.Deposition of 0.1 wt%SNC on the surface of basalt fibers can make the strength consistent with pure epoxy resin.When the adsorption amount of SNC reached 0.5 wt%,the tensile strength of the samples was 23.7%higher than that of pure epoxy resin.This is due to the formation of ether bond homopolymers between the SNC at the fibre-epoxy interface and the epoxy resin,which distorts the originally smooth interface,leading to increased stress concentration and the development of cracks.This enhances the binding of basalt fibers.The conclusions of this paper can provide an effective,simple,low-cost and non-polluting method of interfacial enhancement modification.

Stability behavior of the Lanxi ancient flood control levee after reinforcement with upside-down hanging wells and grouting curtain

The stability of the ancient flood control levees is mainly influenced by water level fluctuations, groundwater concentration and rainfalls. This paper takes the Lanxi ancient levee as a research object to study the evolution laws of its seepage, displacement and stability before and after reinforcement with the upside-down hanging wells and grouting curtain through numerical simulation methods combined with experiments and observations. The study results indicate that the filled soil is less affected by water level fluctuations and groundwater concentration after reinforcement. A high groundwater level is detrimental to the levee's long-term stability, and the drainage issues need to be fully considered. The deformation of the reinforced levee is effectively controlled since the fill deformation is mainly borne by the upside-down hanging wells. The safety factors of the levee before reinforcement vary significantly with the water level. The minimum value of the safety factors is 0.886 during the water level decreasing period, indicating a very high risk of the instability. While it reached 1.478 after reinforcement, the stability of the ancient levee is improved by a large margin.

Nano-scale Reinforcements and Properties of Al-Si-Cu Alloy Processed by High-Pressure Torsion

To improve the comprehensive mechanical properties of Al-Si-Cu alloy,it was treated by a high-pressure torsion process,and the effect of the deformation degree on the microstructure and properties of the Al-Si-Cu alloy was studied.The results show that the reinforcements(β-Si andθ-CuAl_(2)phases)of the Al-Si-Cu alloy are dispersed in theα-Al matrix phase with finer phase size after the treatment.The processed samples exhibit grain sizes in the submicron or even nanometer range,which effectively improves the mechanical properties of the material.The hardness and strength of the deformed alloy are both significantly raised to 268 HV and 390.04 MPa by 10 turns HPT process,and the fracture morphology shows that the material gradually transits from brittle to plastic before and after deformation.The elements interdiffusion at the interface between the phases has also been effectively enhanced.In addition,it is found that the severe plastic deformation at room temperature induces a ternary eutectic reaction,resulting in the formation of ternary Al+Si+CuAl_(2)eutectic.

Elliptical encirclement control capable of reinforcing performances for UAVs around a dynamic target

Most researches associated with target encircling control are focused on moving along a circular orbit under an ideal environment free from external disturbances.However,elliptical encirclement with a time-varying observation radius,may permit a more flexible and high-efficacy enclosing solution,whilst the non-orthogonal property between axial and tangential speed components,non-ignorable environmental perturbations,and strict assignment requirements empower elliptical encircling control to be more challenging,and the relevant investigations are still open.Following this line,an appointed-time elliptical encircling control rule capable of reinforcing circumnavigation performances is developed to enable Unmanned Aerial Vehicles(UAVs)to move along a specified elliptical path within a predetermined reaching time.The remarkable merits of the designed strategy are that the relative distance controlling error can be guaranteed to evolve within specified regions with a designer-specified convergence behavior.Meanwhile,wind perturbations can be online counteracted based on an unknown system dynamics estimator(USDE)with only one regulating parameter and high computational efficiency.Lyapunov tool demonstrates that all involved error variables are ultimately limited,and simulations are implemented to confirm the usability of the suggested control algorithm.

Shear behavior of single-joint bolted sandstone subjected to dryewet cycles:Experimental and analytical approaches

A series of direct shear tests under constant normal loading conditions were carried out on specimens of bolted sandstone single-joint treated with different numbers of dryewet cycles.The experimental results show that the peak shear strength and shear stiffness of bolted sandstone joints were significantly reduced after 12 dryewet cycles.The decrease in the shear strength of rough joints is more significant than that of flat joints.Due to the decrease in the strength of the surrounding rock,the deformation characteristics of the bolts are significantly affected by the number of dryewet cycles performed.With an increase in the number of dryewet cycles,the plastic hinge length of the bolt gradually increases,resulting in an increase in the corresponding shear displacement when the bolt breaks.Compared with the tensileeshear failure mode of the bolts in flat joints,the tensileebending failure mode arises for bolts in rough joints.A shear curve model describing the whole process of bolted rock joints is established based on the deterioration of rock mechanical parameters caused by dry‒wet cycles.The model proposed considers the change in the friction angle of the joint surface with the shear displacement,which is applied to the derivation of the model by introducing the dynamic evolutionary friction angle parameter.The reasonably good agreement between a predicted curve and the corresponding experimental curve indicates that this method can effectively predict the shear strength of a bolted rock joint involving rough joint under dryewet cycling conditions.

Mechanical properties of steel mesh in anchor-mesh support for rocky tunnels

Underground geotechnical engineering encounters persistent challenges in ensuring the stability and safety of surrounding rock structures, particularly within rocky tunnels. Rock reinforcement techniques, including the use of steel mesh, are critical to achieving this goal. However, there exists a knowledge gap regarding the comprehensive understanding of the mechanical behavior and failure mechanisms exhibited by steel mesh under diverse loading conditions. This study thoroughly explored the steel mesh's performance throughout the entire loading-failure process, innovating with detailed analysis and modeling techniques. By integrating advanced numerical modeling with laboratory experiments, the study examines the influence of varying reinforcement levels and geometric parameters on the steel mesh strength and deformation characteristics. Sensitivity analysis, employing gray correlation theory, identifies the key factors affecting the mesh performance, while a BP (Backpropagation) neural network model predicts maximum vertical deformation with high accuracy. The findings underscore the critical role of steel diameter and mesh spacing in optimizing peak load capacity, displacement, and energy absorption, offering practical guidelines for design improvements. The use of a Bayesian Regularization (BR) algorithm further enhances the predictive accuracy compared to traditional methods. This research provides new insights into optimizing steel mesh design for underground applications, offering an innovative approach to enhancing structural safety in geotechnical projects.

Distributed Graph Database Load Balancing Method Based on Deep Reinforcement Learning

This paper focuses on the scheduling problem of workflow tasks that exhibit interdependencies.Unlike indepen-dent batch tasks,workflows typically consist of multiple subtasks with intrinsic correlations and dependencies.It necessitates the distribution of various computational tasks to appropriate computing node resources in accor-dance with task dependencies to ensure the smooth completion of the entire workflow.Workflow scheduling must consider an array of factors,including task dependencies,availability of computational resources,and the schedulability of tasks.Therefore,this paper delves into the distributed graph database workflow task scheduling problem and proposes a workflow scheduling methodology based on deep reinforcement learning(DRL).The method optimizes the maximum completion time(makespan)and response time of workflow tasks,aiming to enhance the responsiveness of workflow tasks while ensuring the minimization of the makespan.The experimental results indicate that the Q-learning Deep Reinforcement Learning(Q-DRL)algorithm markedly diminishes the makespan and refines the average response time within distributed graph database environments.In quantifying makespan,Q-DRL achieves mean reductions of 12.4%and 11.9%over established First-fit and Random scheduling strategies,respectively.Additionally,Q-DRL surpasses the performance of both DRL-Cloud and Improved Deep Q-learning Network(IDQN)algorithms,with improvements standing at 4.4%and 2.6%,respectively.With reference to average response time,the Q-DRL approach exhibits a significantly enhanced performance in the scheduling of workflow tasks,decreasing the average by 2.27%and 4.71%when compared to IDQN and DRL-Cloud,respectively.The Q-DRL algorithm also demonstrates a notable increase in the efficiency of system resource utilization,reducing the average idle rate by 5.02%and 9.30%in comparison to IDQN and DRL-Cloud,respectively.These findings support the assertion that Q-DRL not only upholds a lower average idle rate but also effectively curtails the average response time,thereby substantially improving processing efficiency and optimizing resource utilization within distributed graph database systems.

Combining reinforcement learning with mathematical programming:An approach for optimal design of heat exchanger networks

Heat integration is important for energy-saving in the process industry.It is linked to the persistently challenging task of optimal design of heat exchanger networks(HEN).Due to the inherent highly nonconvex nonlinear and combinatorial nature of the HEN problem,it is not easy to find solutions of high quality for large-scale problems.The reinforcement learning(RL)method,which learns strategies through ongoing exploration and exploitation,reveals advantages in such area.However,due to the complexity of the HEN design problem,the RL method for HEN should be dedicated and designed.A hybrid strategy combining RL with mathematical programming is proposed to take better advantage of both methods.An insightful state representation of the HEN structure as well as a customized reward function is introduced.A Q-learning algorithm is applied to update the HEN structure using theε-greedy strategy.Better results are obtained from three literature cases of different scales.