改进Q-Learning的路径规划算法研究

针对Q-Learning算法学习效率低、收敛速度慢且在动态障碍物的环境下路径规划效果不佳的问题,本文提出一种改进Q-Learning的移动机器人路径规划算法.针对该问题,算法根据概率的突变性引入探索因子来平衡探索和利用以加快学习效率;通过在更新函数中设计深度学习因子以保证算法探索概率;融合遗传算法,避免陷入局部路径最优同时按阶段探索最优迭代步长次数,以减少动态地图探索重复率;最后提取输出的最优路径关键节点采用贝塞尔曲线进行平滑处理,进一步保证路径平滑度和可行性.实验通过栅格法构建地图,对比实验结果表明,改进后的算法效率相较于传统算法在迭代次数和路径上均有较大优化,且能够较好的实现动态地图下的路径规划,进一步验证所提方法的有效性和实用性.

改进的Q-learning蜂群算法求解置换流水车间调度问题

针对置换流水车间调度问题,提出了一种基于改进的Q-learning算法的人工蜂群算法。该算法设计了一种改进的奖励函数作为人工蜂群算法的环境,根据奖励函数的优劣来判断下一代种群的寻优策略,并通过Q-learning智能选择人工蜂群算法的蜜源的更新维度数大小,根据选择的维度数大小对编码进行更新,提高了收敛速度和精度,最后使用不同规模的置换流水车间调度问题的实例来验证所提算法的性能,通过对标准实例的计算与其它算法对比,证明该算法的准确性。

基于改进Q-Learning的移动机器人路径规划算法

随着移动机器人在生产生活中的深入应用,其路径规划能力也需要向快速性和环境适应性兼备发展。为解决现有移动机器人使用强化学习方法进行路径规划时存在的探索前期容易陷入局部最优、反复搜索同一区域,探索后期收敛率低、收敛速度慢的问题,本研究提出一种改进的Q-Learning算法。该算法改进Q矩阵赋值方法,使迭代前期探索过程具有指向性,并降低碰撞的情况;改进Q矩阵迭代方法,使Q矩阵更新具有前瞻性,避免在一个小区域中反复探索;改进随机探索策略,在迭代前期全面利用环境信息,后期向目标点靠近。在不同栅格地图仿真验证结果表明,本文算法在Q-Learning算法的基础上,通过上述改进降低探索过程中的路径长度、减少抖动并提高收敛的速度,具有更高的计算效率。

基于Q-Learning的航空器滑行路径规划研究

针对传统算法规划航空器滑行路径准确度低、不能根据整体场面运行情况进行路径规划的问题,提出一种基于Q-Learning的路径规划方法。通过对机场飞行区网络结构模型和强化学习的仿真环境分析,设置了状态空间和动作空间,并根据路径的合规性和合理性设定了奖励函数,将路径合理性评价值设置为滑行路径长度与飞行区平均滑行时间乘积的倒数。最后,分析了动作选择策略参数对路径规划模型的影响。结果表明,与A*算法和Floyd算法相比,基于Q-Learning的路径规划在滑行距离最短的同时,避开了相对繁忙的区域,路径合理性评价值高。

基于Q-Learning的分簇无线传感网信任管理机制

针对无线传感器网络中存在的安全问题,提出了基于Q-Learning的分簇无线传感网信任管理机制(Q-learning based trust management mechanism for clustered wireless sensor networks,QLTMM-CWSN).该机制主要考虑通信信任、数据信任和能量信任3个方面.在网络运行过程中,基于节点的通信行为、数据分布和能量消耗,使用Q-Learning算法更新节点信任值,并选择簇内信任值最高的节点作为可信簇头节点.当簇中主簇头节点的信任值低于阈值时,可信簇头节点代替主簇头节点管理簇内成员节点,维护正常的数据传输.研究结果表明,QLTMM-CWSN机制能有效抵御通信攻击、伪造本地数据攻击、能量攻击和混合攻击.

基于多步信息辅助的Q-learning路径规划算法

为提升静态环境下移动机器人路径规划能力,解决传统Q-learning算法在路径规划中收敛速度慢的问题,提出一种基于多步信息辅助机制的Q-learning改进算法。利用ε-greedy策略中贪婪动作的多步信息与历史最优路径长度更新资格迹,使有效的资格迹在算法迭代中持续发挥作用,用保存的多步信息解决可能落入的循环陷阱;使用局部多花朵的花授粉算法初始化Q值表,提升机器人前期搜索效率;基于机器人不同探索阶段的目的,结合迭代路径长度的标准差与机器人成功到达目标点的次数设计动作选择策略,以增强算法对环境信息探索与利用的平衡能力。实验结果表明:该算法具有较快的收敛速度,验证了算法的可行性与有效性。

基于softmax的加权Double Q-Learning算法

强化学习作为机器学习的一个分支,用于描述和解决智能体在与环境的交互过程中,通过学习策略以达成回报最大化的问题。Q-Learning作为无模型强化学习的经典方法,存在过估计引起的最大化偏差问题,并且在环境中奖励存在噪声时表现不佳。Double Q-Learning(DQL)的出现解决了过估计问题,但同时造成了低估问题。为解决以上算法的高低估问题,提出了基于softmax的加权Q-Learning算法,并将其与DQL相结合,提出了一种新的基于softmax的加权Double Q-Learning算法(WDQL-Softmax)。该算法基于加权双估计器的构造,对样本期望值进行softmax操作得到权重,使用权重估计动作价值,有效平衡对动作价值的高估和低估问题,使估计值更加接近理论值。实验结果表明,在离散动作空间中,相比于Q-Learning算法、DQL算法和WDQL算法,WDQL-Softmax算法的收敛速度更快且估计值与理论值的误差更小。

Failure mechanism and infrared radiation characteristic of hard siltstone induced by stratification effect

The deformation in sedimentary rock induced by train loads has potential threat to the safe operation of tunnels. This study investigated the influence of stratification structure on the infrared radiation and temporal damage mechanism of hard siltstone. The uniaxial compression tests, coupled with acoustic emission(AE) and infrared radiation temperature(IRT) were conducted on siltstones with different stratification effects. The results revealed that the stratigraphic structure significantly affects the stress-strain response and strength degradation characteristics. The mechanical parameters exhibit anisotropy characteristics, and the stratification effect exhibits a negative correlation with the cracking stress and peak stress. The failure modes caused by the stratification effect show remarkable anisotropic features, including splitting failure(Ⅰ: 0°-22.50°, Ⅱ: 90°), composite failure(45°), and shearing failure(67.50°). The AE temporal sequences demonstrate a stepwise response characteristic to the loading stress level. The AE intensity indicates that the stress sensitivity of shearing failure and composite failure is generally greater than that of splitting failure. The IRT field has spatiotemporal migration and progressive dissimilation with stress loading and its dissimilation degree increases under higher stress levels. The stronger the stratification effect, the greater the dissimilation degree of the IRT field. The abnormal characteristic points of average infrared radiation temperature(AIRT) variance at local stress drop and peak stress can be used as early and late precursors to identify fracture instability. Theoretical analysis shows that the competitive relationship between compaction strengthening and fracturing damage intensifies the dissimilation of the infrared thermal field for an increasing stress level. The present study provides a theoretical reference for disaster warnings in hard sedimentary rock mass.

Regeneration of the heart:f rom molecular mechanisms to clinical therapeutics

Heart injury such as myocardial infarction leads to cardiomyocyte loss,fibrotic tissue deposition,and scar formation.These changes reduce cardiac contractility,resulting in heart failure,which causes a huge public health burden.Military personnel,compared with civilians,is exposed to more stress,a risk factor for heart diseases,making cardiovascular health management and treatment innovation an important topic for military medicine.So far,medical intervention can slow down cardiovascular disease progression,but not yet induce heart regeneration.In the past decades,studies have focused on mechanisms underlying the regenerative capability of the heart and applicable approaches to reverse heart injury.Insights have emerged from studies in animal models and early clinical trials.Clinical interventions show the potential to reduce scar formation and enhance cardiomyocyte proliferation that counteracts the pathogenesis of heart disease.In this review,we discuss the signaling events controlling the regeneration of heart tissue and summarize current therapeutic approaches to promote heart regeneration after injury.

The underlying mechanism of variety–water–nitrogen–stubble damage interactions on yield formation in ratoon rice with low stubble height under mechanized harvesting

Agronomic measures are the key to promote the sustainable development of ratoon rice by reducing the damage from mechanical crushing to the residual stubble of the main crop, thereby mitigating the impact on axillary bud sprouting and yield formation in ratoon rice. This study used widely recommended conventional rice Jiafuzhan and hybrid rice Yongyou 2640 as the test materials to conduct a four-factor block design field experiment in a greenhouse of the experimental farm of Fujian Agricultural and Forestry University, China from 2018 to 2019.The treatments included fertilization and no fertilization, alternate wetting and drying irrigation and continuous water flooding irrigation, and plots with and without artificial crushing damage on the rice stubble. At the same time, a 13C stable isotope in-situ detection technology was used to fertilize the pot experiment. The results showed significant interactions among varieties, water management, nitrogen application and stubble status.Relative to the long-term water flooding treatment, the treatment with sequential application of nitrogen fertilizer coupled with moderate field drought for root-vigor and tiller promotion before and after harvesting of the main crop, significantly improved the effective tillers from low position nodes. This in turn increased the effective panicles per plant and grains per panicle by reducing the influence of artificial crushing damage on rice stubble and achieving a high yield of the regenerated rice. Furthermore, the partitioning of 13C assimilates to the residual stubble and its axillary buds were significantly improved at the mature stage of the main crop, while the translocation rate to roots and rhizosphere soil was reduced at the later growth stage of ratooning season rice. This was triggered by the metabolism of hormones and polyamines at the stem base regulated by the interaction of water and fertilizer at this time. We therefore suggest that to achieve a high yield of ratoon rice with low stubble height under mechanized harvesting, the timely application of nitrogen fertilizer is fundamental,coupled with moderate field drying for root-vigor preservation and tiller promotion before and after the mechanical harvesting of the main crop.

Drug resistance mechanisms in cancers:Execution of prosurvival strategies

One of the quintessential challenges in cancer treatment is drug resistance.Several mechanisms of drug resistance have been described to date,and new modes of drug resistance continue to be discovered.The phenomenon of cancer drug resistance is now widespread,with approximately 90% of cancer-related deaths associated with drug resistance.Despite significant advances in the drug discovery process,the emergence of innate and acquired mechanisms of drug resistance has impeded the progress in cancer therapy.Therefore,understanding the mechanisms of drug resistance and the various pathways involved is integral to treatment modalities.In the present review,I discuss the different mechanisms of drug resistance in cancer cells,including DNA damage repair,epithelial to mesenchymal transition,inhibition of cell death,alteration of drug targets,inactivation of drugs,deregulation of cellular energetics,immune evasion,tumor-promoting inflammation,genome instability,and other contributing epigenetic factors.Furthermore,I highlight available treatment options and conclude with future directions.

To explore the mechanism of Yigong San anti-gastric cancer and immune regulation

BACKGROUND Yigong San(YGS)is a representative prescription for the treatment of digestive disorders,which has been used in clinic for more than 1000 years.However,the mechanism of its anti-gastric cancer and regulate immunity are still remains unclear.AIM To explore the mechanism of YGS anti-gastric cancer and immune regulation.METHODS Firstly,collect the active ingredients and targets of YGS,and the differentially expressed genes of gastric cancer.Secondly,constructed a protein-protein interaction network between the targets of drugs and diseases,and screened hub genes.Then the clinical relevance,mutation and repair,tumor microenvironment and drug sensitivity of the hub gene were analyzed.Finally,molecular docking was used to verify the binding ability of YGS active ingredient and hub genes.RESULTS Firstly,obtained 55 common targets of gastric cancer and YGS.The Kyoto Encyclopedia of Genes and Genomes screened the microtubule-associated protein kinase signaling axis as the key pathway and IL6,EGFR,MMP2,MMP9 and TGFB1 as the hub genes.The 5 hub genes were involved in gastric carcinogenesis,staging,typing and prognosis,and their mutations promote gastric cancer progression.Finally,molecular docking results confirmed that the components of YGS can effectively bind to therapeutic targets.CONCLUSION YGS has the effect of anti-gastric cancer and immune regulation.

多无人机辅助边缘计算场景下基于Q-learning的任务卸载优化

引入多无人机辅助边缘计算系统,由多个无人机和原有边缘服务器共同为移动用户提供通信和计算资源;将优化问题建模为资源竞争和卸载决策约束下的系统总效用最大化问题,系统总效用由用户满意度、任务延迟和系统能耗3个因素组成.由于优化模型是一个具有NP难属性的非凸问题,故采用强化学习方法求解得到使系统总效用最大的最优任务卸载决策集.仿真实验结果表明,与贪心顺序调优卸载方案和随机选择卸载方案相比,该文提出的Q-learning方案的系统总效用分别提高了15%和43%以上.

Deep Insight of Design,Mechanism,and Cancer Theranostic Strategy of Nanozymes

Since the discovery of enzyme-like activity of Fe3O4 nanoparticles in 2007,nanozymes are becoming the promising substitutes for natural enzymes due to their advantages of high catalytic activity,low cost,mild reaction conditions,good stability,and suitable for large-scale production.Recently,with the cross fusion of nanomedicine and nanocatalysis,nanozyme-based theranostic strategies attract great attention,since the enzymatic reactions can be triggered in the tumor microenvironment to achieve good curative effect with substrate specificity and low side effects.Thus,various nanozymes have been developed and used for tumor therapy.In this review,more than 270 research articles are discussed systematically to present progress in the past five years.First,the discovery and development of nanozymes are summarized.Second,classification and catalytic mechanism of nanozymes are discussed.Third,activity prediction and rational design of nanozymes are focused by highlighting the methods of density functional theory,machine learning,biomimetic and chemical design.Then,synergistic theranostic strategy of nanozymes are introduced.Finally,current challenges and future prospects of nanozymes used for tumor theranostic are outlined,including selectivity,biosafety,repeatability and stability,in-depth catalytic mechanism,predicting and evaluating activities.

Instability mechanism of mining roadway passing through fault at different angles in kilometre-deep mine and control measures of roof cutting and NPR cables

The angle α between the fault strike and the axial direction of the roadway produces different damage characteristics. In this paper, the research methodology includes theoretical analyses, numerical simulations and field experiments in the context of the Daqiang coal mine located in Shenyang, China. The stability control countermeasure of "pre-splitting cutting roof + NPR anchor cable"(PSCR-NPR) is simultaneously proposed. According to the different deformation characteristics of the roadway, the faults are innovatively classified into three types, with α of type I being 0°-30°, α of type II being 30°-60°, and α of type III being 60°-90°. The full-cycle stress evolution paths during mining roadway traverses across different types of faults are investigated by numerical simulation. Different pinch angles α lead to high stress concentration areas at different locations in the surrounding rock. The non-uniform stress field formed in the shallow surrounding rock is an important reason for the instability of the roadway. The pre-cracked cut top shifted the high stress region to the deep rock mass and formed a low stress region in the shallow rock mass. The high prestressing NPR anchor cable transforms the non-uniform stress field of the shallow surrounding rock into a uniform stress field. PSCR-NPR is applied in the fault-through roadway of Daqiang mine. The low stress area of the surrounding rock was enlarged by 3-7 times, and the cumulative convergence was reduced by 45%-50%. It provides a reference for the stability control of the deep fault-through mining roadway.

Calcium-chelating peptides from rabbit bone collagen:characterization,identification and mechanism elucidation

This study aimed to characterize and identify calcium-chelating peptides from rabbit bone collagen and explore the underlying chelating mechanism.Collagen peptides and calcium were extracted from rabbit bone by instant ejection steam explosion(ICSE)combined with enzymatic hydrolysis,followed by chelation reaction to prepare rabbit bone peptide-calcium chelate(RBCP-Ca).The chelating sites were further analyzed by liquid chromatography-tandem mass(LC-MS/MS)spectrometry while the chelating mechanism and binding modes were investigated.The structural characterization revealed that RBCP successfully chelated with calcium ions.Furthermore,LC-MS/MS analysis indicated that the binding sites included both acidic amino acids(Asp and Glu)and basic amino acids(Lys and Arg),Interestingly,three binding modes,namely Inter-Linking,Loop-Linking and Mono-Linking were for the first time found,while Inter-Linking mode accounted for the highest proportion(75.1%),suggesting that chelation of calcium ions frequently occurred between two peptides.Overall,this study provides a theoretical basis for the elucidation of chelation mechanism of calcium-chelating peptides.

Molecular Mechanism and Molecular Design of Lubricating Oil Antioxidants

To overcome the limitations of traditional experimental“trial and error”methods in lubricant additive design,a new molecular design method based on molecular structure parameters is established here.The molecular mechanism of the antioxidant reaction of hindered phenol,diphenylamine,and alkyl sulfide are studied via molecular simulations.Calculation results show that the strong electron-donating ability and high hydrogen-donating activity of the antioxidant molecule and the low hydrogen-abstracting activity of free radicals formed after dehydrogenation are the internal molecular causes of the shielding of phenol and diphenylamine from scavenging peroxy free radicals,and the strong electron-donating ability is the internal molecular cause of the high activity of thioether in decomposing alkyl hydrogen peroxide.Based on this antioxidant molecular mechanism,a molecular design rule of antioxidant is proposed,namely“high EHOMO,large Q(S),low bond dissociation energy BDE(O—H)and BDE(N—H)”.Two new antioxidants,PAS-I and PAS-II,are designed and prepared by chemical bonding of hindered phenol,diphenylamine,and sulfur atoms.Experimental results show that these antioxidants both have excellent antioxidant effects in lubricating oil,and that PAS-II is the superior antioxidant,consistent with theoretical predictions.

Recent research progress in the mechanism and suppression of fusion welding-induced liquation cracking of nickel based superalloys

Nickel-based superalloys are extensively used in the crucial hot-section components of industrial gas turbines,aeronautics,and astronautics because of their excellent mechanical properties and corrosion resistance at high temperatures.Fusion welding serves as an effective means for joining and repairing these alloys;however,fusion welding-induced liquation cracking has been a challenging issue.This paper comprehensively reviewed recent liquation cracking,discussing the formation mechanisms,cracking criteria,and remedies.In recent investigations,regulating material composition,changing the preweld heat treatment of the base metal,optimizing the welding process parameters,and applying auxiliary control methods are effective strategies for mitigating cracks.To promote the application of nickel-based superalloys,further research on the combination impact of multiple elements on cracking prevention and specific quantitative criteria for liquation cracking is necessary.

Preliminary discussion on the ignition mechanism of exploding foil initiators igniting boron potassium nitrate

Exploding foil initiator(EFI)is a kind of advanced device for initiating explosives,but its function is unstable when it comes to directly igniting pyrotechnics.To solve the problem,this research aims to reveal the ignition mechanism of EFIs directly igniting pyrotechnics.An oscilloscope,a photon Doppler velocimetry,and a plasma spectrum measurement system were employed to obtain information of electric characteristics,impact pressure,and plasma temperature.The results of the electric characteristics and the impact pressure were inconsistent with ignition results.The only thing that the ignition success tests had in common was that their plasma all had a relatively long period of high-temperature duration(HTD).It eventually concludes that the ignition mechanism in this research is the microconvection heat transfer rather than the shock initiation,which differs from that of exploding foil initiators detonating explosives.Furthermore,the methods for evaluating the ignition success of semiconductor bridge initiators are not entirely applicable to the tests mentioned in this paper.The HTD is the critical parameter for judging the ignition success,and it is influenced by two factors:the late time discharge and the energy of the electric explosion.The longer time of the late time discharge and the more energy of the electric explosion,the easier it is to expand the HTD,which improves the probability of the ignition success.

Unraveling the Fundamental Mechanism of Interface Conductive Network Influence on the Fast‑Charging Performance of SiO‑Based Anode for Lithium‑Ion Batteries

Progress in the fast charging of high-capacity silicon monoxide(SiO)-based anode is currently hindered by insufficient conductivity and notable volume expansion.The construction of an interface conductive network effectively addresses the aforementioned problems;however,the impact of its quality on lithium-ion transfer and structure durability is yet to be explored.Herein,the influence of an interface conductive network on ionic transport and mechanical stability under fast charging is explored for the first time.2D modeling simulation and Cryo-transmission electron microscopy precisely reveal the mitigation of interface polarization owing to a higher fraction of conductive inorganic species formation in bilayer solid electrolyte interphase is mainly responsible for a linear decrease in ionic diffusion energy barrier.Furthermore,atomic force microscopy and Raman shift exhibit substantial stress dissipation generated by a complete conductive network,which is critical to the linear reduction of electrode residual stress.This study provides insights into the rational design of optimized interface SiO-based anodes with reinforced fast-charging performance.