改进MADDPG算法的非凸环境下多智能体自组织协同围捕

针对多智能体在非凸环境下的围捕效率问题,提出基于改进经验回放的多智能体强化学习算法。利用残差网络(ResNet)来改善网络退化问题,并与多智能体深度确定性策略梯度算法(MADDPG)相结合,提出了RW-MADDPG算法。为解决多智能体在训练过程中,经验池数据利用率低的问题,提出两种改善经验池数据利用率的方法;为解决多智能体在非凸障碍环境下陷入障碍物内部的情况(如陷入目标不可达等),通过设计合理的围捕奖励函数使得智能体在非凸障碍物环境下完成围捕任务。基于此算法设计仿真实验,实验结果表明,该算法在训练阶段奖励增加得更快,能更快地完成围捕任务,相比MADDPG算法静态围捕环境下训练时间缩短18.5%,动态环境下训练时间缩短49.5%,而且在非凸障碍环境下该算法训练的围捕智能体的全局平均奖励更高。

基于ATMADDPG算法的多水面无人航行器编队导航

为提高多无人船编队系统的导航能力,提出了一种基于注意力机制的多智能体深度确定性策略梯度(ATMADDPG:Attention Mechanism based Multi-Agent Deep Deterministic Policy Gradient)算法。该算法在训练阶段,通过大量试验训练出最佳策略,并在实验阶段直接使用训练出的最佳策略得到最佳编队路径。仿真实验将4艘相同的“百川号”无人船作为实验对象。实验结果表明,基于ATMADDPG算法的队形保持策略能实现稳定的多无人船编队导航,并在一定程度上满足队形保持的要求。相较于多智能体深度确定性策略梯度(MADDPG:Multi-Agent Depth Deterministic Policy Gradient)算法,所提出的ATMADDPG算法在收敛速度、队形保持能力和对环境变化的适应性等方面表现出更优越的性能,综合导航效率可提高约80%,具有较大的应用潜力。

基于MADDPG的散装物料输送多智能体协同控制

为提升带式输送系统的智能化决策,提高生产效率,降低能耗,应用多智能体深度确定性策略梯度(MADDPG)算法,构建多输送机智能体协同控制系统。系统采用集中式结构控制多输送机,由输送机运行能耗模型,结合MADDPG算法结构,构建多智能体协同控制模型。通过训练模型,寻优输送机运行速度与煤流量最佳匹配关系,得出节能最优速度控制策略。与深度确定性策略梯度(DDPG)算法进行实验对比。结果表明,提出的多输送机智能体算法模型学习效率高,收敛速度快,具有较强的稳定性。

基于MADDPG的多AGVs路径规划算法

针对多辆自动导引车系统(automated guided vehicle system,AGVs)在动态不确定环境下完成货物运送并进行路径规划的问题,提出一种基于多智能体深度确定性策略梯度(MADDPG)的多AGVs路径规划算法。本方法通过状态空间、动作空间、奖励函数和网络结构重新设计MADDPG算法的模型结构,通过OpenAI Gym接口搭建二维仿真环境用作多AGVs (agents)的训练平台。实验结果表明,相比于深度确定性策略梯度(DDPG)算法和双延迟深度确定性策略梯度(TD3)算法,基于MADDPG的多AGVs路径规划算法在智能仓储仿真环境下,多AGVs碰到货架的次数分别减少了21.49%、11.63%,碰到障碍物的次数分别减少了14.69%、10.12%,全部AGVs到达货物装卸点的成功率分别高出了17.22%、10.53%,表明学习后的AGV具有更高效的在线决策能力和自适应能力,能够找到较优的路径。

Gradient coil design with enhanced shielding constraint for a cryogen-free superconducting MRI system

The relatively fragile low-temperature stability of cryogen-free superconducting magnetic resonance imaging(MRI)magnets requires the careful management of exogenous heat sources.A strongly shielded gradient magnetic field is important for the optimal operation of cryogen-free MRI systems.In this study,we present an enhanced shielding method incorporating a regionalized stray field constraining strategy.By optimizing the constraint parameters,we could develop engineering-feasible gradient coil schemes without increasing system complexity but with the stray field intensity reduced by half.In real measurement in an integrated MRI system,the developed gradient assembly demonstrated good performance and supported to output images of excellent quality.Our findings suggested that the proposed method could potentially form a useful design paradigm for cryogen-free MRI magnets.

Climate-growth relationships of Pinus tabuliformis along an altitudinal gradient on Baiyunshan Mountain,Central China

A set of standard chronologies for tree-ring width(TRW),earlywood width(EWW)and latewood width(LWW)in Pinus tabuliformis Carr.along an altitudi-nal gradient(1450,1400,and 1350 m a.s.l.)on Baiyunshan Mountain,Central China to analyze the effect of varying temperature and precipitation on growth along the gradi-ent.Correlation analyses showed that at all three altitudes and the TRW and EWW chronologies generally had signifi-cant negative correlations with mean and maximum tem-peratures in the current April and May and with minimum temperatures in the prior July and August,but significant positive correlations with precipitation in the current May.Correlations were generally significantly negative between LWW chronologies and all temperatures in the prior July and August,indicating that the prior summer temperature had a strong lag effect on the growth of P.tabuliformis that increased with altitude.The correlation with the standard-ized precipitation evapotranspiration index(SPEI)confirmed that wet conditions in the current May promoted growth of TR and EW at all altitudes.Significant altitudinal differences were also found;at 1400 m,there were significant positive correlations between EWW chronologies and SPEI in the current April and significant negative correlations between LWW chronologies and SPEI in the current September,but these correlations were not significant at 1450 m.At 1350 m,there were also significant negative correlations between the TRW and the EWW chronologies and SPEI in the prior October and the current July and between LWW chronology and SPEI in the current August,but these cor-relations were not significant at 1400 m.Moving correlation results showed a stable response of EWW in relation to the SPEI in the current May at all three altitudes and of LWW to maximum temperature in the prior July-August at 1400 m from 2002 to 2018.The EWW chronology at 1400 m and the LWW chronology at 1450 m were identified as more suitable for climate reconstruction.These results provide a strong scientific basis for forest management decisions and climate reconstructions in Central China.

Stress-assisted corrosion mechanism of 3Ni steel by using gradient boosting decision tree machining learning method

Traditional 3Ni weathering steel cannot completely meet the requirements for offshore engineering development,resulting in the design of novel 3Ni steel with the addition of microalloy elements such as Mn or Nb for strength enhancement becoming a trend.The stress-assisted corrosion behavior of a novel designed high-strength 3Ni steel was investigated in the current study using the corrosion big data method.The information on the corrosion process was recorded using the galvanic corrosion current monitoring method.The gradi-ent boosting decision tree(GBDT)machine learning method was used to mine the corrosion mechanism,and the importance of the struc-ture factor was investigated.Field exposure tests were conducted to verify the calculated results using the GBDT method.Results indic-ated that the GBDT method can be effectively used to study the influence of structural factors on the corrosion process of 3Ni steel.Dif-ferent mechanisms for the addition of Mn and Cu to the stress-assisted corrosion of 3Ni steel suggested that Mn and Cu have no obvious effect on the corrosion rate of non-stressed 3Ni steel during the early stage of corrosion.When the corrosion reached a stable state,the in-crease in Mn element content increased the corrosion rate of 3Ni steel,while Cu reduced this rate.In the presence of stress,the increase in Mn element content and Cu addition can inhibit the corrosion process.The corrosion law of outdoor-exposed 3Ni steel is consistent with the law based on corrosion big data technology,verifying the reliability of the big data evaluation method and data prediction model selection.

基于MADDPG算法协同预测的多智能体节能制热饮水机的能效提升

为提升公共场所制热饮水机的群体性能,降低整体能耗,设计了多智能体协同预测的节能制热饮水机。利用季节性ARIMA模型预测饮水机使用人数,结合决策树及遗传算法设定单台饮水机的临界阈值,应用多智能体协同MADDPG算法建立多台饮水机协同控制。通过构建用电成本回归模型测试样机性能,实验结论表明,与市面的商用饮水机相比,新设计的饮水机可节约55%的电费,在提高饮水机能源利用效率方面具有潜在应用价值。

规则耦合下的多异构子网络MADDPG博弈对抗算法

针对多无人机博弈对抗过程中无人机数量动态衰减问题和传统深度强化学习算法中的稀疏奖励问题及无效经验抽取频率过高问题,本文以攻防能力及通信范围受限条件下的多无人机博弈对抗任务为研究背景,构建了红、蓝两方无人机群的博弈对抗模型,在多智能体深度确定性策略梯度(multi-agent deep deterministic policy gradient,MADDPG)算法的Actor-Critic框架下,根据博弈环境的特点对原始的MADDPG算法进行改进。为了进一步提升算法对有效经验的探索和利用,本文构建了规则耦合模块以在无人机的决策过程中对Actor网络进行辅助。仿真实验表明,本文设计的算法在收敛速度、学习效率和稳定性方面都取了一定的提升,异构子网络的引入使算法更适用于无人机数量动态衰减的博弈场景;奖励势函数和重要性权重耦合的优先经验回放方法提升了经验差异的细化程度及优势经验利用率;规则耦合模块的引入实现了无人机决策网络对先验知识的有效利用。

基于EPF-MADDPG算法的多导弹机动策略研究

随着人工智能研究的进一步加深以及在俄乌战场上相关技术的大放异彩,其在军事领域扮演的角色越来越重要;针对日益复杂的战场环境,当前的导弹突防领域存在着信息维度高、指挥反应缓慢、突防机动战术不够灵活等问题;提出了一种基于多智能体深度确定性策略梯度(MADDPG)的训练方法,用以快速制定导弹攻击机动方案,协助军事指挥官进行战场决策;同时改进算法的经验回放策略,添加经验池筛选机制缩短训练的时长,达到现实场景中的快速反应需求;通过设置多目标快速拦截策略,仿真验证了所设计的方法能够突防的机动策略优势,通过协作智能地对目标进行突防打击,并通过比较,验证了该方法相较其他算法可以提升8%的收敛速度以及10%的成功率。

Distribution law and susceptibility of geohazards across a gradient belt of the Western Sichuan Plateau

Across a gradient belt of the Western Sichuan Plateau,geohazards have seriously limited economic and social development.According to incomplete statistics,15,673 geohazards have been recorded in the study area.In order to mitigate the threat of geohazards to human engineering activities in the region,an overall understanding of the distribution pattern of geohazards and susceptibility assessment are necessary.In this paper,a gradient belt of the Western Sichuan Plateau and its zoning criteria were defined.Subsequently,on the basis of relief amplitude,distance to faults,rainfall,and human activities,three indicators of endogenic process were introduced:Bouguer gravity anomaly gradient,vertical deformation gradient,and horizontal deformation gradient.Thereafter,the distribution patterns of geohazards were investigated through mathematical statistics and ArcGIS software.By randomly selecting 10,449 hazards,a geohazard susceptibility map was generated using the Information Value(IV)model.Finally,the IV model was validated against 5224 hazards using the Area Under Curve(AUC)method.The results show that 47.6%of the geohazards were distributed in the zone of steep slope.Geohazards showed strong responses to distance to faults,human activities,and annual rainfall.The distribution of geohazards in the gradient belt of the Western Sichuan Plateau is more sensitive to vertical internal dynamics factors(such as vertical deformation gradient and Bouguer gravity anomaly gradient)without any apparent sensitivity to horizontal internal dynamics factors.The areas of high and very-high risk account for up to 32.22%,mainly distributed in the Longmenshan and Anning River faults.According to the AUC plot,the success rate of the IV model for generating the susceptibility map is 76%.This susceptibility map and geohazard distribution pattern can provide a reference for geological disaster monitoring,preparation of post-disaster emergency measures,and town planning.

Policy Gradient Adaptive Dynamic Programming for Model-Free Multi-Objective Optimal Control

Dear Editor,In this letter,the multi-objective optimal control problem of nonlinear discrete-time systems is investigated.A data-driven policy gradient algorithm is proposed in which the action-state value function is used to evaluate the policy.In the policy improvement process,the policy gradient based method is employed.

Stochastic Gradient Compression for Federated Learning over Wireless Network

As a mature distributed machine learning paradigm,federated learning enables wireless edge devices to collaboratively train a shared AI-model by stochastic gradient descent(SGD).However,devices need to upload high-dimensional stochastic gradients to edge server in training,which cause severe communication bottleneck.To address this problem,we compress the communication by sparsifying and quantizing the stochastic gradients of edge devices.We first derive a closed form of the communication compression in terms of sparsification and quantization factors.Then,the convergence rate of this communicationcompressed system is analyzed and several insights are obtained.Finally,we formulate and deal with the quantization resource allocation problem for the goal of minimizing the convergence upper bound,under the constraint of multiple-access channel capacity.Simulations show that the proposed scheme outperforms the benchmarks.

Wettability Gradient-Induced Diode:MXene-Engineered Membrane for Passive-Evaporative Cooling

Thermoregulatory textiles,leveraging high-emissivity structural materials,have arisen as a promising candidate for personal cooling management;however,their advancement has been hindered by the underperformed water moisture transportation capacity,which impacts on their thermophysiological comfort.Herein,we designed a wettability-gradient-induced-diode(WGID)membrane achieving by MXene-engineered electrospun technology,which could facilitate heat dissipation and moisture-wicking transportation.As a result,the obtained WGID membrane could obtain a cooling temperature of 1.5℃ in the“dry”state,and 7.1℃ in the“wet”state,which was ascribed to its high emissivity of 96.40%in the MIR range,superior thermal conductivity of 0.3349 W m^(-1) K^(-1)(based on radiation-and conduction-controlled mechanisms),and unidirectional moisture transportation property.The proposed design offers an approach for meticulously engineering electrospun membranes with enhanced heat dissipation and moisture transportation,thereby paving the way for developing more efficient and comfortable thermoregulatory textiles in a high-humidity microenvironment.

Bioinspired Multifunctional Self-Sensing Actuated Gradient Hydrogel for Soft-Hard Robot Remote Interaction

The development of bioinspired gradient hydrogels with self-sensing actuated capabilities for remote interaction with soft-hard robots remains a challenging endeavor. Here, we propose a novel multifunctional self-sensing actuated gradient hydrogel that combines ultrafast actuation and high sensitivity for remote interaction with robotic hand. The gradient network structure, achieved through a wettability difference method involving the rapid precipitation of MoO_(2) nanosheets, introduces hydrophilic disparities between two sides within hydrogel. This distinctive approach bestows the hydrogel with ultrafast thermo-responsive actuation(21° s^(-1)) and enhanced photothermal efficiency(increase by 3.7 ℃ s^(-1) under 808 nm near-infrared). Moreover, the local cross-linking of sodium alginate with Ca^(2+) endows the hydrogel with programmable deformability and information display capabilities. Additionally, the hydrogel exhibits high sensitivity(gauge factor 3.94 within a wide strain range of 600%), fast response times(140 ms) and good cycling stability. Leveraging these exceptional properties, we incorporate the hydrogel into various soft actuators, including soft gripper, artificial iris, and bioinspired jellyfish, as well as wearable electronics capable of precise human motion and physiological signal detection. Furthermore, through the synergistic combination of remarkable actuation and sensitivity, we realize a self-sensing touch bioinspired tongue. Notably, by employing quantitative analysis of actuation-sensing, we realize remote interaction between soft-hard robot via the Internet of Things. The multifunctional self-sensing actuated gradient hydrogel presented in this study provides a new insight for advanced somatosensory materials, self-feedback intelligent soft robots and human–machine interactions.

Local Adaptive Gradient Variance Attack for Deep Fake Fingerprint Detection

In recent years,deep learning has been the mainstream technology for fingerprint liveness detection(FLD)tasks because of its remarkable performance.However,recent studies have shown that these deep fake fingerprint detection(DFFD)models are not resistant to attacks by adversarial examples,which are generated by the introduction of subtle perturbations in the fingerprint image,allowing the model to make fake judgments.Most of the existing adversarial example generation methods are based on gradient optimization,which is easy to fall into local optimal,resulting in poor transferability of adversarial attacks.In addition,the perturbation added to the blank area of the fingerprint image is easily perceived by the human eye,leading to poor visual quality.In response to the above challenges,this paper proposes a novel adversarial attack method based on local adaptive gradient variance for DFFD.The ridge texture area within the fingerprint image has been identified and designated as the region for perturbation generation.Subsequently,the images are fed into the targeted white-box model,and the gradient direction is optimized to compute gradient variance.Additionally,an adaptive parameter search method is proposed using stochastic gradient ascent to explore the parameter values during adversarial example generation,aiming to maximize adversarial attack performance.Experimental results on two publicly available fingerprint datasets show that ourmethod achieves higher attack transferability and robustness than existing methods,and the perturbation is harder to perceive.

Strong evidence for latitudinal diversity gradient in mosses across the world

Species richness generally decreases with increasing latitude,a biodiversity gradient that has long been considered as one of the few laws in ecology.This latitudinal diversity gradient has been observed in many major groups of organisms.In plants,the latitudinal diversity gradient has been observed in vascular plants,angiosperms,ferns,and liverworts.However,a conspicuous latitudinal diversity gradient in mosses at a global or continental scale has not been observed until now.Here,we analyze a comprehensive data set including moss species in each band of 20° in latitude worldwide.Our results show that moss species richness decreases strongly with increasing latitude,regardless of whether the globe is considered as a whole or different longitudinal segments(e.g.,Old World versus New World) are considered separately.This result holds when variation in area size among latitudinal bands is taken into account.Pearson's correlation coefficient between latitude and species richness is-0.99 for both the Northern and Southern Hemispheres.Because bryophytes are an extant lineage of early land plants and because mosses not only include most of extant species of bryophytes but also are important constituents of most terrestrial ecosystems,understanding geographic patterns of mosses is particularly important The finding of our study fills a critical knowledge gap.

Changes of growth-climate relationships of Smith fir forests along an altitudinal gradient

Temporal changes in the relationship between tree growth and climate have been observed in numerous forests across the world.The patterns and the possible regu-lators(e.g.,forest community structure)of such changes are,however,not well understood.A vegetation survey and analyses of growth-climate relationships for Abies georgei var.Smithii(Smith fir)forests were carried along an altitudi-nal gradient from 3600 to 4200 m on Meili Snow Mountain,southeastern Tibetan Plateau.The results showed that the associations between growth and temperature have declined since the 1970s over the whole transect,while response to standardized precipitation-evapotranspiration indices(SPEI)strengthened in the mid-and lower-transect.Comparison between growth and vegetation data showed that tree growth was more sensitive to drought in stands with higher species richness and greater shrub cover.Drought stress on growth may be increased by heavy competition from shrub and herb layers.These results show the non-stationary nature of tree growth-climate associations and the linkage to for-est community structures.Vegetation components should be considered in future modeling and forecasting of forest dynamics in relation to climate changes.

Full waveform inversion based on hybrid gradient

The low-wavenumber components in the gradient of full waveform inversion(FWI)play a vital role in the stability of the inversion.However,when FWI is implemented in some high frequencies and current models are not far away from the real velocity model,an excessive number of low-wavenumber components in the gradient will also reduce the convergence rate and inversion accuracy.To solve this problem,this paper firstly derives a formula of scattering angle weighted gradient in FWI,then proposes a hybrid gradient.The hybrid gradient combines the conventional gradient of FWI with the scattering angle weighted gradient in each inversion frequency band based on an empirical formula derived herein.Using weighted hybrid mode,we can retain some low-wavenumber components in the initial lowfrequency inversion to ensure the stability of the inversion,and use more high-wavenumber components in the high-frequency inversion to improve the convergence rate.The results of synthetic data experiment demonstrate that compared to the conventional FWI,the FWI based on the proposed hybrid gradient can effectively reduce the low-wavenumber components in the gradient under the premise of ensuring inversion stability.It also greatly enhances the convergence rate and inversion accuracy,especially in the deep part of the model.And the field marine seismic data experiment also illustrates that the FWI based on hybrid gradient(HGFWI)has good stability and adaptability.

Simultaneous purification of minor components in natural products using twin-column recycling chromatography with a step solvent gradient

The isolation of minor components from complex natural product matrices presents a significant challenge in the field of purification science due to their low concentrations and the presence of structurally similar compounds.This study introduces an optimized twin-column recycling chromatography method for the efficient and simultaneous purification of these elusive constituents.By introducing water at a small flowing rate between the twin columns,a step solvent gradient is created,by which the leading edge of concentration band would migrate at a slower rate than the trailing edge as it flowing from the upstream to downstream column.Hence,the band broadening is counterbalanced,resulting in an enrichment effect for those minor components in separation process.Herein,two target substances,which showed similar peak position in high performance liquid chromatography(HPLC)and did not exceed 1.8%in crude paclitaxel were selected as target compounds for separation.By using the twin-column recycling chromatography with a step solvent gradient,a successful purification was achieved in getting the two with the purity almost 100%.We suggest this method is suitable for the separation of most components in natural produces,which shows higher precision and recovery rate compared with the common lab-operated separation ways for natural products(thin-layer chromatography and prep-HPLC).