Imaginary filtered hindsight experience replay for UAV tracking dynamic targets in large-scale unknown environments

As an advanced combat weapon,Unmanned Aerial Vehicles(UAVs)have been widely used in military wars.In this paper,we formulated the Autonomous Navigation Control(ANC)problem of UAVs as a Markov Decision Process(MDP)and proposed a novel Deep Reinforcement Learning(DRL)method to allow UAVs to perform dynamic target tracking tasks in large-scale unknown environments.To solve the problem of limited training experience,the proposed Imaginary Filtered Hindsight Experience Replay(IFHER)generates successful episodes by reasonably imagining the target trajectory in the failed episode to augment the experiences.The welldesigned goal,episode,and quality filtering strategies ensure that only high-quality augmented experiences can be stored,while the sampling filtering strategy of IFHER ensures that these stored augmented experiences can be fully learned according to their high priorities.By training in a complex environment constructed based on the parameters of a real UAV,the proposed IFHER algorithm improves the convergence speed by 28.99%and the convergence result by 11.57%compared to the state-of-the-art Twin Delayed Deep Deterministic Policy Gradient(TD3)algorithm.The testing experiments carried out in environments with different complexities demonstrate the strong robustness and generalization ability of the IFHER agent.Moreover,the flight trajectory of the IFHER agent shows the superiority of the learned policy and the practical application value of the algorithm.

基于超声衰减和声速动态监测石蜡的相变过程

为研究石蜡在相变过程中内部结构和状态的改变特性,用中心频率为5 MHz的脉冲式超声波动态测量石蜡的相变过程,采集并分析不同升降温速率下的声速和声衰减信号的变化规律,结果与差示扫描量热仪(DSC)测量的热力学性质比较,并拍摄石蜡溶解过程的图像为辅助,探讨了二者由于测量原理不同导致的差异和特点。结果表明,两种方法均得到约50℃的初凝点,且二者信号反映的相变规律一致,表明利用声衰减和声速能够较好的表征石蜡在相变过程中的声学特性。超声可能成为一种新的蜡化物性质原位测量手段。

光学透明技术在植物多尺度成像中的应用

光学透明技术是一种通过各种化学试剂,将原本不透明的生物样本实现透明化,并在光学显微镜下深度成像的技术。结合多种光学显微成像新技术,光学透明技术可对整个组织进行成像和三维重建,深度剖析生物体内部空间特征与形成机制。近年来,多种植物光学透明技术和多尺度成像技术被陆续研发,并取得了丰硕的研究成果。该文综述了生物体光学透明技术的基本原理和一些新技术,重点介绍基于光学透明技术开发的新型成像方法及其在植物成像与细胞生物学中的应用,为后续植物整体、组织或器官的透明、成像与三维重构及功能研究提供理论依据和技术支持。

Relevant experience learning:A deep reinforcement learning method for UAV autonomous motion planning in complex unknown environments

Unmanned Aerial Vehicles(UAVs)play a vital role in military warfare.In a variety of battlefield mission scenarios,UAVs are required to safely fly to designated locations without human intervention.Therefore,finding a suitable method to solve the UAV Autonomous Motion Planning(AMP)problem can improve the success rate of UAV missions to a certain extent.In recent years,many studies have used Deep Reinforcement Learning(DRL)methods to address the AMP problem and have achieved good results.From the perspective of sampling,this paper designs a sampling method with double-screening,combines it with the Deep Deterministic Policy Gradient(DDPG)algorithm,and proposes the Relevant Experience Learning-DDPG(REL-DDPG)algorithm.The REL-DDPG algorithm uses a Prioritized Experience Replay(PER)mechanism to break the correlation of continuous experiences in the experience pool,finds the experiences most similar to the current state to learn according to the theory in human education,and expands the influence of the learning process on action selection at the current state.All experiments are applied in a complex unknown simulation environment constructed based on the parameters of a real UAV.The training experiments show that REL-DDPG improves the convergence speed and the convergence result compared to the state-of-the-art DDPG algorithm,while the testing experiments show the applicability of the algorithm and investigate the performance under different parameter conditions.

Artificial Intelligence Based Smart Energy Community Management: A Reinforcement Learning Approach

This paper presents a smart energy community management approach which is capable of implementing P2P trading and managing household energy storage systems.A smart residential community concept is proposed consisting of domestic users and a local energy pool,in which users are free to trade with the local energy pool and enjoy cheap renewable energy while avoiding the installation of new energy generation equipment.The local energy pool could harvest surplus energy from users and renewable resources,at the same time it sells energy at a higher price than Feed-in-Tariff(FIT)but lower than the retail price.In order to encourage the participation in local energy trading,the electricity price of the energy pool is determined by a real-time demand/supply ratio.Under this pricing mechanism,retail price,users and renewable energy could all affect the electricity price which leads to higher consumers’profits and more optimized utilization of renewable energy.The proposed energy trading process was modeled as a Markov Decision Process(MDP)and a reinforcement learning algorithm was adopted to find the optimal decision in the MDP because of its excellent performance in on-going and model-free tasks.In addition,the fuzzy inference system makes it possible to use Q-learning in continuous state-space problems(Fuzzy Q-learning)considering the infinite possibilities in the energy trading process.To evaluate the performance of the proposed demand side management system,a numerical analysis is conducted in a community comparing the electricity costs before and after using the proposed energy management system.

Plant multiscale networks:charting plant connectivity by multi-level analysis and imaging techniques

In multicellular and even single-celled organisms,individual components are interconnected at multiscale levels to produce enormously complex biological networks that help these systems maintain homeostasis for development and environmental adaptation.Systems biology studies initially adopted network analysis to explore how relationships between individual components give rise to complex biological processes.Network analysis has been applied to dissect the complex connectivity of mammalian brains across different scales in time and space in The Human Brain Project.In plant science,network analysis has similarly been applied to study the connectivity of plant components at the molecular,subcellular,cellular,organic,and organism levels.Analysis of these multiscale networks contributes to our understanding of how genotype determines phenotype.In this review,we summarized the theoretical framework of plant multiscale networks and introduced studies investigating plant networks by various experimental and computational modalities.We next discussed the currently available analytic methodologies and multi-level imaging techniques used to map multiscale networks in plants.Finally,we highlighted some of the technical challenges and key questions remaining to be addressed in this emerging field.