A policy gradient algorithm integrating long and short-term rewards for soft continuum arm control

The soft continuum arm has extensive application in industrial production and human life due to its superior safety and flexibility. Reinforcement learning is a powerful technique for solving soft arm continuous control problems, which can learn an effective control policy with an unknown system model. However, it is often affected by the high sample complexity and requires huge amounts of data to train, which limits its effectiveness in soft arm control. An improved policy gradient method, policy gradient integrating long and short-term rewards denoted as PGLS, is proposed in this paper to overcome this issue. The shortterm rewards provide more dynamic-aware exploration directions for policy learning and improve the exploration efficiency of the algorithm. PGLS can be integrated into current policy gradient algorithms, such as deep deterministic policy gradient(DDPG). The overall control framework is realized and demonstrated in a dynamics simulation environment. Simulation results show that this approach can effectively control the soft arm to reach and track the targets. Compared with DDPG and other model-free reinforcement learning algorithms, the proposed PGLS algorithm has a great improvement in convergence speed and performance. In addition, a fluid-driven soft manipulator is designed and fabricated in this paper, which can verify the proposed PGLS algorithm in real experiments in the future.

Fluorescence and SEM correlative microscopy for nanomanipulation of subcellular structures

Nanomanipulation under scanning electron microscopy(SEM)enables direct interactions of a tool with a sample.We recently developed a nanomanipulation technique for the extraction and identification of DNA contained within sub-nuclear locations of a single cell nucleus.In nanomanipulation of sub-cellular structures,a key step is to identify targets of interest through correlating fluorescence and SEM images.The DNA extraction task must be conducted with low accelerating voltages resulting in low imaging resolutions.This is imposed by the necessity of preserving the biochemical integrity of the sample.Such poor imaging conditions make the identification of nanometer-sized fiducial marks difficult.This paper presents an affine scale-invariant feature transform(ASIFT)based method for correlating SEM images and fluorescence microscopy images.The performance of the image correlation approach under different noise levels and imaging magnifications was quantitatively evaluated.The optimal mean absolute error(MAE)of correlation results is 68634 nm under standard conditions.Compared with manual correlation by skilled operators,the automated correlation approach demonstrates a speed that is higher by an order of magnitude.With the SEM-fluorescence image correlation approach,targeted DNA was successfully extracted via nanomanipulation under SEM conditions.

Rotation Estimation for Mobile Robot Based on Single-axis Gyroscope and Monocular Camera

The rotation matrix estimation problem is a keypoint for mobile robot localization, navigation, and control. Based on the quaternion theory and the epipolar geometry, an extended Kalman filter (EKF) algorithm is proposed to estimate the rotation matrix by using a single-axis gyroscope and the image points correspondence from a monocular camera. The experimental results show that the precision of mobile robot s yaw angle estimated by the proposed EKF algorithm is much better than the results given by the image-only and gyroscope-only method, which demonstrates that our method is a preferable way to estimate the rotation for the autonomous mobile robot applications.

A New Calibration Method for an Inertial and Visual Sensing System

The relative pose between inertial and visual sensors equipped in autonomous robots is calibrated in two steps. In the first step, the sensing system is moved along a line, the orientations in the relative pose are computed from at least five corresponding points in the two images captured before and after the movement. In the second step, the translation parameters in the relative pose are obtained with at least two corresponding points in the two images captured before and after one step motion. Experiments are conducted to verify the effectiveness of the proposed method.

Large-diameter trees dominate snag and surface biomass following reintroduced fire

The reintroduction of fire to landscapes where it was once common is considered a priority to restore historical forest dynamics,including reducing tree density and decreasing levels of woody biomass on the forest floor.However,reintroducing fire causes tree mortality that can have unintended ecological outcomes related to woody biomass,with potential impacts to fuel accumulation,carbon sequestration,subsequent fire severity,and forest management.In this study,we examine the interplay between fire and carbon dynamics by asking how reintroduced fire impacts fuel accumulation,carbon sequestration,and subsequent fire severity potential.Beginning pre-fire,and continuing 6 years post-fire,we tracked all live,dead,and fallen trees≥1 cm in diameter and mapped all pieces of deadwood(downed woody debris)originating from tree boles≥10 cm diameter and≥1 m in length in 25.6 ha of an Abies concolor/Pinus lambertiana forest in the central Sierra Nevada,California,USA.We also tracked surface fuels along 2240 m of planar transects pre-fire,immediately post-fire,and 6 years post-fire.Six years after moderate-severity fire,deadwood≥10 cm diameter was 73 Mg ha^(−1),comprised of 32 Mg ha^(−1) that persisted through fire and 41 Mg ha^(−1) of newly fallen wood(compared to 72 Mg ha^(−1) pre-fire).Woody surface fuel loading was spatially heterogeneous,with mass varying almost four orders of magnitude at the scale of 20 m×20 m quadrats(minimum,0.1 Mg ha^(−1);mean,73 Mg ha^(−1);maximum,497 Mg ha^(−1)).Wood from large-diameter trees(≥60 cm diameter)comprised 57%of surface fuel in 2019,but was 75%of snag biomass,indicating high contributions to current and future fuel loading.Reintroduction of fire does not consume all large-diameter fuel and generates high levels of surface fuels≥10 cm diameter within 6 years.Repeated fires are needed to reduce surface fuel loading.

Novel PIO Algorithm with Multiple Selection Strategies for Many-Objective Optimization Problems

With the increase of problem dimensions,most solutions of existing many-objective optimization algorithms are non-dominant.Therefore,the selection of individuals and the retention of elite individuals are important.Existing algorithms cannot provide sufficient solution precision and guarantee the diversity and convergence of solution sets when solving practical many-objective industrial problems.Thus,this work proposes an improved many-objective pigeon-inspired optimization(ImMAPIO)algorithm with multiple selection strategies to solve many-objective optimization problems.Multiple selection strategies integrating hypervolume,knee point,and vector angles are utilized to increase selection pressure to the true Pareto Front.Thus,the accuracy,convergence,and diversity of solutions are improved.ImMAPIO is applied to the DTLZ and WFG test functions with four to fifteen objectives and compared against NSGA-III,GrEA,MOEA/D,RVEA,and many-objective Pigeon-inspired optimization algorithm.Experimental results indicate the superiority of ImMAPIO on these test functions.