Reinforcement Learning Transfer Based on Subgoal Discovery and Subtask Similarity

This paper studies the problem of transfer learning in the context of reinforcement learning.We propose a novel transfer learning method that can speed up reinforcement learning with the aid of previously learnt tasks.Before performing extensive learning episodes,our method attempts to analyze the learning task via some exploration in the environment,and then attempts to reuse previous learning experience whenever it is possible and appropriate.In particular,our proposed method consists of four stages:1) subgoal discovery,2) option construction,3) similarity searching,and 4) option reusing.Especially,in order to fulfill the task of identifying similar options,we propose a novel similarity measure between options,which is built upon the intuition that similar options have similar stateaction probabilities.We examine our algorithm using extensive experiments,comparing it with existing methods.The results show that our method outperforms conventional non-transfer reinforcement learning algorithms,as well as existing transfer learning methods,by a wide margin.

Room temperature in-situ preparation of hydrazine-linked covalent organic frameworks coated capillaries for separation and determination of polycyclic aromatic hydrocarbons

Covalent organic frameworks(COFs)have been increasingly used in capillary electrochromatography due to their excellent characteristics.In this work,hydrazine-linked TFPB-DHzDS(TFPB:1,3,5-tris(4-formylphenyl)benzene;DHzDS:2,5-bis(3-(ethylthio)propoxy)terephthalohydrazide)was first synthesized by a simpler and easier method at room temperature and introduced into capillary electrochromatography as coating material.The TFPB-DHzDS coated capillaries were prepared by an in-situ growth process at room temperature.After optimizing the coating concentration and experimental conditions of capillary electrochromatography,baseline separation of two groups of polycyclic aromatic hydrocarbons was achieved based on the TFPB-DHzDS coated capillary.And the established method was used successfully to determine PAHs in natural water and soil samples.The spiked recoveries of polycyclic aromatic hydrocarbons in these samples ranged from 90.01%to 111.0%,indicating that the method is reliable and could detect polycyclic aromatic hydrocarbons in natural samples.Finally,molecular simulation was applied to study and visualize the interaction between the analytes and coating COF materials to investigate the molecular level separation mechanism further.

Recent progress in nanomaterial-enhanced fluorescence polarization/anisotropy sensors

As a promising signaling transduction approach, fluorescence polarization(FP)/fluorescence anisotropy(FA), provides a powerful quantitative tool for the rotational motion of fluorescently labeled molecules in chemical or biological homogeneous systems. Unlike fluorescence intensity, FP/FA is almost independent the concentration or quantum of fluorophores, but they are highly dependent on the size or molecular weight of the molecules or materials attached to fluorophores. Recently, significant progress in FP/FA was made, due to the introduction of some nanomaterials as FP/FA enhancers. The detection sensitivity is thus greatly improved by using nanomaterials as FP/FA enhancers, and nanomaterial-based FP/FA is currently used successfully in immunoassay, and analysis of protein, nucleic acid, small molecule and metal ion.Nanomaterial-based FP/FA provides a new kind of strategy to design fluorescent sensors and establishes innovative analytical methods. In this review, we summarize the scientific publications in the field of FP/FA sensor in recent five years, and first introduce the recent progress of FP/FA sensor based on nanomaterial. Subsequently, the various analytical applications of FP/FA based on nanomaterial are discussed. Finally, we provide perspectives on the current challenges and future prospects in this promising field.

Recent advances in DNA glycosylase assays

Genomic deoxyribonucleic acid(DNA)is selected as the ideal carrier for preserving and transmitting the genetic information over the course of evolution.However,the genomic DNA is constantly exposed to various endogenous and environmental threats,causing a diversity of damaged bases,lesions,mismatches and base-pair modifications in the genome,eventually leading to genomic instability and cancers.Base excision repair(BER)is the most important repair mechanism,repairing a variety of DNA damages arising from oxidation,alkylation,methylation,deamination,and hydrolysis reactions.DNA glycosylases are responsible for initiating the first step of the BER pathway through cleaving the N-glycosidic bond between the damaged base and the DNA backbone.However,abnormal DNA glycosylases are associated with a variety of diseases such as cancer,cardiovascular disease,neurological disease and inflammation,suggesting the important role of DNA glycosylases in cancer diagnosis and treatment.Therefore,it is highly desirable to monitor the activity of DNA glycosylases,gaining a deep understanding of the restoration process of damaged DNA and clinical diagnosis.Recently,a series of novel DNA glycosylases detection methods with excellent performance have been developed.In this minireview,we summarize the recent advances in DNA glycosylase assays including amplification-free assay and amplification-assisted assay.Firstly,a brief introduction of amplification-free assay for DNA glycosylase is given.Then,amplification-assisted assays for DNA glycosylases are discussed in detail.Ultimately,the conclusion and prospects of the directions of DNA glycosylase assays are provided.

Research progress on mid-IR nonlinear optical crystals with high laser damage threshold in China

Nonlinear optical(NLO)crystals have been playing an increasingly important role in laser science and technology.The NLO crystals used in the middle infrared(mid-IR)region,compared with the NLO crystals in the other wavelength regions,are still not good enough for the application of high-energy laser.The main defect is that their laser damage thresholds(LDT)are low.Chinese scientists have made a lot of important contributions to the UV and visible NLO crystals.In the last decade,they also did a lot of work on the mid-IR NLO materials.The main purpose of these researches is to increase the LDT and simultaneously balance the other properties.This paper presents a brief summary of their research progress in this topic on three types of materials:chalcogenides,oxides,and halides.The emphasis is put on the design strategy and quality control of the crystals.