Hierarchical Reinforcement Learning With Automatic Sub-Goal Identification

In reinforcement learning an agent may explore ineffectively when dealing with sparse reward tasks where finding a reward point is difficult.To solve the problem,we propose an algorithm called hierarchical deep reinforcement learning with automatic sub-goal identification via computer vision(HADS)which takes advantage of hierarchical reinforcement learning to alleviate the sparse reward problem and improve efficiency of exploration by utilizing a sub-goal mechanism.HADS uses a computer vision method to identify sub-goals automatically for hierarchical deep reinforcement learning.Due to the fact that not all sub-goal points are reachable,a mechanism is proposed to remove unreachable sub-goal points so as to further improve the performance of the algorithm.HADS involves contour recognition to identify sub-goals from the state image where some salient states in the state image may be recognized as sub-goals,while those that are not will be removed based on prior knowledge.Our experiments verified the effect of the algorithm.

N-type core-shell heterostructured Bi2S3@Bi nanorods/polyaniline hybrids for stretchable thermoelectric generator

With the growing need on distributed power supply for portable electronics,energy harvesting from environment becomes a promising solution.Organic thermoelectric(TE)materials have advantages in intrinsic flexibility and low thermal conductivity,thus hold great prospect in applications as a flexible power generator from dissipated heat.Nevertheless,the weak electrical transport behaviors of organic TE materials have severely impeded their development.Moreover,compared with p-type organic TE materials,stable and high-performance n-type counterparts are more difficult to obtain.Here,we developed a n-type polyaniline-based hybrid with core-shell heterostructured Bi;S;@Bi nanorods as fillers,showing a Seebeck coefficient-159.4μV/K at room temperature.Further,a couple of n/p legs from the PANI-based hybrids were integrated into an elastomer substrate forming a stretchable thermoelectric generator(TEG),whose function to output stable voltages responding to temperature differences has been demonstrated.The in situ output performance of the TEG under stretching could withstand up to 75%elongation,and stability test showed little degradation over a one-month period in the air.This study provides a promising strategy to develop stable and high thermopower organic TEGs harvesting heat from environment as long-term power supply.

DeepWMH:无需人工标注训练的脑白质高信号病灶分割工具

White matter hyperintensities(WMHs)on fluid-attenuated inversion recovery(FLAIR)images are imaging features in various neurological diseases and essential markers for clinical impairment and disease progression.WMHs are associated with brain aging and pathological changes in the human brain,such as in Alzheimer’s disease(AD)[1],Parkinson’s disease(PD)[2],cerebral small vessel disease(SVD)[3],multiple sclerosis(MS)[4].

Dissecting PCNA function with a systematically designed mutant library in yeast

Proliferating cell nuclear antigen (PCNA), encoded by POL30 in Saccharomyces cerevisiae, is a key component of DNA metabolism. Here, a library consisting of 304 PCNA mutants was designed and constructed to probe the contribution of each residue to the biological function of PCNA. Five regions with elevated sen sitivity to DNA damaging reagents were identified using high-throughput phe no type screening. Using a series of genetic and biochemical analyses, we demonstrated that one particular mutant, K168A, has defects in the DNA damage tolerance (DDT) pathway by disrupting the interaction between PCNA and Rad5. Subsequent domain analysis showed that the PCNA-Rad5 interaction is a prerequisite for the function of Rad5 in DDT. Our study not only provides a resource in the form of a library of versatile mutants to study the functions of PCNA, but also reveals a key residue on PCNA (K168) which highlights the importance of the PCNA-Rad5 interaction in the template switching (TS) pathway.