Deep reinforcement learning using least-squares truncated temporal-difference

Policy evaluation(PE)is a critical sub-problem in reinforcement learning,which estimates the value function for a given policy and can be used for policy improvement.However,there still exist some limitations in current PE methods,such as low sample efficiency and local convergence,especially on complex tasks.In this study,a novel PE algorithm called Least-Squares Truncated Temporal-Difference learning(LST2D)is proposed.In LST2D,an adaptive truncation mechanism is designed,which effectively takes advantage of the fast convergence property of Least-Squares Temporal Difference learning and the asymptotic convergence property of Temporal Difference learning(TD).Then,two feature pre-training methods are utilised to improve the approximation ability of LST2D.Furthermore,an Actor-Critic algorithm based on LST2D and pre-trained feature representations(ACLPF)is proposed,where LST2D is integrated into the critic network to improve learning-prediction efficiency.Comprehensive simulation studies were conducted on four robotic tasks,and the corresponding results illustrate the effectiveness of LST2D.The proposed ACLPF algorithm outperformed DQN,ACER and PPO in terms of sample efficiency and stability,which demonstrated that LST2D can be applied to online learning control problems by incorporating it into the actor-critic architecture.

Carbon and carbon composites for thermoelectric applications

The urgent need for consistent,reliable,ecofriendly,and stable power sources drives the development of new green energy materials.Thermoelectric(TE)materials receive increasing attention due to their unique capability of realizing the direct energy conversion between heat and electricity,showing diverse applications in harvesting waste heat and low-grade heat.Carbon materials such as carbon nanotubes(CNTs)and graphene have experienced a rapid development as TE materials because of their intrinsic ultrahigh electrical conductivity and light weight.Besides,polymer-based carbon composites are particularly fascinating as the combination of the merits of polymers and filler materials leads to high TE performance and superior flexibility.Herein,the recent TE advances are systematically summarized in the studied popularity of carbon materials(ie,CNTs and graphene)and the category of polymers.The conducting polymer-based carbon materials are particularly highlighted.Finally,the remaining challenges and some tentative suggestions possibly guiding future developments are proposed,which may pave a way for a bright future of carbon and carbon composites in the energy market.

Application of GDAHP on Quality Evaluation of Urban Lake Landscape

The quality evaluation of urban lake landscape (QEULL) is extremely important for the healthy development of lake landscape. In this research, the evaluation model was established with the group decision analytic hierarchy process (GDAHP) method, which consisted of four layers including the target layer, the factor layer, the index layer and the criterion layer, thus forming a model tree based on their subordinate relation-ships. The GDAHP method was employed to determine the weights of constituting factors of each layer in the evaluation model, and the fuzzy method was used to establish the factors remark sets of the criterion layer, thus the single-layer evaluation and comprehensive evaluation of urban lake landscape quality was carried out. Quality evaluation model of urban lake landscape established based on the GDAHP method can provide grounds for planning, design, and renewal of urban lake landscape. This model has been used to evaluate and analyze the artificial lake in People’s Park of Xinxiang City, Henan Province. The results proved that the overall landscape quality of the artificial lake of Peoples Park in Xinxiang city was good.

The Frontiers in Manufacturing Technologies Initiative and University Innovation

Basic innovation,with universities as a key driver,is essential for advancing core technologies in the manufacturing industry.This study used a manufacturing technologies initiative as a natural experiment,combining funding and its outcomes data from the National Natural Science Foundation of China to construct a difference-in-differences model.It found that the Frontiers in Manufacturing Technologies Initiative had a positive effect on university innovation,particularly by increasing the number of university patents.Mechanism analysis showed that the policy mainly encouraged university innovation through university-industry collaboration.The effect was more salient in universities with stronger basic research capabilities and those whose research fields matched strategic emerging industries covered by the initiative.This study underscores the crucial role of the Frontiers in Manufacturing Technologies Initiative in enhancing university-led innovation.It highlights the effectiveness of industry-academia partnerships in advancing technology,particularly in emerging strategic industries.

Toward improved trade-off between thermoelectric and mechanical performances in polycarbonate/single-walled carbon nanotube composite films

Polymer thermoelectric(TE)composites have witnessed explosive developments in recent years,arising from their promising prospect for lightweight flexible electronics and capability of harvesting waste-heat.In sharp contrast with intrinsically conducting polymers(CPs),the insulating thermoplastics have seldom been employed as the matrices for flexible TE composites despite their advantages of low costs,controllable melt-flowing behaviors and excellent mechanical properties.Here,we report flexible films of polycarbonate/single-walled carbon nanotube(PC/SWCNT)composites with improved trade-off between TE and mechanical performances.The SWCNTs with 1D nanostructure were dramatically aligned by PC melt-flowing under hot-pressing in the radial direction.The composite maximum power factor reaches 4.8±0.8μW m^(−1) K^(−2) at 10 wt%SWCNTs in the aligned direction,which is higher than most previously reported thermoplastics-based TE composites at the same SWCNT loading and even comparable to some intrinsically CPs and their composites.In addition,these composites display significantly higher tensile modulus and strength than CPs and their composites.This study paves an effective way to fabricate flexible films of polymer composites with simultaneously high TE and mechanical performances via judicious alignment of SWCNTs in thermoplastic polymers.