Probabilistic Automata-Based Method for Enhancing Performance of Deep Reinforcement Learning Systems

Deep reinforcement learning(DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management.However, due to the model's inherent uncertainty, rigorous validation is requisite for its application in real-world tasks. Specific tests may reveal inadequacies in the performance of pre-trained DRL models, while the “black-box” nature of DRL poses a challenge for testing model behavior. We propose a novel performance improvement framework based on probabilistic automata,which aims to proactively identify and correct critical vulnerabilities of DRL systems, so that the performance of DRL models in real tasks can be improved with minimal model modifications.First, a probabilistic automaton is constructed from the historical trajectory of the DRL system by abstracting the state to generate probabilistic decision-making units(PDMUs), and a reverse breadth-first search(BFS) method is used to identify the key PDMU-action pairs that have the greatest impact on adverse outcomes. This process relies only on the state-action sequence and final result of each trajectory. Then, under the key PDMU, we search for the new action that has the greatest impact on favorable results. Finally, the key PDMU, undesirable action and new action are encapsulated as monitors to guide the DRL system to obtain more favorable results through real-time monitoring and correction mechanisms. Evaluations in two standard reinforcement learning environments and three actual job scheduling scenarios confirmed the effectiveness of the method, providing certain guarantees for the deployment of DRL models in real-world applications.

中国农村基本医疗卫生服务综合评价指标体系构建的定性系统评价

目的系统梳理新一轮医药卫生体制改革以来农村基本医疗卫生服务综合评价指标体系,基于PHCPI概念框架(primary health care performance initiative conceptual framework)探寻目前评价指标普遍关注的内容和可能被忽略的评价内容,为后续科学、全面地评价农村基本医疗卫生服务提供参考依据。方法通过中国知网、万方数据知识服务平台、维普中文科技期刊数据库检索2009—2019年有关农村基本医疗卫生服务综合评价指标体系的文献,基于SPIDER规范制定纳入排除标准,采用批判评估技术方案对纳入文献进行质量评价,运用框架合成法,选取PHCPI概念框架对纳入的文献进行归纳和总结。结果共纳入25篇文献,总体质量良好。评价体系中的指标可分为5个一级领域、16个二级领域、24个三级领域指标。综合对比各领域对应指标的文献数,卫生资金、效率等领域对应指标的文献数较多,均有20篇左右的文献,高于其他多数领域;投入、产出和结果的相应领域中,近85%的领域对应指标的文献在15篇及以上;服务提供的各领域中,仅18%左右的领域对应指标的文献达到15篇,40%以上领域对应指标的文献未超过5篇。结论农村基本医疗卫生服务综合评价体系对经济效益相关指标关注度较高,投入、产出和结果多数领域的指标重合度较高,评价体系对服务提供领域的关注度较低且内容分散。今后可加强对社会效益和服务提供相关指标的探索,以全面、综合地评价农村基本医疗卫生服务。

Metal-organic framework derived magnetic nanoporous carbon as an adsorbent for the magnetic solid-phase extraction of chlorophenols from mushroom sample

In this work, a metal-organic framework derived nanoporous carbon （MOF-5-C） was fabricated and modified with Fe3O4 magnetic nanoparticles. The resulting magnetic MOF-5-derived porous carbon （Fe304@MOF-5-C） was then used for the magnetic solid-phase extraction of chlorophenols （CPs） from mushroom samples prior to high performance liquid chromatography-ultraviolet detection. Scanning electron microscopy, transmission electron microscopy, X-ray diffraction, and N2 adsorption were used to characterize the adsorbent. After experimental optimization, the amount of the adsorbent was chosen as 8.0 mg, extraction time as 10 min, sample volume as 50 mL, desorption solvent as 0.4 mL （0.2 mL × 2） of alkaline methanol, and sample pH as 6. Under the above optimized conditions, good linearity for the analytes was obtained in the range of 0.8-100.0 ng g 1 with the correlation coefficients between 0.9923 and 0.9963. The limits of detection （SIN= 3） were in the range of 0.25-0.30 ng g-1, and the relative standard deviations were below 6.8%. The result showed that the Fe304@MOF-5-C has an excellent adsorption capacity for the analytes.