Actor-Critic是一种强化学习方法,通过与环境在线试错交互收集样本来学习策略,是求解序贯感知决策问题的有效手段.但是,这种在线交互的主动学习范式在一些复杂真实环境中收集样本时会带来成本和安全问题离线强化学习作为一种基于数据驱...Actor-Critic是一种强化学习方法,通过与环境在线试错交互收集样本来学习策略,是求解序贯感知决策问题的有效手段.但是,这种在线交互的主动学习范式在一些复杂真实环境中收集样本时会带来成本和安全问题离线强化学习作为一种基于数据驱动的强化学习范式,强调从静态样本数据集中学习策略,与环境无探索交互,为机器人、自动驾驶、健康护理等真实世界部署应用提供了可行的解决方案,是近年来的研究热点.目前,离线强化学习方法存在学习策略和行为策略之间的分布偏移挑战,针对这个挑战,通常采用策略约束或值函数正则化来限制访问数据集分布之外(Out-Of-Distribution,OOD)的动作,从而导致学习性能过于保守,阻碍了值函数网络的泛化和学习策略的性能提升.为此,本文利用不确定性估计和OOD采样来平衡值函数学习的泛化性和保守性,提出一种基于不确定性估计的离线确定型Actor-Critic方法(Offline Deterministic Actor-Critic based on UncertaintyEstimation,ODACUE).首先,针对确定型策略,给出一种Q值函数的不确定性估计算子定义,理论证明了该算子学到的Q值函数是最优Q值函数的一种悲观估计.然后,将不确定性估计算子应用于确定型Actor-Critic框架中,通过对不确定性估计算子进行凸组合构造Critic学习的目标函数.最后,D4RL基准数据集任务上的实验结果表明:相较于对比算法,ODACUE在11个不同质量等级数据集任务中的总体性能提升最低达9.56%,最高达64.92%.此外,参数分析和消融实验进一步验证了ODACUE的稳定性和泛化能力.展开更多
To investigate the dynamic response problem of the double medium formed by the adherence of sprayed concrete and surrounding rock in the tunnel,a split Hopkinson pressure bar of 75 mm in diameter was adopted at the ag...To investigate the dynamic response problem of the double medium formed by the adherence of sprayed concrete and surrounding rock in the tunnel,a split Hopkinson pressure bar of 75 mm in diameter was adopted at the ages of 3,7 and 10 d.Experimental results showed that dynamic compressive strength and dynamic increase factors(DIF)of the combined bodies increase with the strain rate.With the growth of strain rate,the critical strain of the combined bodies first increases,then deceases.Furthermore,the combined bodies of 3 d reveal the plastic property and brittle property for 7 d and 10 d when the strain rate is over 80/s.The failure characteristic of the sprayed concrete changes from tearing strain damage to crushing damage as the growth of strain rate,and the failure characteristic of rock presents the tensile failure mode as demonstrated by the scanning electron microscope(SEM).展开更多
BACKGROUND In the past 3 years,the global pandemic of coronavirus disease 2019(COVID-19)has posed a great threat to human life and safety.Among the causes of death in COVID-19 patients,combined or secondary bacterial ...BACKGROUND In the past 3 years,the global pandemic of coronavirus disease 2019(COVID-19)has posed a great threat to human life and safety.Among the causes of death in COVID-19 patients,combined or secondary bacterial infection is an important factor.As a special group,pregnant women experience varying degrees of change in their immune status,cardiopulmonary function,and anatomical structure during pregnancy,which puts them at higher risk of contracting COVID-19.COVID-19 infection during pregnancy is associated with increased adverse events such as hospitalisation,admission to the intensive care unit,and mechanical ventilation.Therefore,pregnancy combined with coinfection of COVID-19 and bacteria often leads to critical respiratory failure,posing severe challenges in the diagnosis and treatment process.CASE SUMMARY We report a case of COVID-19 complicated with Staphylococcus aureus(S.aureus)coinfection in a pre-gnant woman at 34 wk of gestation.Her rapid progression of pulmonary lesions caused severe respiratory failure,and she received noninvasive ventilator-assisted respiratory treatment.Subsequently,we delivered a foetus via emergency caesarean section after accelerating the maturity of the foetal pulmonary system,and the respiratory condition of the puerperant woman significantly improved after the delivery of the foetus.Lavage fluid was taken under tracheoscopy to quickly search for pathogens by the metagenomic nextgeneration sequencing(mNGS),and both COVID-19 and S.aureus were detected.After targeted anti-infective treatment,the maternal condition gradually improved,and the patient was discharged from the hospital.CONCLUSION The coinfection of pregnancy with COVID-19 and bacteria often leads to critical respiratory failure,which is a great challenge in the process of diagnosis and treatment.It is crucial to choose the right time to deliver the foetus and to quickly find pathogens by mNGS.展开更多
文摘Actor-Critic是一种强化学习方法,通过与环境在线试错交互收集样本来学习策略,是求解序贯感知决策问题的有效手段.但是,这种在线交互的主动学习范式在一些复杂真实环境中收集样本时会带来成本和安全问题离线强化学习作为一种基于数据驱动的强化学习范式,强调从静态样本数据集中学习策略,与环境无探索交互,为机器人、自动驾驶、健康护理等真实世界部署应用提供了可行的解决方案,是近年来的研究热点.目前,离线强化学习方法存在学习策略和行为策略之间的分布偏移挑战,针对这个挑战,通常采用策略约束或值函数正则化来限制访问数据集分布之外(Out-Of-Distribution,OOD)的动作,从而导致学习性能过于保守,阻碍了值函数网络的泛化和学习策略的性能提升.为此,本文利用不确定性估计和OOD采样来平衡值函数学习的泛化性和保守性,提出一种基于不确定性估计的离线确定型Actor-Critic方法(Offline Deterministic Actor-Critic based on UncertaintyEstimation,ODACUE).首先,针对确定型策略,给出一种Q值函数的不确定性估计算子定义,理论证明了该算子学到的Q值函数是最优Q值函数的一种悲观估计.然后,将不确定性估计算子应用于确定型Actor-Critic框架中,通过对不确定性估计算子进行凸组合构造Critic学习的目标函数.最后,D4RL基准数据集任务上的实验结果表明:相较于对比算法,ODACUE在11个不同质量等级数据集任务中的总体性能提升最低达9.56%,最高达64.92%.此外,参数分析和消融实验进一步验证了ODACUE的稳定性和泛化能力.
基金Supported by the National Key Research Program(2017YFC0804200)the National Key Basic Research Program(2016YFC0600903)the National Natural Science Foundation of China(51274204)
文摘To investigate the dynamic response problem of the double medium formed by the adherence of sprayed concrete and surrounding rock in the tunnel,a split Hopkinson pressure bar of 75 mm in diameter was adopted at the ages of 3,7 and 10 d.Experimental results showed that dynamic compressive strength and dynamic increase factors(DIF)of the combined bodies increase with the strain rate.With the growth of strain rate,the critical strain of the combined bodies first increases,then deceases.Furthermore,the combined bodies of 3 d reveal the plastic property and brittle property for 7 d and 10 d when the strain rate is over 80/s.The failure characteristic of the sprayed concrete changes from tearing strain damage to crushing damage as the growth of strain rate,and the failure characteristic of rock presents the tensile failure mode as demonstrated by the scanning electron microscope(SEM).
文摘BACKGROUND In the past 3 years,the global pandemic of coronavirus disease 2019(COVID-19)has posed a great threat to human life and safety.Among the causes of death in COVID-19 patients,combined or secondary bacterial infection is an important factor.As a special group,pregnant women experience varying degrees of change in their immune status,cardiopulmonary function,and anatomical structure during pregnancy,which puts them at higher risk of contracting COVID-19.COVID-19 infection during pregnancy is associated with increased adverse events such as hospitalisation,admission to the intensive care unit,and mechanical ventilation.Therefore,pregnancy combined with coinfection of COVID-19 and bacteria often leads to critical respiratory failure,posing severe challenges in the diagnosis and treatment process.CASE SUMMARY We report a case of COVID-19 complicated with Staphylococcus aureus(S.aureus)coinfection in a pre-gnant woman at 34 wk of gestation.Her rapid progression of pulmonary lesions caused severe respiratory failure,and she received noninvasive ventilator-assisted respiratory treatment.Subsequently,we delivered a foetus via emergency caesarean section after accelerating the maturity of the foetal pulmonary system,and the respiratory condition of the puerperant woman significantly improved after the delivery of the foetus.Lavage fluid was taken under tracheoscopy to quickly search for pathogens by the metagenomic nextgeneration sequencing(mNGS),and both COVID-19 and S.aureus were detected.After targeted anti-infective treatment,the maternal condition gradually improved,and the patient was discharged from the hospital.CONCLUSION The coinfection of pregnancy with COVID-19 and bacteria often leads to critical respiratory failure,which is a great challenge in the process of diagnosis and treatment.It is crucial to choose the right time to deliver the foetus and to quickly find pathogens by mNGS.
