为解决由于固定温度SAC(Soft Actor Critic)算法中存在的Q函数高估可能会导致算法陷入局部最优的问题,通过深入分析提出了一个稳定且受限的SAC算法(SCSAC:Stable Constrained Soft Actor Critic)。该算法通过改进最大熵目标函数修复固...为解决由于固定温度SAC(Soft Actor Critic)算法中存在的Q函数高估可能会导致算法陷入局部最优的问题,通过深入分析提出了一个稳定且受限的SAC算法(SCSAC:Stable Constrained Soft Actor Critic)。该算法通过改进最大熵目标函数修复固定温度SAC算法中的Q函数高估问题,同时增强算法在测试过程中稳定性的效果。最后,在4个OpenAI Gym Mujoco环境下对SCSAC算法进行了验证,实验结果表明,稳定且受限的SAC算法相比固定温度SAC算法可以有效减小Q函数高估出现的次数并能在测试中获得更加稳定的结果。展开更多
虚拟电厂(virtual power plant,VPP)作为多能流互联的综合能源网络,已成为中国加速实现双碳目标的重要角色。但VPP内部资源协同低碳调度面临多能流的耦合程度紧密、传统碳交易模型参数主观性强、含高维动态参数的优化目标在线求解困难...虚拟电厂(virtual power plant,VPP)作为多能流互联的综合能源网络,已成为中国加速实现双碳目标的重要角色。但VPP内部资源协同低碳调度面临多能流的耦合程度紧密、传统碳交易模型参数主观性强、含高维动态参数的优化目标在线求解困难等问题。针对这些问题,文中提出一种融合注意力机制(attention mechanism,AM)与柔性动作评价(soft actor-critic,SAC)算法的VPP多能流低碳调度方法。首先,根据VPP的随机碳流特性,面向动态参数建立基于贝叶斯优化的改进阶梯型碳交易机制。接着,以经济效益和碳排放量为目标函数构建含氢VPP多能流解耦模型。然后,考虑到该模型具有高维非线性与权重参数实时更新的特征,利用融合AM的改进SAC深度强化学习算法在连续动作空间对模型进行求解。最后,对多能流调度结果进行仿真分析和对比实验,验证了文中方法的可行性及其相较于原SAC算法较高的决策准确性。展开更多
BACKGROUND Conjoined twins are a rare twin malformation commonly presenting as single amniotic sac twinning,with double amniotic sac twinning being extremely rare and poorly reported.Most conjoined twins are females.C...BACKGROUND Conjoined twins are a rare twin malformation commonly presenting as single amniotic sac twinning,with double amniotic sac twinning being extremely rare and poorly reported.Most conjoined twins are females.CASE SUMMARY A woman of childbearing age conceived naturally,and at 8 wk of gestation,transvaginal ultrasonography showed an embryo and cardiac tube pulsation in both amniotic sacs.On dynamic observation,the two embryos were connected in the lower abdomen,with restricted movement.A repeat transvaginal ultrasound at 11 wk showed that the intestinal tubes of both fetuses were connected in the lower abdomen.The pregnancy was terminated and labor was induced.CONCLUSION Transvaginal ultrasound may detect conjoined twin malformations in an early stage.Our case provides diagnostic insights for ultrasonographers and can help develop early therapeutic interventions.展开更多
In mobile edge computing,unmanned aerial vehicles(UAVs)equipped with computing servers have emerged as a promising solution due to their exceptional attributes of high mobility,flexibility,rapid deployment,and terrain...In mobile edge computing,unmanned aerial vehicles(UAVs)equipped with computing servers have emerged as a promising solution due to their exceptional attributes of high mobility,flexibility,rapid deployment,and terrain agnosticism.These attributes enable UAVs to reach designated areas,thereby addressing temporary computing swiftly in scenarios where ground-based servers are overloaded or unavailable.However,the inherent broadcast nature of line-of-sight transmission methods employed by UAVs renders them vulnerable to eavesdropping attacks.Meanwhile,there are often obstacles that affect flight safety in real UAV operation areas,and collisions between UAVs may also occur.To solve these problems,we propose an innovative A*SAC deep reinforcement learning algorithm,which seamlessly integrates the benefits of Soft Actor-Critic(SAC)and A*(A-Star)algorithms.This algorithm jointly optimizes the hovering position and task offloading proportion of the UAV through a task offloading function.Furthermore,our algorithm incorporates a path-planning function that identifies the most energy-efficient route for the UAV to reach its optimal hovering point.This approach not only reduces the flight energy consumption of the UAV but also lowers overall energy consumption,thereby optimizing system-level energy efficiency.Extensive simulation results demonstrate that,compared to other algorithms,our approach achieves superior system benefits.Specifically,it exhibits an average improvement of 13.18%in terms of different computing task sizes,25.61%higher on average in terms of the power of electromagnetic wave interference intrusion into UAVs emitted by different auxiliary UAVs,and 35.78%higher on average in terms of the maximum computing frequency of different auxiliary UAVs.As for path planning,the simulation results indicate that our algorithm is capable of determining the optimal collision-avoidance path for each auxiliary UAV,enabling them to safely reach their designated endpoints in diverse obstacle-ridden environments.展开更多
文摘为解决由于固定温度SAC(Soft Actor Critic)算法中存在的Q函数高估可能会导致算法陷入局部最优的问题,通过深入分析提出了一个稳定且受限的SAC算法(SCSAC:Stable Constrained Soft Actor Critic)。该算法通过改进最大熵目标函数修复固定温度SAC算法中的Q函数高估问题,同时增强算法在测试过程中稳定性的效果。最后,在4个OpenAI Gym Mujoco环境下对SCSAC算法进行了验证,实验结果表明,稳定且受限的SAC算法相比固定温度SAC算法可以有效减小Q函数高估出现的次数并能在测试中获得更加稳定的结果。
文摘虚拟电厂(virtual power plant,VPP)作为多能流互联的综合能源网络,已成为中国加速实现双碳目标的重要角色。但VPP内部资源协同低碳调度面临多能流的耦合程度紧密、传统碳交易模型参数主观性强、含高维动态参数的优化目标在线求解困难等问题。针对这些问题,文中提出一种融合注意力机制(attention mechanism,AM)与柔性动作评价(soft actor-critic,SAC)算法的VPP多能流低碳调度方法。首先,根据VPP的随机碳流特性,面向动态参数建立基于贝叶斯优化的改进阶梯型碳交易机制。接着,以经济效益和碳排放量为目标函数构建含氢VPP多能流解耦模型。然后,考虑到该模型具有高维非线性与权重参数实时更新的特征,利用融合AM的改进SAC深度强化学习算法在连续动作空间对模型进行求解。最后,对多能流调度结果进行仿真分析和对比实验,验证了文中方法的可行性及其相较于原SAC算法较高的决策准确性。
文摘BACKGROUND Conjoined twins are a rare twin malformation commonly presenting as single amniotic sac twinning,with double amniotic sac twinning being extremely rare and poorly reported.Most conjoined twins are females.CASE SUMMARY A woman of childbearing age conceived naturally,and at 8 wk of gestation,transvaginal ultrasonography showed an embryo and cardiac tube pulsation in both amniotic sacs.On dynamic observation,the two embryos were connected in the lower abdomen,with restricted movement.A repeat transvaginal ultrasound at 11 wk showed that the intestinal tubes of both fetuses were connected in the lower abdomen.The pregnancy was terminated and labor was induced.CONCLUSION Transvaginal ultrasound may detect conjoined twin malformations in an early stage.Our case provides diagnostic insights for ultrasonographers and can help develop early therapeutic interventions.
基金supported by the Central University Basic Research Business Fee Fund Project(J2023-027)Open Fund of Key Laboratory of Flight Techniques and Flight Safety,CAAC(No.FZ2022KF06)China Postdoctoral Science Foundation(No.2022M722248).
文摘In mobile edge computing,unmanned aerial vehicles(UAVs)equipped with computing servers have emerged as a promising solution due to their exceptional attributes of high mobility,flexibility,rapid deployment,and terrain agnosticism.These attributes enable UAVs to reach designated areas,thereby addressing temporary computing swiftly in scenarios where ground-based servers are overloaded or unavailable.However,the inherent broadcast nature of line-of-sight transmission methods employed by UAVs renders them vulnerable to eavesdropping attacks.Meanwhile,there are often obstacles that affect flight safety in real UAV operation areas,and collisions between UAVs may also occur.To solve these problems,we propose an innovative A*SAC deep reinforcement learning algorithm,which seamlessly integrates the benefits of Soft Actor-Critic(SAC)and A*(A-Star)algorithms.This algorithm jointly optimizes the hovering position and task offloading proportion of the UAV through a task offloading function.Furthermore,our algorithm incorporates a path-planning function that identifies the most energy-efficient route for the UAV to reach its optimal hovering point.This approach not only reduces the flight energy consumption of the UAV but also lowers overall energy consumption,thereby optimizing system-level energy efficiency.Extensive simulation results demonstrate that,compared to other algorithms,our approach achieves superior system benefits.Specifically,it exhibits an average improvement of 13.18%in terms of different computing task sizes,25.61%higher on average in terms of the power of electromagnetic wave interference intrusion into UAVs emitted by different auxiliary UAVs,and 35.78%higher on average in terms of the maximum computing frequency of different auxiliary UAVs.As for path planning,the simulation results indicate that our algorithm is capable of determining the optimal collision-avoidance path for each auxiliary UAV,enabling them to safely reach their designated endpoints in diverse obstacle-ridden environments.
