Safety-Constrained Multi-Agent Reinforcement Learning for Power Quality Control in Distributed Renewable Energy Networks

This paper examines the difficulties of managing distributed power systems,notably due to the increasing use of renewable energy sources,and focuses on voltage control challenges exacerbated by their variable nature in modern power grids.To tackle the unique challenges of voltage control in distributed renewable energy networks,researchers are increasingly turning towards multi-agent reinforcement learning(MARL).However,MARL raises safety concerns due to the unpredictability in agent actions during their exploration phase.This unpredictability can lead to unsafe control measures.To mitigate these safety concerns in MARL-based voltage control,our study introduces a novel approach:Safety-ConstrainedMulti-Agent Reinforcement Learning(SC-MARL).This approach incorporates a specialized safety constraint module specifically designed for voltage control within the MARL framework.This module ensures that the MARL agents carry out voltage control actions safely.The experiments demonstrate that,in the 33-buses,141-buses,and 322-buses power systems,employing SC-MARL for voltage control resulted in a reduction of the Voltage Out of Control Rate(%V.out)from0.43,0.24,and 2.95 to 0,0.01,and 0.03,respectively.Additionally,the Reactive Power Loss(Q loss)decreased from 0.095,0.547,and 0.017 to 0.062,0.452,and 0.016 in the corresponding systems.

Real-Time Demand Response Management for Controlling Load Using Deep Reinforcement Learning

With the rapid economic growth and improved living standards,electricity has become an indispensable energy source in our lives.Therefore,the stability of the grid power supply and the conservation of electricity is critical.The following are some of the problems facing now:1)During the peak power consumption period,it will pose a threat to the power grid.Enhancing and improving the power distribution infrastructure requires high maintenance costs.2)The user’s electricity schedule is unreasonable due to personal behavior,which will cause a waste of electricity.Controlling load as a vital part of incentive demand response(DR)can achieve rapid response and improve demand-side resilience.Maintaining load by manually formulating rules,some devices are selective to be adjusted during peak power consumption.However,it is challenging to optimize methods based on manual rules.This paper uses SoftActor-Critic(SAC)as a control algorithm to optimize the control strategy.The results show that through the coordination of the SAC to control load in CityLearn,realizes the goal of reducing both the peak load demand and the operation costs on the premise of regulating voltage to the safe limit.

Abnormal State Detection in Lithium-ion Battery Using Dynamic Frequency Memory and Correlation Attention LSTM Autoencoder

This paper addresses the challenge of identifying abnormal states in Lithium-ion Battery(LiB)time series data.As the energy sector increasingly focuses on integrating distributed energy resources,Virtual Power Plants(VPP)have become a vital new framework for energy management.LiBs are key in this context,owing to their high-efficiency energy storage capabilities essential for VPP operations.However,LiBs are prone to various abnormal states like overcharging,over-discharging,and internal short circuits,which impede power transmission efficiency.Traditional methods for detecting such abnormalities in LiB are too broad and lack precision for the dynamic and irregular nature of LiB data.In response,we introduce an innovative method:a Long Short-Term Memory(LSTM)autoencoder based on Dynamic Frequency Memory and Correlation Attention(DFMCA-LSTM-AE).This unsupervised,end-to-end approach is specifically designed for dynamically monitoring abnormal states in LiB data.The method starts with a Dynamic Frequency Fourier Transform module,which dynamically captures the frequency characteristics of time series data across three scales,incorporating a memory mechanism to reduce overgeneralization of abnormal frequencies.This is followed by integrating LSTM into both the encoder and decoder,enabling the model to effectively encode and decode the temporal relationships in the time series.Empirical tests on a real-world LiB dataset demonstrate that DFMCA-LSTM-AE outperforms existing models,achieving an average Area Under the Curve(AUC)of 90.73%and an F1 score of 83.83%.These results mark significant improvements over existing models,ranging from 2.4%–45.3%for AUC and 1.6%–28.9%for F1 score,showcasing the model’s enhanced accuracy and reliability in detecting abnormal states in LiB data.

以氦-4为唯一工质的1.8 K复合制冷机及其应用验证

量子信息技术和深空探测等领域的蓬勃发展,对2 K以下温区高可靠、长寿命、小型轻量化、高制冷效率低温制冷机的需求日益迫切,高频脉冲管耦合Joule-Thomson(JT)的复合制冷循环是实现这一目标的重要手段.目前国际上以该循环获得2 K以下温区的成功实践,均是在脉冲管分系统使用氦-4而JT分系统使用氦-3作为循环工质的情况下获得的.氦-3在地球上存量稀少、价格高昂,是阻碍这一循环在更广范围内实用化的关键瓶颈.本文对以氦-4为唯一工质的四级高频脉冲管耦合JT的复合制冷循环开展了理论与实验研究,分析了基于该循环获取2 K以下温度的关键难点和可行性,从采用间隙密封的直流线性压缩机的低压压力和多级间壁式回热器的低压侧压降损失入手,理论预测出在40 kPa系统充气压力下可实现1.1 kPa的压缩机吸气压力和438.6 Pa的低压侧总压降,从而能获得1.54 kPa的饱和蒸气压,此时采用氦-4节流可实现1.78 K的制冷温度.同时,在氦-4超流态工况下,分析了小界面温差的Kapitza热导对冷头蒸发器内超流氦热传递的影响,并给出了在此基础上JT循环参数优化的限制条件.设计出的制冷机的无负荷温度经过16.5 h从300 K降至1.8 K,且在360 h连续运行时间内温度波动不超过±6 mK,验证了理论的正确性和工质在超流状态下制冷温度的稳定性.随后开展了与实际超导纳米线单光子探测器(superconducting nanowire single-photon detector,SNSPD)的耦合联试,对SNSPD器件的系统探测效率和暗计数率的实际测试表明,所研制复合制冷机在采用氦-4为唯一工质条件下,依然可以为SNSPD提供1.84 K的工作温度以及良好的电环境,使其保持稳定可靠的工作状态.上述理论和实验突破不但将为SNSPD的未来空间应用提供可靠保障,而且也将为彻底打破该类复合制冷循环在更广领域内的实用化瓶颈铺平道路.

Theoretical modeling and experimental verifications of the single-compressor-driven three-stage Stirling-type pulse tube cryocooler

This paper establishes a theoretical model of the single-compressor-driven (SCD) three-stage Stirlingtype pulse tube cryocooler (SPTC) and conducts experimental verifications. The main differences between the SCD type and the multi-compressor-driven (MCD) crycooler are analyzed, such as the distribution of the input acoustic power in each stage and the optimization of the operating parameters, in which both advantages and difficulties of the former are stressed. The effects of the dynamic temperatures are considered to improve the accuracy of the simulation at very low temperatures, and a specific simulation example aiming at 10 K is given in which quantitative analyses are provided. A SCD threestage SPTC is developed based on the theoretical analyses and with a total input acoustic power of 371.58 W, which reaches a no-load temperature of 8.82 K and can simultaneously achieve the cooling capacities of 2.4 W at 70 K, 0.17 W at 25 K, and 0.05 W at 10 K. The performance of the SCD three-stage SPTC is slightly poorer than that of its MCD counterpart developed in the same laboratory, but the advantages of lightweight and compactness make the former more attractive to practical applications.