基于softmax的加权Double Q-Learning算法

强化学习作为机器学习的一个分支,用于描述和解决智能体在与环境的交互过程中,通过学习策略以达成回报最大化的问题。Q-Learning作为无模型强化学习的经典方法,存在过估计引起的最大化偏差问题,并且在环境中奖励存在噪声时表现不佳。Double Q-Learning(DQL)的出现解决了过估计问题,但同时造成了低估问题。为解决以上算法的高低估问题,提出了基于softmax的加权Q-Learning算法,并将其与DQL相结合,提出了一种新的基于softmax的加权Double Q-Learning算法(WDQL-Softmax)。该算法基于加权双估计器的构造,对样本期望值进行softmax操作得到权重,使用权重估计动作价值,有效平衡对动作价值的高估和低估问题,使估计值更加接近理论值。实验结果表明,在离散动作空间中,相比于Q-Learning算法、DQL算法和WDQL算法,WDQL-Softmax算法的收敛速度更快且估计值与理论值的误差更小。

基于Dueling Double DQN的交通信号控制方法

为了提高交叉口通行效率缓解交通拥堵,深入挖掘交通状态信息中所包含的深层次隐含特征信息,提出了一种基于Dueling Double DQN(D3QN)的单交叉口交通信号控制方法;构建了一个基于深度强化学习Double DQN(DDQN)的交通信号控制模型,对动作-价值函数的估计值和目标值迭代运算过程进行了优化,克服基于深度强化学习DQN的交通信号控制模型存在收敛速度慢的问题;设计了一个新的Dueling Network解耦交通状态和相位动作的价值,增强Double DQN(DDQN)提取深层次特征信息的能力;基于微观仿真平台SUMO搭建了一个单交叉口模拟仿真框架和环境,开展仿真测试;仿真测试结果表明,与传统交通信号控制方法和基于深度强化学习DQN的交通信号控制方法相比,所提方法能够有效减少车辆平均等待时间、车辆平均排队长度和车辆平均停车次数,明显提升交叉口通行效率。

Clinical observation of extraction-site incisional hernia after laparoscopic colorectal surgery

BACKGROUND Laparoscopic colorectal cancer surgery increases the risk of incisional hernia(IH)at the tumor extraction site.AIM To investigate the incidence of IH at extraction sites following laparoscopic colo-rectal cancer surgery and identify the risk factors for IH incidence.METHODS This study retrospectively analyzed the data of 1614 patients who underwent la-paroscopic radical colorectal cancer surgery with tumor extraction through the abdominal wall at our center between January 2017 and December 2022.Diffe-rences in the incidence of postoperative IH at different extraction sites and the risk factors for IH incidence were investigated.RESULTS Among the 1614 patients who underwent laparoscopic radical colorectal cancer surgery,303(18.8%),923(57.2%),171(10.6%),and 217(13.4%)tumors were ex-tracted through supraumbilical midline,infraumbilical midline,umbilical,and off-midline incisions.Of these,52 patients developed IH in the abdominal wall,with an incidence of 3.2%.The incidence of postoperative IH was significantly higher in the off-midline incision group(8.8%)than in the middle incision groups[the supraumbilical midline(2.6%),infraumbilical midline(2.2%),and umbilical incision(2.9%)groups](χ^(2)=24.985;P<0.05).Univariate analysis showed that IH occurrence was associated with age,obesity,sex,chronic cough,incision infection,and combined diabetes,anemia,and hypopro-teinemia(P<0.05).Similarly,multivariate analysis showed that off-midline incision,age,sex(female),obesity,incision infection,combined chronic cough,and hypoproteinemia were independent risk factors for IH at the site of laparoscopic colorectal cancer surgery(P<0.05).CONCLUSION The incidence of postoperative IH differs between extraction sites for laparoscopic colorectal cancer surgery.The infraumbilical midline incision is associated with a lower hernia rate and is thus a suitable tumor extraction site.

Influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel

The influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel is investigated.The DarcyBrinkman model is used to characterize the fluid flow dynamics in porous materials.The analytical solutions are obtained for the unidirectional and completely developed flow.Based on a normal mode analysis,the generalized eigenvalue problem under a perturbed state is solved.The eigenvalue problem is then solved by the spectral method.Finally,the critical Rayleigh number with the corresponding wavenumber is evaluated at the assigned values of the other flow-governing parameters.The results show that increasing the Darcy number,the Lewis number,the Dufour parameter,or the Soret parameter increases the stability of the system,whereas increasing the inclination angle of the channel destabilizes the flow.Besides,the flow is the most unstable when the channel is vertically oriented.

Double contrast-enhanced ultrasonography improves diagnostic accuracy of T staging compared with multi-detector computed tomography in gastric cancer patients

BACKGROUND Gastric cancer(GC)is the most common malignant tumor and ranks third for cancer-related deaths among the worldwide.The disease poses a serious public health problem in China,ranking fifth for incidence and third for mortality.Knowledge of the invasive depth of the tumor is vital to treatment decisions.AIM To evaluate the diagnostic performance of double contrast-enhanced ultrasonography(DCEUS)for preoperative T staging in patients with GC by comparing with multi-detector computed tomography(MDCT).METHODS This single prospective study enrolled patients with GC confirmed by preoperative gastroscopy from July 2021 to March 2023.Patients underwent DCEUS,including ultrasonography(US)and intravenous contrast-enhanced ultrasonography(CEUS),and MDCT examinations for the assessment of preoperative T staging.Features of GC were identified on DCEUS and criteria developed to evaluate T staging according to the 8th edition of AJCC cancer staging manual.The diagnostic performance of DCEUS was evaluated by comparing it with that of MDCT and surgical-pathological findings were considered as the gold standard.RESULTS A total of 229 patients with GC(80 T1,33 T2,59 T3 and 57 T4)were included.Overall accuracies were 86.9%for DCEUS and 61.1%for MDCT(P<0.001).DCEUS was superior to MDCT for T1(92.5%vs 70.0%,P<0.001),T2(72.7%vs 51.5%,P=0.041),T3(86.4%vs 45.8%,P<0.001)and T4(87.7%vs 70.2%,P=0.022)staging of GC.CONCLUSION DCEUS improved the diagnostic accuracy of preoperative T staging in patients with GC compared with MDCT,and constitutes a promising imaging modality for preoperative evaluation of GC to aid individualized treatment decision-making.

Enhancing the stability of Ni Fe-layered double hydroxide nanosheet array for alkaline seawater oxidation by Ce doping

Electrocatalytic hydrogen production from seawater holds enormous promise for clean energy generation.Nevertheless,the direct electrolysis of seawater encounters significant challenges due to poor anodic stability caused by detrimental chlorine chemistry.Herein,we present our recent discovery that the incorporation of Ce into Ni Fe layered double hydroxide nanosheet array on Ni foam(Ce-Ni Fe LDH/NF)emerges as a robust electrocatalyst for seawater oxidation.During the seawater oxidation process,CeO_(2)is generated,effectively repelling Cl^(-)and inhibiting the formation of Cl O-,resulting in a notable enhancement in the oxidation activity and stability of alkaline seawater.The prepared Ce-Ni Fe LDH/NF requires only overpotential of 390 m V to achieve the current density of 1 A cm^(-2),while maintaining long-term stability for 500 h,outperforming the performance of Ni Fe LDH/NF(430 m V,150 h)by a significant margin.This study highlights the effectiveness of a Ce-doping strategy in augmenting the activity and stability of materials based on Ni Fe LDH in seawater electrolysis for oxygen evolution.

Correction to:Theoretical analysis of the double-differential cross-sections of neutron,proton,deuteron,3He,and˛for the p+6 Li reaction

Correction to:Nuclear Science and Techniques(2024)35:61 https://doi.org/10.1007/s41365-024-01421-5 In this article,the figures were wrongly numbered.The Fig.7 and 8 should have been Fig.11 and 12.The Fig.9,10,11,and 12 should have been 7,8,9 and 10.The original article has been corrected.

Ten-year observation of corneal densitometry and associated factors following small incision lenticule extraction

●AIM:To investigate the long-term changes of corneal densitometry(CD)and its contributing elements after small incision lenticule extraction(SMILE).●METHODS:Totally 31 eyes of 31 patients with mean spherical equivalent of-6.46±1.50 D and mean age 28.23±7.38y were enrolled.Full-scale examinations were conducted on all patients preoperatively and during followup.Visual acuity,manifest refraction,axial length,corneal thickness,corneal higher-order aberrations,and CD were evaluated.●RESULTS:All surgeries were completed successfully without complications or adverse events.Ten-year safety index was 1.17±0.20 and efficacy 1.04±0.28.CD value of 0–6 mm zones in central layer was statistically significantly lower 10y postoperatively,compared with preoperative values(0–2 mmΔ=-1.62,2–6 mmΔ=-1.24,P<0.01).There were no correlations between CD values and factors evaluated.●CONCLUSION:SMILE is a safe and efficient procedure for myopia on a long-term basis.CD values get lower 10y postoperatively,whose mechanism is to be further discussed.

Diagnosis of indirectly driven double shell targets with point-projection hard x-ray radiography

We present an application of short-pulse laser-generated hard x rays for the diagnosis of indirectly driven double shell targets. Coneinserted double shell targets were imploded through an indirect drive approach on the upgraded SG-II laser facility. Then, based on thepoint-projection hard x-ray radiography technique, time-resolved radiography of the double shell targets, including that of their near-peakcompression, were obtained. The backlighter source was created by the interactions of a high-intensity short pulsed laser with a metalmicrowire target. Images of the target near peak compression were obtained with an Au microwire. In addition, radiation hydrodynamicsimulations were performed, and the target evolution obtained agrees well with the experimental results. Using the radiographic images, arealdensities of the targets were evaluated.

Potential applications of single-incision laparoscopic totally preperitoneal hernioplasty

BACKGROUND The totally preperitoneal(TPP)approach is a new concept that was recently introduced.Although the TPP approach combined with single-incision laparoscopic hernia repair has its own advantages,there is little evidence reflecting the characteristics and feasibility of either approach.AIM To analyze the potential applications of single-incision laparoscopic TPP(SILTPP)inguinal hernia hernioplasty for the treatment of inguinal hernias.METHODS A total of 152 SIL-TPP surgeries were performed at the First Affiliated Hospital of Ningbo University from February 2019 to November 2022.A single-port,named Iconport,and standard laparoscopic instruments were used during the operation.Demographic data,intraoperative parameters and short-term postoperative outcomes were collected and retrospectively analyzed.RESULTS The demographic data of 152 patients underwent SIL-TPP were shown in Table 1.The average age was 49.5 years(range from 21 to 81 years).The average body mass index was 27.7 kg/m^(2)(range from 17.7 kg/m^(2) to 35.6 kg/m^(2)).SIL-TPP were conducted successfully in 147 patients.Three patients were converted to the SILtransabdominal preperitoneal laparoscopic herniorrhaphy at the initial stage of the study due to a lack of experience.In 2 patients with incisional hernias,an auxiliary operation hole was added during the SIL-TPP procedure,as required for surgery.The mean operative time was 64.5 minutes(range:36.0-110.0 minutes)for unilateral direct and femoral hernias and 81.6 minutes for indirect hernias(range:40.0-150.0 minutes).The mean postoperative hospital stay was 3.4 days.CONCLUSION SIL-TPP is feasible and has advantages for inguinal hernia repair.SIL-TPP has potential benefits for patients with various abdominal wall hernias.Consequently,doctors should be encouraged to actively apply the TPP approach combined with a single incision in their daily work.

Changes in posterior corneal elevation after small incision lenticule extraction for different myopic diopters

●AIM:To study the changes and effect factors of posterior corneal surface after small incision lenticule extraction(SMILE)with different myopic diopters.●METHODS:Ninety eyes of 90 patients who underwent SMILE were included in this retrospective study.Patients were allocated into three groups based on the preoperative spherical equivalent(SE):low myopia(SE≥-3.00 D),moderate myopia(-3.00 D>SE>-6.00 D)and high myopia(SE≤-6.00 D).Posterior corneal surfaces were measured by a Scheimpflug camera preoperatively and different postoperative times(1wk,1,3,6mo,and 1y).Posterior mean elevation(PME)at 25 predetermined points of 3 concentric circles(2-,4-,and 6-mm diameter)above the best fit sphere was analyzed.●RESULTS:All surgeries were completed uneventfully and no ectasia was found through the observation.The difference of myopia group was significant at the 2-mm ring at 1 and 3mo postoperatively(1mo:P=0.017;3mo:P=0.018).The effect of time onΔPME was statistically significant(2-mm ring:P=0.001;4-mm ring:P<0.001;6-mm ring:P<0.001).The effect of different corneal locations onΔPME was significant except 1wk postoperatively(1mo:P=0.000;3mo:P=0.000;6mo:P=0.001;1y:P=0.001).Posterior corneal stability was linearly correlated with SE,central corneal thickness,ablation depth,residual bed thickness,percent ablation depth and percent stromal bed thickness.●CONCLUSION:The posterior corneal surface changes dynamically after SMILE.No protrusion is observed on the posterior corneal surface in patients with different degrees of myopia within one year after surgery.SMILE has good stability,accuracy,safety and predictability.

Defective Nickel-Iron Layered Double Hydroxide for Enhanced Photocatalytic NO Oxidation with Significant Alleviation of NO2 Production

Photocatalysis offers a sustainable means for the oxidative removal of low concentrations of NOx(NO,NO2,N2O,N2O5,etc.)from the atmosphere.Layered double hydroxides(LDHs)are promising candidate photocatalysts owing to their unique layered and tunable chemical structures and abundant surface hydroxide(OH)moieties,which are hydroxyl radical(OH)precursors.However,the practical applications of LDHs are limited by their poor charge-separation ability and insufficient active sites.Herein,we developed a facile N_(2)H_(4)-driven etching approach to introduce dual Ni^(2+)and OHvacancies(Niv and OHv,respectively)into NiFe-LDH nanosheets(hereafter referred to as NiFe-LDH-et)to facilitate improved charge-carrier separation and active Lewis acidic site(Fe^(3+)and Ni^(2+)exposed at OHv)formation.In contrast to inert pristine LDH,NiFe-LDH-et actively removed NO under visible-light illumination.Specifically,Ni_(76)Fe_(24)-LDH-et etched with 1.50 mmol·L^(-1)N_(2)H_(4)solution removed 32.8%of the NO in continuously flowing air(NO feed concentration:500 parts per billion(ppb))under visible-light illumination,thereby outperforming most reported catalysts.Experimental and theoretical data revealed that the dual vacancies promoted the production of reactive oxygen species(O_(2)·^(-)andOH)and the adsorption of NO on the LDH.In situ spectroscopy demonstrated that NO was preferentially adsorbed at Lewis acidic sites,particularly exposed Fe^(3+)sites,converted into NO+,and subsequently oxidized to NO3without the notable formation of the more toxic intermediate NO2,thereby alleviating risks associated with its production and emission.

Young's double slit interference with vortex source

The fast and convenient demultiplex of optical vortex(OV) mode is crucial for its further application. We propose a novel approach that combines classic Young's doublet with an OV source to effectively identify the OV mode through the analysis of interference patterns. The interference patterns of the OV source incident on the double slits can be perfectly illustrated by using both the classical double-slit interference method and the Huygens–Fresnel principle. The interference fringes will twist along the negative or positive direction of x axis when topological charge(TC)l>0 or l<0, and the degree of the movement varies with the TC, allowing for a quantitative display of the OV characteristics through the interference patterns. Additionally, we deduce analytically that the zeroth-order interference fringe has a linear relationship with the TC and the vertical position. These findings highlight the ability to identify the OV mode by analyzing the interference patterns produced by Young's doublet.

Research on static characteristic of double redundance double nozzle flapper valve based on AMESim

The feedback spring rod of the armature assembly is cancelled in the double redundance double nozzle flapper valve(DRDNFV),and the difficulty of valve core displacement control is increased.Therefore,this paper intends to study the static characteristic of DRDNFV through the AMESet and AMESim simulation.It is explored under the circumstance of the fixed orifices being clogged and experimentally verified on the test bench.The results show that the pressure gain increases and the flow gain decreases with the increasing clogged degree of the fixed orifices on both sides.The zero bias increases synchronously with the increasing clogged degree of the unilateral fixed orifice.The experimental results are basically consistent with the theoretical curves and the theoretical correctness of the simulation model is effectively verified.The results can provide the theoretical reference for design,debugging,maintenance and fault diagnosis of DRDNFV.

Double quantum images encryption scheme based on chaotic system

This paper explores a double quantum images representation(DNEQR)model that allows for simultaneous storage of two digital images in a quantum superposition state.Additionally,a new type of two-dimensional hyperchaotic system based on sine and logistic maps is investigated,offering a wider parameter space and better chaotic behavior compared to the sine and logistic maps.Based on the DNEQR model and the hyperchaotic system,a double quantum images encryption algorithm is proposed.Firstly,two classical plaintext images are transformed into quantum states using the DNEQR model.Then,the proposed hyperchaotic system is employed to iteratively generate pseudo-random sequences.These chaotic sequences are utilized to perform pixel value and position operations on the quantum image,resulting in changes to both pixel values and positions.Finally,the ciphertext image can be obtained by qubit-level diffusion using two XOR operations between the position-permutated image and the pseudo-random sequences.The corresponding quantum circuits are also given.Experimental results demonstrate that the proposed scheme ensures the security of the images during transmission,improves the encryption efficiency,and enhances anti-interference and anti-attack capabilities.

Compression Properties and Energy Absorption of A Novel Double Curved Beam Negative Stiffness Honeycomb Structures

This paper presents the design of a novel honeycomb structure with a double curved beam.The purpose of this design is to achieve vibration isolation for the main engine of an offshore platform and reduce impact loads.An analytical formula for the force-displacement relationship of the honeycomb single-cell structure is presented based on the modal superposition method.This formula provides a theoretical basis for predicting the compression performance of honeycomb structures.The effects of structural geometric parameters,series and parallel connection methods on the mechanical and energy absorption properties are investigated through mathematical modeling and experimental methods.Furthermore,the study focuses on the vibration isolation and impact resistance performance of honeycomb panels.The results show that the designed honeycomb structure has good mechanical and energy absorption performance,and its energy absorption effect is related to the geometric parameters and series and parallel connection methods of the structure.The isolation efficiency of the honeycomb with 4 rows and 3 columns reaches 38%.The initial isolation frequency of the isolator is 11.7 Hz.

基于Double DQN的双模式多目标信号配时方法

近年来深度强化学习作为一种高效可靠的机器学习方法被广泛应用在交通信号控制领域。目前,现有交通信号配时方法通常忽略了特殊车辆(例如救护车、消防车等)的优先通行;此外,基于传统深度强化学习的信号配时方法优化目标较为单一,导致其在复杂交通场景中性能不佳。针对上述问题,基于Double DQN提出一种融合特殊车辆优先通行的双模式多目标信号配时方法(Dual-mode Multi-objective signal timing method based on Double DQN,DMDD),以提高不同交通场景下路口的通行效率。该方法首先基于路口的饱和状态选择信号控制模式,特殊车辆在紧急控制模式下被赋予更高的通行权重,有利于其更快通过路口;接着针对等待时长、队列长度和CO 2排放量3个指标分别设计神经网络进行奖励计算;最后利用Double DQN进行最优信号相位的选择,通过灵活切换信号相位以提升通行效率。基于SUMO的实验结果表明,DMDD与对比方法相比能有效缩短路口处特殊车辆的等待时长、队列长度和CO 2排放量,特殊车辆能够更快通过路口,有效地提高了通行效率。

On power series statistical convergence and new uniform integrability of double sequences

In the present paper,we mostly focus on P_(p)^(2)-statistical convergence.We will look into the uniform integrability via the power series method and its characterizations for double sequences.Also,the notions of P_(p)^(2)-statistically Cauchy sequence,P_(p)^(2)-statistical boundedness and core for double sequences will be described in addition to these findings.

Effect of Single- versus Double-Layer Uterine Closure during Caesarean Section on Niche Formation and Menstrual Irregularity

Background: The myometrium at the location of the CS (caesarean section) scars, also known as residual myometrium thickness (RMT), is larger after a double-layer uterine closure procedure than following a single-layer one. It may lessen the formation of a niche that is the myometrium’s disruption at the location of the scar of the uterus. Gynecological manifestations, obstetric problems in a future pregnancy and birth, and maybe subfertility are linked to thin RMT and a niche. Objective: To ascertain if double-layer unlocked closure of the uterus is better than single-layer one in terms of post-menstrual spotting and niche development following a first CS. Patients and Methods: In this randomized clinical study, 287 patients were evaluated for qualifying. Of all eligible individuals, 57 patients were excluded from the study based on the inclusion criteria. Results: The variation in ages, gestational age, body mass index (BMI), and cesarean section indications between the two assigned groups is statistically insignificant. However, postmenstrual spotting was statistically significantly more common in single-layer group compared to in double-group. The current study revealed ultrasound findings suggestive of niche formation was statistically significantly more common in single-layer group compared to in double-layer group. Conclusion: As evident from the current study, it demonstrates the advantages of double-layer unlocked closure of the uterus over single-layer one in terms of post-menstrual spotting and niche development following first-time cs. Thus, we deduced that fewer niches are formed, and fewer menstrual spotting occurs in the presence of double unlocked layers closure. To ascertain the impact of uterus closure method on post-operative niche development and the risk of obstetrics and gynaecological problems, further prospective trials with extended follow-up periods are required.

Micro-power negative pressure wound technique reduces risk of incision infection following loop ileostomy closure

BACKGROUND Prophylactic loop ileostomy is an effective way to reduce the clinical severity of anastomotic leakage following radical resection of rectal cancer.Incisional surgical site infection(SSI)is a common complication after ileostomy closure.AIM To evaluate the efficacy and safety of the micro-power negative pressure wound technique(MPNPWT)in preventing incisional SSI.METHODS This was a prospective,randomized controlled clinical trial conducted at a single center.A total of 101 consecutive patients who underwent ileostomy closure after rectal cancer surgery with a prophylactic ileostomy were enrolled from January 2019 to December 2021.Patients were randomly allocated into an MPNPWT group and a control group.The MPNPWT group underwent intermittent suturing of the surgical incision with 2-0 Prolene and was covered with a micro-power negative pressure dressing.The surgical outcomes were compared between the MPNPWT(n=50)and control(n=51)groups.Risk factors for incisional SSI were identified using logistic regression.RESULTS There were no differences in baseline characteristics between the MPNPWT(n=50)and control groups(n=51).The incisional SSI rate was significantly higher in the control group than in the MPNPWT group(15.7%vs 2.0%,P=0.031).However,MPNPWT did not affect other surgical outcomes,including intra-abdominal complications,operative time,and blood loss.Postoperative hospital stay length and hospitalization costs did not differ significantly between the two groups(P=0.069 and 0.843,respectively).None of the patients experienced adverse effects of MPNPWT,including skin allergy,dermatitis,and pain.MPNPWT also helped heal the infected incision.Our study indicated that MPNPWT was an independent protective factor[odds ratio(OR)=0.005,P=0.025)]and diabetes was a risk factor(OR=26.575,P=0.029)for incisional SSI.CONCLUSION MPNPWT is an effective and safe way to prevent incisional SSI after loop ileostomy closure.