基于softmax的加权Double Q-Learning算法

强化学习作为机器学习的一个分支,用于描述和解决智能体在与环境的交互过程中,通过学习策略以达成回报最大化的问题。Q-Learning作为无模型强化学习的经典方法,存在过估计引起的最大化偏差问题,并且在环境中奖励存在噪声时表现不佳。Double Q-Learning(DQL)的出现解决了过估计问题,但同时造成了低估问题。为解决以上算法的高低估问题,提出了基于softmax的加权Q-Learning算法,并将其与DQL相结合,提出了一种新的基于softmax的加权Double Q-Learning算法(WDQL-Softmax)。该算法基于加权双估计器的构造,对样本期望值进行softmax操作得到权重,使用权重估计动作价值,有效平衡对动作价值的高估和低估问题,使估计值更加接近理论值。实验结果表明,在离散动作空间中,相比于Q-Learning算法、DQL算法和WDQL算法,WDQL-Softmax算法的收敛速度更快且估计值与理论值的误差更小。

基于Dueling Double DQN的交通信号控制方法

为了提高交叉口通行效率缓解交通拥堵,深入挖掘交通状态信息中所包含的深层次隐含特征信息,提出了一种基于Dueling Double DQN(D3QN)的单交叉口交通信号控制方法;构建了一个基于深度强化学习Double DQN(DDQN)的交通信号控制模型,对动作-价值函数的估计值和目标值迭代运算过程进行了优化,克服基于深度强化学习DQN的交通信号控制模型存在收敛速度慢的问题;设计了一个新的Dueling Network解耦交通状态和相位动作的价值,增强Double DQN(DDQN)提取深层次特征信息的能力;基于微观仿真平台SUMO搭建了一个单交叉口模拟仿真框架和环境,开展仿真测试;仿真测试结果表明,与传统交通信号控制方法和基于深度强化学习DQN的交通信号控制方法相比,所提方法能够有效减少车辆平均等待时间、车辆平均排队长度和车辆平均停车次数,明显提升交叉口通行效率。

Influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel

The influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel is investigated.The DarcyBrinkman model is used to characterize the fluid flow dynamics in porous materials.The analytical solutions are obtained for the unidirectional and completely developed flow.Based on a normal mode analysis,the generalized eigenvalue problem under a perturbed state is solved.The eigenvalue problem is then solved by the spectral method.Finally,the critical Rayleigh number with the corresponding wavenumber is evaluated at the assigned values of the other flow-governing parameters.The results show that increasing the Darcy number,the Lewis number,the Dufour parameter,or the Soret parameter increases the stability of the system,whereas increasing the inclination angle of the channel destabilizes the flow.Besides,the flow is the most unstable when the channel is vertically oriented.

Double contrast-enhanced ultrasonography improves diagnostic accuracy of T staging compared with multi-detector computed tomography in gastric cancer patients

BACKGROUND Gastric cancer(GC)is the most common malignant tumor and ranks third for cancer-related deaths among the worldwide.The disease poses a serious public health problem in China,ranking fifth for incidence and third for mortality.Knowledge of the invasive depth of the tumor is vital to treatment decisions.AIM To evaluate the diagnostic performance of double contrast-enhanced ultrasonography(DCEUS)for preoperative T staging in patients with GC by comparing with multi-detector computed tomography(MDCT).METHODS This single prospective study enrolled patients with GC confirmed by preoperative gastroscopy from July 2021 to March 2023.Patients underwent DCEUS,including ultrasonography(US)and intravenous contrast-enhanced ultrasonography(CEUS),and MDCT examinations for the assessment of preoperative T staging.Features of GC were identified on DCEUS and criteria developed to evaluate T staging according to the 8th edition of AJCC cancer staging manual.The diagnostic performance of DCEUS was evaluated by comparing it with that of MDCT and surgical-pathological findings were considered as the gold standard.RESULTS A total of 229 patients with GC(80 T1,33 T2,59 T3 and 57 T4)were included.Overall accuracies were 86.9%for DCEUS and 61.1%for MDCT(P<0.001).DCEUS was superior to MDCT for T1(92.5%vs 70.0%,P<0.001),T2(72.7%vs 51.5%,P=0.041),T3(86.4%vs 45.8%,P<0.001)and T4(87.7%vs 70.2%,P=0.022)staging of GC.CONCLUSION DCEUS improved the diagnostic accuracy of preoperative T staging in patients with GC compared with MDCT,and constitutes a promising imaging modality for preoperative evaluation of GC to aid individualized treatment decision-making.

Enhancing the stability of Ni Fe-layered double hydroxide nanosheet array for alkaline seawater oxidation by Ce doping

Electrocatalytic hydrogen production from seawater holds enormous promise for clean energy generation.Nevertheless,the direct electrolysis of seawater encounters significant challenges due to poor anodic stability caused by detrimental chlorine chemistry.Herein,we present our recent discovery that the incorporation of Ce into Ni Fe layered double hydroxide nanosheet array on Ni foam(Ce-Ni Fe LDH/NF)emerges as a robust electrocatalyst for seawater oxidation.During the seawater oxidation process,CeO_(2)is generated,effectively repelling Cl^(-)and inhibiting the formation of Cl O-,resulting in a notable enhancement in the oxidation activity and stability of alkaline seawater.The prepared Ce-Ni Fe LDH/NF requires only overpotential of 390 m V to achieve the current density of 1 A cm^(-2),while maintaining long-term stability for 500 h,outperforming the performance of Ni Fe LDH/NF(430 m V,150 h)by a significant margin.This study highlights the effectiveness of a Ce-doping strategy in augmenting the activity and stability of materials based on Ni Fe LDH in seawater electrolysis for oxygen evolution.

Diagnosis of indirectly driven double shell targets with point-projection hard x-ray radiography

We present an application of short-pulse laser-generated hard x rays for the diagnosis of indirectly driven double shell targets. Coneinserted double shell targets were imploded through an indirect drive approach on the upgraded SG-II laser facility. Then, based on thepoint-projection hard x-ray radiography technique, time-resolved radiography of the double shell targets, including that of their near-peakcompression, were obtained. The backlighter source was created by the interactions of a high-intensity short pulsed laser with a metalmicrowire target. Images of the target near peak compression were obtained with an Au microwire. In addition, radiation hydrodynamicsimulations were performed, and the target evolution obtained agrees well with the experimental results. Using the radiographic images, arealdensities of the targets were evaluated.

Defective Nickel-Iron Layered Double Hydroxide for Enhanced Photocatalytic NO Oxidation with Significant Alleviation of NO2 Production

Photocatalysis offers a sustainable means for the oxidative removal of low concentrations of NOx(NO,NO2,N2O,N2O5,etc.)from the atmosphere.Layered double hydroxides(LDHs)are promising candidate photocatalysts owing to their unique layered and tunable chemical structures and abundant surface hydroxide(OH)moieties,which are hydroxyl radical(OH)precursors.However,the practical applications of LDHs are limited by their poor charge-separation ability and insufficient active sites.Herein,we developed a facile N_(2)H_(4)-driven etching approach to introduce dual Ni^(2+)and OHvacancies(Niv and OHv,respectively)into NiFe-LDH nanosheets(hereafter referred to as NiFe-LDH-et)to facilitate improved charge-carrier separation and active Lewis acidic site(Fe^(3+)and Ni^(2+)exposed at OHv)formation.In contrast to inert pristine LDH,NiFe-LDH-et actively removed NO under visible-light illumination.Specifically,Ni_(76)Fe_(24)-LDH-et etched with 1.50 mmol·L^(-1)N_(2)H_(4)solution removed 32.8%of the NO in continuously flowing air(NO feed concentration:500 parts per billion(ppb))under visible-light illumination,thereby outperforming most reported catalysts.Experimental and theoretical data revealed that the dual vacancies promoted the production of reactive oxygen species(O_(2)·^(-)andOH)and the adsorption of NO on the LDH.In situ spectroscopy demonstrated that NO was preferentially adsorbed at Lewis acidic sites,particularly exposed Fe^(3+)sites,converted into NO+,and subsequently oxidized to NO3without the notable formation of the more toxic intermediate NO2,thereby alleviating risks associated with its production and emission.

Young's double slit interference with vortex source

The fast and convenient demultiplex of optical vortex(OV) mode is crucial for its further application. We propose a novel approach that combines classic Young's doublet with an OV source to effectively identify the OV mode through the analysis of interference patterns. The interference patterns of the OV source incident on the double slits can be perfectly illustrated by using both the classical double-slit interference method and the Huygens–Fresnel principle. The interference fringes will twist along the negative or positive direction of x axis when topological charge(TC)l>0 or l<0, and the degree of the movement varies with the TC, allowing for a quantitative display of the OV characteristics through the interference patterns. Additionally, we deduce analytically that the zeroth-order interference fringe has a linear relationship with the TC and the vertical position. These findings highlight the ability to identify the OV mode by analyzing the interference patterns produced by Young's doublet.

Research on static characteristic of double redundance double nozzle flapper valve based on AMESim

The feedback spring rod of the armature assembly is cancelled in the double redundance double nozzle flapper valve(DRDNFV),and the difficulty of valve core displacement control is increased.Therefore,this paper intends to study the static characteristic of DRDNFV through the AMESet and AMESim simulation.It is explored under the circumstance of the fixed orifices being clogged and experimentally verified on the test bench.The results show that the pressure gain increases and the flow gain decreases with the increasing clogged degree of the fixed orifices on both sides.The zero bias increases synchronously with the increasing clogged degree of the unilateral fixed orifice.The experimental results are basically consistent with the theoretical curves and the theoretical correctness of the simulation model is effectively verified.The results can provide the theoretical reference for design,debugging,maintenance and fault diagnosis of DRDNFV.

Double quantum images encryption scheme based on chaotic system

This paper explores a double quantum images representation(DNEQR)model that allows for simultaneous storage of two digital images in a quantum superposition state.Additionally,a new type of two-dimensional hyperchaotic system based on sine and logistic maps is investigated,offering a wider parameter space and better chaotic behavior compared to the sine and logistic maps.Based on the DNEQR model and the hyperchaotic system,a double quantum images encryption algorithm is proposed.Firstly,two classical plaintext images are transformed into quantum states using the DNEQR model.Then,the proposed hyperchaotic system is employed to iteratively generate pseudo-random sequences.These chaotic sequences are utilized to perform pixel value and position operations on the quantum image,resulting in changes to both pixel values and positions.Finally,the ciphertext image can be obtained by qubit-level diffusion using two XOR operations between the position-permutated image and the pseudo-random sequences.The corresponding quantum circuits are also given.Experimental results demonstrate that the proposed scheme ensures the security of the images during transmission,improves the encryption efficiency,and enhances anti-interference and anti-attack capabilities.

基于Double DQN的双模式多目标信号配时方法

近年来深度强化学习作为一种高效可靠的机器学习方法被广泛应用在交通信号控制领域。目前,现有交通信号配时方法通常忽略了特殊车辆(例如救护车、消防车等)的优先通行;此外,基于传统深度强化学习的信号配时方法优化目标较为单一,导致其在复杂交通场景中性能不佳。针对上述问题,基于Double DQN提出一种融合特殊车辆优先通行的双模式多目标信号配时方法(Dual-mode Multi-objective signal timing method based on Double DQN,DMDD),以提高不同交通场景下路口的通行效率。该方法首先基于路口的饱和状态选择信号控制模式,特殊车辆在紧急控制模式下被赋予更高的通行权重,有利于其更快通过路口;接着针对等待时长、队列长度和CO 2排放量3个指标分别设计神经网络进行奖励计算;最后利用Double DQN进行最优信号相位的选择,通过灵活切换信号相位以提升通行效率。基于SUMO的实验结果表明,DMDD与对比方法相比能有效缩短路口处特殊车辆的等待时长、队列长度和CO 2排放量,特殊车辆能够更快通过路口,有效地提高了通行效率。

On power series statistical convergence and new uniform integrability of double sequences

In the present paper,we mostly focus on P_(p)^(2)-statistical convergence.We will look into the uniform integrability via the power series method and its characterizations for double sequences.Also,the notions of P_(p)^(2)-statistically Cauchy sequence,P_(p)^(2)-statistical boundedness and core for double sequences will be described in addition to these findings.

Effect of Single- versus Double-Layer Uterine Closure during Caesarean Section on Niche Formation and Menstrual Irregularity

Background: The myometrium at the location of the CS (caesarean section) scars, also known as residual myometrium thickness (RMT), is larger after a double-layer uterine closure procedure than following a single-layer one. It may lessen the formation of a niche that is the myometrium’s disruption at the location of the scar of the uterus. Gynecological manifestations, obstetric problems in a future pregnancy and birth, and maybe subfertility are linked to thin RMT and a niche. Objective: To ascertain if double-layer unlocked closure of the uterus is better than single-layer one in terms of post-menstrual spotting and niche development following a first CS. Patients and Methods: In this randomized clinical study, 287 patients were evaluated for qualifying. Of all eligible individuals, 57 patients were excluded from the study based on the inclusion criteria. Results: The variation in ages, gestational age, body mass index (BMI), and cesarean section indications between the two assigned groups is statistically insignificant. However, postmenstrual spotting was statistically significantly more common in single-layer group compared to in double-group. The current study revealed ultrasound findings suggestive of niche formation was statistically significantly more common in single-layer group compared to in double-layer group. Conclusion: As evident from the current study, it demonstrates the advantages of double-layer unlocked closure of the uterus over single-layer one in terms of post-menstrual spotting and niche development following first-time cs. Thus, we deduced that fewer niches are formed, and fewer menstrual spotting occurs in the presence of double unlocked layers closure. To ascertain the impact of uterus closure method on post-operative niche development and the risk of obstetrics and gynaecological problems, further prospective trials with extended follow-up periods are required.

Fluorescent Double Network Hydrogels with Ionic Responsiveness and High Mechanical Properties for Visual Detection

We developed a fluorescent double network hydrogel with ionic responsiveness and high mechanical properties for visual detection.The nanocomposite hydrogel of laponite and polyacrylamide serves as the first network,while the ionic cross-linked hydrogel of terbium ions and sodium alginate serves as the second network.The double-network structure,the introduction of nanoparticles and the reversible ionic crosslinked interactions confer high mechanical properties to the hydrogel.Terbium ions are not only used as the ionic cross-linked points,but also used as green emitters to endow hydrogels with fluorescent properties.On the basis of the “antenna effect” of terbium ions and the ion exchange interaction,the fluorescence of the hydrogels can make selective responses to various ions(such as organic acid radical ions,transition metal ions) in aqueous solutions,which enables a convenient strategy for visual detection toward ions.Consequently,the fluorescent double network hydrogel fabricated in this study is promising for use in the field of visual sensor detection.

Internal collision double ionization of molecules driven by co-rotating two-color circularly polarized laser pulses

The double ionization process of molecules driven by co-rotating two-color circularly polarized fields is investigated with a three-dimensional classical ensemble model. Numerical results indicate that a considerable part of the sequential double ionization(DI) events of molecules occur through internal collision double ionization(ICD), and the ICD recollision mechanism is significantly different from that in non-sequential double ionization(NSDI). By analyzing the results of internuclear distances R = 5 a.u. and 2 a.u., these two recollision mechanisms are studied in depth. It is found that the dynamic behaviors of the recollision mechanisms of NSDI and ICD are similar. For NSDI, the motion range of electrons after the ionization is relatively large, and the electrons will return to the core after a period of time. In the ICD process,electrons will rotate around the parent ion before ionization, and the distance of the electron motion is relatively small. After a period of time, the electrons will come back to the core and collide with another electron. Furthermore, the molecular internuclear distance has a significant effect on the electron dynamic behavior of the two ionization mechanisms. This study will help to understand the multi-electron ionization process of complex systems.

Transformer-Aided Deep Double Dueling Spatial-Temporal Q-Network for Spatial Crowdsourcing Analysis

With the rapid development ofmobile Internet,spatial crowdsourcing has becomemore andmore popular.Spatial crowdsourcing consists of many different types of applications,such as spatial crowd-sensing services.In terms of spatial crowd-sensing,it collects and analyzes traffic sensing data from clients like vehicles and traffic lights to construct intelligent traffic prediction models.Besides collecting sensing data,spatial crowdsourcing also includes spatial delivery services like DiDi and Uber.Appropriate task assignment and worker selection dominate the service quality for spatial crowdsourcing applications.Previous research conducted task assignments via traditional matching approaches or using simple network models.However,advanced mining methods are lacking to explore the relationship between workers,task publishers,and the spatio-temporal attributes in tasks.Therefore,in this paper,we propose a Deep Double Dueling Spatial-temporal Q Network(D3SQN)to adaptively learn the spatialtemporal relationship between task,task publishers,and workers in a dynamic environment to achieve optimal allocation.Specifically,D3SQNis revised through reinforcement learning by adding a spatial-temporal transformer that can estimate the expected state values and action advantages so as to improve the accuracy of task assignments.Extensive experiments are conducted over real data collected fromDiDi and ELM,and the simulation results verify the effectiveness of our proposed models.

Transcriptome and morphological analyses of double flower formation in Dianthus chinensis

The double flower developmental process is regulated via a complex transcriptional regulatory network.To understand this highly dynamic and complex developmental process of Dianthus spp.,we performed a comparative analysis of floral morphology and transcriptome dynamics in simple flowers and double flowers.We found that the primordium of double flowers of‘X’was larger in size compared to that of simple flowers of‘L’in Dianthus chinensis.RNA-seq and Weighted Gene Co-expression Network Analysis(WGCNA)during flower development,identified stage-specific gene network modules.Expression analysis by RNA-seq indicated that a group of genes related to floral meristem identity,primordia position and polarity were highly expressed in double flowers genotypes compared to simple flowers genotypes,suggesting their roles in double-petal formation.A total of 21 DEGs related to petal number were identified between simple and double flowers.The experiments of in situ hybridization revealed that DcaAP2L,DcaLFY and DcaUFO genes were expressed in the intra-sepal boundary and petal boundary.We proposed a potential transcriptional regulatory network for simple and double flower development.This study provides novel insights into the molecular mechanism underlying double flower formation in Dianthus spp.

The microbial community,nutrient supply and crop yields differ along a potassium fertilizer gradient under wheat-maize double-cropping systems

Soil microorganisms play critical roles in ecosystem function.However,the relative impact of the potassium(K)fertilizer gradient on the microbial community in wheat-maize double-cropping systems remains unclear.In this long-term field experiment(2008-2019),we researched bacterial and fungal diversity,composition,and community assemblage in the soil along a K fertilizer gradient in the wheat season(K0,no K fertilizer;K1,45 kg ha^(-1) K_(2)O;K_(2),90 kg ha^(-1)K_(2)O;K3,135 kg ha^(-1)K_(2)O)and in the maize season(K0,no K fertilizer;K_(1),150 kg ha^(-1) K_(2)O;K_(2),300 kg ha^(-1)K_(2)O;K_(3),450 kg ha^(-1)K_(2)O)using bacterial 16S rRNA and fungal internally transcribed spacer(ITS)data.We observed that environmental variables,such as mean annual soil temperature(MAT)and precipitation,available K,ammonium,nitrate,and organic matter,impacted the soil bacterial and fungal communities,and their impacts varied with fertilizer treatments and crop species.Furthermore,the relative abundance of bacteria involved in soil nutrient transformation(phylum Actinobacteria and class Alphaproteobacteria)in the wheat season was significantly increased compared to the maize season,and the optimal K fertilizer dosage(K2 treatment)boosted the relative bacterial abundance of soil nutrient transformation(genus Lactobacillus)and soil denitrification(phylum Proteobacteria)bacteria in the wheat season.The abundance of the soil bacterial community promoting root growth and nutrient absorption(genus Herbaspirillum)in the maize season was improved compared to the wheat season,and the K2 treatment enhanced the bacterial abundance of soil nutrient transformation(genus MND1)and soil nitrogen cycling(genus Nitrospira)genera in the maize season.The results indicated that the bacterial and fungal communities in the double-cropping system exhibited variable sensitivities and assembly mechanisms along a K fertilizer gradient,and microhabitats explained the largest amount of the variation in crop yields,and improved wheat?maize yields by 11.2-22.6 and 9.2-23.8%with K addition,respectively.These modes are shaped contemporaneously by the different meteorological factors and soil nutrient changes in the K fertilizer gradients.

Supramolecular polymer-based gel fracturing fluid with a double network applied in ultra-deep hydraulic fracturing

A gel based on polyacrylamide,exhibiting delayed crosslinking characteristics,emerges as the preferred solution for mitigating degradation under conditions of high temperature and extended shear in ultralong wellbores.High viscosity/viscoelasticity of the fracturing fluid was required to maintain excellent proppant suspension properties before gelling.Taking into account both the cost and the potential damage to reservoirs,polymers with lower concentrations and molecular weights are generally preferred.In this work,the supramolecular action was integrated into the polymer,resulting in significant increases in the viscosity and viscoelasticity of the synthesized supramolecular polymer system.The double network gel,which is formed by the combination of the supramolecular polymer system and a small quantity of Zr-crosslinker,effectively resists temperature while minimizing permeability damage to the reservoir.The results indicate that the supramolecular polymer system with a molecular weight of(268—380)×10^(4)g/mol can achieve the same viscosity and viscoelasticity at 0.4 wt%due to the supramolecular interaction between polymers,compared to the 0.6 wt%traditional polymer(hydrolyzed polyacrylamide,molecular weight of 1078×10^(4)g/mol).The supramolecular polymer system possessed excellent proppant suspension properties with a 0.55 cm/min sedimentation rate at 0.4 wt%,whereas the0.6 wt%traditional polymer had a rate of 0.57 cm/min.In comparison to the traditional gel with a Zrcrosslinker concentration of 0.6 wt%and an elastic modulus of 7.77 Pa,the double network gel with a higher elastic modulus(9.00 Pa)could be formed only at 0.1 wt%Zr-crosslinker,which greatly reduced the amount of residue of the fluid after gel-breaking.The viscosity of the double network gel was66 m Pa s after 2 h shearing,whereas the traditional gel only reached 27 m Pa s.

Manipulating the electron dynamics in the non-sequential double ionization process of Ar atoms by an orthogonal two-color laser field

Electron dynamics during non-sequential double ionization(NSDI) is one of the most attractive areas of research in the field of laser–atom or laser–molecule interaction. Based on the classic two-dimensional model, we study the process of NSDI of argon atoms driven by a few-cycle orthogonal two-color laser field composed of 800 nm and 400 nm laser pulses. By changing the relative phase of the two laser pulses, a localized enhancement of NSDI yield is observed at 0.5πand 1.5π, which could be attributed to a rapid and substantial increase in the number of electrons returning to the parent ion within extremely short time intervals at these specific phases. Through the analysis of the electron–electron momentum correlations within different time windows of NSDI events and the angular distributions of emitted electrons in different channels, we observe a more pronounced electron–electron correlation phenomenon in the recollision-induced ionization(RII) channel. This is attributed to the shorter delay time in the RII channel.