基于softmax的加权Double Q-Learning算法

强化学习作为机器学习的一个分支,用于描述和解决智能体在与环境的交互过程中,通过学习策略以达成回报最大化的问题。Q-Learning作为无模型强化学习的经典方法,存在过估计引起的最大化偏差问题,并且在环境中奖励存在噪声时表现不佳。Double Q-Learning(DQL)的出现解决了过估计问题,但同时造成了低估问题。为解决以上算法的高低估问题,提出了基于softmax的加权Q-Learning算法,并将其与DQL相结合,提出了一种新的基于softmax的加权Double Q-Learning算法(WDQL-Softmax)。该算法基于加权双估计器的构造,对样本期望值进行softmax操作得到权重,使用权重估计动作价值,有效平衡对动作价值的高估和低估问题,使估计值更加接近理论值。实验结果表明,在离散动作空间中,相比于Q-Learning算法、DQL算法和WDQL算法,WDQL-Softmax算法的收敛速度更快且估计值与理论值的误差更小。

基于Dueling Double DQN的交通信号控制方法

为了提高交叉口通行效率缓解交通拥堵,深入挖掘交通状态信息中所包含的深层次隐含特征信息,提出了一种基于Dueling Double DQN(D3QN)的单交叉口交通信号控制方法;构建了一个基于深度强化学习Double DQN(DDQN)的交通信号控制模型,对动作-价值函数的估计值和目标值迭代运算过程进行了优化,克服基于深度强化学习DQN的交通信号控制模型存在收敛速度慢的问题;设计了一个新的Dueling Network解耦交通状态和相位动作的价值,增强Double DQN(DDQN)提取深层次特征信息的能力;基于微观仿真平台SUMO搭建了一个单交叉口模拟仿真框架和环境,开展仿真测试;仿真测试结果表明,与传统交通信号控制方法和基于深度强化学习DQN的交通信号控制方法相比,所提方法能够有效减少车辆平均等待时间、车辆平均排队长度和车辆平均停车次数,明显提升交叉口通行效率。

Unsupervised Time Series Segmentation: A Survey on Recent Advances

Time series segmentation has attracted more interests in recent years,which aims to segment time series into different segments,each reflects a state of the monitored objects.Although there have been many surveys on time series segmentation,most of them focus more on change point detection(CPD)methods and overlook the advances in boundary detection(BD)and state detection(SD)methods.In this paper,we categorize time series segmentation methods into CPD,BD,and SD methods,with a specific focus on recent advances in BD and SD methods.Within the scope of BD and SD,we subdivide the methods based on their underlying models/techniques and focus on the milestones that have shaped the development trajectory of each category.As a conclusion,we found that:(1)Existing methods failed to provide sufficient support for online working,with only a few methods supporting online deployment;(2)Most existing methods require the specification of parameters,which hinders their ability to work adaptively;(3)Existing SD methods do not attach importance to accurate detection of boundary points in evaluation,which may lead to limitations in boundary point detection.We highlight the ability to working online and adaptively as important attributes of segmentation methods,the boundary detection accuracy as a neglected metrics for SD methods.

Diagnosis of indirectly driven double shell targets with point-projection hard x-ray radiography

We present an application of short-pulse laser-generated hard x rays for the diagnosis of indirectly driven double shell targets. Coneinserted double shell targets were imploded through an indirect drive approach on the upgraded SG-II laser facility. Then, based on thepoint-projection hard x-ray radiography technique, time-resolved radiography of the double shell targets, including that of their near-peakcompression, were obtained. The backlighter source was created by the interactions of a high-intensity short pulsed laser with a metalmicrowire target. Images of the target near peak compression were obtained with an Au microwire. In addition, radiation hydrodynamicsimulations were performed, and the target evolution obtained agrees well with the experimental results. Using the radiographic images, arealdensities of the targets were evaluated.

Defective Nickel-Iron Layered Double Hydroxide for Enhanced Photocatalytic NO Oxidation with Significant Alleviation of NO2 Production

Photocatalysis offers a sustainable means for the oxidative removal of low concentrations of NOx(NO,NO2,N2O,N2O5,etc.)from the atmosphere.Layered double hydroxides(LDHs)are promising candidate photocatalysts owing to their unique layered and tunable chemical structures and abundant surface hydroxide(OH)moieties,which are hydroxyl radical(OH)precursors.However,the practical applications of LDHs are limited by their poor charge-separation ability and insufficient active sites.Herein,we developed a facile N_(2)H_(4)-driven etching approach to introduce dual Ni^(2+)and OHvacancies(Niv and OHv,respectively)into NiFe-LDH nanosheets(hereafter referred to as NiFe-LDH-et)to facilitate improved charge-carrier separation and active Lewis acidic site(Fe^(3+)and Ni^(2+)exposed at OHv)formation.In contrast to inert pristine LDH,NiFe-LDH-et actively removed NO under visible-light illumination.Specifically,Ni_(76)Fe_(24)-LDH-et etched with 1.50 mmol·L^(-1)N_(2)H_(4)solution removed 32.8%of the NO in continuously flowing air(NO feed concentration:500 parts per billion(ppb))under visible-light illumination,thereby outperforming most reported catalysts.Experimental and theoretical data revealed that the dual vacancies promoted the production of reactive oxygen species(O_(2)·^(-)andOH)and the adsorption of NO on the LDH.In situ spectroscopy demonstrated that NO was preferentially adsorbed at Lewis acidic sites,particularly exposed Fe^(3+)sites,converted into NO+,and subsequently oxidized to NO3without the notable formation of the more toxic intermediate NO2,thereby alleviating risks associated with its production and emission.

Young's double slit interference with vortex source

The fast and convenient demultiplex of optical vortex(OV) mode is crucial for its further application. We propose a novel approach that combines classic Young's doublet with an OV source to effectively identify the OV mode through the analysis of interference patterns. The interference patterns of the OV source incident on the double slits can be perfectly illustrated by using both the classical double-slit interference method and the Huygens–Fresnel principle. The interference fringes will twist along the negative or positive direction of x axis when topological charge(TC)l>0 or l<0, and the degree of the movement varies with the TC, allowing for a quantitative display of the OV characteristics through the interference patterns. Additionally, we deduce analytically that the zeroth-order interference fringe has a linear relationship with the TC and the vertical position. These findings highlight the ability to identify the OV mode by analyzing the interference patterns produced by Young's doublet.

An Innovative Deep Architecture for Flight Safety Risk Assessment Based on Time Series Data

With the development of the integration of aviation safety and artificial intelligence,research on the combination of risk assessment and artificial intelligence is particularly important in the field of risk management,but searching for an efficient and accurate risk assessment algorithm has become a challenge for the civil aviation industry.Therefore,an improved risk assessment algorithm(PS-AE-LSTM)based on long short-term memory network(LSTM)with autoencoder(AE)is proposed for the various supervised deep learning algorithms in flight safety that cannot adequately address the problem of the quality on risk level labels.Firstly,based on the normal distribution characteristics of flight data,a probability severity(PS)model is established to enhance the quality of risk assessment labels.Secondly,autoencoder is introduced to reconstruct the flight parameter data to improve the data quality.Finally,utilizing the time-series nature of flight data,a long and short-termmemory network is used to classify the risk level and improve the accuracy of risk assessment.Thus,a risk assessment experimentwas conducted to analyze a fleet landing phase dataset using the PS-AE-LSTMalgorithm to assess the risk level associated with aircraft hard landing events.The results show that the proposed algorithm achieves an accuracy of 86.45%compared with seven baseline models and has excellent risk assessment capability.

Multivariate Time Series Anomaly Detection Based on Spatial-Temporal Network and Transformer in Industrial Internet of Things

In the Industrial Internet of Things(IIoT),sensors generate time series data to reflect the working state.When the systems are attacked,timely identification of outliers in time series is critical to ensure security.Although many anomaly detection methods have been proposed,the temporal correlation of the time series over the same sensor and the state(spatial)correlation between different sensors are rarely considered simultaneously in these methods.Owing to the superior capability of Transformer in learning time series features.This paper proposes a time series anomaly detection method based on a spatial-temporal network and an improved Transformer.Additionally,the methods based on graph neural networks typically include a graph structure learning module and an anomaly detection module,which are interdependent.However,in the initial phase of training,since neither of the modules has reached an optimal state,their performance may influence each other.This scenario makes the end-to-end training approach hard to effectively direct the learning trajectory of each module.This interdependence between the modules,coupled with the initial instability,may cause the model to find it hard to find the optimal solution during the training process,resulting in unsatisfactory results.We introduce an adaptive graph structure learning method to obtain the optimal model parameters and graph structure.Experiments on two publicly available datasets demonstrate that the proposed method attains higher anomaly detection results than other methods.

Periodic signal extraction of GNSS height time series based on adaptive singular spectrum analysis

Singular spectrum analysis is widely used in geodetic time series analysis.However,when extracting time-varying periodic signals from a large number of Global Navigation Satellite System(GNSS)time series,the selection of appropriate embedding window size and principal components makes this method cumbersome and inefficient.To improve the efficiency and accuracy of singular spectrum analysis,this paper proposes an adaptive singular spectrum analysis method by combining spectrum analysis with a new trace matrix.The running time and correlation analysis indicate that the proposed method can adaptively set the embedding window size to extract the time-varying periodic signals from GNSS time series,and the extraction efficiency of a single time series is six times that of singular spectrum analysis.The method is also accurate and more suitable for time-varying periodic signal analysis of global GNSS sites.

Research on static characteristic of double redundance double nozzle flapper valve based on AMESim

The feedback spring rod of the armature assembly is cancelled in the double redundance double nozzle flapper valve(DRDNFV),and the difficulty of valve core displacement control is increased.Therefore,this paper intends to study the static characteristic of DRDNFV through the AMESet and AMESim simulation.It is explored under the circumstance of the fixed orifices being clogged and experimentally verified on the test bench.The results show that the pressure gain increases and the flow gain decreases with the increasing clogged degree of the fixed orifices on both sides.The zero bias increases synchronously with the increasing clogged degree of the unilateral fixed orifice.The experimental results are basically consistent with the theoretical curves and the theoretical correctness of the simulation model is effectively verified.The results can provide the theoretical reference for design,debugging,maintenance and fault diagnosis of DRDNFV.

A Time Series Short-Term Prediction Method Based on Multi-Granularity Event Matching and Alignment

Accurate forecasting of time series is crucial across various domains.Many prediction tasks rely on effectively segmenting,matching,and time series data alignment.For instance,regardless of time series with the same granularity,segmenting them into different granularity events can effectively mitigate the impact of varying time scales on prediction accuracy.However,these events of varying granularity frequently intersect with each other,which may possess unequal durations.Even minor differences can result in significant errors when matching time series with future trends.Besides,directly using matched events but unaligned events as state vectors in machine learning-based prediction models can lead to insufficient prediction accuracy.Therefore,this paper proposes a short-term forecasting method for time series based on a multi-granularity event,MGE-SP(multi-granularity event-based short-termprediction).First,amethodological framework for MGE-SP established guides the implementation steps.The framework consists of three key steps,including multi-granularity event matching based on the LTF(latest time first)strategy,multi-granularity event alignment using a piecewise aggregate approximation based on the compression ratio,and a short-term prediction model based on XGBoost.The data from a nationwide online car-hailing service in China ensures the method’s reliability.The average RMSE(root mean square error)and MAE(mean absolute error)of the proposed method are 3.204 and 2.360,lower than the respective values of 4.056 and 3.101 obtained using theARIMA(autoregressive integratedmoving average)method,as well as the values of 4.278 and 2.994 obtained using k-means-SVR(support vector regression)method.The other experiment is conducted on stock data froma public data set.The proposed method achieved an average RMSE and MAE of 0.836 and 0.696,lower than the respective values of 1.019 and 0.844 obtained using the ARIMA method,as well as the values of 1.350 and 1.172 obtained using the k-means-SVR method.

Double quantum images encryption scheme based on chaotic system

This paper explores a double quantum images representation(DNEQR)model that allows for simultaneous storage of two digital images in a quantum superposition state.Additionally,a new type of two-dimensional hyperchaotic system based on sine and logistic maps is investigated,offering a wider parameter space and better chaotic behavior compared to the sine and logistic maps.Based on the DNEQR model and the hyperchaotic system,a double quantum images encryption algorithm is proposed.Firstly,two classical plaintext images are transformed into quantum states using the DNEQR model.Then,the proposed hyperchaotic system is employed to iteratively generate pseudo-random sequences.These chaotic sequences are utilized to perform pixel value and position operations on the quantum image,resulting in changes to both pixel values and positions.Finally,the ciphertext image can be obtained by qubit-level diffusion using two XOR operations between the position-permutated image and the pseudo-random sequences.The corresponding quantum circuits are also given.Experimental results demonstrate that the proposed scheme ensures the security of the images during transmission,improves the encryption efficiency,and enhances anti-interference and anti-attack capabilities.

TSCND:Temporal Subsequence-Based Convolutional Network with Difference for Time Series Forecasting

Time series forecasting plays an important role in various fields, such as energy, finance, transport, and weather. Temporal convolutional networks (TCNs) based on dilated causal convolution have been widely used in time series forecasting. However, two problems weaken the performance of TCNs. One is that in dilated casual convolution, causal convolution leads to the receptive fields of outputs being concentrated in the earlier part of the input sequence, whereas the recent input information will be severely lost. The other is that the distribution shift problem in time series has not been adequately solved. To address the first problem, we propose a subsequence-based dilated convolution method (SDC). By using multiple convolutional filters to convolve elements of neighboring subsequences, the method extracts temporal features from a growing receptive field via a growing subsequence rather than a single element. Ultimately, the receptive field of each output element can cover the whole input sequence. To address the second problem, we propose a difference and compensation method (DCM). The method reduces the discrepancies between and within the input sequences by difference operations and then compensates the outputs for the information lost due to difference operations. Based on SDC and DCM, we further construct a temporal subsequence-based convolutional network with difference (TSCND) for time series forecasting. The experimental results show that TSCND can reduce prediction mean squared error by 7.3% and save runtime, compared with state-of-the-art models and vanilla TCN.

Influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel

The influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel is investigated.The DarcyBrinkman model is used to characterize the fluid flow dynamics in porous materials.The analytical solutions are obtained for the unidirectional and completely developed flow.Based on a normal mode analysis,the generalized eigenvalue problem under a perturbed state is solved.The eigenvalue problem is then solved by the spectral method.Finally,the critical Rayleigh number with the corresponding wavenumber is evaluated at the assigned values of the other flow-governing parameters.The results show that increasing the Darcy number,the Lewis number,the Dufour parameter,or the Soret parameter increases the stability of the system,whereas increasing the inclination angle of the channel destabilizes the flow.Besides,the flow is the most unstable when the channel is vertically oriented.

Internal collision double ionization of molecules driven by co-rotating two-color circularly polarized laser pulses

The double ionization process of molecules driven by co-rotating two-color circularly polarized fields is investigated with a three-dimensional classical ensemble model. Numerical results indicate that a considerable part of the sequential double ionization(DI) events of molecules occur through internal collision double ionization(ICD), and the ICD recollision mechanism is significantly different from that in non-sequential double ionization(NSDI). By analyzing the results of internuclear distances R = 5 a.u. and 2 a.u., these two recollision mechanisms are studied in depth. It is found that the dynamic behaviors of the recollision mechanisms of NSDI and ICD are similar. For NSDI, the motion range of electrons after the ionization is relatively large, and the electrons will return to the core after a period of time. In the ICD process,electrons will rotate around the parent ion before ionization, and the distance of the electron motion is relatively small. After a period of time, the electrons will come back to the core and collide with another electron. Furthermore, the molecular internuclear distance has a significant effect on the electron dynamic behavior of the two ionization mechanisms. This study will help to understand the multi-electron ionization process of complex systems.

Fluorescent Double Network Hydrogels with Ionic Responsiveness and High Mechanical Properties for Visual Detection

We developed a fluorescent double network hydrogel with ionic responsiveness and high mechanical properties for visual detection.The nanocomposite hydrogel of laponite and polyacrylamide serves as the first network,while the ionic cross-linked hydrogel of terbium ions and sodium alginate serves as the second network.The double-network structure,the introduction of nanoparticles and the reversible ionic crosslinked interactions confer high mechanical properties to the hydrogel.Terbium ions are not only used as the ionic cross-linked points,but also used as green emitters to endow hydrogels with fluorescent properties.On the basis of the “antenna effect” of terbium ions and the ion exchange interaction,the fluorescence of the hydrogels can make selective responses to various ions(such as organic acid radical ions,transition metal ions) in aqueous solutions,which enables a convenient strategy for visual detection toward ions.Consequently,the fluorescent double network hydrogel fabricated in this study is promising for use in the field of visual sensor detection.

Transcriptome and morphological analyses of double flower formation in Dianthus chinensis

The double flower developmental process is regulated via a complex transcriptional regulatory network.To understand this highly dynamic and complex developmental process of Dianthus spp.,we performed a comparative analysis of floral morphology and transcriptome dynamics in simple flowers and double flowers.We found that the primordium of double flowers of‘X’was larger in size compared to that of simple flowers of‘L’in Dianthus chinensis.RNA-seq and Weighted Gene Co-expression Network Analysis(WGCNA)during flower development,identified stage-specific gene network modules.Expression analysis by RNA-seq indicated that a group of genes related to floral meristem identity,primordia position and polarity were highly expressed in double flowers genotypes compared to simple flowers genotypes,suggesting their roles in double-petal formation.A total of 21 DEGs related to petal number were identified between simple and double flowers.The experiments of in situ hybridization revealed that DcaAP2L,DcaLFY and DcaUFO genes were expressed in the intra-sepal boundary and petal boundary.We proposed a potential transcriptional regulatory network for simple and double flower development.This study provides novel insights into the molecular mechanism underlying double flower formation in Dianthus spp.

Effect of Single- versus Double-Layer Uterine Closure during Caesarean Section on Niche Formation and Menstrual Irregularity

Background: The myometrium at the location of the CS (caesarean section) scars, also known as residual myometrium thickness (RMT), is larger after a double-layer uterine closure procedure than following a single-layer one. It may lessen the formation of a niche that is the myometrium’s disruption at the location of the scar of the uterus. Gynecological manifestations, obstetric problems in a future pregnancy and birth, and maybe subfertility are linked to thin RMT and a niche. Objective: To ascertain if double-layer unlocked closure of the uterus is better than single-layer one in terms of post-menstrual spotting and niche development following a first CS. Patients and Methods: In this randomized clinical study, 287 patients were evaluated for qualifying. Of all eligible individuals, 57 patients were excluded from the study based on the inclusion criteria. Results: The variation in ages, gestational age, body mass index (BMI), and cesarean section indications between the two assigned groups is statistically insignificant. However, postmenstrual spotting was statistically significantly more common in single-layer group compared to in double-group. The current study revealed ultrasound findings suggestive of niche formation was statistically significantly more common in single-layer group compared to in double-layer group. Conclusion: As evident from the current study, it demonstrates the advantages of double-layer unlocked closure of the uterus over single-layer one in terms of post-menstrual spotting and niche development following first-time cs. Thus, we deduced that fewer niches are formed, and fewer menstrual spotting occurs in the presence of double unlocked layers closure. To ascertain the impact of uterus closure method on post-operative niche development and the risk of obstetrics and gynaecological problems, further prospective trials with extended follow-up periods are required.

Defect Detection Model Using Time Series Data Augmentation and Transformation

Time-series data provide important information in many fields,and their processing and analysis have been the focus of much research.However,detecting anomalies is very difficult due to data imbalance,temporal dependence,and noise.Therefore,methodologies for data augmentation and conversion of time series data into images for analysis have been studied.This paper proposes a fault detection model that uses time series data augmentation and transformation to address the problems of data imbalance,temporal dependence,and robustness to noise.The method of data augmentation is set as the addition of noise.It involves adding Gaussian noise,with the noise level set to 0.002,to maximize the generalization performance of the model.In addition,we use the Markov Transition Field(MTF)method to effectively visualize the dynamic transitions of the data while converting the time series data into images.It enables the identification of patterns in time series data and assists in capturing the sequential dependencies of the data.For anomaly detection,the PatchCore model is applied to show excellent performance,and the detected anomaly areas are represented as heat maps.It allows for the detection of anomalies,and by applying an anomaly map to the original image,it is possible to capture the areas where anomalies occur.The performance evaluation shows that both F1-score and Accuracy are high when time series data is converted to images.Additionally,when processed as images rather than as time series data,there was a significant reduction in both the size of the data and the training time.The proposed method can provide an important springboard for research in the field of anomaly detection using time series data.Besides,it helps solve problems such as analyzing complex patterns in data lightweight.

CNN-LSTM based incremental attention mechanism enabled phase-space reconstruction for chaotic time series prediction

To improve the prediction accuracy of chaotic time series and reconstruct a more reasonable phase space structure of the prediction network,we propose a convolutional neural network-long short-term memory(CNN-LSTM)prediction model based on the incremental attention mechanism.Firstly,a traversal search is conducted through the traversal layer for finite parameters in the phase space.Then,an incremental attention layer is utilized for parameter judgment based on the dimension weight criteria(DWC).The phase space parameters that best meet DWC are selected and fed into the input layer.Finally,the constructed CNN-LSTM network extracts spatio-temporal features and provides the final prediction results.The model is verified using Logistic,Lorenz,and sunspot chaotic time series,and the performance is compared from the two dimensions of prediction accuracy and network phase space structure.Additionally,the CNN-LSTM network based on incremental attention is compared with long short-term memory(LSTM),convolutional neural network(CNN),recurrent neural network(RNN),and support vector regression(SVR)for prediction accuracy.The experiment results indicate that the proposed composite network model possesses enhanced capability in extracting temporal features and achieves higher prediction accuracy.Also,the algorithm to estimate the phase space parameter is compared with the traditional CAO,false nearest neighbor,and C-C,three typical methods for determining the chaotic phase space parameters.The experiments reveal that the phase space parameter estimation algorithm based on the incremental attention mechanism is superior in prediction accuracy compared with the traditional phase space reconstruction method in five networks,including CNN-LSTM,LSTM,CNN,RNN,and SVR.