基于softmax的加权Double Q-Learning算法

强化学习作为机器学习的一个分支,用于描述和解决智能体在与环境的交互过程中,通过学习策略以达成回报最大化的问题。Q-Learning作为无模型强化学习的经典方法,存在过估计引起的最大化偏差问题,并且在环境中奖励存在噪声时表现不佳。Double Q-Learning(DQL)的出现解决了过估计问题,但同时造成了低估问题。为解决以上算法的高低估问题,提出了基于softmax的加权Q-Learning算法,并将其与DQL相结合,提出了一种新的基于softmax的加权Double Q-Learning算法(WDQL-Softmax)。该算法基于加权双估计器的构造,对样本期望值进行softmax操作得到权重,使用权重估计动作价值,有效平衡对动作价值的高估和低估问题,使估计值更加接近理论值。实验结果表明,在离散动作空间中,相比于Q-Learning算法、DQL算法和WDQL算法,WDQL-Softmax算法的收敛速度更快且估计值与理论值的误差更小。

Underwater four-quadrant dual-beam circumferential scanning laser fuze using nonlinear adaptive backscatter filter based on pauseable SAF-LMS algorithm

The phenomenon of a target echo peak overlapping with the backscattered echo peak significantly undermines the detection range and precision of underwater laser fuzes.To overcome this issue,we propose a four-quadrant dual-beam circumferential scanning laser fuze to distinguish various interference signals and provide more real-time data for the backscatter filtering algorithm.This enhances the algorithm loading capability of the fuze.In order to address the problem of insufficient filtering capacity in existing linear backscatter filtering algorithms,we develop a nonlinear backscattering adaptive filter based on the spline adaptive filter least mean square(SAF-LMS)algorithm.We also designed an algorithm pause module to retain the original trend of the target echo peak,improving the time discrimination accuracy and anti-interference capability of the fuze.Finally,experiments are conducted with varying signal-to-noise ratios of the original underwater target echo signals.The experimental results show that the average signal-to-noise ratio before and after filtering can be improved by more than31 d B,with an increase of up to 76%in extreme detection distance.

Influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel

The influence of variable viscosity and double diffusion on the convective stability of a nanofluid flow in an inclined porous channel is investigated.The DarcyBrinkman model is used to characterize the fluid flow dynamics in porous materials.The analytical solutions are obtained for the unidirectional and completely developed flow.Based on a normal mode analysis,the generalized eigenvalue problem under a perturbed state is solved.The eigenvalue problem is then solved by the spectral method.Finally,the critical Rayleigh number with the corresponding wavenumber is evaluated at the assigned values of the other flow-governing parameters.The results show that increasing the Darcy number,the Lewis number,the Dufour parameter,or the Soret parameter increases the stability of the system,whereas increasing the inclination angle of the channel destabilizes the flow.Besides,the flow is the most unstable when the channel is vertically oriented.

Double contrast-enhanced ultrasonography improves diagnostic accuracy of T staging compared with multi-detector computed tomography in gastric cancer patients

BACKGROUND Gastric cancer(GC)is the most common malignant tumor and ranks third for cancer-related deaths among the worldwide.The disease poses a serious public health problem in China,ranking fifth for incidence and third for mortality.Knowledge of the invasive depth of the tumor is vital to treatment decisions.AIM To evaluate the diagnostic performance of double contrast-enhanced ultrasonography(DCEUS)for preoperative T staging in patients with GC by comparing with multi-detector computed tomography(MDCT).METHODS This single prospective study enrolled patients with GC confirmed by preoperative gastroscopy from July 2021 to March 2023.Patients underwent DCEUS,including ultrasonography(US)and intravenous contrast-enhanced ultrasonography(CEUS),and MDCT examinations for the assessment of preoperative T staging.Features of GC were identified on DCEUS and criteria developed to evaluate T staging according to the 8th edition of AJCC cancer staging manual.The diagnostic performance of DCEUS was evaluated by comparing it with that of MDCT and surgical-pathological findings were considered as the gold standard.RESULTS A total of 229 patients with GC(80 T1,33 T2,59 T3 and 57 T4)were included.Overall accuracies were 86.9%for DCEUS and 61.1%for MDCT(P<0.001).DCEUS was superior to MDCT for T1(92.5%vs 70.0%,P<0.001),T2(72.7%vs 51.5%,P=0.041),T3(86.4%vs 45.8%,P<0.001)and T4(87.7%vs 70.2%,P=0.022)staging of GC.CONCLUSION DCEUS improved the diagnostic accuracy of preoperative T staging in patients with GC compared with MDCT,and constitutes a promising imaging modality for preoperative evaluation of GC to aid individualized treatment decision-making.

Rao Algorithms-Based Structure Optimization for Heterogeneous Wireless Sensor Networks

The structural optimization of wireless sensor networks is a critical issue because it impacts energy consumption and hence the network’s lifetime.Many studies have been conducted for homogeneous networks,but few have been performed for heterogeneouswireless sensor networks.This paper utilizes Rao algorithms to optimize the structure of heterogeneous wireless sensor networks according to node locations and their initial energies.The proposed algorithms lack algorithm-specific parameters and metaphorical connotations.The proposed algorithms examine the search space based on the relations of the population with the best,worst,and randomly assigned solutions.The proposed algorithms can be evaluated using any routing protocol,however,we have chosen the well-known routing protocols in the literature:Low Energy Adaptive Clustering Hierarchy(LEACH),Power-Efficient Gathering in Sensor Information Systems(PEAGSIS),Partitioned-based Energy-efficient LEACH(PE-LEACH),and the Power-Efficient Gathering in Sensor Information Systems Neural Network(PEAGSIS-NN)recent routing protocol.We compare our optimized method with the Jaya,the Particle Swarm Optimization-based Energy Efficient Clustering(PSO-EEC)protocol,and the hybrid Harmony Search Algorithm and PSO(HSA-PSO)algorithms.The efficiencies of our proposed algorithms are evaluated by conducting experiments in terms of the network lifetime(first dead node,half dead nodes,and last dead node),energy consumption,packets to cluster head,and packets to the base station.The experimental results were compared with those obtained using the Jaya optimization algorithm.The proposed algorithms exhibited the best performance.The proposed approach successfully prolongs the network lifetime by 71% for the PEAGSIS protocol,51% for the LEACH protocol,10% for the PE-LEACH protocol,and 73% for the PEGSIS-NN protocol;Moreover,it enhances other criteria such as energy conservation,fitness convergence,packets to cluster head,and packets to the base station.

Falcon Optimization Algorithm-Based Energy Efficient Communication Protocol for Cluster-Based Vehicular Networks

Rapid development in Information Technology(IT)has allowed several novel application regions like large outdoor vehicular networks for Vehicle-to-Vehicle(V2V)transmission.Vehicular networks give a safe and more effective driving experience by presenting time-sensitive and location-aware data.The communication occurs directly between V2V and Base Station(BS)units such as the Road Side Unit(RSU),named as a Vehicle to Infrastructure(V2I).However,the frequent topology alterations in VANETs generate several problems with data transmission as the vehicle velocity differs with time.Therefore,the scheme of an effectual routing protocol for reliable and stable communications is significant.Current research demonstrates that clustering is an intelligent method for effectual routing in a mobile environment.Therefore,this article presents a Falcon Optimization Algorithm-based Energy Efficient Communication Protocol for Cluster-based Routing(FOA-EECPCR)technique in VANETS.The FOA-EECPCR technique intends to group the vehicles and determine the shortest route in the VANET.To accomplish this,the FOA-EECPCR technique initially clusters the vehicles using FOA with fitness functions comprising energy,distance,and trust level.For the routing process,the Sparrow Search Algorithm(SSA)is derived with a fitness function that encompasses two variables,namely,energy and distance.A series of experiments have been conducted to exhibit the enhanced performance of the FOA-EECPCR method.The experimental outcomes demonstrate the enhanced performance of the FOA-EECPCR approach over other current methods.

Enhancing the stability of Ni Fe-layered double hydroxide nanosheet array for alkaline seawater oxidation by Ce doping

Electrocatalytic hydrogen production from seawater holds enormous promise for clean energy generation.Nevertheless,the direct electrolysis of seawater encounters significant challenges due to poor anodic stability caused by detrimental chlorine chemistry.Herein,we present our recent discovery that the incorporation of Ce into Ni Fe layered double hydroxide nanosheet array on Ni foam(Ce-Ni Fe LDH/NF)emerges as a robust electrocatalyst for seawater oxidation.During the seawater oxidation process,CeO_(2)is generated,effectively repelling Cl^(-)and inhibiting the formation of Cl O-,resulting in a notable enhancement in the oxidation activity and stability of alkaline seawater.The prepared Ce-Ni Fe LDH/NF requires only overpotential of 390 m V to achieve the current density of 1 A cm^(-2),while maintaining long-term stability for 500 h,outperforming the performance of Ni Fe LDH/NF(430 m V,150 h)by a significant margin.This study highlights the effectiveness of a Ce-doping strategy in augmenting the activity and stability of materials based on Ni Fe LDH in seawater electrolysis for oxygen evolution.

Product quality prediction based on RBF optimized by firefly algorithm

With the development of information technology,a large number of product quality data in the entire manufacturing process is accumulated,but it is not explored and used effectively.The traditional product quality prediction models have many disadvantages,such as high complexity and low accuracy.To overcome the above problems,we propose an optimized data equalization method to pre-process dataset and design a simple but effective product quality prediction model:radial basis function model optimized by the firefly algorithm with Levy flight mechanism(RBFFALM).First,the new data equalization method is introduced to pre-process the dataset,which reduces the dimension of the data,removes redundant features,and improves the data distribution.Then the RBFFALFM is used to predict product quality.Comprehensive expe riments conducted on real-world product quality datasets validate that the new model RBFFALFM combining with the new data pre-processing method outperforms other previous me thods on predicting product quality.

Path Planning for AUVs Based on Improved APF-AC Algorithm

With the increase in ocean exploration activities and underwater development,the autonomous underwater vehicle(AUV)has been widely used as a type of underwater automation equipment in the detection of underwater environments.However,nowadays AUVs generally have drawbacks such as weak endurance,low intelligence,and poor detection ability.The research and implementation of path-planning methods are the premise of AUVs to achieve actual tasks.To improve the underwater operation ability of the AUV,this paper studies the typical problems of path-planning for the ant colony algorithm and the artificial potential field algorithm.In response to the limitations of a single algorithm,an optimization scheme is proposed to improve the artificial potential field ant colony(APF-AC)algorithm.Compared with traditional ant colony and comparative algorithms,the APF-AC reduced the path length by 1.57%and 0.63%(in the simple environment),8.92%and 3.46%(in the complex environment).The iteration time has been reduced by approximately 28.48%and 18.05%(in the simple environment),18.53%and 9.24%(in the complex environment).Finally,the improved APF-AC algorithm has been validated on the AUV platform,and the experiment is consistent with the simulation.Improved APF-AC algorithm can effectively reduce the underwater operation time and overall power consumption of the AUV,and shows a higher safety.

Algorithm Selection Method Based on Coupling Strength for Partitioned Analysis of Structure-Piezoelectric-Circuit Coupling

In this study, we propose an algorithm selection method based on coupling strength for the partitioned analysis ofstructure-piezoelectric-circuit coupling, which includes two types of coupling or inverse and direct piezoelectriccoupling and direct piezoelectric and circuit coupling. In the proposed method, implicit and explicit formulationsare used for strong and weak coupling, respectively. Three feasible partitioned algorithms are generated, namely(1) a strongly coupled algorithm that uses a fully implicit formulation for both types of coupling, (2) a weaklycoupled algorithm that uses a fully explicit formulation for both types of coupling, and (3) a partially stronglycoupled and partially weakly coupled algorithm that uses an implicit formulation and an explicit formulation forthe two types of coupling, respectively.Numerical examples using a piezoelectric energy harvester,which is a typicalstructure-piezoelectric-circuit coupling problem, demonstrate that the proposed method selects the most costeffectivealgorithm.

A Review of Image Steganography Based on Multiple Hashing Algorithm

Steganography is a technique for hiding secret messages while sending and receiving communications through a cover item.From ancient times to the present,the security of secret or vital information has always been a significant problem.The development of secure communication methods that keep recipient-only data transmissions secret has always been an area of interest.Therefore,several approaches,including steganography,have been developed by researchers over time to enable safe data transit.In this review,we have discussed image steganography based on Discrete Cosine Transform(DCT)algorithm,etc.We have also discussed image steganography based on multiple hashing algorithms like the Rivest–Shamir–Adleman(RSA)method,the Blowfish technique,and the hash-least significant bit(LSB)approach.In this review,a novel method of hiding information in images has been developed with minimal variance in image bits,making our method secure and effective.A cryptography mechanism was also used in this strategy.Before encoding the data and embedding it into a carry image,this review verifies that it has been encrypted.Usually,embedded text in photos conveys crucial signals about the content.This review employs hash table encryption on the message before hiding it within the picture to provide a more secure method of data transport.If the message is ever intercepted by a third party,there are several ways to stop this operation.A second level of security process implementation involves encrypting and decrypting steganography images using different hashing algorithms.

Hybrid Optimization Algorithm for Handwritten Document Enhancement

The Gannet Optimization Algorithm (GOA) and the Whale Optimization Algorithm (WOA) demonstrate strong performance;however, there remains room for improvement in convergence and practical applications. This study introduces a hybrid optimization algorithm, named the adaptive inertia weight whale optimization algorithm and gannet optimization algorithm (AIWGOA), which addresses challenges in enhancing handwritten documents. The hybrid strategy integrates the strengths of both algorithms, significantly enhancing their capabilities, whereas the adaptive parameter strategy mitigates the need for manual parameter setting. By amalgamating the hybrid strategy and parameter-adaptive approach, the Gannet Optimization Algorithm was refined to yield the AIWGOA. Through a performance analysis of the CEC2013 benchmark, the AIWGOA demonstrates notable advantages across various metrics. Subsequently, an evaluation index was employed to assess the enhanced handwritten documents and images, affirming the superior practical application of the AIWGOA compared with other algorithms.

Representation of an Integer by a Quadratic Form through the Cornacchia Algorithm

Cornachia’s algorithm can be adapted to the case of the equation x2+dy2=nand even to the case of ax2+bxy+cy2=n. For the sake of completeness, we have given modalities without proofs (the proof in the case of the equation x2+y2=n). Starting from a quadratic form with two variables f(x,y)=ax2+bxy+cy2and n an integer. We have shown that a primitive positive solution (u,v)of the equation f(x,y)=nis admissible if it is obtained in the following way: we take α modulo n such that f(α,1)≡0modn, u is the first of the remainders of Euclid’s algorithm associated with n and α that is less than 4cn/| D |) (possibly α itself) and the equation f(x,y)=n. has an integer solution u in y. At the end of our work, it also appears that the Cornacchia algorithm is good for the form n=ax2+bxy+cy2if all the primitive positive integer solutions of the equation f(x,y)=nare admissible, i.e. computable by the algorithmic process.

Diagnosis of indirectly driven double shell targets with point-projection hard x-ray radiography

We present an application of short-pulse laser-generated hard x rays for the diagnosis of indirectly driven double shell targets. Coneinserted double shell targets were imploded through an indirect drive approach on the upgraded SG-II laser facility. Then, based on thepoint-projection hard x-ray radiography technique, time-resolved radiography of the double shell targets, including that of their near-peakcompression, were obtained. The backlighter source was created by the interactions of a high-intensity short pulsed laser with a metalmicrowire target. Images of the target near peak compression were obtained with an Au microwire. In addition, radiation hydrodynamicsimulations were performed, and the target evolution obtained agrees well with the experimental results. Using the radiographic images, arealdensities of the targets were evaluated.

A Hybrid Dung Beetle Optimization Algorithm with Simulated Annealing for the Numerical Modeling of Asymmetric Wave Equations

In the generalized continuum mechanics(GCM)theory framework,asymmetric wave equations encompass the characteristic scale parameters of the medium,accounting for microstructure interactions.This study integrates two theoretical branches of the GCM,the modified couple stress theory(M-CST)and the one-parameter second-strain-gradient theory,to form a novel asymmetric wave equation in a unified framework.Numerical modeling of the asymmetric wave equation in a unified framework accurately describes subsurface structures with vital implications for subsequent seismic wave inversion and imaging endeavors.However,employing finite-difference(FD)methods for numerical modeling may introduce numerical dispersion,adversely affecting the accuracy of numerical modeling.The design of an optimal FD operator is crucial for enhancing the accuracy of numerical modeling and emphasizing the scale effects.Therefore,this study devises a hybrid scheme called the dung beetle optimization(DBO)algorithm with a simulated annealing(SA)algorithm,denoted as the SA-based hybrid DBO(SDBO)algorithm.An FD operator optimization method under the SDBO algorithm was developed and applied to the numerical modeling of asymmetric wave equations in a unified framework.Integrating the DBO and SA algorithms mitigates the risk of convergence to a local extreme.The numerical dispersion outcomes underscore that the proposed SDBO algorithm yields FD operators with precision errors constrained to 0.5‱while encompassing a broader spectrum coverage.This result confirms the efficacy of the SDBO algorithm.Ultimately,the numerical modeling results demonstrate that the new FD method based on the SDBO algorithm effectively suppresses numerical dispersion and enhances the accuracy of elastic wave numerical modeling,thereby accentuating scale effects.This result is significant for extracting wavefield perturbations induced by complex microstructures in the medium and the analysis of scale effects.

Multi-Objective Optimization of VBHF in Deep Drawing Based on the Improved QO-Jaya Algorithm

Blank holder force(BHF)is a crucial parameter in deep drawing,having close relation with the forming quality of sheet metal.However,there are different BHFs maintaining the best forming effect in different stages of deep drawing.The variable blank holder force(VBHF)varying with the drawing stage can overcome this problem at an extent.The optimization of VBHF is to determine the optimal BHF in every deep drawing stage.In this paper,a new heuristic optimization algorithm named Jaya is introduced to solve the optimization efficiently.An improved“Quasi-oppositional”strategy is added to Jaya algorithm for improving population diversity.Meanwhile,an innovated stop criterion is added for better convergence.Firstly,the quality evaluation criteria for wrinkling and tearing are built.Secondly,the Kriging models are developed to approximate and quantify the relation between VBHF and forming defects under random sampling.Finally,the optimization models are established and solved by the improved QO-Jaya algorithm.A VBHF optimization example of component with complicated shape and thin wall is studied to prove the effectiveness of the improved Jaya algorithm.The optimization results are compared with that obtained by other algorithms based on the TOPSIS method.

Defective Nickel-Iron Layered Double Hydroxide for Enhanced Photocatalytic NO Oxidation with Significant Alleviation of NO2 Production

Photocatalysis offers a sustainable means for the oxidative removal of low concentrations of NOx(NO,NO2,N2O,N2O5,etc.)from the atmosphere.Layered double hydroxides(LDHs)are promising candidate photocatalysts owing to their unique layered and tunable chemical structures and abundant surface hydroxide(OH)moieties,which are hydroxyl radical(OH)precursors.However,the practical applications of LDHs are limited by their poor charge-separation ability and insufficient active sites.Herein,we developed a facile N_(2)H_(4)-driven etching approach to introduce dual Ni^(2+)and OHvacancies(Niv and OHv,respectively)into NiFe-LDH nanosheets(hereafter referred to as NiFe-LDH-et)to facilitate improved charge-carrier separation and active Lewis acidic site(Fe^(3+)and Ni^(2+)exposed at OHv)formation.In contrast to inert pristine LDH,NiFe-LDH-et actively removed NO under visible-light illumination.Specifically,Ni_(76)Fe_(24)-LDH-et etched with 1.50 mmol·L^(-1)N_(2)H_(4)solution removed 32.8%of the NO in continuously flowing air(NO feed concentration:500 parts per billion(ppb))under visible-light illumination,thereby outperforming most reported catalysts.Experimental and theoretical data revealed that the dual vacancies promoted the production of reactive oxygen species(O_(2)·^(-)andOH)and the adsorption of NO on the LDH.In situ spectroscopy demonstrated that NO was preferentially adsorbed at Lewis acidic sites,particularly exposed Fe^(3+)sites,converted into NO+,and subsequently oxidized to NO3without the notable formation of the more toxic intermediate NO2,thereby alleviating risks associated with its production and emission.

RRT Autonomous Detection Algorithm Based on Multiple Pilot Point Bias Strategy and Karto SLAM Algorithm

A Rapid-exploration Random Tree(RRT)autonomous detection algorithm based on the multi-guide-node deflection strategy and Karto Simultaneous Localization and Mapping(SLAM)algorithm was proposed to solve the problems of low efficiency of detecting frontier boundary points and drift distortion in the process of map building in the traditional RRT algorithm in the autonomous detection strategy of mobile robot.Firstly,an RRT global frontier boundary point detection algorithm based on the multi-guide-node deflection strategy was put forward,which introduces the reference value of guide nodes’deflection probability into the random sampling function so that the global search tree can detect frontier boundary points towards the guide nodes according to random probability.After that,a new autonomous detection algorithm for mobile robots was proposed by combining the graph optimization-based Karto SLAM algorithm with the previously improved RRT algorithm.The algorithm simulation platform based on the Gazebo platform was built.The simulation results show that compared with the traditional RRT algorithm,the proposed RRT autonomous detection algorithm can effectively reduce the time of autonomous detection,plan the length of detection trajectory under the condition of high average detection coverage,and complete the task of autonomous detection mapping more efficiently.Finally,with the help of the ROS-based mobile robot experimental platform,the performance of the proposed algorithm was verified in the real environment of different obstacles.The experimental results show that in the actual environment of simple and complex obstacles,the proposed RRT autonomous detection algorithm was superior to the traditional RRT autonomous detection algorithm in the time of detection,length of detection trajectory,and average coverage,thus improving the efficiency and accuracy of autonomous detection.

Synergistic Swarm Optimization Algorithm

This research paper presents a novel optimization method called the Synergistic Swarm Optimization Algorithm(SSOA).The SSOA combines the principles of swarmintelligence and synergistic cooperation to search for optimal solutions efficiently.A synergistic cooperation mechanism is employed,where particles exchange information and learn from each other to improve their search behaviors.This cooperation enhances the exploitation of promising regions in the search space while maintaining exploration capabilities.Furthermore,adaptive mechanisms,such as dynamic parameter adjustment and diversification strategies,are incorporated to balance exploration and exploitation.By leveraging the collaborative nature of swarm intelligence and integrating synergistic cooperation,the SSOAmethod aims to achieve superior convergence speed and solution quality performance compared to other optimization algorithms.The effectiveness of the proposed SSOA is investigated in solving the 23 benchmark functions and various engineering design problems.The experimental results highlight the effectiveness and potential of the SSOA method in addressing challenging optimization problems,making it a promising tool for a wide range of applications in engineering and beyond.Matlab codes of SSOA are available at:https://www.mathworks.com/matlabcentral/fileexchange/153466-synergistic-swarm-optimization-algorithm.

MCWOA Scheduler:Modified Chimp-Whale Optimization Algorithm for Task Scheduling in Cloud Computing

Cloud computing provides a diverse and adaptable resource pool over the internet,allowing users to tap into various resources as needed.It has been seen as a robust solution to relevant challenges.A significant delay can hamper the performance of IoT-enabled cloud platforms.However,efficient task scheduling can lower the cloud infrastructure’s energy consumption,thus maximizing the service provider’s revenue by decreasing user job processing times.The proposed Modified Chimp-Whale Optimization Algorithm called Modified Chimp-Whale Optimization Algorithm(MCWOA),combines elements of the Chimp Optimization Algorithm(COA)and the Whale Optimization Algorithm(WOA).To enhance MCWOA’s identification precision,the Sobol sequence is used in the population initialization phase,ensuring an even distribution of the population across the solution space.Moreover,the traditional MCWOA’s local search capabilities are augmented by incorporating the whale optimization algorithm’s bubble-net hunting and random search mechanisms into MCWOA’s position-updating process.This study demonstrates the effectiveness of the proposed approach using a two-story rigid frame and a simply supported beam model.Simulated outcomes reveal that the new method outperforms the original MCWOA,especially in multi-damage detection scenarios.MCWOA excels in avoiding false positives and enhancing computational speed,making it an optimal choice for structural damage detection.The efficiency of the proposed MCWOA is assessed against metrics such as energy usage,computational expense,task duration,and delay.The simulated data indicates that the new MCWOA outpaces other methods across all metrics.The study also references the Whale Optimization Algorithm(WOA),Chimp Algorithm(CA),Ant Lion Optimizer(ALO),Genetic Algorithm(GA)and Grey Wolf Optimizer(GWO).