Improved IChOA-Based Reinforcement Learning for Secrecy Rate Optimization in Smart Grid Communications

In the evolving landscape of the smart grid(SG),the integration of non-organic multiple access(NOMA)technology has emerged as a pivotal strategy for enhancing spectral efficiency and energy management.However,the open nature of wireless channels in SG raises significant concerns regarding the confidentiality of critical control messages,especially when broadcasted from a neighborhood gateway(NG)to smart meters(SMs).This paper introduces a novel approach based on reinforcement learning(RL)to fortify the performance of secrecy.Motivated by the need for efficient and effective training of the fully connected layers in the RL network,we employ an improved chimp optimization algorithm(IChOA)to update the parameters of the RL.By integrating the IChOA into the training process,the RL agent is expected to learn more robust policies faster and with better convergence properties compared to standard optimization algorithms.This can lead to improved performance in complex SG environments,where the agent must make decisions that enhance the security and efficiency of the network.We compared the performance of our proposed method(IChOA-RL)with several state-of-the-art machine learning(ML)algorithms,including recurrent neural network(RNN),long short-term memory(LSTM),K-nearest neighbors(KNN),support vector machine(SVM),improved crow search algorithm(I-CSA),and grey wolf optimizer(GWO).Extensive simulations demonstrate the efficacy of our approach compared to the related works,showcasing significant improvements in secrecy capacity rates under various network conditions.The proposed IChOA-RL exhibits superior performance compared to other algorithms in various aspects,including the scalability of the NOMA communication system,accuracy,coefficient of determination(R2),root mean square error(RMSE),and convergence trend.For our dataset,the IChOA-RL architecture achieved coefficient of determination of 95.77%and accuracy of 97.41%in validation dataset.This was accompanied by the lowest RMSE(0.95),indicating very precise predictions with minimal error.

Identification of Crop Diseases Based on Improved Genetic Algorithm and Extreme Learning Machine

As an indispensable task in crop protection,the detection of crop diseases directly impacts the income of farmers.To address the problems of low crop-disease identification precision and detection abilities,a new method of detection is proposed based on improved genetic algorithm and extreme learning machine.Taking five different typical diseases with common crops as the objects,this method first preprocesses the images of crops and selects the optimal features for fusion.Then,it builds a model of crop disease identification for extreme learning machine,introduces the hill-climbing algorithm to improve the traditional genetic algorithm,optimizes the initial weights and thresholds of the machine,and acquires the approximately optimal solution.And finally,a data set of crop diseases is used for verification,demonstrating that,compared with several other common machine learning methods,this method can effectively improve the crop-disease identification precision and detection abilities and provide a basis for the identification of other crop diseases.

Improved Bat Algorithm with Deep Learning-Based Biomedical ECG Signal Classification Model

With new developments experienced in Internet of Things(IoT),wearable,and sensing technology,the value of healthcare services has enhanced.This evolution has brought significant changes from conventional medicine-based healthcare to real-time observation-based healthcare.Biomedical Electrocardiogram(ECG)signals are generally utilized in examination and diagnosis of Cardiovascular Diseases(CVDs)since it is quick and non-invasive in nature.Due to increasing number of patients in recent years,the classifier efficiency gets reduced due to high variances observed in ECG signal patterns obtained from patients.In such scenario computer-assisted automated diagnostic tools are important for classification of ECG signals.The current study devises an Improved Bat Algorithm with Deep Learning Based Biomedical ECGSignal Classification(IBADL-BECGC)approach.To accomplish this,the proposed IBADL-BECGC model initially pre-processes the input signals.Besides,IBADL-BECGC model applies NasNet model to derive the features from test ECG signals.In addition,Improved Bat Algorithm(IBA)is employed to optimally fine-tune the hyperparameters related to NasNet approach.Finally,Extreme Learning Machine(ELM)classification algorithm is executed to perform ECG classification method.The presented IBADL-BECGC model was experimentally validated utilizing benchmark dataset.The comparison study outcomes established the improved performance of IBADL-BECGC model over other existing methodologies since the former achieved a maximum accuracy of 97.49%.

Improved Fruitfly Optimization with Stacked Residual Deep Learning Based Email Classification

Applied linguistics means a wide range of actions which include addressing a few language-based problems or solving some language-based concerns.Emails stay in the leading positions for business as well as personal use.This popularity grabs the interest of individuals with malevolent inten-tions—phishing and spam email assaults.Email filtering mechanisms were developed incessantly to follow unwanted,malicious content advancement to protect the end-users.But prevailing solutions were focused on phishing email filtering and spam and whereas email labelling and analysis were not fully advanced.Thus,this study provides a solution related to email message body text automatic classification into phishing and email spam.This paper presents an Improved Fruitfly Optimization with Stacked Residual Recurrent Neural Network(IFFO-SRRNN)based on Applied Linguistics for Email Classification.The presented IFFO-SRRNN technique examines the intrinsic features of email for the identification of spam emails.At the preliminary level,the IFFO-SRRNN model follows the email pre-processing stage to make it compatible with further computation.Next,the SRRNN method can be useful in recognizing and classifying spam emails.As hyperparameters of the SRRNN model need to be effectually tuned,the IFFO algorithm can be utilized as a hyperparameter optimizer.To investigate the effectual email classification results of the IFFO-SRDL technique,a series of simulations were taken placed on public datasets,and the comparison outcomes highlight the enhancements of the IFFO-SRDL method over other recent approaches with an accuracy of 98.86%.

A Novel Tuning Method for Predictive Control of VAV Air Conditioning System Based on Machine Learning and Improved PSO

The variable air volume(VAV)air conditioning system is with strong coupling and large time delay,for which model predictive control(MPC)is normally used to pursue performance improvement.Aiming at the difficulty of the parameter selection of VAV MPC controller which is difficult to make the system have a desired response,a novel tuning method based on machine learning and improved particle swarm optimization(PSO)is proposed.In this method,the relationship between MPC controller parameters and time domain performance indices is established via machine learning.Then the PSO is used to optimize MPC controller parameters to get better performance in terms of time domain indices.In addition,the PSO algorithm is further modified under the principle of population attenuation and event triggering to tune parameters of MPC and reduce the computation time of tuning method.Finally,the effectiveness of the proposed method is validated via a hardware-in-the-loop VAV system.

A dynamic fusion path planning algorithm for mobile robots incorporating improved IB-RRT∗and deep reinforcement learning

Dynamic path planning is crucial for mobile robots to navigate successfully in unstructured envi-ronments.To achieve globally optimal path and real-time dynamic obstacle avoidance during the movement,a dynamic path planning algorithm incorporating improved IB-RRT∗and deep reinforce-ment learning(DRL)is proposed.Firstly,an improved IB-RRT∗algorithm is proposed for global path planning by combining double elliptic subset sampling and probabilistic central circle target bi-as.Then,to tackle the slow response to dynamic obstacles and inadequate obstacle avoidance of tra-ditional local path planning algorithms,deep reinforcement learning is utilized to predict the move-ment trend of dynamic obstacles,leading to a dynamic fusion path planning.Finally,the simulation and experiment results demonstrate that the proposed improved IB-RRT∗algorithm has higher con-vergence speed and search efficiency compared with traditional Bi-RRT∗,Informed-RRT∗,and IB-RRT∗algorithms.Furthermore,the proposed fusion algorithm can effectively perform real-time obsta-cle avoidance and navigation tasks for mobile robots in unstructured environments.

Improved Ant Colony Optimization and Machine Learning Based Ensemble Intrusion Detection Model

Internet of things(IOT)possess cultural,commercial and social effect in life in the future.The nodes which are participating in IOT network are basi-cally attracted by the cyber-attack targets.Attack and identification of anomalies in IoT infrastructure is a growing problem in the IoT domain.Machine Learning Based Ensemble Intrusion Detection(MLEID)method is applied in order to resolve the drawback by minimizing malicious actions in related botnet attacks on Message Queue Telemetry Transport(MQTT)and Hyper-Text Transfer Proto-col(HTTP)protocols.The proposed work has two significant contributions which are a selection of features and detection of attacks.New features are chosen from Improved Ant Colony Optimization(IACO)in the feature selection,and then the detection of attacks is carried out based on a combination of their possible proper-ties.The IACO approach is focused on defining the attacker’s important features against HTTP and MQTT.In the IACO algorithm,the constant factor is calculated against HTTP and MQTT based on the mean function for each element.Attack detection,the performance of several machine learning models are Distance Deci-sion Tree(DDT),Adaptive Neuro-Fuzzy Inference System(ANFIS)and Mahala-nobis Distance Support Vector Machine(MDSVM)were compared with predicting accurate attacks on the IoT network.The outcomes of these classifiers are combined into the ensemble model.The proposed MLEID strategy has effec-tively established malicious incidents.The UNSW-NB15 dataset is used to test the MLEID technique using data from simulated IoT sensors.Besides,the pro-posed MLEID technique has a greater detection rate and an inferior rate of false-positive compared to other conventional techniques.

Assessment of compressive strength of jet grouting by machine learning

Jet grouting is one of the most popular soil improvement techniques,but its design usually involves great uncertainties that can lead to economic cost overruns in construction projects.The high dispersion in the properties of the improved material leads to designers assuming a conservative,arbitrary and unjustified strength,which is even sometimes subjected to the results of the test fields.The present paper presents an approach for prediction of the uniaxial compressive strength(UCS)of jet grouting columns based on the analysis of several machine learning algorithms on a database of 854 results mainly collected from different research papers.The selected machine learning model(extremely randomized trees)relates the soil type and various parameters of the technique to the value of the compressive strength.Despite the complex mechanism that surrounds the jet grouting process,evidenced by the high dispersion and low correlation of the variables studied,the trained model allows to optimally predict the values of compressive strength with a significant improvement with respect to the existing works.Consequently,this work proposes for the first time a reliable and easily applicable approach for estimation of the compressive strength of jet grouting columns.

An Underwater Target Detection Algorithm Based on Attention Mechanism and Improved YOLOv7

For underwater robots in the process of performing target detection tasks,the color distortion and the uneven quality of underwater images lead to great difficulties in the feature extraction process of the model,which is prone to issues like error detection,omission detection,and poor accuracy.Therefore,this paper proposed the CER-YOLOv7(CBAM-EIOU-RepVGG-YOLOv7)underwater target detection algorithm.To improve the algorithm’s capability to retain valid features from both spatial and channel perspectives during the feature extraction phase,we have added a Convolutional Block Attention Module(CBAM)to the backbone network.The Reparameterization Visual Geometry Group(RepVGG)module is inserted into the backbone to improve the training and inference capabilities.The Efficient Intersection over Union(EIoU)loss is also used as the localization loss function,which reduces the error detection rate and missed detection rate of the algorithm.The experimental results of the CER-YOLOv7 algorithm on the UPRC(Underwater Robot Prototype Competition)dataset show that the mAP(mean Average Precision)score of the algorithm is 86.1%,which is a 2.2%improvement compared to the YOLOv7.The feasibility and validity of the CER-YOLOv7 are proved through ablation and comparison experiments,and it is more suitable for underwater target detection.

Hybrid Gene Selection Methods for High-Dimensional Lung Cancer Data Using Improved Arithmetic Optimization Algorithm

Lung cancer is among the most frequent cancers in the world,with over one million deaths per year.Classification is required for lung cancer diagnosis and therapy to be effective,accurate,and reliable.Gene expression microarrays have made it possible to find genetic biomarkers for cancer diagnosis and prediction in a high-throughput manner.Machine Learning(ML)has been widely used to diagnose and classify lung cancer where the performance of ML methods is evaluated to identify the appropriate technique.Identifying and selecting the gene expression patterns can help in lung cancer diagnoses and classification.Normally,microarrays include several genes and may cause confusion or false prediction.Therefore,the Arithmetic Optimization Algorithm(AOA)is used to identify the optimal gene subset to reduce the number of selected genes.Which can allow the classifiers to yield the best performance for lung cancer classification.In addition,we proposed a modified version of AOA which can work effectively on the high dimensional dataset.In the modified AOA,the features are ranked by their weights and are used to initialize the AOA population.The exploitation process of AOA is then enhanced by developing a local search algorithm based on two neighborhood strategies.Finally,the efficiency of the proposed methods was evaluated on gene expression datasets related to Lung cancer using stratified 4-fold cross-validation.The method’s efficacy in selecting the optimal gene subset is underscored by its ability to maintain feature proportions between 10%to 25%.Moreover,the approach significantly enhances lung cancer prediction accuracy.For instance,Lung_Harvard1 achieved an accuracy of 97.5%,Lung_Harvard2 and Lung_Michigan datasets both achieved 100%,Lung_Adenocarcinoma obtained an accuracy of 88.2%,and Lung_Ontario achieved an accuracy of 87.5%.In conclusion,the results indicate the potential promise of the proposed modified AOA approach in classifying microarray cancer data.

Dynamic plugging regulating strategy of pipeline robot based on reinforcement learning

Pipeline isolation plugging robot (PIPR) is an important tool in pipeline maintenance operation. During the plugging process, the violent vibration will occur by the flow field, which can cause serious damage to the pipeline and PIPR. In this paper, we propose a dynamic regulating strategy to reduce the plugging-induced vibration by regulating the spoiler angle and plugging velocity. Firstly, the dynamic plugging simulation and experiment are performed to study the flow field changes during dynamic plugging. And the pressure difference is proposed to evaluate the degree of flow field vibration. Secondly, the mathematical models of pressure difference with plugging states and spoiler angles are established based on the extreme learning machine (ELM) optimized by improved sparrow search algorithm (ISSA). Finally, a modified Q-learning algorithm based on simulated annealing is applied to determine the optimal strategy for the spoiler angle and plugging velocity in real time. The results show that the proposed method can reduce the plugging-induced vibration by 19.9% and 32.7% on average, compared with single-regulating methods. This study can effectively ensure the stability of the plugging process.

Probabilistic Automata-Based Method for Enhancing Performance of Deep Reinforcement Learning Systems

Deep reinforcement learning(DRL) has demonstrated significant potential in industrial manufacturing domains such as workshop scheduling and energy system management.However, due to the model's inherent uncertainty, rigorous validation is requisite for its application in real-world tasks. Specific tests may reveal inadequacies in the performance of pre-trained DRL models, while the “black-box” nature of DRL poses a challenge for testing model behavior. We propose a novel performance improvement framework based on probabilistic automata,which aims to proactively identify and correct critical vulnerabilities of DRL systems, so that the performance of DRL models in real tasks can be improved with minimal model modifications.First, a probabilistic automaton is constructed from the historical trajectory of the DRL system by abstracting the state to generate probabilistic decision-making units(PDMUs), and a reverse breadth-first search(BFS) method is used to identify the key PDMU-action pairs that have the greatest impact on adverse outcomes. This process relies only on the state-action sequence and final result of each trajectory. Then, under the key PDMU, we search for the new action that has the greatest impact on favorable results. Finally, the key PDMU, undesirable action and new action are encapsulated as monitors to guide the DRL system to obtain more favorable results through real-time monitoring and correction mechanisms. Evaluations in two standard reinforcement learning environments and three actual job scheduling scenarios confirmed the effectiveness of the method, providing certain guarantees for the deployment of DRL models in real-world applications.

Improvement of learning and memory abilities and motor function in rats with cerebral infarction by intracerebral transplantation of neuron-like cells derived from bone marrow stromal cells

BACKGROUND： Transplantation of fetal cell suspension or blocks of fetal tissue can ameliorate the nerve function after the injury or disease in the central nervous system, and it has been used to treat neurodegenerative disorders induced by Parkinson disease. OBJECTIVE： To observe the effects of the transplantation of neuron-like cells derived from bone marrow stromal cells （rMSCs） into the brain in restoring the dysfunctions of muscle strength and balance as well as learning and memory in rat models of cerebral infarction. DESIGN ： A randomized controlled experiment.SETTING ： Department of Pathophysiology, Zhongshan Medical College of Sun Yat-sen University.MATERIALS ： Twenty-four male SD rats （3-4 weeks of age, weighing 200-220 g） were used in this study （Certification number：2001A027）. METHODS： The experiments were carried out in Zhongshan Medical College of Sun Yat-sen University between December 2003 and December 2004. ① Twenty-four male SD rats randomized into three groups with 8 rats in each： experimental group, control group and sham-operated group. Rats in the experiment al group and control group were induced into models of middle cerebral artery occlusion （MCAO）. After in vitro cultured, purified and identified with digestion, the Fischer344 rMSCs were induced to differentiate by tanshinone IIA, which was locally injected into the striate cortex （18 area） of rats in the experimental group, and the rats in the control group were injected by L-DMEM basic culture media （without serum） of the same volume to the corresponding brain area. In the sham-operated group, only muscle and vessel of neck were separated. ② At 2 and 8 weeks after the transplantation, the rats were given the screen test, prehensile-traction test, balance beam test and Morris water-maze test. ③ The survival and distribution of the induced cells in corresponding brain area were observed with Nissl stained with toluidine blue and hematoxylin and eosin （HE） staining in the groups.MAIN OUTCOME MEASURES ： ① Results of the behavioral tests （time of the Morris water-maze test screen test, prehensile-traction test, balance beam test）; ② Survival and distribution of the induced cells.RESULTS： All the 24 rats were involved in the analysis of results. ① Two weeks after transplantation, rats with neuron-like cells grafts in the experimental group had significant improvement on their general muscle strength than those in the control group [screen test： （9.4±1.7）, （4.7±1.0） s, P 〈 0.01]; forelimb muscle strength [prehensile-traction test： （7.6±1.4）, （5.2±1.2） s, P 〈 0.01], ability to keep balance [balance beam test： （7.9±0.74）, （6.1±0.91） s, P 〈 0.01] and abilities of learning and memory [latency to find the platform： （35.8±5.9）, （117.5±11.6） s, P 〈 0.01; distance： （623.1±43.4）, （1 902.3±98.6） cm, P 〈 0.01] as compared with those in the control group. The functional performances in the experimental group at 8 weeks were better than those at two weeks, which were still obviously different from those in the sham-operated group （P 〈 0.05）. ② The HE and Nissl stained brain tissue section showed that there was nerve cell proliferation at the infarcted cortex in the experiment group, the density was higher than that in the control group, plenty of aggregative or scattered cells could be observed at the site where needle was inserted for transplantation, the cells migrated directively towards the area around them, the cerebral vascular walls were wrapped by plenty of cells; In the control group, most of the cortices were destroyed, karyopyknosis and necrosis of neurons were observed, normal nervous tissue structure disappeared induced by edema, only some nerve fibers and glial cells remained.CONCLUSION： The rMSCs transplantation can obviously enhance the motor function and the abilities of learning and memory in rat models of cerebral infarction.

基于改进Q-learning算法的移动机器人路径规划

针对传统Q-learning算法应用在路径规划中存在收敛速度慢、运行时间长、学习效率差等问题,提出一种将人工势场法和传统Q-learning算法结合的改进Q-learning算法。该算法引入人工势场法的引力函数与斥力函数,通过对比引力函数动态选择奖励值,以及对比斥力函数计算姿值,动态更新Q值,使移动机器人具有目的性的探索,并且优先选择离障碍物较远的位置移动。通过仿真实验证明,与传统Q-learning算法、引入引力场算法对比,改进Q-learning算法加快了收敛速度,缩短了运行时间,提高了学习效率,降低了与障碍物相撞的概率,使移动机器人能够快速地找到一条无碰撞通路。

Studying Design and Use of Healthcare Technologies in Interaction: The Social Learning Perspective in a Dutch Quality Improvement Collaborative Program

Designing technologies is a process that relies on multiple interactions between design and use contexts. These interactions are essential to the development and establishment of technologies. This article seeks to understand the attempts of healthcare organisations to integrate use contexts into the design of healthcare technologies following insights of the theoretical approaches of social learning and user representations. We present a multiple case study of three healthcare technologies involved in improving elderly care practice. These cases were part of a Dutch quality improvement collaborative program, which urged that development of these technologies was not “just” development, but should occur in close collaboration with other parts of the collaborative program, which were more focused on implementation. These cases illustrate different ways to develop technologies in interaction with use contexts and users. Despite the infrastructure of the collaborative program, interactions were not without problems. We conclude by arguing that interactions between design and use are not naturally occurring phenomena, but must be actively organised in order to create effects.

A Study on the improvement strategies to Effectiveness of Online Teaching and Learning in EFL Classes in College

English online learning has been common trend in the world, how to teach and learn effectively in EFL classes through online environment is an urgent study. The purpose of the study to analyze the factors of affecting the effectiveness of online teaching and learning in EFL classes in college. We build up a three-dimensional model in the perspective of teacher, learner and technology. And we propose the strategies of improving the effectiveness of online teaching and learning in EFL classes in college in the dimensions of teacher, learner and technology.

Improvement of Stochastic Competitive Learning for Social Network

As an unsupervised learning method,stochastic competitive learning is commonly used for community detection in social network analysis.Compared with the traditional community detection algorithms,it has the advantage of realizing the time-series community detection by simulating the community formation process.In order to improve the accuracy and solve the problem that several parameters in stochastic competitive learning need to be pre-set,the author improves the algorithms and realizes improved stochastic competitive learning by particle position initialization,parameter optimization and particle domination ability self-adaptive.The experiment result shows that each improved method improves the accuracy of the algorithm,and the F1 score of the improved algorithm is 9.07%higher than that of original algorithm.

改进Q-Learning输电线路超声驱鸟设备参数优化研究

超声波驱鸟是一种解决输电设备鸟害的重要手段,但现场使用超声波驱鸟器工作模式较单一,易产生鸟类适应问题。提出了一种改进Q-Learning输电线路超声驱鸟设备参数优化方法,针对涉鸟故障历史数据量少以及鸟类的适应性问题,将强化学习算法应用于输电线路超声驱鸟设备参数优化;针对传统强化学习算法在设备终端应用中存在收敛慢、耗时长的缺点,提出一种基于动态学习率的改进Q-Learning算法,对不同频段超声波的权重进行自适应优化。实验结果显示,改进Q-Learning算法最优参数的迭代收敛速度大幅提高,优化后驱鸟设备的驱鸟成功率达到了76%,优于传统强化学习算法模式,较好地解决了鸟类适应性问题。

改进麻雀算法和Q-Learning优化集成学习轨道电路故障诊断

无绝缘轨道电路的故障具有复杂性与随机性,采用单一的模型进行故障诊断,其性能评价指标难以提高。而采用集成学习方式,则存在各基学习器结构、参数设计盲目,集成模型中各基学习器组合权重难以分配的问题。针对以上问题,提出一种改进麻雀算法和Q-Learning优化集成学习的轨道电路故障诊断新方法,该方法有机地将集成学习与计算智能和强化学习相结合,充分挖掘轨道电路故障特征,提高性能评价指标。首先,使用卷积神经网络、长短期记忆网络和多层感知器深度学习模型,以及支持向量机和随机森林传统机器学习模型,共同构成集成学习基学习器,解决单一学习模型的不足,不同基学习器的使用保证集成学习的多样性。从自动化机器学习角度出发,采用改进麻雀算法优化该集成学习模型的结构和参数,克服其结构和参数难以确定的问题。在此之上,引入强化学习Q-learning对集成模型中各基学习器组合权重进行优化,智能地确定集成学习各基学习器的组合权重。最后,将集成学习模型的预测结果与真实结果比较后得到误差,再采用BP神经网络对预测结果进行补偿修正,进一步提高轨道电路的故障诊断性能评价指标。仿真结果表明,利用所提方法进一步改善了轨道电路故障诊断的准确度、精确度、召回率和F1值等性能评价指标。