基于Q-Learning的分簇无线传感网信任管理机制

针对无线传感器网络中存在的安全问题,提出了基于Q-Learning的分簇无线传感网信任管理机制(Q-learning based trust management mechanism for clustered wireless sensor networks,QLTMM-CWSN).该机制主要考虑通信信任、数据信任和能量信任3个方面.在网络运行过程中,基于节点的通信行为、数据分布和能量消耗,使用Q-Learning算法更新节点信任值,并选择簇内信任值最高的节点作为可信簇头节点.当簇中主簇头节点的信任值低于阈值时,可信簇头节点代替主簇头节点管理簇内成员节点,维护正常的数据传输.研究结果表明,QLTMM-CWSN机制能有效抵御通信攻击、伪造本地数据攻击、能量攻击和混合攻击.

An Incentive Mechanism for Federated Learning:A Continuous Zero-Determinant Strategy Approach

As a representative emerging machine learning technique, federated learning(FL) has gained considerable popularity for its special feature of “making data available but not visible”. However, potential problems remain, including privacy breaches, imbalances in payment, and inequitable distribution.These shortcomings let devices reluctantly contribute relevant data to, or even refuse to participate in FL. Therefore, in the application of FL, an important but also challenging issue is to motivate as many participants as possible to provide high-quality data to FL. In this paper, we propose an incentive mechanism for FL based on the continuous zero-determinant(CZD) strategies from the perspective of game theory. We first model the interaction between the server and the devices during the FL process as a continuous iterative game. We then apply the CZD strategies for two players and then multiple players to optimize the social welfare of FL, for which we prove that the server can keep social welfare at a high and stable level. Subsequently, we design an incentive mechanism based on the CZD strategies to attract devices to contribute all of their high-accuracy data to FL.Finally, we perform simulations to demonstrate that our proposed CZD-based incentive mechanism can indeed generate high and stable social welfare in FL.

Prediction of lime utilization ratio of dephosphorization in BOF steelmaking based on online sequential extreme learning machine with forgetting mechanism

The machine learning models of multiple linear regression(MLR),support vector regression(SVR),and extreme learning ma-chine(ELM)and the proposed ELM models of online sequential ELM(OS-ELM)and OS-ELM with forgetting mechanism(FOS-ELM)are applied in the prediction of the lime utilization ratio of dephosphorization in the basic oxygen furnace steelmaking process.The ELM model exhibites the best performance compared with the models of MLR and SVR.OS-ELM and FOS-ELM are applied for sequential learning and model updating.The optimal number of samples in validity term of the FOS-ELM model is determined to be 1500,with the smallest population mean absolute relative error(MARE)value of 0.058226 for the population.The variable importance analysis reveals lime weight,initial P content,and hot metal weight as the most important variables for the lime utilization ratio.The lime utilization ratio increases with the decrease in lime weight and the increases in the initial P content and hot metal weight.A prediction system based on FOS-ELM is applied in actual industrial production for one month.The hit ratios of the predicted lime utilization ratio in the error ranges of±1%,±3%,and±5%are 61.16%,90.63%,and 94.11%,respectively.The coefficient of determination,MARE,and root mean square error are 0.8670,0.06823,and 1.4265,respectively.The system exhibits desirable performance for applications in actual industrial pro-duction.

Novel Static Security and Stability Control of Power Systems Based on Artificial Emotional Lazy Q-Learning

The stability problem of power grids has become increasingly serious in recent years as the size of novel power systems increases.In order to improve and ensure the stable operation of the novel power system,this study proposes an artificial emotional lazy Q-learning method,which combines artificial emotion,lazy learning,and reinforcement learning for static security and stability analysis of power systems.Moreover,this study compares the analysis results of the proposed method with those of the small disturbance method for a stand-alone power system and verifies that the proposed lazy Q-learning method is able to effectively screen useful data for learning,and improve the static security stability of the new type of power system more effectively than the traditional proportional-integral-differential control and Q-learning methods.

Decoding topological XYZ^(2) codes with reinforcement learning based on attention mechanisms

Quantum error correction, a technique that relies on the principle of redundancy to encode logical information into additional qubits to better protect the system from noise, is necessary to design a viable quantum computer. For this new topological stabilizer code-XYZ^(2) code defined on the cellular lattice, it is implemented on a hexagonal lattice of qubits and it encodes the logical qubits with the help of stabilizer measurements of weight six and weight two. However topological stabilizer codes in cellular lattice quantum systems suffer from the detrimental effects of noise due to interaction with the environment. Several decoding approaches have been proposed to address this problem. Here, we propose the use of a state-attention based reinforcement learning decoder to decode XYZ^(2) codes, which enables the decoder to more accurately focus on the information related to the current decoding position, and the error correction accuracy of our reinforcement learning decoder model under the optimisation conditions can reach 83.27% under the depolarizing noise model, and we have measured thresholds of 0.18856 and 0.19043 for XYZ^(2) codes at code spacing of 3–7 and 7–11, respectively. our study provides directions and ideas for applications of decoding schemes combining reinforcement learning attention mechanisms to other topological quantum error-correcting codes.

Enhancing Image Description Generation through Deep Reinforcement Learning:Fusing Multiple Visual Features and Reward Mechanisms

Image description task is the intersection of computer vision and natural language processing,and it has important prospects,including helping computers understand images and obtaining information for the visually impaired.This study presents an innovative approach employing deep reinforcement learning to enhance the accuracy of natural language descriptions of images.Our method focuses on refining the reward function in deep reinforcement learning,facilitating the generation of precise descriptions by aligning visual and textual features more closely.Our approach comprises three key architectures.Firstly,it utilizes Residual Network 101(ResNet-101)and Faster Region-based Convolutional Neural Network(Faster R-CNN)to extract average and local image features,respectively,followed by the implementation of a dual attention mechanism for intricate feature fusion.Secondly,the Transformer model is engaged to derive contextual semantic features from textual data.Finally,the generation of descriptive text is executed through a two-layer long short-term memory network(LSTM),directed by the value and reward functions.Compared with the image description method that relies on deep learning,the score of Bilingual Evaluation Understudy(BLEU-1)is 0.762,which is 1.6%higher,and the score of BLEU-4 is 0.299.Consensus-based Image Description Evaluation(CIDEr)scored 0.998,Recall-Oriented Understudy for Gisting Evaluation(ROUGE)scored 0.552,the latter improved by 0.36%.These results not only attest to the viability of our approach but also highlight its superiority in the realm of image description.Future research can explore the integration of our method with other artificial intelligence(AI)domains,such as emotional AI,to create more nuanced and context-aware systems.

Reinforcement learning based adaptive control for uncertain mechanical systems with asymptotic tracking

This paper mainly focuses on the development of a learning-based controller for a class of uncertain mechanical systems modeled by the Euler-Lagrange formulation.The considered system can depict the behavior of a large class of engineering systems,such as vehicular systems,robot manipulators and satellites.All these systems are often characterized by highly nonlinear characteristics,heavy modeling uncertainties and unknown perturbations,therefore,accurate-model-based nonlinear control approaches become unavailable.Motivated by the challenge,a reinforcement learning(RL)adaptive control methodology based on the actor-critic framework is investigated to compensate the uncertain mechanical dynamics.The approximation inaccuracies caused by RL and the exogenous unknown disturbances are circumvented via a continuous robust integral of the sign of the error(RISE)control approach.Different from a classical RISE control law,a tanh(·)function is utilized instead of a sign(·)function to acquire a more smooth control signal.The developed controller requires very little prior knowledge of the dynamic model,is robust to unknown dynamics and exogenous disturbances,and can achieve asymptotic output tracking.Eventually,co-simulations through ADAMS and MATLAB/Simulink on a three degrees-of-freedom(3-DOF)manipulator and experiments on a real-time electromechanical servo system are performed to verify the performance of the proposed approach.

Development of a Bias Compensating Q-Learning Controller for a Multi-Zone HVAC Facility

We present the development of a bias compensating reinforcement learning(RL)algorithm that optimizes thermal comfort(by minimizing tracking error)and control utilization(by penalizing setpoint deviations)in a multi-zone heating,ventilation,and air-conditioning(HVAC)lab facility subject to unmeasurable disturbances and unknown dynamics.It is shown that the presence of unmeasurable disturbance results in an inconsistent learning equation in traditional RL controllers leading to parameter estimation bias(even with integral action support),and in the extreme case,the divergence of the learning algorithm.We demonstrate this issue by applying the popular Q-learning algorithm to linear quadratic regulation(LQR)of a multi-zone HVAC environment and showing that,even with integral support,the algorithm exhibits bias issue during the learning phase when the HVAC disturbance is unmeasurable due to unknown heat gains,occupancy variations,light sources,and outside weather changes.To address this difficulty,we present a bias compensating learning equation that learns a lumped bias term as a result of disturbances(and possibly other sources)in conjunction with the optimal control parameters.Experimental results show that the proposed scheme not only recovers the bias-free optimal control parameters but it does so without explicitly learning the dynamic model or estimating the disturbances,demonstrating the effectiveness of the algorithm in addressing the above challenges.

Gas liquid cylindrical cyclone flow regime identification using machine learning combined with experimental mechanism explanation

The flow regimes of GLCC with horizon inlet and a vertical pipe are investigated in experiments,and the velocities and pressure drops data labeled by the corresponding flow regimes are collected.Combined with the flow regimes data of other GLCC positions from other literatures in existence,the gas and liquid superficial velocities and pressure drops are used as the input of the machine learning algorithms respectively which are applied to identify the flow regimes.The choosing of input data types takes the availability of data for practical industry fields into consideration,and the twelve machine learning algorithms are chosen from the classical and popular algorithms in the area of classification,including the typical ensemble models,SVM,KNN,Bayesian Model and MLP.The results of flow regimes identification show that gas and liquid superficial velocities are the ideal type of input data for the flow regimes identification by machine learning.Most of the ensemble models can identify the flow regimes of GLCC by gas and liquid velocities with the accuracy of 0.99 and more.For the pressure drops as the input of each algorithm,it is not the suitable as gas and liquid velocities,and only XGBoost and Bagging Tree can identify the GLCC flow regimes accurately.The success and confusion of each algorithm are analyzed and explained based on the experimental phenomena of flow regimes evolution processes,the flow regimes map,and the principles of algorithms.The applicability and feasibility of each algorithm according to different types of data for GLCC flow regimes identification are proposed.

A dynamic incentive and reputation mechanism for energy-efficient federated learning in 6G

As 5G becomes commercial,researchers have turned attention toward the Sixth-Generation(6G)network with the vision of connecting intelligence in a green energy-efficient manner.Federated learning triggers an upsurge of green intelligent services such as resources orchestration of communication infrastructures while preserving privacy and increasing communication efficiency.However,designing effective incentives in federated learning is challenging due to the dynamic available clients and the correlation between clients'contributions during the learning process.In this paper,we propose a dynamic incentive and reputation mechanism to improve energy efficiency and training performance of federated learning.The proposed incentive based on the Stackelberg game can timely adjust optimal energy consumption with changes in available clients during federated learning.Meanwhile,clients’contributions in reputation management are formulated based on the cooperative game to capture the correlation between tasks,which satisfies availability,fairness,and additivity.The simulation results show that the proposed scheme can significantly motivate high-performance clients to participate in federated learning and improve the accuracy and energy efficiency of the federated learning model.

Exploration on the mechanism of therapeutic and toxic bidirectional effects of Haizao Yuhu decoction based on machine learning and data mining

Objective:To explore the bidirectional mechanism of Haizao Yuhu decoction(HYD)on goiter and drug-induced liver injury(DILI)based on machine learning and data mining.Methods:Firstly,compounds of HYD were selected from the TCMSP,TCMIP,and BATMAN databases,then the TCMSP was used to acquire the targets of compounds.Targets of goiter and DILI were obtained from the GeneCards database.Secondly,common targets of“HYD-goiter”and“HYD-DILI”as well as related compounds were used to construct the networks and perform Random Walk with Restart(RWR)algorithm and network stability test.Finally,core targets in the“HYD-goiter”and“HYD-DILI”networks were used for molecular docking with core compounds and searched for validation on PubChem,and the relevant experimental data of our group were quoted to verify the analysis results.Results:There were 22 intersection targets of HYD and DILI,326 of HYD and goiter.RWR analysis showed that MAPK1,MAPK3,AKT1,etc.may be the core targets of HYD treating goiter,RELA,TNF,IL4,etc.may be the core targets of the bidirectional effect,and eckol may be the core compound in bidirectional effect.Network stability test indicated that the HYD had a high stability on treating goiter and playing a bidirectional effect.The core targets and core compounds docked well,and 37.3%of targets had been confirmed by experiments and 29.8%core targets had been confirmed.Our previous experimental result confirmed that the HYD could treat goiter usefully by reducing the expression levels of PI3K and AKT mRNA,and down-regulating the expression of Cyclin D1 and Bcl-2 mRNA.Conclusion:HYD containing“sargassum-liquorice”combination may have a bidirectional effect on treating goiter and causing DILI.We offered a new way for more explorations on the therapeutic and toxic bidirectional mechanisms based on machine learning and data mining.

Machine Learning‑Enhanced Flexible Mechanical Sensing

To realize a hyperconnected smart society with high productivity,advances in flexible sensing technology are highly needed.Nowadays,flexible sensing technology has witnessed improvements in both the hardware performances of sensor devices and the data processing capabilities of the device’s software.Significant research efforts have been devoted to improving materials,sensing mechanism,and configurations of flexible sensing systems in a quest to fulfill the requirements of future technology.Meanwhile,advanced data analysis methods are being developed to extract useful information from increasingly complicated data collected by a single sensor or network of sensors.Machine learning(ML)as an important branch of artificial intelligence can efficiently handle such complex data,which can be multi-dimensional and multi-faceted,thus providing a powerful tool for easy interpretation of sensing data.In this review,the fundamental working mechanisms and common types of flexible mechanical sensors are firstly presented.Then how ML-assisted data interpretation improves the applications of flexible mechanical sensors and other closely-related sensors in various areas is elaborated,which includes health monitoring,human-machine interfaces,object/surface recognition,pressure prediction,and human posture/motion identification.Finally,the advantages,challenges,and future perspectives associated with the fusion of flexible mechanical sensing technology and ML algorithms are discussed.These will give significant insights to enable the advancement of next-generation artificial flexible mechanical sensing.

Transfer learning framework for multi-scale crack type classification with sparse microseismic networks

Rock fracture mechanisms can be inferred from moment tensors(MT)inverted from microseismic events.However,MT can only be inverted for events whose waveforms are acquired across a network of sensors.This is limiting for underground mines where the microseismic stations often lack azimuthal coverage.Thus,there is a need for a method to invert fracture mechanisms using waveforms acquired by a sparse microseismic network.Here,we present a novel,multi-scale framework to classify whether a rock crack contracts or dilates based on a single waveform.The framework consists of a deep learning model that is initially trained on 2400000+manually labelled field-scale seismic and microseismic waveforms acquired across 692 stations.Transfer learning is then applied to fine-tune the model on 300000+MT-labelled labscale acoustic emission waveforms from 39 individual experiments instrumented with different sensor layouts,loading,and rock types in training.The optimal model achieves over 86%F-score on unseen waveforms at both the lab-and field-scale.This model outperforms existing empirical methods in classification of rock fracture mechanisms monitored by a sparse microseismic network.This facilitates rapid assessment of,and early warning against,various rock engineering hazard such as induced earthquakes and rock bursts.

Enhancing Deep Learning Soil Moisture Forecasting Models by Integrating Physics-based Models

Accurate soil moisture(SM)prediction is critical for understanding hydrological processes.Physics-based(PB)models exhibit large uncertainties in SM predictions arising from uncertain parameterizations and insufficient representation of land-surface processes.In addition to PB models,deep learning(DL)models have been widely used in SM predictions recently.However,few pure DL models have notably high success rates due to lacking physical information.Thus,we developed hybrid models to effectively integrate the outputs of PB models into DL models to improve SM predictions.To this end,we first developed a hybrid model based on the attention mechanism to take advantage of PB models at each forecast time scale(attention model).We further built an ensemble model that combined the advantages of different hybrid schemes(ensemble model).We utilized SM forecasts from the Global Forecast System to enhance the convolutional long short-term memory(ConvLSTM)model for 1–16 days of SM predictions.The performances of the proposed hybrid models were investigated and compared with two existing hybrid models.The results showed that the attention model could leverage benefits of PB models and achieved the best predictability of drought events among the different hybrid models.Moreover,the ensemble model performed best among all hybrid models at all forecast time scales and different soil conditions.It is highlighted that the ensemble model outperformed the pure DL model over 79.5%of in situ stations for 16-day predictions.These findings suggest that our proposed hybrid models can adequately exploit the benefits of PB model outputs to aid DL models in making SM predictions.

Automatic recognition of landslides based on YOLOv7 and attention mechanism

Landslide disasters comprise the majority of geological incidents on slopes,posing severe threats to the safety of human lives and property while exerting a significant impact on the geological environment.The rapid identification of landslides is important for disaster prevention and control;however,currently,landslide identification relies mainly on the manual interpretation of remote sensing images.Manual interpretation and feature recognition methods are time-consuming,labor-intensive,and challenging when confronted with complex scenarios.Consequently,automatic landslide recognition has emerged as a pivotal avenue for future development.In this study,a dataset comprising 2000 landslide images was constructed using open-source remote sensing images and datasets.The YOLOv7 model was enhanced using data augmentation algorithms and attention mechanisms.Three optimization models were formulated to realize automatic landslide recognition.The findings demonstrate the commendable performance of the optimized model in automatic landslide recognition,achieving a peak accuracy of 95.92%.Subsequently,the optimized model was applied to regional landslide identification,co-seismic landslide identification,and landslide recognition at various scales,all of which showed robust recognition capabilities.Nevertheless,the model exhibits limitations in detecting small targets,indicating areas for refining the deep-learning algorithms.The results of this research offer valuable technical support for the swift identification,prevention,and mitigation of landslide disasters.

Dynamic plugging regulating strategy of pipeline robot based on reinforcement learning

Pipeline isolation plugging robot (PIPR) is an important tool in pipeline maintenance operation. During the plugging process, the violent vibration will occur by the flow field, which can cause serious damage to the pipeline and PIPR. In this paper, we propose a dynamic regulating strategy to reduce the plugging-induced vibration by regulating the spoiler angle and plugging velocity. Firstly, the dynamic plugging simulation and experiment are performed to study the flow field changes during dynamic plugging. And the pressure difference is proposed to evaluate the degree of flow field vibration. Secondly, the mathematical models of pressure difference with plugging states and spoiler angles are established based on the extreme learning machine (ELM) optimized by improved sparrow search algorithm (ISSA). Finally, a modified Q-learning algorithm based on simulated annealing is applied to determine the optimal strategy for the spoiler angle and plugging velocity in real time. The results show that the proposed method can reduce the plugging-induced vibration by 19.9% and 32.7% on average, compared with single-regulating methods. This study can effectively ensure the stability of the plugging process.

An Image Fingerprint and Attention Mechanism Based Load Estimation Algorithm for Electric Power System

With the rapid development of electric power systems,load estimation plays an important role in system operation and planning.Usually,load estimation techniques contain traditional,time series,regression analysis-based,and machine learning-based estimation.Since the machine learning-based method can lead to better performance,in this paper,a deep learning-based load estimation algorithm using image fingerprint and attention mechanism is proposed.First,an image fingerprint construction is proposed for training data.After the data preprocessing,the training data matrix is constructed by the cyclic shift and cubic spline interpolation.Then,the linear mapping and the gray-color transformation method are proposed to form the color image fingerprint.Second,a convolutional neural network(CNN)combined with an attentionmechanism is proposed for training performance improvement.At last,an experiment is carried out to evaluate the estimation performance.Compared with the support vector machine method,CNN method and long short-term memory method,the proposed algorithm has the best load estimation performance.

An Improved Solov2 Based on Attention Mechanism and Weighted Loss Function for Electrical Equipment Instance Segmentation

The current existing problem of deep learning framework for the detection and segmentation of electrical equipment is dominantly related to low precision.Because of the reliable,safe and easy-to-operate technology provided by deep learning-based video surveillance for unmanned inspection of electrical equipment,this paper uses the bottleneck attention module(BAM)attention mechanism to improve the Solov2 model and proposes a new electrical equipment segmentation mode.Firstly,the BAM attention mechanism is integrated into the feature extraction network to adaptively learn the correlation between feature channels,thereby improving the expression ability of the feature map;secondly,the weighted sum of CrossEntropy Loss and Dice loss is designed as the mask loss to improve the segmentation accuracy and robustness of the model;finally,the non-maximal suppression(NMS)algorithm to better handle the overlap problem in instance segmentation.Experimental results show that the proposed method achieves an average segmentation accuracy of mAP of 80.4% on three types of electrical equipment datasets,including transformers,insulators and voltage transformers,which improve the detection accuracy by more than 5.7% compared with the original Solov2 model.The segmentation model proposed can provide a focusing technical means for the intelligent management of power systems.

Multimodal Sentiment Analysis Based on a Cross-Modal Multihead Attention Mechanism

Multimodal sentiment analysis aims to understand people’s emotions and opinions from diverse data.Concate-nating or multiplying various modalities is a traditional multi-modal sentiment analysis fusion method.This fusion method does not utilize the correlation information between modalities.To solve this problem,this paper proposes amodel based on amulti-head attention mechanism.First,after preprocessing the original data.Then,the feature representation is converted into a sequence of word vectors and positional encoding is introduced to better understand the semantic and sequential information in the input sequence.Next,the input coding sequence is fed into the transformer model for further processing and learning.At the transformer layer,a cross-modal attention consisting of a pair of multi-head attention modules is employed to reflect the correlation between modalities.Finally,the processed results are input into the feedforward neural network to obtain the emotional output through the classification layer.Through the above processing flow,the model can capture semantic information and contextual relationships and achieve good results in various natural language processing tasks.Our model was tested on the CMU Multimodal Opinion Sentiment and Emotion Intensity(CMU-MOSEI)and Multimodal EmotionLines Dataset(MELD),achieving an accuracy of 82.04% and F1 parameters reached 80.59% on the former dataset.

An Underwater Target Detection Algorithm Based on Attention Mechanism and Improved YOLOv7

For underwater robots in the process of performing target detection tasks,the color distortion and the uneven quality of underwater images lead to great difficulties in the feature extraction process of the model,which is prone to issues like error detection,omission detection,and poor accuracy.Therefore,this paper proposed the CER-YOLOv7(CBAM-EIOU-RepVGG-YOLOv7)underwater target detection algorithm.To improve the algorithm’s capability to retain valid features from both spatial and channel perspectives during the feature extraction phase,we have added a Convolutional Block Attention Module(CBAM)to the backbone network.The Reparameterization Visual Geometry Group(RepVGG)module is inserted into the backbone to improve the training and inference capabilities.The Efficient Intersection over Union(EIoU)loss is also used as the localization loss function,which reduces the error detection rate and missed detection rate of the algorithm.The experimental results of the CER-YOLOv7 algorithm on the UPRC(Underwater Robot Prototype Competition)dataset show that the mAP(mean Average Precision)score of the algorithm is 86.1%,which is a 2.2%improvement compared to the YOLOv7.The feasibility and validity of the CER-YOLOv7 are proved through ablation and comparison experiments,and it is more suitable for underwater target detection.