Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

Prediction of collapse process and tipping points for mutualistic and competitive networks with k-core method

Ecosystems generally have the self-adapting ability to resist various external pressures or disturbances,which is always called resilience.However,once the external disturbances exceed the tipping points of the system resilience,the consequences would be catastrophic,and eventually lead the ecosystem to complete collapse.We capture the collapse process of ecosystems represented by plant-pollinator networks with the k-core nested structural method,and find that a sufficiently weak interaction strength or a sufficiently large competition weight can cause the structure of the ecosystem to collapse from its smallest k-core towards its largest k-core.Then we give the tipping points of structure and dynamic collapse of the entire system from the one-dimensional dynamic function of the ecosystem.Our work provides an intuitive and precise description of the dynamic process of ecosystem collapse under multiple interactions,and provides theoretical insights into further avoiding the occurrence of ecosystem collapse.

Influencer identification of dynamical networks based on an information entropy dimension reduction method

Identifying critical nodes or sets in large-scale networks is a fundamental scientific problem and one of the key research directions in the fields of data mining and network science when implementing network attacks, defense, repair and control.Traditional methods usually begin from the centrality, node location or the impact on the largest connected component after node destruction, mainly based on the network structure.However, these algorithms do not consider network state changes.We applied a model that combines a random connectivity matrix and minimal low-dimensional structures to represent network connectivity.By using mean field theory and information entropy to calculate node activity,we calculated the overlap between the random parts and fixed low-dimensional parts to quantify the influence of node impact on network state changes and ranked them by importance.We applied this algorithm and the proposed importance algorithm to the overall analysis and stratified analysis of the C.elegans neural network.We observed a change in the critical entropy of the network state and by utilizing the proposed method we can calculate the nodes that indirectly affect muscle cells through neural layers.

Online Learning-Based Offloading Decision and Resource Allocation in Mobile Edge Computing-Enabled Satellite-Terrestrial Networks

Mobile edge computing(MEC)-enabled satellite-terrestrial networks(STNs)can provide Internet of Things(IoT)devices with global computing services.Sometimes,the network state information is uncertain or unknown.To deal with this situation,we investigate online learning-based offloading decision and resource allocation in MEC-enabled STNs in this paper.The problem of minimizing the average sum task completion delay of all IoT devices over all time periods is formulated.We decompose this optimization problem into a task offloading decision problem and a computing resource allocation problem.A joint optimization scheme of offloading decision and resource allocation is then proposed,which consists of a task offloading decision algorithm based on the devices cooperation aided upper confidence bound(UCB)algorithm and a computing resource allocation algorithm based on the Lagrange multiplier method.Simulation results validate that the proposed scheme performs better than other baseline schemes.

Detection of UAV Target Based on Continuous Radon Transform and Matched Filtering Process for Passive Bistatic Radar

Long-time integration technique is an effective way of improving target detection performance for unmanned aerial vehicle(UAV)in the passive bistatic radar(PBR),while range migration(RM)and Doppler frequency migration(DFM)may have a major effect due to the target maneuverability.This paper proposed an innovative long-time coherent integration approach,regarded as Continuous Radon-matched filtering process(CRMFP),for low-observable UAV target in passive bistatic radar.It not only mitigates the RM by collaborative research in range and velocity dimensions but also compensates the DFM and ensures the coherent integration through the matched filtering process(MFP).Numerical and real-life data following detailed analysis verify that the proposed method can overcome the Doppler mismatch influence and acquire comparable detection performance.

Detection of EEG signals in normal and epileptic seizures with multiscale multifractal analysis approach via weighted horizontal visibility graph

Electroencephalogram(EEG) signals contain important information about the regulation of brain system. Thus, automatic detection of epilepsy by analyzing the characteristics obtained from EEG signals has important research implications in the field of clinical medicine. In this paper, the horizontal visibility graph(HVG) algorithm is used to map multifractal EEG signals into complex networks. Then, we study the structure of the networks and explore the nonlinear dynamics properties of the EEG signals inherited from these networks. In order to better describe complex brain behaviors, we use the angle between two connected nodes as the edge weight of the network and construct the weighted horizontal visibility graph(WHVG). In our studies, fractality and multifractality of WHVG are innovatively used to analyze the structure of related networks. However, these methods only analyze the reconstructed dynamical system in general characterizations,they are not sufficient to describe the complex behavior and cannot provide a comprehensive picture of the system. To this effect, we propose an improved multiscale multifractal analysis(MMA) for network, which extends the description of the network dynamics features by focusing on the relationship between the multifractality and the measured scale-free intervals.Furthermore, neural networks are applied to train the above-mentioned parameters for the classification and identification of three kinds of EEG signals, i.e., health, interictal phase, and ictal phase. By evaluating our experimental results, the classification accuracy is 99.0%, reflecting the effectiveness of the WHVG algorithm in extracting the potential dynamic characteristics of EEG signals.

Channel Capacity and Power Allocation of MIMO Visible Light Communication System

In this paper,the channel capacity of the multiple-input multiple-output(MIMO)visible light communication(VLC)system is investigated under the peak,average optical and electrical power constraints.Finding the channel capacity of MIMO VLC is shown to be a mixed integer programming problem.To address this open problem,we propose an inexact gradient projection method to find the channel capacity-achieving discrete input distribution and the channel capacity of MIMO VLC.Also we derive both upper and lower bounds of the capacity of MIMO VLC with the closed-form expressions.Furthermore,by considering practical discrete constellation inputs,we develop the optimal power allocation scheme to maximize transmission rate of MIMO VLC system.Simulation results show that more discrete points are needed to achieve the channel capacity as SNR increases.Both the upper and lower bounds of channel capacity are tight at low SNR region.In addition,comparing the equal power allocation,the proposed power allocation scheme can significantly increase the rate for the low-order modulation inputs.

Detection of healthy and pathological heartbeat dynamics in ECG signals using multivariate recurrence networks with multiple scale factors

The electrocardiogram(ECG)is one of the physiological signals applied in medical clinics to determine health status.The physiological complexity of the cardiac system is related to age,disease,etc.For the investigation of the effects of age and cardiovascular disease on the cardiac system,we then construct multivariate recurrence networks with multiple scale factors from multivariate time series.We propose a new concept of cross-clustering coefficient entropy to construct a weighted network,and calculate the average weighted path length and the graph energy of the weighted network to quantitatively probe the topological properties.The obtained results suggest that these two network measures show distinct changes between different subjects.This is because,with aging or cardiovascular disease,a reduction in the conductivity or structural changes in the myocardium of the heart contributes to a reduction in the complexity of the cardiac system.Consequently,the complexity of the cardiac system is reduced.After that,the support vector machine(SVM)classifier is adopted to evaluate the performance of the proposed approach.Accuracy of 94.1%and 95.58%between healthy and myocardial infarction is achieved on two datasets.Therefore,this method can be adopted for the development of a noninvasive and low-cost clinical prognostic system to identify heart-related diseases and detect hidden state changes in the cardiac system.

The Construction of Linux System and Network Programming Course to Develop Engineering Ability

To meet society’s needs for undergraduate students to have engineering skills and to develop students’ability to operate Linux and engage in network software development,this paper proposes the construction of a new specialized course for network engineering major--Linux system and network programming.This paper analyzes the course’s advantages,presents the contents of this course,designs a series of teaching methods aimed at improving students’engineering ability,proposes a course assessment method that will encourage students to practice,lists the development requirements for an examination software designed for this course,and finally,presents the results of our practice in teaching this course.

Design of a novel hybrid quantum deep neural network in INEQR images classification

We redesign the parameterized quantum circuit in the quantum deep neural network, construct a three-layer structure as the hidden layer, and then use classical optimization algorithms to train the parameterized quantum circuit, thereby propose a novel hybrid quantum deep neural network(HQDNN) used for image classification. After bilinear interpolation reduces the original image to a suitable size, an improved novel enhanced quantum representation(INEQR) is used to encode it into quantum states as the input of the HQDNN. Multi-layer parameterized quantum circuits are used as the main structure to implement feature extraction and classification. The output results of parameterized quantum circuits are converted into classical data through quantum measurements and then optimized on a classical computer. To verify the performance of the HQDNN, we conduct binary classification and three classification experiments on the MNIST(Modified National Institute of Standards and Technology) data set. In the first binary classification, the accuracy of 0 and 4 exceeds98%. Then we compare the performance of three classification with other algorithms, the results on two datasets show that the classification accuracy is higher than that of quantum deep neural network and general quantum convolutional neural network.

Remote sensing image encryption algorithm based on novel hyperchaos and an elliptic curve cryptosystem

Remote sensing images carry crucial ground information,often involving the spatial distribution and spatiotemporal changes of surface elements.To safeguard this sensitive data,image encryption technology is essential.In this paper,a novel Fibonacci sine exponential map is designed,the hyperchaotic performance of which is particularly suitable for image encryption algorithms.An encryption algorithm tailored for handling the multi-band attributes of remote sensing images is proposed.The algorithm combines a three-dimensional synchronized scrambled diffusion operation with chaos to efficiently encrypt multiple images.Moreover,the keys are processed using an elliptic curve cryptosystem,eliminating the need for an additional channel to transmit the keys,thus enhancing security.Experimental results and algorithm analysis demonstrate that the algorithm offers strong security and high efficiency,making it suitable for remote sensing image encryption tasks.

Recurrent neural network decoding of rotated surface codes based on distributed strategy

Quantum error correction is a crucial technology for realizing quantum computers.These computers achieve faulttolerant quantum computing by detecting and correcting errors using decoding algorithms.Quantum error correction using neural network-based machine learning methods is a promising approach that is adapted to physical systems without the need to build noise models.In this paper,we use a distributed decoding strategy,which effectively alleviates the problem of exponential growth of the training set required for neural networks as the code distance of quantum error-correcting codes increases.Our decoding algorithm is based on renormalization group decoding and recurrent neural network decoder.The recurrent neural network is trained through the ResNet architecture to improve its decoding accuracy.Then we test the decoding performance of our distributed strategy decoder,recurrent neural network decoder,and the classic minimum weight perfect matching(MWPM)decoder for rotated surface codes with different code distances under the circuit noise model,the thresholds of these three decoders are about 0.0052,0.0051,and 0.0049,respectively.Our results demonstrate that the distributed strategy decoder outperforms the other two decoders,achieving approximately a 5%improvement in decoding efficiency compared to the MWPM decoder and approximately a 2%improvement compared to the recurrent neural network decoder.

Survey and Prospect for Applying Knowledge Graph in Enterprise Risk Management

Enterprise risk management holds significant importance in fostering sustainable growth of businesses and in serving as a critical element for regulatory bodies to uphold market order.Amidst the challenges posed by intricate and unpredictable risk factors,knowledge graph technology is effectively driving risk management,leveraging its ability to associate and infer knowledge from diverse sources.This review aims to comprehensively summarize the construction techniques of enterprise risk knowledge graphs and their prominent applications across various business scenarios.Firstly,employing bibliometric methods,the aim is to uncover the developmental trends and current research hotspots within the domain of enterprise risk knowledge graphs.In the succeeding section,systematically delineate the technical methods for knowledge extraction and fusion in the standardized construction process of enterprise risk knowledge graphs.Objectively comparing and summarizing the strengths and weaknesses of each method,we provide recommendations for addressing the existing challenges in the construction process.Subsequently,categorizing the applied research of enterprise risk knowledge graphs based on research hotspots and risk category standards,and furnishing a detailed exposition on the applicability of technical routes and methods.Finally,the future research directions that still need to be explored in enterprise risk knowledge graphs were discussed,and relevant improvement suggestions were proposed.Practitioners and researchers can gain insights into the construction of technical theories and practical guidance of enterprise risk knowledge graphs based on this foundation.

Adaptive Optimal Output Regulation of Interconnected Singularly Perturbed Systems With Application to Power Systems

This article studies the adaptive optimal output regulation problem for a class of interconnected singularly perturbed systems(SPSs) with unknown dynamics based on reinforcement learning(RL).Taking into account the slow and fast characteristics among system states,the interconnected SPS is decomposed into the slow time-scale dynamics and the fast timescale dynamics through singular perturbation theory.For the fast time-scale dynamics with interconnections,we devise a decentralized optimal control strategy by selecting appropriate weight matrices in the cost function.For the slow time-scale dynamics with unknown system parameters,an off-policy RL algorithm with convergence guarantee is given to learn the optimal control strategy in terms of measurement data.By combining the slow and fast controllers,we establish the composite decentralized adaptive optimal output regulator,and rigorously analyze the stability and optimality of the closed-loop system.The proposed decomposition design not only bypasses the numerical stiffness but also alleviates the high-dimensionality.The efficacy of the proposed methodology is validated by a load-frequency control application of a two-area power system.

An Improved UNet Lightweight Network for Semantic Segmentation of Weed Images in Corn Fields

In cornfields,factors such as the similarity between corn seedlings and weeds and the blurring of plant edge details pose challenges to corn and weed segmentation.In addition,remote areas such as farmland are usually constrained by limited computational resources and limited collected data.Therefore,it becomes necessary to lighten the model to better adapt to complex cornfield scene,and make full use of the limited data information.In this paper,we propose an improved image segmentation algorithm based on unet.Firstly,the inverted residual structure is introduced into the contraction path to reduce the number of parameters in the training process and improve the feature extraction ability;secondly,the pyramid pooling module is introduced to enhance the network’s ability of acquiring contextual information as well as the ability of dealing with the small target loss problem;and lastly,Finally,to further enhance the segmentation capability of the model,the squeeze and excitation mechanism is introduced in the expansion path.We used images of corn seedlings collected in the field and publicly available corn weed datasets to evaluate the improved model.The improved model has a total parameter of 3.79 M and miou can achieve 87.9%.The fps on a single 3050 ti video card is about 58.9.The experimental results show that the network proposed in this paper can quickly segment corn weeds in a cornfield scenario with good segmentation accuracy.

Analysis of learnability of a novel hybrid quantum-classical convolutional neural network in image classification

We design a new hybrid quantum-classical convolutional neural network(HQCCNN)model based on parameter quantum circuits.In this model,we use parameterized quantum circuits(PQCs)to redesign the convolutional layer in classical convolutional neural networks,forming a new quantum convolutional layer to achieve unitary transformation of quantum states,enabling the model to more accurately extract hidden information from images.At the same time,we combine the classical fully connected layer with PQCs to form a new hybrid quantum-classical fully connected layer to further improve the accuracy of classification.Finally,we use the MNIST dataset to test the potential of the HQCCNN.The results indicate that the HQCCNN has good performance in solving classification problems.In binary classification tasks,the classification accuracy of numbers 5 and 7 is as high as 99.71%.In multivariate classification,the accuracy rate also reaches 98.51%.Finally,we compare the performance of the HQCCNN with other models and find that the HQCCNN has better classification performance and convergence speed.

Analysis of multiple-faults of high-voltage circuit breakers based on non-negative matrix decomposition

High-voltage circuit breakers are the core equipment in power networks,and to a certain extent,are related to the safe and reliable operation of power systems.However,their core components are prone to mechanical faults.This study proposes a component separation method to detect multiple mechanical faults in circuit breakers that can achieve online real-time monitoring.First,a model and strategy are presented for obtaining mechanical voiceprint signals from circuit breakers.Subsequently,the component separation method was used to decompose the voiceprint signals of multiple faults into individual component signals.Based on this,the recognition of the features of a single-fault voiceprint signal can be achieved.Finally,multiple faults in high-voltage circuit breakers were identified through an experimental simulation and verification of the circuit breaker voiceprint signals collected from the substation site.The research results indicate that the proposed method exhibits excellent performance for multiple mechanical faults,such as spring structures and loose internal components of circuit breakers.In addition,it provides a reference method for the real-time online monitoring of high-voltage circuit breakers.

Hybrid Optimization Algorithm for Handwritten Document Enhancement

The Gannet Optimization Algorithm (GOA) and the Whale Optimization Algorithm (WOA) demonstrate strong performance;however, there remains room for improvement in convergence and practical applications. This study introduces a hybrid optimization algorithm, named the adaptive inertia weight whale optimization algorithm and gannet optimization algorithm (AIWGOA), which addresses challenges in enhancing handwritten documents. The hybrid strategy integrates the strengths of both algorithms, significantly enhancing their capabilities, whereas the adaptive parameter strategy mitigates the need for manual parameter setting. By amalgamating the hybrid strategy and parameter-adaptive approach, the Gannet Optimization Algorithm was refined to yield the AIWGOA. Through a performance analysis of the CEC2013 benchmark, the AIWGOA demonstrates notable advantages across various metrics. Subsequently, an evaluation index was employed to assess the enhanced handwritten documents and images, affirming the superior practical application of the AIWGOA compared with other algorithms.

Research on Performance Optimization of Spark Distributed Computing Platform

Spark,a distributed computing platform,has rapidly developed in the field of big data.Its in-memory computing feature reduces disk read overhead and shortens data processing time,making it have broad application prospects in large-scale computing applications such as machine learning and image processing.However,the performance of the Spark platform still needs to be improved.When a large number of tasks are processed simultaneously,Spark’s cache replacementmechanismcannot identify high-value data partitions,resulting inmemory resources not being fully utilized and affecting the performance of the Spark platform.To address the problem that Spark’s default cache replacement algorithm cannot accurately evaluate high-value data partitions,firstly the weight influence factors of data partitions are modeled and evaluated.Then,based on this weighted model,a cache replacement algorithm based on dynamic weighted data value is proposed,which takes into account hit rate and data difference.Better integration and usage strategies are implemented based on LRU(LeastRecentlyUsed).Theweight update algorithm updates the weight value when the data partition information changes,accurately measuring the importance of the partition in the current job;the cache removal algorithm clears partitions without useful values in the cache to releasememory resources;the weight replacement algorithm combines partition weights and partition information to replace RDD partitions when memory remaining space is insufficient.Finally,by setting up a Spark cluster environment,the algorithm proposed in this paper is experimentally verified.Experiments have shown that this algorithmcan effectively improve cache hit rate,enhance the performance of the platform,and reduce job execution time by 7.61%compared to existing improved algorithms.

Extraction algorithm for longitudinal and transverse mechanical information of AFM

The atomic force microscope(AFM)can measure nanoscale morphology and mechanical properties and has a wide range of applications.The traditional method for measuring the mechanical properties of a sample does so for the longitudinal and transverse properties separately,ignoring the coupling between them.In this paper,a data processing and multidimensional mechanical information extraction algorithm for the composite mode of peak force tapping and torsional resonance is proposed.On the basis of a tip–sample interaction model for the AFM,longitudinal peak force data are used to decouple amplitude and phase data of transverse torsional resonance,accurately identify the tip–sample longitudinal contact force in each peak force cycle,and synchronously obtain the corresponding characteristic images of the transverse amplitude and phase.Experimental results show that the measured longitudinal mechanical characteristics are consistent with the transverse amplitude and phase characteristics,which verifies the effectiveness of the method.Thus,a new method is provided for the measurement of multidimensional mechanical characteristics using the AFM.