Pure State Feedback Switching Control Based on the Online Estimated State for Stochastic Open Quantum Systems

For the n-qubit stochastic open quantum systems,based on the Lyapunov stability theorem and LaSalle’s invariant set principle,a pure state switching control based on on-line estimated state feedback(short for OQST-SFC)is proposed to realize the state transition the pure state of the target state including eigenstate and superposition state.The proposed switching control consists of a constant control and a control law designed based on the Lyapunov method,in which the Lyapunov function is the state distance of the system.The constant control is used to drive the system state from an initial state to the convergence domain only containing the target state,and a Lyapunov-based control is used to make the state enter the convergence domain and then continue to converge to the target state.At the same time,the continuous weak measurement of quantum system and the quantum state tomography method based on the on-line alternating direction multiplier(QST-OADM)are used to obtain the system information and estimate the quantum state which is used as the input of the quantum system controller.Then,the pure state feedback switching control method based on the on-line estimated state feedback is realized in an n-qubit stochastic open quantum system.The complete derivation process of n-qubit QST-OADM algorithm is given;Through strict theoretical proof and analysis,the convergence conditions to ensure any initial state of the quantum system to converge the target pure state are given.The proposed control method is applied to a 2-qubit stochastic open quantum system for numerical simulation experiments.Four possible different position cases between the initial estimated state and that of the controlled system are studied and discussed,and the performances of the state transition under the corresponding cases are analyzed.

Event-based performance guaranteed tracking control for constrained nonlinear system via adaptive dynamic programming method

An optimal tracking control problem for a class of nonlinear systems with guaranteed performance and asymmetric input constraints is discussed in this paper.The control policy is implemented by adaptive dynamic programming(ADP)algorithm under two event-based triggering mechanisms.It is often challenging to design an optimal control law due to the system deviation caused by asymmetric input constraints.First,a prescribed performance control technique is employed to guarantee the tracking errors within predetermined boundaries.Subsequently,considering the asymmetric input constraints,a discounted non-quadratic cost function is introduced.Moreover,in order to reduce controller updates,an event-triggered control law is developed for ADP algorithm.After that,to further simplify the complexity of controller design,this work is extended to a self-triggered case for relaxing the need for continuous signal monitoring by hardware devices.By employing the Lyapunov method,the uniform ultimate boundedness of all signals is proved to be guaranteed.Finally,a simulation example on a mass–spring–damper system subject to asymmetric input constraints is provided to validate the effectiveness of the proposed control scheme.

Recent Progress in Reinforcement Learning and Adaptive Dynamic Programming for Advanced Control Applications

Reinforcement learning(RL) has roots in dynamic programming and it is called adaptive/approximate dynamic programming(ADP) within the control community. This paper reviews recent developments in ADP along with RL and its applications to various advanced control fields. First, the background of the development of ADP is described, emphasizing the significance of regulation and tracking control problems. Some effective offline and online algorithms for ADP/adaptive critic control are displayed, where the main results towards discrete-time systems and continuous-time systems are surveyed, respectively.Then, the research progress on adaptive critic control based on the event-triggered framework and under uncertain environment is discussed, respectively, where event-based design, robust stabilization, and game design are reviewed. Moreover, the extensions of ADP for addressing control problems under complex environment attract enormous attention. The ADP architecture is revisited under the perspective of data-driven and RL frameworks,showing how they promote ADP formulation significantly.Finally, several typical control applications with respect to RL and ADP are summarized, particularly in the fields of wastewater treatment processes and power systems, followed by some general prospects for future research. Overall, the comprehensive survey on ADP and RL for advanced control applications has d emonstrated its remarkable potential within the artificial intelligence era. In addition, it also plays a vital role in promoting environmental protection and industrial intelligence.

基于Online-GRU信道预测的星上自适应功率控制方法

针对传统卫星功率控制方法存在资源浪费、时延长的问题,提出一种基于在线-门控循环单元(Online-GRU)信道预测的星上自适应功率控制方法,通过在线训练更新网络参数来解决离线预测算法存在的累积误差的问题。仿真结果表明,提出的在线训练算法比离线算法预测精确度提升了38.30%,相比在线-长短期记忆网络(Online-LSTM)节约了63.21%的训练时间;提出的自适应功率控制方法比固定发射功率的方法节约了55.74%的发射功率;同时,相比基于地面定时反馈信道状态的自适应功率控制方法具备更好的鲁棒性。

Policy Iteration for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.

QoS-Aware Congestion Control with Online Learning

In emerging applications such as industrial control and autonomous driving,end-to-end deterministic quality of service(QoS)transmission guarantee has become an urgent problem to be solved.Internet congestion control algorithms are essential to the performance of applications.However,existing congestion control schemes follow the best-effort principle of data transmission without the perception of application QoS requirements.To enable data delivery within application QoS constraints,we leverage an online learning mechanism to design Crimson,a novel congestion control algorithm in which each sender continuously observes the gap between current performance and pre-defined QoS.Crimson can change rates adaptively that satisfy application QoS requirements as a result.Across many emulation environments and real-world experiments,our proposed scheme can efficiently balance the different trade-offs between throughput,delay and loss rate.Crimson also achieves consistent performance over a wide range of QoS constraints under diverse network scenarios.

Stability of Nonlinear Systems Using Optimal Fuzzy Controllers and Its Simulation by Java Programming

In this paper, at first, the single input rule modules(SIRMs) dynamically connected fuzzy inference model is used to stabilize a double inverted pendulum system. Then, a multiobjective particle swarm optimization(MOPSO) is implemented to optimize the fuzzy controller parameters in order to decrease the distance error of the cart and summation of the angle errors of the pendulums, simultaneously. The feasibility and efficiency of the proposed Pareto front is assessed in comparison with results reported in literature and obtained from other algorithms.Finally, the Java programming with applets is utilized to simulate the stability of the nonlinear system and explain the internetbased control.

Online Optimization in Power Systems With High Penetration of Renewable Generation:Advances and Prospects

Traditionally,offline optimization of power systems is acceptable due to the largely predictable loads and reliable generation.The increasing penetration of fluctuating renewable generation and internet-of-things devices allowing for fine-grained controllability of loads have led to the diminishing applicability of offline optimization in the power systems domain,and have redirected attention to online optimization methods.However,online optimization is a broad topic that can be applied in and motivated by different settings,operated on different time scales,and built on different theoretical foundations.This paper reviews the various types of online optimization techniques used in the power systems domain and aims to make clear the distinction between the most common techniques used.In particular,we introduce and compare four distinct techniques used covering the breadth of online optimization techniques used in the power systems domain,i.e.,optimization-guided dynamic control,feedback optimization for single-period problems,Lyapunov-based optimization,and online convex optimization techniques for multi-period problems.Lastly,we recommend some potential future directions for online optimization in the power systems domain.

Optimal Fixed-Point Tracking Control for Discrete-Time Nonlinear Systems via ADP

Based on adaptive dynamic programming(ADP),the fixed-point tracking control problem is solved by a value iteration(VI) algorithm. First, a class of discrete-time(DT)nonlinear system with disturbance is considered. Second, the convergence of a VI algorithm is given. It is proven that the iterative cost function precisely converges to the optimal value,and the control input and disturbance input also converges to the optimal values. Third, a novel analysis pertaining to the range of the discount factor is presented, where the cost function serves as a Lyapunov function. Finally, neural networks(NNs) are employed to approximate the cost function, the control law, and the disturbance law. Simulation examples are given to illustrate the effective performance of the proposed method.

A Novel Distributed Optimal Adaptive Control Algorithm for Nonlinear Multi-Agent Differential Graphical Games

In this paper, an online optimal distributed learning algorithm is proposed to solve leader-synchronization problem of nonlinear multi-agent differential graphical games. Each player approximates its optimal control policy using a single-network approximate dynamic programming(ADP) where only one critic neural network(NN) is employed instead of typical actorcritic structure composed of two NNs. The proposed distributed weight tuning laws for critic NNs guarantee stability in the sense of uniform ultimate boundedness(UUB) and convergence of control policies to the Nash equilibrium. In this paper, by introducing novel distributed local operators in weight tuning laws, there is no more requirement for initial stabilizing control policies. Furthermore, the overall closed-loop system stability is guaranteed by Lyapunov stability analysis. Finally, Simulation results show the effectiveness of the proposed algorithm.

On stochastic optimal control of partially observable nonlinear quasi Hamiltonian systems

A stochastic optimal control strategy for partially observable nonlinear quasi Hamiltonian systems is proposed. The optimal control forces consist of two parts. The first part is determined by the conditions under which the stochastic optimal control problem of a partially observable nonlinear system is converted into that of a completely observable linear system. The second part is determined by solving the dynamical programming equation derived by applying the stochastic averaging method and stochastic dynamical programming principle to the completely observable linear control system. The response of the optimally controlled quasi Hamiltonian system is predicted by solving the averaged Fokker-Planck-Kolmogorov equation associated with the optimally controlled completely observable linear system and solving the Riccati equation for the estimated error of system states. An example is given to illustrate the procedure and effectiveness of the proposed control strategy.

Online Learning Control for Harmonics Reduction Based on Current Controlled Voltage Source Power Inverters

Nonlinear loads in the power distribution system cause non-sinusoidal currents and voltages with harmonic components.Shunt active filters(SAF) with current controlled voltage source inverters(CCVSI) are usually used to obtain balanced and sinusoidal source currents by injecting compensation currents.However,CCVSI with traditional controllers have a limited transient and steady state performance.In this paper,we propose an adaptive dynamic programming(ADP) controller with online learning capability to improve transient response and harmonics.The proposed controller works alongside existing proportional integral(PI) controllers to efficiently track the reference currents in the d-q domain.It can generate adaptive control actions to compensate the PI controller.The proposed system was simulated under different nonlinear(three-phase full wave rectifier) load conditions.The performance of the proposed approach was compared with the traditional approach.We have also included the simulation results without connecting the traditional PI control based power inverter for reference comparison.The online learning based ADP controller not only reduced average total harmonic distortion by 18.41%,but also outperformed traditional PI controllers during transients.

Data-based Fault Tolerant Control for Affine Nonlinear Systems Through Particle Swarm Optimized Neural Networks

In this paper, a data-based fault tolerant control(FTC) scheme is investigated for unknown continuous-time(CT)affine nonlinear systems with actuator faults. First, a neural network(NN) identifier based on particle swarm optimization(PSO) is constructed to model the unknown system dynamics. By utilizing the estimated system states, the particle swarm optimized critic neural network(PSOCNN) is employed to solve the Hamilton-Jacobi-Bellman equation(HJBE) more efficiently.Then, a data-based FTC scheme, which consists of the NN identifier and the fault compensator, is proposed to achieve actuator fault tolerance. The stability of the closed-loop system under actuator faults is guaranteed by the Lyapunov stability theorem. Finally, simulations are provided to demonstrate the effectiveness of the developed method.

Neural-Network-Based Control for Discrete-Time Nonlinear Systems with Input Saturation Under Stochastic Communication Protocol

In this paper,an adaptive dynamic programming(ADP)strategy is investigated for discrete-time nonlinear systems with unknown nonlinear dynamics subject to input saturation.To save the communication resources between the controller and the actuators,stochastic communication protocols(SCPs)are adopted to schedule the control signal,and therefore the closed-loop system is essentially a protocol-induced switching system.A neural network(NN)-based identifier with a robust term is exploited for approximating the unknown nonlinear system,and a set of switch-based updating rules with an additional tunable parameter of NN weights are developed with the help of the gradient descent.By virtue of a novel Lyapunov function,a sufficient condition is proposed to achieve the stability of both system identification errors and the update dynamics of NN weights.Then,a value iterative ADP algorithm in an offline way is proposed to solve the optimal control of protocol-induced switching systems with saturation constraints,and the convergence is profoundly discussed in light of mathematical induction.Furthermore,an actor-critic NN scheme is developed to approximate the control law and the proposed performance index function in the framework of ADP,and the stability of the closed-loop system is analyzed in view of the Lyapunov theory.Finally,the numerical simulation results are presented to demonstrate the effectiveness of the proposed control scheme.

Discussion on neighborhood optimal trajectory online correction algorithm and its application range

This paper presents a neighborhood optimal trajectory online correction algorithm considering terminal time variation,and investigates its application range.Firstly,the motion model of midcourse guidance is established,and the online trajectory correction-regenerating strategy is introduced.Secondly,based on the neighborhood optimal control theory,a neighborhood optimal trajectory online correction algorithm considering the terminal time variation is proposed by adding the consideration of terminal time variation to the traditional neighborhood optimal trajectory correction method.Thirdly,the Monte Carlo simulation method is used to analyze the application range of the algorithm,which provides a basis for the division of application domain of the online correction algorithm and the online regeneration algorithm of midcourse guidance trajectory.Finally,the simulation results show that the algorithm has high real-time performance,and the online correction trajectory can meet the requirements of terminal constraint change.The application range of the algorithm is obtained through Monte Carlo simulation.

Adaptive Control of Discrete-time Nonlinear Systems Using ITF-ORVFL

Random vector functional ink(RVFL)networks belong to a class of single hidden layer neural networks in which some parameters are randomly selected.Their network structure in which contains the direct links between inputs and outputs is unique,and stability analysis and real-time performance are two difficulties of the control systems based on neural networks.In this paper,combining the advantages of RVFL and the ideas of online sequential extreme learning machine(OS-ELM)and initial-training-free online extreme learning machine(ITFOELM),a novel online learning algorithm which is named as initial-training-free online random vector functional link algo rithm(ITF-ORVFL)is investigated for training RVFL.The link vector of RVFL network can be analytically determined based on sequentially arriving data by ITF-ORVFL with a high learning speed,and the stability for nonlinear systems based on this learning algorithm is analyzed.The experiment results indicate that the proposed ITF-ORVFL is effective in coping with nonparametric uncertainty.

Analyzing Student Behavior in Online Programming Courses

Rather than maintaining the classic teaching approach, a growing number of schools use the blended learning system in higher education. The traditional method of teaching focuses on the result of students' progress. However, many student activities are recorded by an online programming learning platform at present. In this paper, we focus on student behavior when completing an online open-ended programming task. First, we conduct statistical analysis to examine student behavior on the basis of test times and completed time. By combining these two factors, we then classify student behavior into four types by using k-means algorithm. The results are useful for teachers to enhance their understanding of student learning and for students to know their learning style in depth. The findings are also valuable to re-design the learning platform.

Behavior analysis and formative assessments in online oral medicine education during the COVID-19 pandemic

BACKGROUND During the coronavirus disease 2019(COVID-19)pandemic,traditional teaching methods were disrupted and online teaching became a new topic in education reform and informatization.In this context,it is important to investigate the necessity and effectiveness of online teaching methods for medical students.This study explored stomatology education in China to evaluate the development and challenges facing the field using massive open online courses(MOOCs)for oral medicine education during the pandemic.AIM To investigate the current situation and challenges facing stomatology education in China,and to assess the necessity and effectiveness of online teaching methods among medical students.METHODS Online courses were developed and offered on personal computers and mobile terminals.Behavioral analysis and formative assessments were conducted to evaluate the learning status of students.RESULTS The results showed that most learners had already completed MOOCs and achieved better results.Course behavior analysis and student surveys indicated that students enjoyed the learning experience.However,the development of oral MOOCs during the COVID-19 pandemic faced significant challenges.CONCLUSION This study provides insights into the potential of using MOOCs to support online professional learning and future teaching innovation,but emphasizes the need for careful design and positive feedback to ensure their success.

A novel Q-based online model updating strategy and its application in statistical process control for rubber mixing

To overcome the large time-delay in measuring the hardness of mixed rubber, rheological parameters were used to predict the hardness. A novel Q-based model updating strategy was proposed as a universal platform to track time-varying properties. Using a few selected support samples to update the model, the strategy could dramat- ically save the storage cost and overcome the adverse influence of low signal-to-noise ratio samples. Moreover, it could be applied to any statistical process monitoring system without drastic changes to them, which is practical for industrial practices. As examples, the Q-based strategy was integrated with three popular algorithms （partial least squares （PIE）, recursive PIE （RPLS）, and kernel PIE （KPIE）） to form novel regression ones, QPLS, QRPIE and QKPLS, respectively. The applications for predicting mixed rubber hardness on a large-scale tire plant in east China prove the theoretical considerations.