Policy Iteration for Optimal Control of Discrete-Time Time-Varying Nonlinear Systems

Aimed at infinite horizon optimal control problems of discrete time-varying nonlinear systems,in this paper,a new iterative adaptive dynamic programming algorithm,which is the discrete-time time-varying policy iteration(DTTV)algorithm,is developed.The iterative control law is designed to update the iterative value function which approximates the index function of optimal performance.The admissibility of the iterative control law is analyzed.The results show that the iterative value function is non-increasingly convergent to the Bellman-equation optimal solution.To implement the algorithm,neural networks are employed and a new implementation structure is established,which avoids solving the generalized Bellman equation in each iteration.Finally,the optimal control laws for torsional pendulum and inverted pendulum systems are obtained by using the DTTV policy iteration algorithm,where the mass and pendulum bar length are permitted to be time-varying parameters.The effectiveness of the developed method is illustrated by numerical results and comparisons.

Iteration dependent interval based open‐closed‐loop iterative learning control for time varying systems with vector relative degree

For linear time varying(LTV)multiple input multiple output(MIMO)systems with vector relative degree,an open‐closed‐loop iterative learning control(ILC)strategy is developed in this article,where the time interval of operation is iteration dependent.To compensate the missing tracking signal caused by iteration dependent interval,the feedback control is introduced in ILC design.As the tracking signal of many continuous iterations is lost in a certain interval,the feedback control part can employ the tracking signal of current iteration for compensation.Under the assumption that the initial state vibrates around the desired initial state uniformly in mathematical expectation sense,the expectation of ILC tracking error can converge to zero as the number of iteration tends to infinity.Under the circumstance that the initial state varies around the desired initial state with a bound,as the number of iteration tends to infinity,the expectation of ILC tracking error can be driven to a bounded range,whose upper bound is proportional to the fluctuation.It is revealed that the convergence condition is dependent on the feed-forward control gains,while the feedback control can accelerate convergence speed by selecting appropriate feedback control gains.As a special case,the controlled system with integrated high relative degree is also addressed by proposing a simplified iteration dependent interval based open‐closed‐loop ILC method.Finally,the effectiveness of the developed iteration dependent interval based open‐closed‐loop ILC is illustrated by a simulation example with two cases on initial state.

Iteration and evaluation of shale oil development technology for continental rift lake basins

By benchmarking with the iteration of drilling technology,fracturing technology and well placement mode for shale oil and gas development in the United States and considering the geological characteristics and development difficulties of shale oil in the Jiyang continental rift lake basin,East China,the development technology system suitable for the geological characteristics of shale oil in continental rift lake basins has been primarily formed through innovation and iteration of the development,drilling and fracturing technologies.The technology system supports the rapid growth of shale oil production and reduces the development investment cost.By comparing it with the shale oil development technology in the United States,the prospect of the shale oil development technology iteration in continental rift lake basins is proposed.It is suggested to continuously strengthen the overall three-dimensional development,improve the precision level of engineering technology,upgrade the engineering technical indicator system,accelerate the intelligent optimization of engineering equipment,explore the application of complex structure wells,form a whole-process integrated quality management system from design to implementation,and constantly innovate the concept and technology of shale oil development,so as to promote the realization of extensive,beneficial and high-quality development of shale oil in continental rift lake basins.

Fast and Accurate Predictor-Corrector Methods Using Feedback-Accelerated Picard Iteration for Strongly Nonlinear Problems

Although predictor-corrector methods have been extensively applied,they might not meet the requirements of practical applications and engineering tasks,particularly when high accuracy and efficiency are necessary.A novel class of correctors based on feedback-accelerated Picard iteration(FAPI)is proposed to further enhance computational performance.With optimal feedback terms that do not require inversion of matrices,significantly faster convergence speed and higher numerical accuracy are achieved by these correctors compared with their counterparts;however,the computational complexities are comparably low.These advantages enable nonlinear engineering problems to be solved quickly and accurately,even with rough initial guesses from elementary predictors.The proposed method offers flexibility,enabling the use of the generated correctors for either bulk processing of collocation nodes in a domain or successive corrections of a single node in a finite difference approach.In our method,the functional formulas of FAPI are discretized into numerical forms using the collocation approach.These collocated iteration formulas can directly solve nonlinear problems,but they may require significant computational resources because of the manipulation of high-dimensionalmatrices.To address this,the collocated iteration formulas are further converted into finite difference forms,enabling the design of lightweight predictor-corrector algorithms for real-time computation.The generality of the proposed method is illustrated by deriving new correctors for three commonly employed finite-difference approaches:the modified Euler approach,the Adams-Bashforth-Moulton approach,and the implicit Runge-Kutta approach.Subsequently,the updated approaches are tested in solving strongly nonlinear problems,including the Matthieu equation,the Duffing equation,and the low-earth-orbit tracking problem.The numerical findings confirm the computational accuracy and efficiency of the derived predictor-corrector algorithms.

Value Iteration-Based Cooperative Adaptive Optimal Control for Multi-Player Differential Games With Incomplete Information

This paper presents a novel cooperative value iteration(VI)-based adaptive dynamic programming method for multi-player differential game models with a convergence proof.The players are divided into two groups in the learning process and adapt their policies sequentially.Our method removes the dependence of admissible initial policies,which is one of the main drawbacks of the PI-based frameworks.Furthermore,this algorithm enables the players to adapt their control policies without full knowledge of others’ system parameters or control laws.The efficacy of our method is illustrated by three examples.

Calculation of Mass Concrete Temperature Containing Cooling Water Pipe Based on Substructure and Iteration Algorithm

Mathematical physics equations are often utilized to describe physical phenomena in various fields of science and engineering.One such equation is the Fourier equation,which is a commonly used and effective method for evaluating the effectiveness of temperature control measures for mass concrete.One important measure for temperature control in mass concrete is the use of cooling water pipes.However,the mismatch of grids between large-scale concrete models and small-scale cooling pipe models can result in a significant waste of calculation time when using the finite element method.Moreover,the temperature of the water in the cooling pipe needs to be iteratively calculated during the thermal transfer process.The substructure method can effectively solve this problem,and it has been validated by scholars.The Abaqus/Python secondary development technology provides engineers with enough flexibility to combine the substructure method with an iteration algorithm,which enables the creation of a parametric modeling calculation for cooling water pipes.This paper proposes such a method,which involves iterating the water pipe boundary and establishing the water pipe unit substructure to numerically simulate the concrete temperature field that contains a cooling water pipe.To verify the feasibility and accuracy of the proposed method,two classic numerical examples were analyzed.The results showed that this method has good applicability in cooling pipe calculations.When the value of the iteration parameterαis 0.4,the boundary temperature of the cooling water pipes can meet the accuracy requirements after 4∼5 iterations,effectively improving the computational efficiency.Overall,this approach provides a useful tool for engineers to analyze the temperature control measures accurately and efficiently for mass concrete,such as cooling water pipes,using Abaqus/Python secondary development.

Fault Estimation for a Class of Markov Jump Piecewise-Affine Systems: Current Feedback Based Iterative Learning Approach

In this paper, the issues of stochastic stability analysis and fault estimation are investigated for a class of continuoustime Markov jump piecewise-affine(PWA) systems against actuator and sensor faults. Firstly, a novel mode-dependent PWA iterative learning observer with current feedback is designed to estimate the system states and faults, simultaneously, which contains both the previous iteration information and the current feedback mechanism. The auxiliary feedback channel optimizes the response speed of the observer, therefore the estimation error would converge to zero rapidly. Then, sufficient conditions for stochastic stability with guaranteed performance are demonstrated for the estimation error system, and the equivalence relations between the system information and the estimated information can be established via iterative accumulating representation.Finally, two illustrative examples containing a class of tunnel diode circuit systems are presented to fully demonstrate the effectiveness and superiority of the proposed iterative learning observer with current feedback.

Noise-Tolerant ZNN-Based Data-Driven Iterative Learning Control for Discrete Nonaffine Nonlinear MIMO Repetitive Systems

Aiming at the tracking problem of a class of discrete nonaffine nonlinear multi-input multi-output(MIMO) repetitive systems subjected to separable and nonseparable disturbances, a novel data-driven iterative learning control(ILC) scheme based on the zeroing neural networks(ZNNs) is proposed. First, the equivalent dynamic linearization data model is obtained by means of dynamic linearization technology, which exists theoretically in the iteration domain. Then, the iterative extended state observer(IESO) is developed to estimate the disturbance and the coupling between systems, and the decoupled dynamic linearization model is obtained for the purpose of controller synthesis. To solve the zero-seeking tracking problem with inherent tolerance of noise,an ILC based on noise-tolerant modified ZNN is proposed. The strict assumptions imposed on the initialization conditions of each iteration in the existing ILC methods can be absolutely removed with our method. In addition, theoretical analysis indicates that the modified ZNN can converge to the exact solution of the zero-seeking tracking problem. Finally, a generalized example and an application-oriented example are presented to verify the effectiveness and superiority of the proposed process.

Low-complexity signal detection for massive MIMO systems via trace iterative method

Linear minimum mean square error(MMSE)detection has been shown to achieve near-optimal performance for massive multiple-input multiple-output(MIMO)systems but inevitably involves complicated matrix inversion,which entails high complexity.To avoid the exact matrix inversion,a considerable number of implicit and explicit approximate matrix inversion based detection methods is proposed.By combining the advantages of both the explicit and the implicit matrix inversion,this paper introduces a new low-complexity signal detection algorithm.Firstly,the relationship between implicit and explicit techniques is analyzed.Then,an enhanced Newton iteration method is introduced to realize an approximate MMSE detection for massive MIMO uplink systems.The proposed improved Newton iteration significantly reduces the complexity of conventional Newton iteration.However,its complexity is still high for higher iterations.Thus,it is applied only for first two iterations.For subsequent iterations,we propose a novel trace iterative method(TIM)based low-complexity algorithm,which has significantly lower complexity than higher Newton iterations.Convergence guarantees of the proposed detector are also provided.Numerical simulations verify that the proposed detector exhibits significant performance enhancement over recently reported iterative detectors and achieves close-to-MMSE performance while retaining the low-complexity advantage for systems with hundreds of antennas.

Mean Dimension for Non-autonomous Iterated Function Systems

In this paper we introduce the notions of mean dimension and metric mean dimension for non-autonomous iterated function systems(NAIFSs for short)on countably infinite alphabets which can be regarded as generalizations of the mean dimension and the Lindenstrauss metric mean dimension for non-autonomous iterated function systems.We also show the relationship between the mean topological dimension and the metric mean dimension.

Another SSOR Iteration Method

Kellogg gave a version of the Peaceman-Radford method. In this paper, we introduce a SSOR iteration method which uses Kellogg’s method. The new algorithm has some advantages over the traditional SSOR algorithm. A Cyclic Reduction algorithm is introduced via a decoupling in Kellogg’s method.

Low-Complexity Signal Detection Based on Relaxation Iteration Method in Massive MIMO Systems

Minimum mean square error(MMSE) detection algorithm can achieve nearly optimal performance when the number of antennas at the base station(BS) is large enough compared to the number of users. But the traditional MMSE involves complicated matrix inversion. In this paper, we propose a modified MMSE algorithm which exploits the channel characteristics occurring in massive multiple-input multipleoutput(MIMO) channels and the relaxation iteration(RI) method to avoid the matrix inversion. A proper initial solution is given to accelerate the convergence speed. In addition, we point out that the channel estimation scheme used in channel hardening-exploiting message passing(CHEMP) receiver is very appropriate for our proposed detection algorithm. Simulation results verify that the proposed algorithm can achieve very close performance of the traditional MMSE algorithm with a small number of iterations.

Adaptive Optimal Control of Space Tether System for Payload Capture via Policy Iteration

The libration control problem of space tether system(STS)for post-capture of payload is studied.The process of payload capture will cause tether swing and deviation from the nominal position,resulting in the failure of capture mission.Due to unknown inertial parameters after capturing the payload,an adaptive optimal control based on policy iteration is developed to stabilize the uncertain dynamic system in the post-capture phase.By introducing integral reinforcement learning(IRL)scheme,the algebraic Riccati equation(ARE)can be online solved without known dynamics.To avoid computational burden from iteration equations,the online implementation of policy iteration algorithm is provided by the least-squares solution method.Finally,the effectiveness of the algorithm is validated by numerical simulations.

Application of a simultaneous iterations reconstruction technique for a 3-D water vapor tomography system

The simultaneous iterations rithms of the ART family. It is used reconstruction technique （SIRT） widely in tomography because of is one of several reconstruction algoits convenience in dealing with large sparse matrices. Its theoretical background and iteration model are discussed at the beginning of this paper. Then, the implementation of the SIRT to reconstruct the three-dimensional distribution of water vapor by simulation is discussed. The results show that the SIRT can function effectively in water vapor tomography, obtain rapid convergence, and be implemented more easily than inversion.

Using Walsh-m Composite Sequence and LDPC-QPSK Joint Iteration to Reduce ISI in Long-range Underwater Acoustic Systems

To reduce inter-symbol-interference (ISI) in underwater acoustic (UWA) communication systems, a method based on LDPC-QPSK joint iteration and Walsh-m composite sequence is proposed in this paper. The method is intended for use in long-range and low signal-to-noise ratio (SNR) UWA communications. At the transmitter, Walsh-m composite sequence is introduced to resist multipath effect. At the receiver, a soft-input soft-output (SISO) module is implemented in a joint iterative process between QPSK demodulator and LDPC decoder. This method is demonstrated in three types of UWA channel models: positive, negative and invariable sound velocity gradients channels. It is shown that through contrastive simulation experiments, this method is more efficient than conventional methods based on independent decoding and demodulation. After two rounds of joint iteration, the proposed method can obtain 2.5 dB over conventional method at BER of 10-5. Numerical results verify that the proposed method is a good candidate for long-range underwater acoustic communication systems.

An Iteration Method Based on Homotopy Function for Solving Polynomial Systems and Application to Mechanisms Problems

A new iterating method based on homotopy function is developed in this paper. All solutions can be found easily without the need of choosing proper initial values. Compared to the homotopy continuation method, the solution process of the present method is simplified, and the computation efficiency as well as the reliability for obtaining all solutions is also improved. By application of the method to the mechanisms problems, the results are satisfactory.

Reinforcement learning-based scheduling of multi-battery energy storage system

In this paper, a reinforcement learning-based multibattery energy storage system(MBESS) scheduling policy is proposed to minimize the consumers ’ electricity cost. The MBESS scheduling problem is modeled as a Markov decision process(MDP) with unknown transition probability. However, the optimal value function is time-dependent and difficult to obtain because of the periodicity of the electricity price and residential load. Therefore, a series of time-independent action-value functions are proposed to describe every period of a day. To approximate every action-value function, a corresponding critic network is established, which is cascaded with other critic networks according to the time sequence. Then, the continuous management strategy is obtained from the related action network. Moreover, a two-stage learning protocol including offline and online learning stages is provided for detailed implementation in real-time battery management. Numerical experimental examples are given to demonstrate the effectiveness of the developed algorithm.

Complex Mode Frequency Iteration Method for Flutter Analysis of 2-DOF Systems

For a vibration system with 2 DOF of bend and torsion, its critical flutter wind speed can be calculated by using complex mode frequency iteration (CMFI) method based on MatLab 5.2, the results of which are in agree with those acquired by wind tunnel test. Not only critical flutter wind speed, but also vibration characteristic of a system under different wind speeds can be determined. CMFI method is suitable for both of separated flow torsional flutter and classic coupling flutter analysis, which is presented by flutter analysis of an ideal thin plate and a bluff bridge deck. Furthermore, it is proved through the investigation of the relationship between flutter derivatives and its critical flutter wind speed that coupling aerodynamic derivatives are necessary for classic coupling flutter to occur.]

Variational Iteration Method for Solving Time Fractional Burgers Equation Using Maple

The Time Fractional Burger equation was solved in this study using the Mabel software and the Variational Iteration approach. where a number of instances of the Time Fractional Burger Equation were handled using this technique. Tables and images were used to present the collected numerical results. The difference between the exact and numerical solutions demonstrates the effectiveness of the Mabel program’s solution, as well as the accuracy and closeness of the results this method produced. It also demonstrates the Mabel program’s ability to quickly and effectively produce the numerical solution.

A novel three-step implicit iteration process for threeΦ-hemicontractive mapping in the intermediate sense

In this paper,we introduce a three-step composite implicit iteration process for approximating the common fixed point of three uniformly continuous and asymptotically generalizedΦ-hemicontractive mappings in the intermediate sense.We prove that our proposed iteration process converges to the common fixed point of three finite family of asymptotically generalizedΦ-hemicontractive mappings in the intermediate sense.Our results extends,improves and complements several known results in literature.