Using approximate dynamic programming for multi-ESM scheduling to track ground moving targets

This paper researches the adaptive scheduling problem of multiple electronic support measures(multi-ESM) in a ground moving radar targets tracking application. It is a sequential decision-making problem in uncertain environment. For adaptive selection of appropriate ESMs, we generalize an approximate dynamic programming(ADP) framework to the dynamic case. We define the environment model and agent model, respectively. To handle the partially observable challenge, we apply the unsented Kalman filter(UKF) algorithm for belief state estimation. To reduce the computational burden, a simulation-based approach rollout with a redesigned base policy is proposed to approximate the long-term cumulative reward. Meanwhile, Monte Carlo sampling is combined into the rollout to estimate the expectation of the rewards. The experiments indicate that our method outperforms other strategies due to its better performance in larger-scale problems.

Approximate Dynamic Programming for Stochastic Resource Allocation Problems

A stochastic resource allocation model, based on the principles of Markov decision processes(MDPs), is proposed in this paper. In particular, a general-purpose framework is developed, which takes into account resource requests for both instant and future needs. The considered framework can handle two types of reservations(i.e., specified and unspecified time interval reservation requests), and implement an overbooking business strategy to further increase business revenues. The resulting dynamic pricing problems can be regarded as sequential decision-making problems under uncertainty, which is solved by means of stochastic dynamic programming(DP) based algorithms. In this regard, Bellman’s backward principle of optimality is exploited in order to provide all the implementation mechanisms for the proposed reservation pricing algorithm. The curse of dimensionality, as the inevitable issue of the DP both for instant resource requests and future resource reservations,occurs. In particular, an approximate dynamic programming(ADP) technique based on linear function approximations is applied to solve such scalability issues. Several examples are provided to show the effectiveness of the proposed approach.

THE CONVERGENCE OF APPROACH PENALTY FUNCTION METHOD FOR APPROXIMATE BILEVEL PROGRAMMING PROBLEM

In this paper, a new algorithm-approximate penalty function method is designed, which can be used to solve a bilevel optimization problem with linear constrained function. In this kind of bilevel optimization problem. the evaluation of the objective function is very difficult, so that only their approximate values can be obtained. This algorithm is obtained by combining penalty function method and approximation in bilevel programming. The presented algorithm is completely different from existing methods. That convergence for this algorithm is proved.

Approximate Dynamic Programming for Self-Learning Control

This paper introduces a self-learning control approach based on approximate dynamic programming. Dynamic programming was introduced by Bellman in the 1950's for solving optimal control problems of nonlinear dynamical systems. Due to its high computational complexity, the applications of dynamic programming have been limited to simple and small problems. The key step in finding approximate solutions to dynamic programming is to estimate the performance index in dynamic programming. The optimal control signal can then be determined by minimizing (or maximizing) the performance index. Artificial neural networks are very efficient tools in representing the performance index in dynamic programming. This paper assumes the use of neural networks for estimating the performance index in dynamic programming and for generating optimal control signals, thus to achieve optimal control through self-learning.

Two-stage robust optimization of power cost minimization problem in gunbarrel natural gas networks by approximate dynamic programming

In short-term operation of natural gas network,the impact of demand uncertainty is not negligible.To address this issue we propose a two-stage robust model for power cost minimization problem in gunbarrel natural gas networks.The demands between pipelines and compressor stations are uncertain with a budget parameter,since it is unlikely that all the uncertain demands reach the maximal deviation simultaneously.During solving the two-stage robust model we encounter a bilevel problem which is challenging to solve.We formulate it as a multi-dimensional dynamic programming problem and propose approximate dynamic programming methods to accelerate the calculation.Numerical results based on real network in China show that we obtain a speed gain of 7 times faster in average without compromising optimality compared with original dynamic programming algorithm.Numerical results also verify the advantage of robust model compared with deterministic model when facing uncertainties.These findings offer short-term operation methods for gunbarrel natural gas network management to handle with uncertainties.

Call for papers Journal of Control Theory and Applications Special issue on Approximate dynamic programming and reinforcement learning

Approximate dynamic programming (ADP) is a general and effective approach for solving optimal control and estimation problems by adapting to uncertain and nonconvex environments over time.

ASYMPTOTIC APPROXIMATION METHOD AND ITS CONVERGENCE ON SEMI-INFINITE PROGRAMMING

The aim of this article is to discuss an asymptotic approximation model and its convergence for the minimax semi-infinite programming problem. An asymptotic surrogate constraints method for the minimax semi-infinite programming problem is presented by making use of two general discrete approximation methods. Simultaneously, the consistence and the epi-convergence of the asymptotic approximation problem are discussed.

An Overview of Sequential Approximation in Topology Optimization of Continuum Structure

This paper offers an extensive overview of the utilization of sequential approximate optimization approaches in the context of numerically simulated large-scale continuum structures.These structures,commonly encountered in engineering applications,often involve complex objective and constraint functions that cannot be readily expressed as explicit functions of the design variables.As a result,sequential approximation techniques have emerged as the preferred strategy for addressing a wide array of topology optimization challenges.Over the past several decades,topology optimization methods have been advanced remarkably and successfully applied to solve engineering problems incorporating diverse physical backgrounds.In comparison to the large-scale equation solution,sensitivity analysis,graphics post-processing,etc.,the progress of the sequential approximation functions and their corresponding optimizersmake sluggish progress.Researchers,particularly novices,pay special attention to their difficulties with a particular problem.Thus,this paper provides an overview of sequential approximation functions,related literature on topology optimization methods,and their applications.Starting from optimality criteria and sequential linear programming,the other sequential approximate optimizations are introduced by employing Taylor expansion and intervening variables.In addition,recent advancements have led to the emergence of approaches such as Augmented Lagrange,sequential approximate integer,and non-gradient approximation are also introduced.By highlighting real-world applications and case studies,the paper not only demonstrates the practical relevance of these methods but also underscores the need for continued exploration in this area.Furthermore,to provide a comprehensive overview,this paper offers several novel developments that aim to illuminate potential directions for future research.

Stochastic level-value approximation for quadratic integer convex programming

We propose a stochastic level value approximation method for a quadratic integer convex minimizing problem in this paper. This method applies an importance sampling technique, and make use of the cross-entropy method to update the sample density functions. We also prove the asymptotic convergence of this algorithm, and report some numerical results to illuminate its effectiveness.

Policy iteration optimal tracking control for chaotic systems by using an adaptive dynamic programming approach

A policy iteration algorithm of adaptive dynamic programming（ADP） is developed to solve the optimal tracking control for a class of discrete-time chaotic systems. By system transformations, the optimal tracking problem is transformed into an optimal regulation one. The policy iteration algorithm for discrete-time chaotic systems is first described. Then,the convergence and admissibility properties of the developed policy iteration algorithm are presented, which show that the transformed chaotic system can be stabilized under an arbitrary iterative control law and the iterative performance index function simultaneously converges to the optimum. By implementing the policy iteration algorithm via neural networks,the developed optimal tracking control scheme for chaotic systems is verified by a simulation.

A Tabu Search Algorithm for Quadratic 0-1 Programming Problem

In this paper,quadratic 0-1 programming problem (I) is considered, in terms of its features quadratic 0-1 programming problem is solved by linear approxity heurstic algrothm and a developed tabu search ahgrothm .

Optimal Constrained Self-learning Battery Sequential Management in Microgrid Via Adaptive Dynamic Programming

This paper concerns a novel optimal self-learning battery sequential control scheme for smart home energy systems. The main idea is to use the adaptive dynamic programming U+0028 ADP U+0029 technique to obtain the optimal battery sequential control iteratively. First, the battery energy management system model is established, where the power efficiency of the battery is considered. Next, considering the power constraints of the battery, a new non-quadratic form performance index function is established, which guarantees that the value of the iterative control law cannot exceed the maximum charging/discharging power of the battery to extend the service life of the battery. Then, the convergence properties of the iterative ADP algorithm are analyzed, which guarantees that the iterative value function and the iterative control law both reach the optimums. Finally, simulation and comparison results are given to illustrate the performance of the presented method. © 2017 Chinese Association of Automation.

METHOD BASED ON DUAL-QUADRATIC PROGRAMMING FOR FRAME STRUCTURAL OPTIMIZATION WITH LARGE SCALE

The optimality criteria （OC） method and mathematical programming （MP） were combined to found the sectional optimization model of frame structures. Different methods were adopted to deal with the different constraints. The stress constraints as local constraints were approached by zero-order approximation and transformed into movable sectional lower limits with the full stress criterion. The displacement constraints as global constraints were transformed into explicit expressions with the unit virtual load method. Thus an approximate explicit model for the sectional optimization of frame structures was built with stress and displacement constraints. To improve the resolution efficiency, the dual-quadratic programming was adopted to transform the original optimization model into a dual problem according to the dual theory and solved iteratively in its dual space. A method called approximate scaling step was adopted to reduce computations and smooth the iterative process. Negative constraints were deleted to reduce the size of the optimization model. With MSC/Nastran software as structural solver and MSC/Patran software as developing platform, the sectional optimization software of frame structures was accomplished, considering stress and displacement constraints. The examples show that the efficiency and accuracy are improved.

A quadratic programming method for optimal degree reduction of Bézier curves with G^1-continuity

This paper presents a quadratic programming method for optimal multi-degree reduction of B6zier curves with G^1-continuity. The L2 and I2 measures of distances between the two curves are used as the objective functions. The two additional parameters, available from the coincidence of the oriented tangents, are constrained to be positive so as to satisfy the solvability condition. Finally, degree reduction is changed to solve a quadratic problem of two parameters with linear constraints. Applications of degree reduction of Bezier curves with their parameterizations close to arc-length parameterizations are also discussed.

Chebyshev Approximate Solution to Allocation Problem in Multiple Objective Surveys with Random Costs

In this paper, we consider an allocation problem in multivariate surveys as a convex programming problem with non-linear objective functions and a single stochastic cost constraint. The stochastic constraint is converted into an equivalent deterministic one by using chance constrained programming. The resulting multi-objective convex programming problem is then solved by Chebyshev approximation technique. A numerical example is presented to illustrate the computational procedure.

新能源电力系统不确定优化调度方法研究现状及展望

风电场和光伏电站出力的不确定性给电力系统优化调度带来很大技术挑战。文中主要介绍了考虑新能源不确定性的电力系统优化调度方法的研究现状及后续研究方向展望。首先,重点论述了各种不确定优化调度(UOD)方法,包括随机优化方法、鲁棒优化方法、随机鲁棒优化结合方法和基于人工智能技术的方法。其中,随机优化方法包括场景法、机会约束规划法和近似动态规划法;鲁棒优化方法包括传统鲁棒优化法和分布鲁棒优化法;随机鲁棒优化结合方法包括采样鲁棒优化法和分布鲁棒机会约束规划法。然后,介绍了每一种方法的优化模型形式、模型的转化和求解原理及其优缺点。最后,对UOD的后续重点研究方向进行展望,包括兼顾多个目标的UOD问题及多目标不确定优化方法、输配系统UOD问题及分布式不确定优化方法、考虑稳定性约束的UOD问题及含常微分方程约束的不确定优化方法、考虑管道传输动态的综合能源系统UOD问题及含偏微分方程约束的不确定优化方法。

近似动态规划在电力系统优化运行中的应用综述

高比例可再生能源的接入和日益扩大的系统规模给电力系统优化运行带来了巨大挑战。近似动态规划(ADP)能够有效处理维数灾问题,在求解复杂系统动态优化问题上具备强大优势,近年来成为运筹学领域的一大研究热点。文中对ADP及其在电力优化运行中的应用进行了综述。首先,阐述了ADP的基本思想、主要特点及其近似策略。其次,介绍了不同类型的ADP在电力系统优化运行中的应用现状。最后,讨论了当前ADP在理论和应用中的不足,并对其未来发展方向作了进一步的展望。

凸近似避障及采样区优化的智能车辆轨迹规划

针对结构化道路下作匀速运动的智能车辆避障轨迹规划问题,提出一种基于凸近似避障原理及采样区域优化的智能车辆轨迹规划方法。引入凸近似避障原理,得到轨迹可行域范围;将采样区域分为静态采样区、动态采样区两部分,并根据障碍物运动状态,另外划分动态、静态障碍物采样区;采用“动态规划(DP)+二次规划(QP)”思想求解轨迹:利用五次多项式对采样点依次连接,建立动态规划代价函数并筛选得到粗略轨迹;通过二次规划及约束条件的构造,对粗略轨迹进行平滑,最终得到最优轨迹。仿真结果表明:对于静态、低速、动态三种障碍物,该车能够有效地得到平滑轨迹并避开障碍物。

沿海区域综合能源生产单元随机优化调度

为实现沿海区域的海上风电场、海上采气平台和陆上热电联供燃气电厂等多种能源生产子单元的协同化运行,考虑可再生能源出力和氢负荷的随机波动,提出沿海区域综合能源生产单元(coastal integrated energy production units,CIEPU)随机优化调度模型。采用参数化代价函数近似(parametric cost function approximation,PCFA)的动态规划算法求解随机优化调度模型。通过一种基于梯度下降的求解方法--Adadelta法,获得策略函数的一阶信息,并计算梯度平方的指数衰减平均值,以更新策略函数的迭代步长;对随机优化调度模型进行策略参数逼近,从而得到近似最优的策略参数,并逐一时段求解出CIEPU的最优调度计划。最后,以某个CIEPU为例,分析计算结果表明,所提出方法获得的优化调度方案可以提高CIEPU运行的经济性并降低碳排放量,验证了所提方法的准确性和高效性。