This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-d...This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an （infinite-dimensional） linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.展开更多
This paper is concerned with the convergence of a sequence of discrete-time Markov decision processes(DTMDPs)with constraints,state-action dependent discount factors,and possibly unbounded costs.Using the convex analy...This paper is concerned with the convergence of a sequence of discrete-time Markov decision processes(DTMDPs)with constraints,state-action dependent discount factors,and possibly unbounded costs.Using the convex analytic approach under mild conditions,we prove that the optimal values and optimal policies of the original DTMDPs converge to those of the"limit"one.Furthermore,we show that any countablestate DTMDP can be approximated by a sequence of finite-state DTMDPs,which are constructed using the truncation technique.Finally,we illustrate the approximation by solving a controlled queueing system numerically,and give the corresponding error bound of the approximation.展开更多
To enhance dynamic tracking performance and anti-disturbance capacity of finite impulse response(FIR) filters, variable discount factors are introduced to the recursive least squares(RLS) algorithm. By employing impro...To enhance dynamic tracking performance and anti-disturbance capacity of finite impulse response(FIR) filters, variable discount factors are introduced to the recursive least squares(RLS) algorithm. By employing improved FIR filters to conduct modelling of industrial robot drive systems, dynamic characteristics of the target systems are identified. Then the fault detection for a target system can be utilized by analyzing the coefficients of the FIR filter. Finally, an application of the fault detection scheme to a kind of brushless DC motor drive system is described. Compared with reference methods, the proposed scheme achieves effective fault detection and performs better in dynamic tracking and robustness according to the final simulation results.展开更多
永磁同步电机(permanent magnet synchronous motor,PMSM)的磁链准确辨识是实现高性能电机控制的基础。针对传统递推最小二乘(recursive least squares,RLS)法受噪声影响小但存在数据饱和,影响辨识精度和动态性问题,以及遗忘最小二乘(re...永磁同步电机(permanent magnet synchronous motor,PMSM)的磁链准确辨识是实现高性能电机控制的基础。针对传统递推最小二乘(recursive least squares,RLS)法受噪声影响小但存在数据饱和,影响辨识精度和动态性问题,以及遗忘最小二乘(recursive least squares with forgetting factor,FRLS)法避免数据饱和但存在参数估计误差与动态跟踪性能矛盾的问题,文章提出一种基于折息最小二乘(recursive least squares with discount factor,DRLS)法的磁链辨识方法。该算法在FRLS法中引入加权因子构成折息因子,采用递推方法进行磁链辨识,减小参数估计误差,提高磁链辨识精度及动态跟踪能力。通过MATLAB仿真及半实物仿真试验,验证所提磁链识别方法的有效性。展开更多
基金This work was supported in part by the National Natural Science Foundation of China （Grant Nos. 61374067, 41271076）.
文摘This paper focuses on the constrained optimality problem （COP） of first passage discrete-time Markov decision processes （DTMDPs） in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an （infinite-dimensional） linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.
基金supported by National Natural Science Foundation of China (Grant Nos. 61374067 and 41271076)
文摘This paper is concerned with the convergence of a sequence of discrete-time Markov decision processes(DTMDPs)with constraints,state-action dependent discount factors,and possibly unbounded costs.Using the convex analytic approach under mild conditions,we prove that the optimal values and optimal policies of the original DTMDPs converge to those of the"limit"one.Furthermore,we show that any countablestate DTMDP can be approximated by a sequence of finite-state DTMDPs,which are constructed using the truncation technique.Finally,we illustrate the approximation by solving a controlled queueing system numerically,and give the corresponding error bound of the approximation.
基金Supported by the Provincial Training Program of Innovation and Entrepreneurship for Undergraduates (202013571002Z)。
文摘To enhance dynamic tracking performance and anti-disturbance capacity of finite impulse response(FIR) filters, variable discount factors are introduced to the recursive least squares(RLS) algorithm. By employing improved FIR filters to conduct modelling of industrial robot drive systems, dynamic characteristics of the target systems are identified. Then the fault detection for a target system can be utilized by analyzing the coefficients of the FIR filter. Finally, an application of the fault detection scheme to a kind of brushless DC motor drive system is described. Compared with reference methods, the proposed scheme achieves effective fault detection and performs better in dynamic tracking and robustness according to the final simulation results.
文摘永磁同步电机(permanent magnet synchronous motor,PMSM)的磁链准确辨识是实现高性能电机控制的基础。针对传统递推最小二乘(recursive least squares,RLS)法受噪声影响小但存在数据饱和,影响辨识精度和动态性问题,以及遗忘最小二乘(recursive least squares with forgetting factor,FRLS)法避免数据饱和但存在参数估计误差与动态跟踪性能矛盾的问题,文章提出一种基于折息最小二乘(recursive least squares with discount factor,DRLS)法的磁链辨识方法。该算法在FRLS法中引入加权因子构成折息因子,采用递推方法进行磁链辨识,减小参数估计误差,提高磁链辨识精度及动态跟踪能力。通过MATLAB仿真及半实物仿真试验,验证所提磁链识别方法的有效性。
