DENSITY OF MARKOV SYSTEMS AND ZEROS OF CHEBYSHEV POLYNOMIALS

We raise and partly answer the question: whether there exists a Markov system with respect to which the zeros of the Chebyshev polynomials are dense, but the maximum length of a zero free interval of the nth Chebyshev polynomial does not tends to zero. We also draw the conclu- tion that a Markov system, under an additional assumption, is dense if and only if the maxi- mum length of a zero free interval of the nth associated Chebyshev polynomial tends to zero.

Performance sensitivities for parameterized Markov systems

It is known that the performance potentials (or equivalentiy, perturbation realization factors) can be used as building blocks for performance sensitivities of Markov systems. In parameterized systems, the changes in parameters may only affect some states, and the explicit transition probability matrix may not be known. In this paper, we use an example to show that we can use potentials to construct performance sensitivities in a more flexible way; only the potentials at the affected states need to be estimated, and the transition probability matrix need not be known. Policy iteration algorithms, which are simpler than the standard one, can be established.

Control of 2-D Semi-Markov Jump Systems:A View from Mode Generation Mechanism

Dear Editor,Two-dimensional(2-D) systems have wide applications in image data processing,gas absorption and fluid dynamics analysis [1]-[3].When there exist abrupt changes in 2-D systems,they are usually modeled by 2-D Markov jump systems(MJSs) or 2-D semi-Markov jump systems(SMJSs).This letter investigates the control of 2-D SMJSs based on a novel mode generation mechanism,which could avoid mode ambiguousness phenomenon caused by the evolution of system mode in two different directions.The criterion that guarantees the almost surely exponential stability of the system is obtained.A thermal process is studied to demonstrate the availability of the proposed method.

Fault Estimation for a Class of Markov Jump Piecewise-Affine Systems: Current Feedback Based Iterative Learning Approach

In this paper, the issues of stochastic stability analysis and fault estimation are investigated for a class of continuoustime Markov jump piecewise-affine(PWA) systems against actuator and sensor faults. Firstly, a novel mode-dependent PWA iterative learning observer with current feedback is designed to estimate the system states and faults, simultaneously, which contains both the previous iteration information and the current feedback mechanism. The auxiliary feedback channel optimizes the response speed of the observer, therefore the estimation error would converge to zero rapidly. Then, sufficient conditions for stochastic stability with guaranteed performance are demonstrated for the estimation error system, and the equivalence relations between the system information and the estimated information can be established via iterative accumulating representation.Finally, two illustrative examples containing a class of tunnel diode circuit systems are presented to fully demonstrate the effectiveness and superiority of the proposed iterative learning observer with current feedback.

基于改进Markov算法的电力线载波通信网络安全态势感知仿真研究

针对电力线载波通信网络安全态势感知单位运算时间较长且误差较大等问题,基于改进Markov算法研究一种新型通信网络安全态势感知方法。采用分区采集与降维运算数据预处理,去除电力线载波信号干扰因素。利用隶属关联矩阵挖掘网络安全要素特征,构建层次化Markov网络安全态势感知模型。利用BW算法寻找目标参数最优解,来确定感知目标点位置,缩短挖掘时间,提高感知精准度。经过试验验证,所提方法单位感知时间只有60~90 ms,多组并行感知均方误差不超过2%,表明所提方法能够满足电力线载波通信网络安全态势感知应用需求。

D-S理论和Markov链组合的桥梁性能退化预测研究

为准确预测桥梁性能退化,考虑到数据随机性和微小扰动发生状态跳跃,提出了一种D-S(Dempster-Shafer)证据理论和Markov链组合的桥梁性能退化组合预测模型和性能退化率的概念.该模型基于指数平滑(exponential smoothing,ES)方法获得新的预测数据序列,并利用Markov链和D-S理论不断进行优化,从而实现桥梁性能退化的组合预测.实际工程的应用结果表明:性能退化率可以直观地表征在梁性能退化的速度.其次,该模型的平均相对误差为1.54%,较于回归、灰色和模糊加权Markov链模型,精度分别提高了1.11%,0.88%和2.8%,而后验差比值为0.242,小于0.35;模型的标准差为9.021,相比其他模型分别减小了3.978,3.405和7.500,而变异系数为0.109,均小于其他模型,验证了组合预测模型在精度和稳定性方面的优越性,可为在役桥梁结构性能退化预测与维护提供理论基础.

非线性广义时滞Markov跳变系统的H_(∞)滑模控制

研究了一类转移速率部分未知的广义时滞Markov跳变系统的H_(∞)滑模控制问题.首先,对广义时滞Markov跳变系统设计了积分型滑模面,并求出对应的等效控制器.之后,构建Lyapunov泛函,通过线性矩阵不等式方法得到闭环系统随机容许且满足H_(∞)性能γ的充分条件,运用加减项方法以及引入自由权矩阵,使得到的结果降低了保守性;对滑模力学分析,设计了滑模控制器,使得系统能在有限时间内到达滑模面上.最后,通过数值算例与仿真验证了理论的可行性.

转移概率部分未知的离散时间Markov跳变系统Nash微分博弈

考虑到转移概率矩阵元素无法完全获悉,如何在转移概率部分未知的情境下研究离散时间Markov跳变系统Nash微分博弈是有待解决的问题之一,这一问题可以为转移概率部分未知的Markov跳变系统Nash微分博弈理论在管理问题上的应用提供理论支撑。基于此,本文首先研究单人博弈情形,即ε-次优控制问题,借助自由连接权矩阵和配方法,得到了ε-次优控制策略存在的充分性条件,并给出了成本函数上界的显式表达;然后延伸至双人博弈进行分析,得到了ε-次优Nash均衡策略存在的条件等价于求解双线性矩阵不等式和矩阵不等式的优化问题,并通过启发式算法求解优化问题得到ε-次优Nash均衡策略;最后通过数值算例证明了主要结论的有效性。

自适应事件触发机制下分布时滞Markov跳变网络化系统H_(∞)故障检测

传统事件触发机制中阈值的选择会对信号传输效率产生很大影响。针对此问题,研究了自适应事件触发机制下具有分布时滞的Markov网络化系统的故障检测问题。设计了一种模态相关的自适应事件触发机制,通过动态调整阈值,提高了网络资源利用率。考虑网络时延和数据丢包现象,同步设计触发器和故障检测滤波器。构造模态相关的Lyapunov泛数,根据积分不等式技术推导出使系统随机稳定且具有H_(∞)性能的充分条件。利用解耦方法和LMI技术求解出了故障检测滤波器参数。最后通过数值仿真验证了方案在节省网络资源上的优势。

采样控制下的Markov跳变系统稳定性与安全性分析

研究针对连续Markov跳变系统的稳定性和安全性问题通过状态采样控制器保证了其稳定性和安全性。首先,与通常只考虑Markov系统的稳定性不同的是,系统的安全性问题也被考虑了。其次,系统的安全性通过引入控制障碍函数的方法得到解决。因此,系统分析及计算变得极其复杂。最后,通过设计采样控制器保证了系统的稳定性和安全性,并提供了一个有效的数值例子验证了提出方法的有效性。

基于Markov模型的三种方案治疗胰腺癌的药物经济学分析

目的通过建立Markov模型,对三种治疗胰腺癌的方案进行药物经济学评价,为临床用药提供循证依据。方法参考药品价格、治疗成本、健康效用值、状态转移概率建立马尔可夫(Markov)模型,对兰州大学第一医院2017年6月~2022年5月三种方案的有效性和经济性进行评价,并进行敏感性分析。结果Markov模型模拟回乘分析结果显示60个周期(5年)后,吉西他滨联合白蛋白结合型紫杉醇方案组(AG方案)的成本-效果比为437903.43元/11.80质量调整生命月(QALMs);吉西他滨联合卡培他滨方案组(GX方案)的成本-效果比为162234.52元/6.58 QALMs;吉西他滨联合替吉奥方案组(GS方案)的成本-效果比为429029.92元/15.64 QALMs。与GX方案相比较,AG方案的成本高出275668.91元,效果高出5.22 QALMs,ICER却为负值,处于绝对劣势;GS方案比GX方案效果高出9.06 QALMs,ICER为正值。结论从成本-效果的角度来看,方案AG的成本最高,治疗效果也相对较差,处于绝对劣势;GS方案比GX方案效果高,ICER也为正值,但成本较高,GX方案为优选方案。从药物经济学的角度来看,GS方案和GX方案相对于AG方案具有更高的性价比,患者在接受治疗后能够获得更长的生存期和更好的生活质量,从而提高整体的经济效果。在实际应用中,需要结合患者的具体情况和医生的专业建议来制定最佳治疗方案。

Intrusion detection based on system calls and homogeneous Markov chains

A novel method for detecting anomalous program behavior is presented, which is applicable to hostbased intrusion detection systems that monitor system call activities. The method constructs a homogeneous Markov chain model to characterize the normal behavior of a privileged program, and associates the states of the Markov chain with the unique system calls in the training data. At the detection stage, the probabilities that the Markov chain model supports the system call sequences generated by the program are computed. A low probability indicates an anomalous sequence that may result from intrusive activities. Then a decision rule based on the number of anomalous sequences in a locality frame is adopted to classify the program＇s behavior. The method gives attention to both computational efficiency and detection accuracy, and is especially suitable for on-line detection. It has been applied to practical host-based intrusion detection systems.

Markov repairable systems with stochastic regimes switching

Compared with the classical Markov repairable system, the Markov repairable system with stochastic regimes switching introduced in the paper provides a more realistic description of the practical system. The system can be used to model the dynamics of a repairable system whose performance regimes switch according to the external conditions. For example, to satisfy the demand variation that is typical for the power and communication systems and reduce the cost, these systems usually adjust their operating regimes. The transition rate matrices under distinct operating regimes are assumed to be different and the sojourn times in distinct regimes are governed by a finite state Markov chain. By using the theory of Markov process, Ion channel theory, and Laplace transforms, the up time of the system are studied. A numerical example is given to illustrate the obtained results. The effect of sojourn times in distinct regimes on the availability and the up time are also discussed in the numerical example.

OPTIMAL CONTROL OF MARKOVIAN SWITCHING SYSTEMS WITH APPLICATIONS TO PORTFOLIO DECISIONS UNDER INFLATION

This article is concerned with a class of control systems with Markovian switching, in which an It5 formula for Markov-modulated processes is derived. Moreover, an optimal control law satisfying the generalized Hamilton-Jacobi-Bellman （HJB） equation with Markovian switching is characterized. Then, through the generalized HJB equation, we study an optimal consumption and portfolio problem with the financial markets of Markovian switching and inflation. Thus, we deduce the optimal policies and show that a modified Mutual Fund Theorem consisting of three funds holds. Finally, for the CRRA utility function, we explicitly give the optimal consumption and portfolio policies. Numerical examples are included to illustrate the obtained results.

Exponential stability of impulsive jump linear systems with Markov process

The exponential stability is investigated for a class of continuous time linear systems with a finite state Markov chain form process and the impulsive jump at switching moments. The conditions, based on the average dwell time and the ratio of expectation of the total time running on all unstable subsystems to the expectation of the total time running on all stable subsystems,assure the exponential stability with a desired stability degree of the system irrespective of the impact of impulsive jump. The uniformly bounded result is realized for the case in which switched system is subjected to the impulsive effect of the excitation signal at some switching moments.

Modeling and Design of Real-Time Pricing Systems Based on Markov Decision Processes

A real-time pricing system of electricity is a system that charges different electricity prices for different hours of the day and for different days, and is effective for reducing the peak and flattening the load curve. In this paper, using a Markov decision process (MDP), we propose a modeling method and an optimal control method for real-time pricing systems. First, the outline of real-time pricing systems is explained. Next, a model of a set of customers is derived as a multi-agent MDP. Furthermore, the optimal control problem is formulated, and is reduced to a quadratic programming problem. Finally, a numerical simulation is presented.

Accuracy analysis of a single-fault Markov model for FADEC system

Time-limited dispatching(TLD)analysis of the full authority digital engine control(FADEC)systems is an important part of the aircraft system safety analysis and a necessary task for the certification of commercial aircraft and aeroengines.In the time limited dispatch guidance document ARP5107B,a single-fault Markov model(MM)approach is proposed for TLD analysis.However,ARP5107B also requires that the loss of thrust control(LOTC)rate error calculated by applying the single-fault MM must be less than 5%when performing airworthiness certification.Firstly,the sources of accuracy errors in three kinds of MM are analyzed and specified through a case study of the general FADEC system,and secondly a two-fault MM considering maintenance policy is established through analyzing and calculating the expected repair time when two related faults happen.Finally,a specific FADEC system is given to study on the influence factors of accuracy error in the single-fault MM,and the results show that the accuracy error of the single-fault MM decreases with the increase of short or long prescribed dispatch time,and the range values of short time(ST)and long time(LT)are determined to satisfy the requirement of accuracy error within 5%.

Robust reliable H_∞ control for discrete-time Markov jump linear systems with actuator failures

The robust reliable H∞ control problem for discrete-time Markovian jump systems with actuator failures is studied. A more practical model of actuator failures than outage is considered. Based on the state feedback method, the resulting closed-loop systems are reliable in that they remain robust stochastically stable and satisfy a certain level of H∞ disturbance attenuation not only when all actuators are operational, but also in case of some actuator failures, The solvability condition of controllers can be equivalent to a feasibility problem of coupled linear matrix inequalities （LMIs）. A numerical example is also given to illustrate the design procedures and their effectiveness.

Robust H_∞ Control for Uncertain Markovian Jump Linear Time-Delay Systems

This paper studies the robust stochastic stabilization and robust H∞ control for linear time-delay systems with both Markovian jump parameters and unknown norm-bounded parameter uncertainties. This problem can be solved on the basis of stochastic Lyapunov approach and linear matrix inequality (LMI) technique. Sufficient conditions for the existence of stochastic stabilization and robust H∞ state feedback controller are presented in terms of a set of solutions of coupled LMIs. Finally, a numerical example is included to demonstrate the practicability of the proposed methods.

H_∞ Control for Nonlinear Stochastic Markov Systems with Time-Delay and Multiplicative Noise

This paper is concerned with the H_∞ control problem for a class of nonlinear stochastic Markov jump systems with time-delay and system state-, control input-and external disturbancedependent noise. Firstly, by solving a set of Hamilton-Jacobi inequalities(HJIs), the exponential mean square H_∞ controller design of delayed nonlinear stochastic Markov systems is presented. Secondly,by using fuzzy T-S model approach, the H_∞ controller can be designed via solving a set of linear matrix inequalities(LMIs) instead of HJIs. Finally, two numerical examples are provided to show the effectiveness of the proposed design methods.