在概率阈值准则下马尔可夫策略的最优化算法

在一种新的概率阈值准则下讨论马尔可夫决策的最优解的算法问题.采用基于增益的过去累积值的方法,求解马尔可夫最优策略.

基于多步回溯Q(λ)的PSS最优控制方法的研究

电力系统稳定器(PSS)是用来产生能抑制低频电力系统振荡的励磁系统辅助控制信号,具备自学习和参数在线整定能力是未来智能电网PSS控制器的一个发展趋势。提出一种基于多步回溯Q(λ)学习的新颖电力系统稳定器设计方法。利用多步回溯Q(λ)控制器代替整个传统PSS作为励磁附加控制,并与传统PSS和Q学习控制器进行比较。仿真研究显示,引入基于多步回溯Q(λ)学习的PSS控制后显著增强了整个系统的鲁棒性,有效提高了系统抑制低频电力系统振荡的能力,较好地解决了Q学习控制器收敛速度慢的问题。

基于同步控制的重载列车纵向力仿真与研究

针对解决列车运输能力局限性的问题,对开行万吨重载列车进行了研究。通过建立列车纵向力模型,主要考虑车钩间隙与空气阻力两个影响因素,利用Matlab仿真的方法研究了车钩的受力情况与规律;在仿真的结果上对重载列车的同步控制性能进行了评价,引入了Markov策略以优化同步无线传输性能;结合1+1+1万吨列车的实际试验结果与数据,比较了仿真模拟结果与试验值,研究结果表明,两者基本上吻合,在实际运行线路中加入优化策略是可行的,能够实现对同步控制性能大幅度地优化,为进一步优化长大重载列车的同步控制性能奠定了基础。

Evolutionary Game Dynamics in a Fitness-Dependent Wright-Fisher Process with Noise

Evolutionary game dynamics in finite size populations can be described by a fitness-dependent Wright- Fisher process. We consider symmetric 2×2 games in a well-mixed population. In our model, two parameters to describe the level of player＇s rationality and noise intensity in environment are introduced. In contrast with the fixation probability method that used in a noiseless case, the introducing of the noise intensity parameter makes the process an ergodic Markov process and based on the limit distribution of the process, we can analysis the evolutionary stable strategy （ESS） of the games. We illustrate the effects of the two parameters on the ESS of games using the Prisoner＇s dilemma games （PDG） and the snowdrift games （SG）. We also compare the ESS of our model with that of the replicator dynamics in infinite size populations. The results are determined by simulation experiments.

Modeling the Sojourn Time of Items for In-Networ Cache Based on LRU Policy

To reduce network redundancy,innetwork caching is considered in many future Internet architectures,such as Information Centric Networking.In in-network caching system,the item sojourn time of LRU(Least Recently Used) replacement policy is an important issue for two reasons:firstly,LRU is one of the most common used cache policy;secondly,item sojourn time is positively correlated to the hit probability,so this metric parameter could be useful to design the caching system.However,to the best of our knowledge,the sojourn time hasn't been studied theoretically so far.In this paper,we first model the LRU cache policy by Markov chain.Then an approximate closedform expression of the item expectation sojourn time is provided through the theory of stochastic service system,which is a function of the item request rates and cache size.Finally,extensive simulation results are illustrated to show that the expression is a good approximation of the item sojourn time.

Evaluation and Optimization of the Mixed Redundancy Strategy in Cloud-Based Systems

Mixed redundancy strategies are generally used in cloud-based systems,with different node switch mechanisms from traditional fault-tolerant strategies.Existing studies often concentrate on optimizing a single strategy in cloud computing environment and ignore the impact of mixed redundancy strategies.Therefore,a model is proposed to evaluate and optimize the reliability and performance of cloud-based degraded systems subject to a mixed active and cold standby redundancy strategy.In this strategy,node switching is triggered by a continual monitoring and detection mechanism when active nodes fail.To evaluate the transient availability and the expected job completion rate of systems with such kind of strategy,a continuous-time Markov chain model is built on the state transition process and a numerical method is used to solve the model.To choose the optimal redundancy for the mixed strategy under system constraints,a greedy search algorithm is proposed after sensitivity analysis.Illustrative examples were presented to explain the process of calculating the transient probability of each system state and in turn,the availability and performance of the whole system.It was shown that the near-optimal redundancy solution could be obtained using the optimizationmethod.The comparison with optimization of the traditional mixed redundancy strategy proved that the system behavior was different using different kinds of mixed strategies and less redundancy was assigned for the new type of mixed strategy under the same system constraint.

Analysis Markov Delay Control Strategy for Smart Home Systems

with the development of science and technology, smart home systems require better, faster to meet the needs of human. In order to achieve this goal, the human-machine-items all need to interact each other with understand, efficient and speedy. Cps could unify combination with the human-machine-items; realize the interaction between the physical nformation and the cyber world. However, information interaction and the control task needs to be completed in a valid time. Therefore, the transform delay control strategy becomes more and more important. This paper analysis Markov delay control strategy for smart home systems, which might help the system decrease the transmission delay.

MPC-based energy management with adaptive Markov-chain prediction for a dual-mode hybrid electric vehicle

The energy management strategy is an important part of a hybrid electrical vehicle design. It is used to improve fuel economy and to sustain a proper battery state of charge by controlling the power components while satisfying various constraints and driving demands. However, achieving an optimal control performance is challenging due to the nonlinearities of the hybrid powertrain, the time varying constraints, and the dilemma in which controller complexity and real-time capability are generally conflicting objectives. In this paper, a real-time capable cascaded control strategy is proposed for a dual-mode hybrid electric vehicle that considers nonlinearities of the system and complies with all time-varying constraints. The strategy consists of a supervisory controller based on a non-linear model predictive control （MPC） with a long sampling time interval and a coordinating controller based on linear model predictive control with a short sampling time interval to deal with different dynamics of the system. Additionally, a novel data based methodology using adaptive Markov chains to predict future load demand is introduced. The predictive future information is used to improve controller performance. The proposed strategy is implemented on a real test-bed and experimental trials using unknown driving cycles are conducted. The results demonstrate the validity of the proposed approach and show that fuel economy is significantly improved compared with other methods.