On a risk model with Markovian arrivals and tax

A risk model with Markovian arrivals and tax payments is considered.When the insurer is in a profitable situation,the insurer may pay a certain proportion of the premium income as tax payments.First,the Laplace transform of the time to cross a certain level before ruin is discussed.Second,explicit formulas for a generalized Gerber-Shiu function are established in terms of the＇original＇Gerber-Shiu function without tax and the Laplace transform of the first passage time before ruin.Finally,the differential equations satisfied by the expected accumulated discounted tax payments until ruin are derived.An explicit expression for the discounted tax payments is also given.

The MAP/PH(PH/PH)/1 Discrete-time Queuing System with Repairable Server

In this paper, we discuss a discrete time repairable queuing system with Markovian arrival process, where lifetime of server, service time and repair time of server are all discrete phase type random variables. Using the theory of matrix geometric solution, we give the steady state distribution of queue length and waiting time. In addition, the stable availability of the system is also provided.

Price-Based Residential Demand Response Management in Smart Grids:A Reinforcement Learning-Based Approach

This paper studies price-based residential demand response management(PB-RDRM)in smart grids,in which non-dispatchable and dispatchable loads(including general loads and plug-in electric vehicles(PEVs))are both involved.The PB-RDRM is composed of a bi-level optimization problem,in which the upper-level dynamic retail pricing problem aims to maximize the profit of a utility company(UC)by selecting optimal retail prices(RPs),while the lower-level demand response(DR)problem expects to minimize the comprehensive cost of loads by coordinating their energy consumption behavior.The challenges here are mainly two-fold:1)the uncertainty of energy consumption and RPs;2)the flexible PEVs’temporally coupled constraints,which make it impossible to directly develop a model-based optimization algorithm to solve the PB-RDRM.To address these challenges,we first model the dynamic retail pricing problem as a Markovian decision process(MDP),and then employ a model-free reinforcement learning(RL)algorithm to learn the optimal dynamic RPs of UC according to the loads’responses.Our proposed RL-based DR algorithm is benchmarked against two model-based optimization approaches(i.e.,distributed dual decomposition-based(DDB)method and distributed primal-dual interior(PDI)-based method),which require exact load and electricity price models.The comparison results show that,compared with the benchmark solutions,our proposed algorithm can not only adaptively decide the RPs through on-line learning processes,but also achieve larger social welfare within an unknown electricity market environment.

CONSTRUCTION OF CONTINUOUS TIME MARKOVIAN ARRIVAL PROCESSES

Markovian arrival processes were introduced by Neuts in 1979 （Neuts 1979） and have been used extensively in the stochastic modeling of queueing, inventory, reliability, risk, and telecommunications systems. In this paper, we introduce a constructive approach to define continuous time Markovian arrival processes. The construction is based on Poisson processes, and is simple and intuitive. Such a construction makes it easy to interpret the parameters of Markovian arrival processes. The construction also makes it possible to establish rigorously basic equations, such as Kolmogorov differential equations, for Markovian arrival processes, using only elementary properties of exponential distributions and Poisson processes. In addition, the approach can be used to construct continuous time Markov chains with a finite number of states

Total duration of negative surplus for a MAP risk model

In this paper, we study the risk model with Markovian arrivals where we allow the surplus process to continue if the surplus falls below zero. We first derive expressions for the severity of ruin. Then by using the strong Markovian property of a two-dimensional Markov process and the expression for the severity of ruin, we obtain the Laplace transform of the total duration of negative surplus.

Performance Evaluation of Mobility Anchor Point with Guard Load Reservation in Hierarchical Mobile IPv6

Hierarchical mobile IPv6 （HMIPv6） introduces a mobility anchor point to reduce the signaling overhead and handoff latency. In this paper, we apply the matrix-analytical approach to explore the performance measures of the ongoing mobile nodes （MNs） drop and new MNs block probabilities of mobility anchor point with a guard bandwidth reservation scheme. We apply the Markovian arrival process （MAP） to model ongoing MNs and new MNs. Five related performance measures are derived, including the long-term new MN block and ongoing MN drop probabilities, and the three short-term measures of average length of a block period and a non-block period, as well as the conditional ongoing MN drop probability during a block period. These performance measures greatly assist the guard bandwidth reservation mechanism in determining a proper threshold guard bandwidth. The results presented in this paper can provide guidelines for designing adaptive algorithms to adjust the threshold in the guard bandwidth reservation scheme.

Determining Cost-Efficiency and Regulation Policy for Water Transfer of Lake Balaton by Stochastic Dynamic Programming

The present research is based upon a comprehensive survey which discusses the slightly tolerable water level of Balaton between 2000 and 2003. The low water level of the extreme period caused considerable problems in recreation. Our goal was to investigate the possible water transfer policies and the water level regulation policy of Lake Balaton by applying the dynamic programming of Markov chains. This iteration supports the cost-benefit analysis of different scenarios and also provides information about the best water governing policy. As a basis of our scientific analysis, Markov chains were created by ARMA （autoregressive moving average） synthetic data generator. Profit was joined to each transition-probability for the economic analysis. In our case the profit was negative, because the harmful effects of the low water level should be estimated, which is based on the calculated willingness-to-pay for improving the water quality of Lake Balaton. In addition, the profit includes the cost of different water supplement scenarios. After computer programming, the method proved to be an efficient tool to buttress the cost-benefit analysis of water supplement scenarios. The result highlights the importance of further climate change monitoring. Calculation confirmed water transfer to be cost-effective, yet scenarios with less ecological risk are also effective, thus preferable.

SINGLE SERVER QUEUES WITH A BATCH MARKOVIAN ARRIVAL PROCESS AND BULK RENEWAL OR NON-RENEWAL SERVICE

We first consider an infinite-buffer single server queue where arrivals occur according to a batch Markovian arrival process （BMAP）. The server serves customers in batches of maximum size ＇b＇ with a minimum threshold size ＇a＇. The service time of each batch follows general distribution independent of each other as well as the arrival process. The proposed analysis is based on the use of matrix-analytic procedure to obtain queue-length distribution at a post-departure epoch. Next we obtain queue-length distributions at various other epochs such as, pre-arrival, arbitrary and pre-service using relations with post-departure epoch. Later we also obtain the system-length distributions at post-departure and arbitrary epochs using queue-length distribution at post-departure epoch. Some important performance measures, like mean queue-lengths and mean waiting times have been obtained Total expected cost function per trait time is also derived to determine the locally optimal values of a and b. Secondly, we perform similar analysis for the corresponding infinite-buffer single server queue where arrivals occur according to a BMAP and service process in this case follows a non-renewal one, namely, Markovian service process （MSP）.

The MAP/PH/N Retrial Queue in a Random Environment

We consider the MAP/PH/N retrial queue with a finite number of sources operating in a finite state Markovian random environment. Two different types of multi-dimensional Markov chains are investigated describing the behavior of the system based on state space arrangements. The special features of the two formulations are discussed. The algorithms for calculating the stationary state probabilities are elaborated, based on which the main performance measures are obtained, and numerical examples are presented as well.

On a BMAP/G/1 G-queue with Setup Times and Multiple Vacations

In this paper, we consider a BMAP/G/1 G-queue with setup times and multiple vacations. Arrivals of positive customers and negative customers follow a batch Markovian arrival process （BMAP） and Markovian arrival process （MAP） respectively. The arrival of a negative customer removes all the customers in the system when the server is working. The server leaves for a vacation as soon as the system empties and is allowed to take repeated （multiple） vacations. By using the supplementary variables method and the censoring technique, we obtain the queue length distributions. We also obtain the mean of the busy period based on the renewal theory.

Critical links detection in stochastic networks: application to the transport networks

Purpose–The purpose of this paper is to study a multiple-origin-multiple-destination variant of dynamic critical nodes detection problem(DCNDP)and dynamic critical links detection problem(DCLDP)in stochastic networks.DCNDP and DCLDP consist of identifying the subset of nodes and links,respectively,whose deletion maximizes the stochastic shortest paths between all origins–destinations pairs,in the graph modeling the transport network.The identification of such nodes(or links)helps to better control the road traffic and predict the necessary measures to avoid congestion.Design/methodology/approach–A Markovian decision process is used to model the shortest path problem underdynamic trafficconditions.Effectivealgorithmstodeterminethe criticalnodes(links)whileconsideringthe dynamicity of the traffic network are provided.Also,sensitivity analysis toward capacity reduction for critical links is studied.Moreover,the complexity of the underlying algorithms is analyzed and the computational efficiency resulting from the decomposition operation of the network into communities is highlighted.Findings–The numerical results demonstrate that the use of dynamic shortest path(time dependency)as a metric has a significant impact on the identification of critical nodes/links and the experiments conducted on real world networks highlight the importance of sensitive links to dynamically detect critical links and elaborate smart transport plans.Research limitations/implications–The research in this paper also revealed several challenges,which call for future investigations.First,the authors have restricted our experimentation to a small network where the only focus is on the model behavior,in the absence of historical data.The authors intend to extend this study to very large network using real data.Second,the authors have considered only congestion to assess network’s criticality;future research on this topic may include other factors,mainly vulnerability.Practical implications–Taking into consideration the dynamic and stochastic nature in problem modeling enables to be effective tools for real-time control of transportation networks.This leads to design optimized smart transport plans particularly in disaster management,to improve the emergency evacuation effeciency.Originality/value–The paper provides a novel approach to solve critical nodes/links detection problems.In contrast to the majority of research works in the literature,the proposed model considers dynamicity and betweennesswhiletakingintoaccount the stochasticaspectof transportnetworks.Thisenables theapproach to guide the traffic and analyze transport networks mainly under disaster conditions in which networks become highly dynamic.

A Stochastic Inventory System with Postponed Demands and Infinite Pool in Discrete-Time Setup

In this article,we consider a discrete-time inventory model in which demands arrive according to a discrete Markovian arrival process.The inventory is replenished according to an es;ST policy,and the lead time is assumed to follow a discrete phase-type distribution.The demands that occur during stock-out periods either enter a pool which has an infinite capacity or leave the system with a predefined probability.The demands in the pool are selected one by one,if the on-hand inventory level is above s t 1;and the interval time between any two successive selections is assumed to have a discrete phase-type distribution.The joint probability distribution of the number of customers in the pool and the inventory level is obtained in the steady-state case.We derive the system performance measures under steady state and using these measures,the total expected cost rate of the system is calculated.The impacts of arrival rate on the performance measures are graphically illustrated.Finally,we study the impact of cost on the optimal values of the total expected cost rate,inventory level and the reorder point.